JP2013246654A - Log management device, log management method and log management program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a log management device, a log management method and a log management program that enable efficient collation processing of received log data in accordance with an environment.SOLUTION: Log collation processing means 20 after receiving each of collected messages from log receiving means 10 transmits a collated message caused in association with collation processing to priority data update processing means 50. The priority data update processing means 50 after receiving this refers to a setting information storage part 80 to determine the presence or absence of message definition information to be stored in a priority list 71, and updates the priority list 71 on the basis of information relating to the determination result. Statistical data update processing means 60 updates a statistics list 72 using a counting list created on the basis of information in a log storage part 30. The log collation processing means 20 sequentially refers to the dynamically-updated priority list 71 and the regularly-updated statistics list 72 to execute efficient collation processing.

Description

  The present invention relates to a log management apparatus that is connected to various log collection target devices on a network and that efficiently collates log data acquired therefrom, a management method thereof, and a program thereof.

  An integrated log management system that processes log messages in an integrated manner is used as a response to internal data control, system audits, and subsequent investigations when an incident occurs in an in-house network. The integrated log management system refers to various types of devices such as log collection target devices such as client terminals and servers connected to the corporate network, applications installed in them, and network devices such as switches and routers. It is a system that collects, manages, and analyzes operation log data (logs) as a target.

  The integrated log management system collects and manages all corporate logs, enabling audits and systems such as audit trails stipulated by laws and regulations, status monitoring of collection target devices, and subsequent investigations when security incidents occur. It becomes possible to improve the management level in terms of operation and security.

  Here, in the following, one entry (one line) in the log data is referred to as a message. In other words, a set of messages is log data. In addition, the information that indicates “where and what information exists in the message” and “in what format” is called message definition information, and a set of message definition information. This is referred to as log format definition information. Further, the “where and what information is present in the message” is referred to as “the meaning of each message”.

  By the way, in order to perform analysis such as system audits and incident investigations on collected log data, it is necessary to understand the meaning of each message in the log management device. The collected log format definition information is used to collate messages collected from log collection target devices (hereinafter referred to as devices).

  Conventionally, this message collation process is performed sequentially in the order described in the log format definition information. In the message matching process, the message definition information to be compared and the message definition information to be compared are nothing but character string data, so each message definition information is defined in a fixed order in this log format definition information. ing. In other words, conventionally, collation processing depending on log format definition information in which each message definition information is arranged in a fixed order has been performed.

  In such a sequential collation process, collation efficiency greatly depends on the number of the corresponding message in the log format definition information. For example, when the message definition information related to an arbitrary message X is defined at the end of the log format definition information (for example, 10000th), the collation process (collation process with the 9999 message definition information) between reaching the end is performed. Since it is necessary, it takes an enormous amount of time to complete the processing. As a more specific example, in an environment where 1000 messages corresponding to the message definition information described in the 10000th of the log format definition information are generated in 1 second, collation to be performed in 1 second. Since the number of times is 10 million times (10000 × 1000), problems such as lost log data may occur.

  In addition, considering the fact that “when and what messages are generated from which messages and how much (amount)” depends on the environment in which the log management device is installed, as in the above example In addition, in an environment where a large number of messages corresponding to the message definition information at the end of the log format definition information are generated, inefficient collation processing is repeated many times. In this case, not only does the verification process take time, but there is a possibility that a message that should be collected is missed (log data is lost). Furthermore, an increase in the load of such inefficient collation processing affects analysis processing such as log data search.

  Under such circumstances, devices for log management have been improved conventionally. For example, the following technical contents (Patent Document 1 or 2) are known.

  The known example described in Patent Document 1 discloses a technique for determining whether or not security is required by managing log data for a plurality of different devices to be managed in an integrated manner. Further, in the known example described in Patent Document 2, when managing output log data, a storage device is used while ensuring security by using a table that can be organized for each log level (degree of information amount). The technique which aimed at effective use of is disclosed.

JP2010-205081 JP2008-15733

  However, in the prior art including Patent Documents 1 and 2 described above, each message definition information in the log format definition information is ranked and collated after considering the features appearing in the log data and the number of received messages. There is no disclosure of a technique that allows the operating environment of each log collection target device to be grasped flexibly and quickly by referring to the processing when executing the processing.

(Object of invention)
The present invention is to improve the inconvenience of the above-described conventional example, and in particular, to provide a log management device, a management method thereof, and a program thereof that efficiently perform matching processing of each received message according to the environment. And its purpose.

  In order to achieve the above object, the log management apparatus according to the present invention is a log management apparatus that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the operating environment of each device, Log receiving means for receiving a plurality of messages constituting log data, log matching processing means for executing matching processing for each message according to a predetermined order, operating environment and data transmission time zone for each log collection target device A log definition storage unit in which log format definition information classified based on the log definition storage unit is stored in advance, and the log collation processing unit performs collation processing among a plurality of log format definition information stored in the log definition storage unit. It is configured to execute by referring to any one of the above.

  Further, in the log management method according to the present invention, a log management apparatus that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the operating environment of each device, the plurality of log data constituting the log data Log received by the log receiving means, and then each message is sent by the log receiving means to the log matching processing means, and the log classified based on the operating environment and data transmission time zone for each log collection target device. The log processing unit selects log format definition information based on each message from the log definition storage unit in which the format definition information is stored in advance, and refers to the selected log format definition information. Is characterized by performing a collation process on each message received from the log receiving means.

  Furthermore, the log management program according to the present invention is a log management apparatus that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the operating environment of each device, and a plurality of log data constituting the log data Log receiving function that the log receiving means receives the message, log transmission function that the log receiving means sends the message to the log verification processing means, the operating environment and the data transmission time zone for each log collection target device Log definition selection function for selecting the log format definition information based on each message from the log definition storage unit in which the log format definition information is stored in advance, and the log format definition information selected by this log definition selection function By referring to the log verification processing means, the verification of each message received from the log reception means It logs collation processing function of performing management, characterized in that to realize the computer.

  As described above, the present invention employs a configuration that refers to log format definition information classified according to the environment of each log collection target device when collating each received message, and further, the log format definition information Since the message definition information in the list is arranged in advance, the matching process of each received message is efficiently executed according to the environment, and the message to be originally collected is efficiently collected and managed. It is possible to provide an excellent log management device, a management method thereof, and a program thereof.

It is the block diagram which showed the structure of the log management apparatus concerning embodiment of this invention. It is the block diagram which showed the structure of the log collation process means concerning embodiment of this invention. It is the block diagram which showed the structure of the priority data processing means concerning embodiment of this invention. It is the figure which showed the network structure of the log management apparatus concerning its embodiment of this invention, and its peripheral device. It is the schematic which illustrated the internal structure of the log format definition storage part concerning embodiment of this invention. It is the schematic which illustrated the internal structure of the setting information storage part concerning 1st Embodiment (when using an event) of this invention. It is the sequence diagram which showed the operation | movement of the collation process (when using an event) concerning 1st Embodiment of this invention. It is the figure which illustrated the message appearance frequency count result concerning embodiment of this invention. It is the sequence diagram which showed the update process of the regular statistical definition data concerning 1st Embodiment of this invention. It is the schematic which illustrated the internal structure of the setting information storage part concerning 2nd Embodiment (when an exceptional state is utilized) of this invention. It is the sequence diagram which showed the operation | movement of the collation process (when an exceptional state is utilized) concerning 2nd Embodiment of this invention.

[First embodiment: Collation processing when an event is used]
A log management device according to a first embodiment of the present invention will be described with reference to FIGS. Although the characteristics of log data are various, in the first embodiment, update of priority definition data using prediction of subsequent messages by events and usage thereof will be described. An event here is a specific phenomenon composed of a plurality of messages. Further, when using an event, it is necessary to define information related to the event in advance as in the setting information storage unit 80 illustrated in FIG.

(Overall configuration)
In FIG. 1, reference numeral 90 denotes a log management apparatus according to the first embodiment that performs collation processing of log data received from a plurality of log collection target devices (devices) and manages them. The log management device 90 stores a log receiving unit 10 that collects log data from various devices, a log collation processing unit 20 that collates each message related to log data received from the log receiving unit 10, and a collated message. Log storage unit 30 as a storage area to be stored, and log access processing unit 40 that stores the collated message received from log collation processing unit 20 in log storage unit 30 and delivers it appropriately, should be collated preferentially Priority data update processing means 50 for updating priority definition data 71A as message definition information, statistical data update processing means 60 for updating statistical definition data 72A in which all message definition information of the corresponding device is aligned, Based on the operating environment (system type) and data transmission time zone (unit time ID) for each device A log definition storage unit 70 that stores classified log format definition information, a setting information storage unit 80 that stores setting information necessary for the operation of the log management apparatus 90 according to the first embodiment, It is comprised by.

  The log definition storage unit 70 according to the first embodiment includes a plurality of areas divided based on the operating environment and data transmission time zone for each device. Each of the plurality of areas is provided with a priority list 71 in which priority definition data 71A is stored and a statistics list 72 in which statistical definition data 72A is stored. That is, the log format definition information according to the first embodiment includes priority definition data 71A and statistical definition data 72A.

  The priority definition data 71A is configured to be updated by the priority data update processing unit 50 as the collation processing by the log collation processing unit 20 proceeds. Also. The order of the message definition information in the statistical definition data 72A is updated by the statistical data update processing means 60. Further, the log collation processing unit 20 described above first refers to the message definition information (priority definition data 71A) in the priority list 71 when performing collation processing of the message received from the log reception unit 10, and then performs statistical processing. A function for referring to the message definition information (statistical definition data 72A) in the list 72 is provided.

  The setting information storage unit 80 stores the storage determination information 81, which is information necessary for the priority data update processing means 50 to determine whether or not there is message definition information to be the priority definition data 71A, and the priority list 71. Priority data exclusion information 82, which is information necessary for the priority data update processing means 50 to exclude the message definition information as the priority definition data 71A, and information for setting the unit time for holding the log format definition information Unit time setting information 83 is stored. The statistical definition data 72A described above is created by the statistical data update processing means 60 every unit time.

  The priority data update processing unit 50 determines whether or not “message definition information to be used as the priority definition data 71A (message definition information to be stored in the priority list 71) exists based on the collated message input from the log verification processing unit 20”. The priority data storage determination unit 51 that performs the “determination” by referring to the storage determination information 81, and message definition information corresponding to the information related to the determination result transmitted from the priority data storage determination unit 51 is displayed in the priority list 71. A priority data update unit 52 that performs storage processing as priority definition data 71A (a specific storage area in the log definition storage unit). The priority data storage determination unit 51 updates the storage determination information 81 depending on the result of the determination.

  The priority data update unit 52 has a configuration for updating the priority data exclusion information 82 (FIG. 3: exclusion information storage processing function 52A, exclusion information deletion function 52C), and priority when the exclusion time is reached. A configuration (FIG. 3: message exclusion function 52B) that excludes the message definition information from the priority list 71 by referring to the data exclusion information 82 is adopted.

  The log access processing unit 40 acquires the log data designated by the statistical data update processing unit 60 from the log storage unit 30 and delivers the log data to the statistical data update processing unit 60, and the log collation processing unit 20. The log storage processing unit 42 stores the collated message input from the log storage unit 30.

  When the end time of the unit time is reached, the statistical data update processing means 60 requests the log acquisition processing unit 41 to acquire all the log data collected in the current unit time and updates the statistical information based on the corresponding log data acquired there. A statistical information calculation processing unit 61 that performs the calculation, and a statistical data update unit 62 that updates the statistical definition data 72A based on the calculation result of the statistical information input from the statistical information calculation processing unit 61 are provided. The statistical data update unit 62 updates the storage determination information 81 and the unit time setting information 83 as the unit time ends, and notifies the statistical information calculation processing unit 61 of the next unit time information. It has become.

  Here, the collation process performed by the log collation processing means 20 will be described. The log collation processing unit 20 uses the unit time setting information 83 in the setting information storage unit 80 and the log format definition information stored in the log definition storage unit 70 when collating the collected message received from the log receiving unit 10. It is a configuration to refer to. With the collected message matching process, it is possible to uniquely determine in the log management apparatus 90 which message definition information the received message corresponds to in the log format definition information.

  In addition, the log collation processing unit 20 has a function of transmitting each collated message to the log storage processing unit 42 and the priority data update processing unit 50 of the log access processing unit 40. In the first embodiment, for convenience, a unique message ID is assigned to all message definition information. Therefore, the log matching processing unit 20 can determine the message ID of the received message.

  Next, the internal configuration of the log matching processing unit 20 shown in FIG. 2 will be described.

  The log verification processing means 20 described above includes the collection target IP address (device IP address), the operating environment (system type), and the unit time ID corresponding to the data transmission time confirmed by referring to the setting information storage unit 80. Specific data for selecting the log format definition information corresponding to the combination from the log definition storage unit 70 and executing the matching process by sequentially referring to the priority definition data 71A and the statistical definition data 72A in the selected log format definition information A verification unit 20A is provided. Further, the log collation processing unit 20 includes an update information transmission unit 20B that transmits the message ID, the collected control IP address, and the operating environment for the collated message to the priority data update processing unit 50. The information transmitted by the update information transmitting unit 20B is information necessary for storing or excluding specific message definition information (priority definition data 71A) in the priority list 71.

  As described above, the log collation processing unit 20 employs a configuration in which collation processing of each message is performed and information provision collation to the priority data update processing unit 50 is performed in parallel with this. According to this, it is possible to sequentially refer to the priority definition data 71A and the statistical definition data 72A updated by the rapid operation of each component based on the provided information, and an efficient collation process can be realized.

  Next, the priority data update processing means 50 shown in FIG. 3 will be described.

  As described above, the priority data update processing unit 50 determines whether or not there is a special situation such as an event or an exceptional state (failure state) based on the collated message input from the log collation processing unit 20. A priority data storage determination unit 51 and a priority data update unit 52 that updates the priority definition data 71A based on information related to the determination that is notified when the priority data storage determination unit 51 determines that the situation is a special situation. And.

  Further, the priority data storage determination unit 51 calculates an exclusion time calculation function 51A that calculates a time (exclusion time) at which message definition information is excluded from the priority definition data 71A based on a preset time set in the setting information storage unit 80. It has.

  Further, the priority data update unit 52 stores in the setting information storage unit 80 priority data exclusion information in which the exclusion time is added to the information related to the priority definition data 71A to be excluded from the priority list 71. From the function 52A, the message exclusion function 52B that excludes the message definition information as the priority definition data 71A from the priority list 71 as soon as the exclusion time is reached, and the setting information storage unit 80 immediately after the exclusion operation by the message exclusion function 52B. An exclusion information deletion function 52C for deleting the priority data exclusion information.

  As described above, the priority data update processing means 50 has a configuration specialized for updating the priority list 71 (updating the priority definition data 71A) in which only message definition information suitable for the message generation status is stored. Further, the log collation processing unit 20 has a configuration specialized for collation processing by appropriately referring to the priority definition data 71A and the like updated by the priority data update processing unit 50. Therefore, the priority data update processing means 50 and the log collation processing means 20 can perform the respective processes quickly and can smoothly coordinate the transmission / reception of the processed data. In other words, the log management apparatus 90 according to the first embodiment can implement efficient collation processing by referring to appropriate message definition information at an optimal timing by functional role sharing for each configuration.

  Here, the network configuration of the log management apparatus according to the first embodiment will be described with reference to FIG.

  In FIG. 4, reference numerals 91 to 93 denote various log collection target devices including a network device, a PC installed with an OS / application, and the like. Reference numeral 90 denotes a log management apparatus according to the first embodiment that collects log data from various log collection target devices (91 to 93) and executes analysis / calculation processing, collation processing, and the like. These devices and apparatuses are connected on a network 94 as shown in FIG. With this configuration, the log management apparatus 90 according to the first embodiment recognizes various received messages (collation with information defining the message format) and executes these managements.

  In a normal log management device, when collecting log data, a verification process of “fixed order log format definition information held by the device itself” and “each received message” are performed to check the received message. The log management device 90 according to the first embodiment adopts characteristic operations such as storage processing of the priority definition data 71A and periodic update processing of the statistical definition data 72A when log data is received. Therefore, it is possible to increase the efficiency of the collation process.

  Next, the log definition storage unit 70 illustrated in FIG. 5 will be described.

  As shown in FIG. 5, the log definition storage unit 70 includes a log format definition master storage area (log definition master storage area) 70 A that manages a log format definition master for each system type, and a log collation processing unit 20. A collation log format definition storage area 70B in which log format definition information (priority definition data 71A and statistical definition data 72A) used in actual collation processing is stored is provided.

  The log format definition master storage area 70A has a unique definition for each system type (operating environment), and this is expanded in the verification log format definition storage area 70B. That is, the collation log format definition storage area 70B has a hierarchical structure in the order of the collection target IP address, system type, and unit time ID for various devices, and each of the hierarchical structures has log format definition information. doing.

  In the first embodiment that employs priority definition data 71A and statistical definition data 72A as log format definition information, as shown in FIG. 5, similarly to the collection target IP address, system type, and unit time ID, Each is stored in the priority list 71 and the statistics list 72 in the log format definition area classified based on each.

  The system here is software related to a computer such as an information system (including an operating system (OS) as a component of the information system), a network system, etc. In the first embodiment, the system type and operation Synonymous with environment. In addition, a database (DB) directly constructed on a file system provided by the OS, a database management system, and the like are included in the system here.

  Incidentally, the meaning that can be read from the log format definition information illustrated in FIG. 5 is “log format definition information in a certain time zone of a certain system type existing at a certain IP address”. For example, it is structured as “log format definition information to be referenced for a Windows (registered trademark) OS message sent from 13:00 to 14:00 from the IP address of 192.168.0.15”.

  The log format definition information in the conventional log management apparatus is generally unique for each system type (operating environment). However, the log management apparatus according to the first embodiment has the same system type. However, considering the fact that the message generation status varies depending on the installation environment, a configuration is adopted in which the log format definition information is retained for each collection target IP address and for each unit time.

  According to the division management for each collection target IP address, it is possible to flexibly cope with a difference in log output tendency that may occur particularly when the same system is used for different purposes. For example, the same Windows (registered trademark) OS may be used as an OA machine or operated for 24 hours.

  In addition, according to the division management for each unit time, it is possible to flexibly cope with a log output tendency that is different for each time zone. For example, due to the situation such as "A lot of login logs are generated on each client machine during the office hours from 8:00 to 9:00, but there are few login logs during working hours from 15:00 to 16:00" This is the case.

  Further, each log format definition information includes statistical definition data 72A in which the message format information of the log format definition master related to the corresponding system is rearranged in descending order of the number of receptions within the corresponding unit time, and the message of the statistical definition data 72A. Priority definition data 71A, which is to be collated by temporarily raising the priority among the definition information, is provided. The statistical definition data 72A and the priority definition data 71A are stored in the statistical list 72 and the priority list 71, which are specific storage areas, respectively. The message definition information as the priority definition data 71A stored in the priority list 71 is configured to be updated by the priority data update processing means 50 using the characteristics of the log data.

  Next, the setting information storage unit 80 illustrated in FIG. 6 will be described.

  The set time storage unit 80 stores storage determination information (event definition information 81A) for determining whether or not there is message definition information that the priority data storage determination unit 51 should use as the priority definition data 71A based on the information about each collated message. ), Unit time setting information 83A that is information relating to the cycle and unit time ID, and priority data exclusion information 82A that the priority data update unit 52 refers to when the priority definition data 71A is excluded from the priority list 71. Stored.

  The unit time setting information 83 includes “unit time ID”, “start time”, “end time”, “valid flag” for the “cycle” and each unit time as in the unit time setting information 83A shown in FIG. "Is held. The period (repetition period every fixed time) is a period until the unit time goes around. The unit time ID is unique identification information assigned to each unit time.

  Furthermore, the start and end items represent the start time and end time of each unit time. In the column for the start and end items, an arbitrary time is designated from the outside. Further, flag information indicating which unit time the current time belongs to is written in the column related to the valid flag. In the unit time setting information 83A according to the first embodiment, the unit time is set in units of one hour with a period of one day. That is, one cycle is divided into 24 unit times.

(Description of operation: Update processing of priority definition data 71A)
Here, of the operations related to the log management device 90 disclosed in FIG. 1, refer to FIG. 7 (steps S700 to S715) for the update of the priority definition data 71A using the prediction of the subsequent message by the event and the usage method. While explaining.

  First, the log management device 90 receives log data (a set of messages) from each device by the log receiving means 10 and confirms a collection target IP address (source IP address) for each received message (FIG. 7). : S700). Here, the log receiving unit 10 associates (links) the message with the collection target IP address related to the message. Further, the log receiving unit 10 transmits all the received messages and the collection target IP address to the log matching processing unit 20 (FIG. 7: S701). The collection target IP address is an identification number assigned to each device.

  In order to obtain the unit time ID corresponding to the current time, the log verification processing unit 20 refers to the valid flag in the unit time setting information 83 in the setting information storage unit 80 and confirms the valid unit time ID (see FIG. 7: S702). When a message is received for the first time from the collection target IP address received from the log receiving means 10, it is necessary to determine the system type (operating environment) before determining the message. In the first embodiment, this is already the case. It is assumed that the system type can be recognized after the system type determination process is completed.

  Subsequently, the log verification processing unit 20 uses the collection target IP address and unit time setting information 83 confirmed by the log reception unit 10 in the log format definition storage area 70B for verification (FIG. 5) of the log definition storage unit 70. Reference is made to the log format definition information of a specific area that matches the acquired unit time ID and system type combination (FIG. 7: S703).

  When there are a plurality of systems at the same IP address, the log format definition information for each system is referred to. For example, it is a case like the “area of the collection target IP address 1 including the area of the system A and the area of the system B” shown in FIG. Further, as shown in FIG. 5, the areas related to the system A and the system B are divided by unit time IDs, and log format definition information is provided in the divided areas. That is, priority definition data 71A and statistical definition data 72A as log format definition information are stored in the priority list 71 and the statistics list 72 in the area related to all log format definitions.

  The log collation processing means 20 refers to the collation log format definition storage area 70B, selects log format definition information classified based on the operating environment and data transmission time for each log collection target device, and uses this Then, collation processing is performed on the message received from the log receiving means 10.

  In the first embodiment, as described above, the priority list 71 and the statistics list 72 are provided in all corresponding log format definition areas, and log verification processing means is used within a temporary priority verification period to be described later. 20 sequentially refers to priority definition data 71A and statistical definition data 72A in order, and executes collation processing between the message definition information defined as the character string list and the acquired message. That is, the log verification processing means 20 performs message verification (matching processing) sequentially from the upper message definition information to the lower message definition information in the log format definition information (FIG. 7: S704).

  With respect to a message for which the collation processing has been completed in such a process, the log management device 90 is in a state where it can uniquely determine “which message information is in the log format definition”. Since a plurality of systems exist at the same collection target IP address, when the log format definition information for each system is specified, the same matching process is performed based on each log format definition information (FIG. 7). : S704).

  In the description of this operation, for the sake of convenience, it is assumed that a uniquely defined message ID exists for each message definition information in the log format definition master of each system. Therefore, in such message collation processing, it is always possible to determine the message ID, and also in the first embodiment, the log collation processing means 20 determines the message ID here. Note that by determining the message ID, it is possible to refer to the message definition information corresponding to the corresponding message ID, so that the information held by the message is found and analysis processing is possible (FIG. 7: S704).

  By the way, in the first embodiment as an operation explanation when using an event, priority definition data 71A is not stored in the priority list 71 as a specific storage area immediately after the start of collation (stored message definition). Since the information is excluded by the message exclusion function 52B after a predetermined time has elapsed), the log matching processing means 20 first refers to the statistical definition data 72A in the same log format definition area. However, if specific message definition information that makes use of characteristics appearing in the log data is stored in the priority definition data 71A by a process described later, the priority definition data 71A and the statistical definition data 72A are used until this is excluded. Are sequentially referred to in this order, and collation processing for each message is executed. Whether or not an event can be used is determined in step S509 (storage determination step) described later.

  Further, as described above, the statistical definition data 72A updated every unit time exists in all areas classified by the environment for each device, and stores all message definition information for each device. Yes. Furthermore, since the message definition information in the statistics list 72 is rearranged in the order of the number of receptions, according to the log management device 90 according to the first embodiment using this, the message definition information is performed in a fixed order. As a result, the matching process can be performed more efficiently.

  In the first embodiment, the following steps (steps S705 to S715) are sequentially executed with the progress of the collation processing by the log collation processing unit 20 described above.

  The log collation processing unit 20 transmits the message ID determined by the collation processing and the collated message to the log storage processing unit 42 of the log access processing unit 40 (FIG. 7: S705). The log storage processing unit 42 stores the information received from the log format verification unit 20 in the log storage unit 30 (FIG. 7: S706).

  The collated message stored in the log storage unit 30 in steps S705 and S706 is used in a process related to update processing of statistical definition data 72A described later (FIG. 9: S900 to S907). The statistical definition data 72A in the statistical list 72 is periodically updated by the processing of the statistical data update processing means 60 based on the unit time setting information 83.

  Subsequently, the log collation processing means 20 receives the message ID and log received by referring to the prioritized data storage determination section 51 in the prioritized data update processing means 50 by referring to the collation log format definition storage area 70B (FIG. 5). The collection target IP address confirmed by the means 10 and the system type that can be discriminated by the collation process are transmitted (FIG. 7: S707). These pieces of information are information necessary for the priority data storage determination unit 51 to specify message definition information to be used as the priority definition data 71A.

  When the priority data storage determination unit 51 receives the message ID or the like from the log format verification unit 20, the priority data storage determination unit 51 refers to the storage determination information 81 and the unit time setting information 83 in the setting information storage unit 80 (FIG. 7: S708).

  In the reference process by the priority data storage determination unit 51, first, the event definition information 81A (FIG. 6) in the setting information storage unit 80 is referred to. The priority data storage determination unit 51 refers to each event definition in the event definition information 81A for each piece of information received from the log collation processing unit 20, thereby “message definition information to be used as priority definition data 71A (stored in the priority list 71”). Whether or not message definition information to be present exists ”is confirmed (FIG. 7: S709).

  Specifically, the combination of the collection target IP address, the system type, and the message ID related to the collated message received from the log collation processing unit 20 is any one of the origin messages in the event definition information 81A (FIG. 6). If they match, the information related to the subsequent message corresponding to the starting message is recognized as the information related to the message definition information to be preferentially checked. That is, the priority data storage determination unit 51 determines that there is message definition information to be the priority definition data 71A when there is a starting message including the above three information in the referenced event definition information 81A. It is determined that the message definition information relating to the subsequent message of the event is to be stored in the priority list 71.

Here, an operation description relating to determination that there is message definition information to be the priority definition data 71A for the specific example of the event shown in FIG. 6 will be added. Examples of events include
“Search by Web system A” listed in the event definition 81A is used. The setting information storage unit 80 stores event definition information and the like in advance.

  First, when a search process is executed in the Web system A, the Web system A outputs a message indicating that “search is executed”. Thereafter, a database search instructed by the Web system A is performed in a database (Oracle (registered trademark) in the first embodiment) which is a data reference destination of the Web system A, indicating that “database search has been performed”. A message is output. Therefore, in the “search in the Web system A” event, the “search execution message in the Web system A” and the subsequent “search execution message in the database” as the starting point are defined as a series of events.

  In other words, when the “search execution message in the Web system A” that is the origin message is generated, the possibility that the “search execution message in the database” will be generated very soon is very high. It is more efficient to collate subsequent messages with priority. In view of this point, in the first embodiment, priority definition data 71A is provided in the log format definition storage unit 70, and it is applied preferentially, thereby enabling more efficient collation processing.

  By the way, the message definition information to be set as the priority definition data 71A (to be stored in the priority list 71 as a specific storage area) is set for each event, so that the verification efficiency is higher than that in the normal statistical order. Is better only when the relevant event occurs. Accordingly, if this event is used until the corresponding event has not occurred, the collation efficiency is lowered.

  Therefore, the priority data storage determination unit 51 can confirm the information related to the message definition information to be the priority definition data 71A (when the origin message containing the above three informations can be confirmed in the event definition information 81A). In addition, a time (exclusion time) at which the priority definition data 71A is excluded from the priority list 71 in the log definition storage unit 70 is calculated (FIG. 7: S709). In calculating the exclusion time to be set for each event, the contents of the set time (10 seconds in the first embodiment) in the event definition information 81A shown in FIG. 6 are taken into consideration.

  The set time set for each corresponding event is the time until the message definition information stored for temporarily improving the collation efficiency is excluded from the priority list 71. That is, the priority data storage determination unit 51 calculates an exclusion time that is a time after the set time has elapsed from the current time by adding the set time to the current time (FIG. 7: S709). A function related to calculation of the exclusion time included in the priority data storage determination unit 51 is referred to as an exclusion time calculation function 51A (FIG. 3). The period from the current time to the exclusion time is the priority verification period.

  In the first embodiment, since the event definition information stored in advance in the setting information storage unit 80 is used, the priority data storage determination unit 51 performs the next operation without updating the event definition information 81A. Run. In the second embodiment to be described later, it is necessary to update.

  Subsequently, the priority data storage determination unit 51 refers to the collection target IP address (device IP address), system type, unit time ID, and unit time setting acquired as information related to the subsequent message of the event when referring to the event definition information 81A. The priority data update unit 52 uses the message ID acquired when referring to the information 83A and the exclusion time calculated as described above as information used to update the priority definition data 71A (store in the priority list 71 and exclude from it). Transmit (FIG. 7: S710).

  The priority data update unit 52 updates the priority data exclusion information 82A (FIG. 6) in the setting information storage unit 80 with the information related to the subsequent message received from the priority data storage determination unit 51 (FIG. 7: S711). The priority data exclusion information 82A is used when the priority data updating unit 52 excludes the message definition information stored for temporarily improving the collation efficiency from the priority list 71.

  Further, the priority data update unit 52 stores the message definition information corresponding to the system type and message ID received from the priority data storage determination unit 51 in the log corresponding to the corresponding system type in the log format definition master storage area 70A (FIG. 5). Obtained from the format definition information (FIG. 7: S712).

  Further, the priority data update unit 52 uses the log corresponding to the collection target IP address, system type, and unit time ID received from the priority data storage determination unit 51 for the message definition information acquired in the log format definition master storage area 70A. Stored as priority definition data 71A in the priority list 71 (specific storage area) in the definition storage unit 70 (FIG. 7: S713).

  When specific message definition information (priority definition data 71A) is stored in the corresponding priority list 71 by this process, the log matching processing means 20 performs efficient matching with the priority definition data 71A having the highest priority thereafter. Processing can be executed. The collation process with priority application of the priority definition data 71A is performed until the exclusion time calculated as described above is reached.

  As described above, the priority data updating unit 52 excludes the message definition information as the priority definition data 71A stored in the priority list 71 when each exclusion time in the priority data exclusion information 82A is reached (FIG. 3: A message exclusion function 52B) is provided. Specifically, the corresponding IP address, system type, unit time ID, and message ID related to the message definition information stored in the corresponding priority list 71 are excluded (FIG. 7: S714). At that time, the priority data update unit 52 deletes the information related to the corresponding message definition from the priority data exclusion information 82A (FIG. 7: S715).

(Description of operation: update processing of statistical definition data 72A)
FIG. 9 shows an operation when the statistical definition data 72A is periodically updated. The statistical definition data 72A is obtained by rearranging the message definition information in descending order of the number of occurrences, and is updated as needed every unit time. Here, the process of updating the statistical definition data 72A by the log management apparatus 90 according to the first embodiment shown in FIG. 1 will be described with reference to the sequence diagram of FIG. In addition, FIG. 8 is referred as needed.

  The statistical information calculation processing unit 61 of the statistical data update processing means 60 instructs the log acquisition processing unit 41 to acquire log data of all devices collected during the completed unit time when the end time of the unit time is reached. (Log acquisition request). Further, the statistical information calculation processing unit 61 transmits the start time and the end time of the corresponding unit time along with the log acquisition request (FIG. 9: S900). In the description of the operation according to the first embodiment, the completed unit time will be described as the unit time UT001.

  The log acquisition processing unit 41 uses the start time and end time received from the statistical information calculation processing unit 61 to acquire log data of all devices collected from the log storage unit 30 in the corresponding time zone. That is, in the first embodiment, log data in the unit time UT001 is acquired (FIG. 9: S901).

  Here, the information acquired by the log acquisition processing unit 41 is obtained by the log storage processing unit 42 receiving the collated message that has undergone the collation processing by the log collation processing unit 20 and storing it in the log storage unit 30. (FIG. 7: S705, S706). The verification process is executed over time, and the verification message is stored in the log storage unit 30 each time.

  Subsequently, the log acquisition processing unit 41 sends the log data in the unit time UT001 acquired from the log storage unit 30 to the statistical information calculation processing unit 61 of the statistical data update processing means 60 (FIG. 9: S902).

  The statistical information calculation processing unit 61 counts the number of messages acquired during the unit time UT001 set for each system type of the collection target IP address for the log data received from the log acquisition processing unit 41, and Sort (sort) in order of most messages. Specifically, a list of message counts in the unit time UT001 of each system (hereinafter referred to as a count list) is created as in the message appearance count 84 shown in FIG. 8 (FIG. 9: S903).

  The statistical information calculation processing unit 61 sends the created count list (calculation result) of each system to the statistical data update unit 62 (FIG. 9: S904).

  The statistical data update unit 62 refers to the log format definition master storage area 70A (FIG. 5) divided for each system type, and based on the count list of each system received from the statistical information calculation processing unit 61, unit time UT001. The statistical definition data 72A is re-created (the statistical definition data 72A stored in the statistical list 72 in the verification log format definition storage area 70B is updated). That is, the message definition information in the statistics list 72 is rearranged again. Specifically, the count number of the count list is added to the count number of the corresponding message definition information in the statistical definition data 72A, and then this is sorted (aligned) in the descending order of the number of times. If the message count is 0, it is added to the end of the statistical definition data 72A (FIG. 9: S905).

  As described above, the statistical data update processing means 60 classifies the collated messages that have undergone the collation processing in the statistical information calculation processing unit 61 based on the collection target IP address and the operating environment, and then calculates the number of occurrences for each message ID. It has a function of counting and creating a count list in which the counts are arranged in descending order at the end of a preset unit time (FIG. 9: S904). Further, the statistical data update unit 62 adds the count list to the count list. It can be summarized that it has a function of updating the statistical definition data 72A based on it (FIG. 9: S905).

  Subsequently, the statistical data update unit 62 updates the unit time setting information 83A (FIG. 6) in the setting information storage unit 80 (FIG. 9: S906). That is, here, since the unit time UT001 has been completed, the valid flag of the valid unit time UT001 is invalidated and the valid flag of the next unit time UT002 is validated. Since the first embodiment is an example using an event, it is not necessary to update the event definition information 81A (FIG. 9: S906).

  Further, the statistical data update unit 62 notifies the statistical information calculation processing unit 61 of information relating to the unit time UT002 as the next update time of the statistical definition data 72A (FIG. 9: S907). As a result, the statistical information calculation processing unit 61 can sequentially execute the above-described steps for the unit time UT002 as soon as the end time of the unit time UT002 is reached.

  By using statistical definition data 72A that is accurately updated every fixed period in this way, message definition information with a higher appearance rate can be referred to for each scene in advance, so that priority definition data 71A is not used. Also, the efficiency of the collation process can be improved. In a situation where priority definition data 71A that is dynamically updated at the time of log collection can be preferentially applied, message definition information listed based on empirical knowledge unique to log management devices can be referred to effectively. More efficient matching process can be realized according to.

  Further, the execution contents of each step in each of the above steps S100 to S116 (FIG. 11) and each of steps S900 to 907 (FIG. 9) are programmed, and this series of each control program is realized by a computer. Also good.

[Second Embodiment: Collation Processing when Failure Occurs]
A second embodiment (exception state utilization method) of the log management apparatus according to the present invention will be described with reference to FIG. 1, FIG. 5, FIG. 10, and FIG.

  In an environment where the log management device is operated in a normal state, such an error log (log at the time of failure) is rarely output. However, when a failure occurs, a log during normal operation is not output, and there is a high possibility that a large number of error logs will occur. Despite this situation, if only the statistical definition data 72A in which an infrequent error log is described at the end is used, an efficient collation process cannot be realized. Therefore, in the second embodiment, a description will be given of how to update the priority definition data 71A and how to effectively use it when a failure state (exception state) occurs.

  By the way, in the storage process of the priority definition data 71A at the time of log data reception in the first embodiment described above, information is stored by predicting a message that will be output when an event occurs. In the storage processing of the priority definition data 71A at the time of log data reception in the second embodiment, the message definition to be used as the priority definition data 71A after detecting the occurrence of a failure state by the determination based on a threshold set in advance from the outside. A configuration that identifies information was adopted.

  The overall configuration according to the second embodiment is the same as that in the first embodiment shown in FIGS. In addition, the same is true in that the following steps (steps S105 to S116) are sequentially executed in parallel with the collation processing (FIG. 11: S104) by the log collation processing means 20.

  Here, in the second embodiment, the setting information storage unit 80 preliminarily defines and stores exceptional state information. Specifically, when the exceptional state is used as a storage determination condition for the priority definition data 71A in the priority list 71, message appearance count information is stored in advance as the storage determination information 81 as in the setting information storage unit 80 shown in FIG. 81B and exception determination setting information 81C are stored. Further, the start time and the end time in the unit time setting information 83B can be arbitrarily set by external input. In the second embodiment, a unit time of one hour period and 30 minutes is set. In such a case, as shown in FIG. 10, one cycle is divided into two unit times.

(Description of operation: Update processing of priority definition data 71A)
Here, the update and use method of the priority definition data 71A using the exceptional state among the operations of the log management apparatus disclosed in FIG. 1 will be described along the sequence diagram of FIG. 11 (steps S100 to S116). The content of the process concerning step S100-S107 is the same as the content (step S700-S707) about 1st Embodiment demonstrated along FIG. For this reason, below, the other operation | movement content is demonstrated.

  The setting information storage unit 80 has message appearance number information in which a collated message that has undergone classification processing based on a collection target IP address, an operating environment, and a preset period is counted for each message ID. In the message appearance frequency information, a current cycle count and a previous cycle count relating to the immediately preceding cycle are provided. The previous cycle counts are arranged in descending order of the number of counts. Also, the current cycle count and the previous cycle count are obtained by counting and organizing each message definition information based on the message ID.

  When the priority data storage determination unit 51 receives the message ID from the log matching processing unit 20, the message appearance count information 81B, the exception determination setting information 81C, and the unit time setting information 83B in the setting information storage unit 80 shown in FIG. Refer to Since the second embodiment is a case where the exceptional state is used as the storage determination condition of the priority definition data 71A, the priority data storage determination unit 51 first refers to the message appearance count information 81B and the exception determination setting information 81C, Information such as the set time is acquired (FIG. 11: S108).

  The priority data storage determination unit 51 should be the priority definition data 71A based on the information received from the log collation processing unit 20, the message appearance count information 81B and the exception determination setting information 81C referred to in the setting information storage unit 80. It is confirmed whether or not message definition information exists (FIG. 11: S109).

  Specifically, the ratio of the current cycle count to the previous cycle count (the number of messages received in the corresponding unit time in the previous cycle) is calculated, and exceeds the exception determination threshold referenced in the exception determination setting information 81C. If it is determined that there is message definition information to be the priority definition data 71A, the information related to the message ID that has reached the determination is confirmed. That is, when the calculated value of “(current cycle count) / (previous cycle count) × 100” exceeds the threshold set in percentage (ratio), it is determined that the state is an exceptional state.

  Here, specific determination by the priority data storage determination unit 51 will be described using the numerical values illustrated in FIG. According to the message appearance count 85 of the system A illustrated in FIG. 10, the previous cycle count of the message ID MSG003 is 31 and the current cycle count is 7. Further, according to the exception determination setting information 81C, the threshold set in the second embodiment is 500%. Then, based on the above-described calculation formula, since 31/7 × 100 ≦ 500, the calculated value does not exceed the threshold value, so it is determined as “not in an exceptional state”.

  On the other hand, in the message ID MSG00N (N is an arbitrary number), the previous cycle count is 10, and the current cycle count is 51. Then, according to the comparison with the threshold value based on the same calculation formula as described above, since 51/10 × 100 ≧ 500, the ratio between the previous cycle count and the current cycle count exceeds the threshold value. The storage determination unit 51 determines that the message is “exceptional” in which the corresponding message occurs frequently (FIG. 11: 109). That is, the information related to the message ID (here, MSG00N) that has been a factor for determining that the state is an exceptional state is recognized as the information related to the priority definition data 71A.

  Here, the priority data storage determination unit 51 that has determined that the state is an exceptional state calculates a time (exclusion time) at which the message definition information is excluded from the priority list 71. In the priority list 71 in the second embodiment, as in the case of the first embodiment, message definition information (priority definition data 71A) to be preferentially collated using the characteristics found during log data collation. Is temporarily listed. For this reason, as in the event occurrence, it is necessary to calculate in advance the timing for excluding the exception state.

  The priority data storage determination unit 51 uses the set time (300 seconds in the second embodiment) of the exception determination setting information 81C illustrated in FIG. 10 in calculating the exclusion time. In this set time, a time until the stored message is excluded from the priority list 71 in the exceptional state is set. That is, the priority data storage determination unit 51 calculates an exclusion time that is a time after the set time has elapsed from the current time, based on the set time acquired with reference to the exception determination setting information 81C. The calculation method here is the same as in the first embodiment (FIG. 11: 109). Similarly, a period from the current time to the exclusion time is set as a priority collation period.

  In addition, since the second embodiment uses an exceptional state, the priority data storage determination unit 51 uses the collated message received from the log collation processing unit 20 in the message appearance count information 81B (FIG. 10). The current cycle count (number of messages received within the current unit time in the current cycle) in the appearance count area corresponding to the collection target IP address, system type, and message ID is the number of occurrences of the corresponding message. Add (+1) minutes. As a result, the current cycle count relating to the message ID in the appearance count area corresponding to the verified message is updated (FIG. 11: S110).

  Furthermore, the priority data storage determination unit 51 transmits the collection target IP address, system type, unit time ID, message ID, and exclusion time as information related to the determination result to the priority data update unit 52 (FIG. 11: S111). . In the second embodiment using the exceptional state, the collection target IP address, the system type, and the message ID are acquired from the log format verification unit 20 as information related to the verified message. Further, the unit time ID is acquired when the unit time setting information 83B in the setting information storage unit 80 is referred to.

  The priority data update unit 52 updates the priority data exclusion information 82B (FIG. 10) in the setting information storage unit 80 with the information related to the determination result received from the priority data storage determination unit 51 (FIG. 11: S112). Specifically, the exclusion time calculated by the priority data storage determination unit 51, the unit time ID acquired by the setting information storage unit 80, the collection target IP address received from the log verification processing unit 20, the system type, and the message ID are Stored as priority data exclusion information 82B in the setting information storage unit 80 (FIG. 11: S112). The priority data exclusion information 82B is used when the priority data update unit 52 excludes the message definition information stored for temporarily improving the collation efficiency from the priority list 71.

  As described above, the occurrence of this exceptional state is temporary. By using the priority definition data 71A stored in the priority list 71, the collation efficiency is improved as compared with the case where the normal statistical order is used. Only when this exceptional condition occurs. Therefore, the priority data updating unit 52 excludes the priority definition data 71A from the priority list 71 with reference to the priority data exclusion information 82B in a process described later.

  When specific message definition information (priority definition data 71A) is stored in the corresponding priority list 71 by this process, the log matching processing means 20 performs efficient matching with the priority definition data 71A having the highest priority thereafter. Processing can be executed. The collation process with priority application of the priority definition data 71A is performed until the exclusion time calculated as described above is reached.

  Further, the priority data update unit 52 stores the message definition information corresponding to the system type and message ID received from the priority data storage determination unit 51 in the log corresponding to the corresponding system type in the log format definition master storage area 70A (FIG. 5). Obtained from the format definition information (FIG. 11: S113).

  Further, the priority data update unit 52 uses the log corresponding to the collection target IP address, system type, and unit time ID received from the priority data storage determination unit 51 for the message definition information acquired in the log format definition master storage area 70A. The priority definition data 71A is stored in the priority list 71 (specific storage area) in the definition storage unit 70 (FIG. 11: S114).

  Thereby, the log collation processing means 20 can efficiently perform the collation processing with reference to the priority list 71 in which the message definition information suitable for the exceptional state is stored thereafter. The collation process with priority application of the priority definition data 71A is performed until the exclusion time calculated as described above is reached.

  Further, the priority data updating unit 52 excludes the message definition information as the priority definition data 71A stored in the priority list 71 when each exclusion time in the priority data exclusion information 82B is reached (FIG. 3: message). Exclusion function 52B) is provided. Specifically, the corresponding IP address, system type, unit time ID, and message ID related to the message definition information stored in the corresponding priority list 71 are excluded (FIG. 7: S714). At that time, the priority data update unit 52 deletes the information related to the corresponding message definition from the priority data exclusion information 82A (FIG. 11: S116).

(Description of operation: update processing of statistical definition data 72A)
Subsequently, also in the second embodiment, the operation will be described along the sequence diagram for the update processing of the statistical definition data 72A shown in FIG. Note that the basic operations in steps S900 to S907 are the same as the contents in the first embodiment described above, so only the differences between the contents of the operations will be described here.

  As described above, in the case of using an event, there is no need to update the event definition information 81A. However, in the case of the second embodiment using the exceptional state, it is necessary to update the event definition information 81B. Specifically, updating is performed based on the count list information of each system acquired by the log format verification unit 20. Further, all the current cycle counts are initialized to 0, and the previous cycle count portion is updated to the count value acquired by the log format verification unit 20 (FIG. 9: S906).

  In addition, the execution contents of each step in each of the above steps S700 to S715 (FIG. 7) and each of steps S900 to 907 (FIG. 9) are programmed, and this series of each control program is realized by a computer. Also good.

(Effect of embodiment)
In the first and second embodiments of the present invention, a priority list 71 in which message definition information to be preferentially collated is listed as priority definition data 71A in order to effectively change the collation order of messages, and each message A statistical list 72 storing statistical definition data 72A in which definition information is rearranged in descending order of the number of occurrences is employed. Since the priority definition data 71A is dynamically updated at the time of log data collection, the log matching processing unit 20 can execute a flexible matching order according to the situation. Further, since the statistical definition data 72A is updated every unit time, the log collation processing unit 20 can also flexibly execute the collation order. That is, in the log management device 90 according to the first and second embodiments of the present invention, by efficiently using these two definition data, it is efficient to grasp the operating environment of each log collection target device. Verification processing is possible.

  The embodiment described above is a preferable specific example in the log management device, the management method, and the program thereof, and may have various technically preferable limitations. However, the technical scope of the present invention is as follows. Unless otherwise specifically limited, the present invention is not limited to these embodiments.

  The following summarizes the main points of the new technical contents of the above-described embodiment, but the present invention is not necessarily limited to this.

(Appendix 1)
A log management device that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the environment of each device,
Log receiving means for receiving a plurality of messages constituting the log data;
Log verification processing means for executing verification processing for each message according to a predetermined order;
A log definition storage unit in which log format definition information classified based on the operating environment and data transmission time zone for each log collection target device is stored in advance,
The log collation processing unit executes the collation processing by referring to any one of the plurality of log format definition information stored in the log definition storage unit. .

(Appendix 2)
In the log management device according to attachment 1,
The log format definition information is temporarily associated with statistical definition data in which the message definition information identified by the collation process is arranged in descending order of reception, and reference information relating to preset characteristics of the message. Priority definition data that is the message definition information to be collated preferentially, and
The log collation processing unit selects the log format definition information that matches a combination of a collection target IP address for the log collection target device, the operating environment, and a unit time ID corresponding to the data transmission time zone. A specific data collating unit that refers to the statistical definition data in the selected log format definition information and sequentially executes the collation processing with reference to the priority definition data prior to the statistical definition data in the priority collation period A log management apparatus comprising:

(Appendix 3)
In the log management device according to Appendix 2,
The log verification processing means includes:
An update information transmission unit that includes a priority data update processing unit that controls the update of the priority definition data, and transmits a message ID of the collated message, the collected control IP address, and the operating environment to the priority data update processing unit With
The priority data update processing means includes:
A priority data storage determination unit that determines the presence or absence of the message definition information to be the priority definition data based on the reference information;
A log management apparatus comprising: a priority data update unit that updates the priority definition data in the log definition storage unit based on a determination result indicating that the message definition information to be used as the priority definition data is present.

(Appendix 4)
In the log management device according to attachment 3,
The reference information is a plurality of event definition information defined based on the collection target IP address, the operating environment, and the message ID,
Each event definition information is composed of a plurality of origin messages for event determination and subsequent messages corresponding thereto,
The priority data storage determination unit, when the combination of the collection control IP address, the operating environment, and the message ID sent from the log verification processing unit matches any of the origin messages, A log management apparatus characterized by determining that there is the message definition information to be transmitted and transmitting information related to the subsequent message corresponding to the matched origin message to the priority data update unit.

(Appendix 5)
In the log management device according to attachment 3,
The reference information includes a message appearance in which the collated message that has undergone a classification process based on the collection target IP address, the operating environment, and a preset repetition period for every predetermined time is counted for each message ID. Count information,
This message appearance number information includes the current cycle count and the previous cycle count immediately before it.
The priority data storage determination unit calculates a ratio of the current cycle count to the previous cycle count, and the message definition to be used as the priority definition data when the calculated value exceeds a preset threshold value from the outside. A log management apparatus characterized by determining that there is information and transmitting information related to the determination result to the priority data update unit.

(Appendix 6)
In the log management device according to any one of appendices 3 to 5,
The priority data update unit obtains the message definition information corresponding to the collection control IP address and the message ID sent from the priority data storage determination unit from a log definition master storage area in the log definition storage unit. The log management apparatus, wherein the message definition information acquired here is stored as the priority definition data in another specific storage area in the log definition storage unit.

(Appendix 7)
In the log management device according to appendix 6,
The storage area in the log definition storage unit is specified based on a combination of the collection reference IP address, the operating environment, and the unit time ID received by the priority data update unit from the priority data storage determination unit. Log management device characterized by

(Appendix 8)
In the log management device according to any one of appendices 3 to 5,
The priority data update unit stores the priority definition data in the log when a preset time as a reference of the priority collation period set in advance has elapsed since the priority definition data was stored in the log storage unit. A log management apparatus comprising a message exclusion function for exclusion from a department.

(Appendix 9)
In the log management device according to appendix 8,
The priority data update processing means is provided with a setting information storage unit for storing information related to the update of the priority definition data,
The priority data storage determination unit includes an exclusion time calculation function for calculating an exclusion time for excluding the priority definition data from the log definition storage unit by adding the set time to a current time,
The priority data update unit includes:
An exclusion information storage processing function for storing, in the setting information storage unit, priority data exclusion information in which the exclusion time is added to information related to the priority definition data to be excluded from the log definition storage unit;
An exclusion information deletion function for deleting the priority data exclusion information from the setting information storage unit when the exclusion time is reached,
The log management apparatus, wherein the message exclusion function excludes the priority definition data from the log definition storage unit by referring to the priority data exclusion information.

(Appendix 10)
In the log management device according to any one of appendices 3 to 5,
The log definition storage unit is provided with statistical data update processing means for updating the statistical definition data,
The statistical data update processing means
The collated messages classified based on the collection target IP address and the operating environment are counted for each message ID, and a count list in which the counts are arranged in descending order at the end of a preset unit time is created. A statistical data update processing unit;
And a statistical data updating unit that updates the statistical definition data based on the count list.

(Appendix 11)
In a log management device that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the environment of each device,
The log receiving means receives a plurality of messages constituting the log data,
Next, each of the messages is sent by the log receiving means to the log matching processing means,
From the log definition storage unit in which the log format definition information classified based on the operating environment and the data transmission time zone for each log collection target device is stored in advance, the verification processing unit is configured to output the log format based on each message. Select definition information,
A log management method, wherein the log collation processing unit performs collation processing of each message received from the log reception unit by referring to the selected log format definition information.

(Appendix 12)
In the log management method according to appendix 11,
The log format definition information is temporarily associated with statistical definition data in which the message definition information identified by the collation process is arranged in descending order of reception, and reference information relating to preset characteristics of the message. And priority definition data that is the message definition information to be collated preferentially,
The log collation processing means selects the log format definition information that matches the combination of the collection target IP address for the log collection target device, the operating environment, and the unit time ID corresponding to the data transmission time zone. ,
The log verification processing means sequentially refers to the statistical definition data in the log format definition information selected here and refers to the priority definition data prior to the statistical definition data within the priority verification period. A log management method characterized by being executed.

(Appendix 13)
In the log management method according to appendix 12,
The log collation processing unit provided with a priority data update processing unit that controls the update of the priority definition data transmits a message ID of the collated message, the collected control IP address, and the operating environment to the priority data update processing unit. And
The priority data update processing means determines the presence / absence of the message definition information to be the priority definition data based on preset reference information,
Based on the determination result that there is the message definition information to be the priority definition data, the priority data update processing means updates the priority definition data in the log definition storage unit,
A log management method characterized by sequentially executing the contents of each series of steps as the collation processing by the log collation processing means proceeds.

(Appendix 14)
In the log management method according to appendix 13,
The reference information is a plurality of event definition information defined based on the collection target IP address, the operating environment, and the message ID,
Each event definition information is composed of a plurality of origin messages for event determination and subsequent messages corresponding thereto.
When the combination of the collection control IP address, the operating environment, and the message ID received from the log matching processing unit matches any of the origin messages, the priority data storage determination unit sets the priority definition data as the priority definition data. Determining that the message definition information should be present,
Sending information related to the subsequent message corresponding to the matched origin message to the priority data update unit,
The priority data update unit receives the information relating to the subsequent message,
The priority data update unit acquires the message definition information corresponding to the collection reference IP address and the message ID included in the received information related to the subsequent message from the log definition master storage area in the log definition storage unit. ,
Other storage areas related to the priority definition data in the log definition storage unit are stored in the collection control IP address, the operating environment, and the unit time ID received by the priority data update unit from the priority data storage determination unit. Identify based on the combination,
The log management method, wherein the priority data update unit stores the corresponding message definition information acquired as described above as the priority definition data in the storage area specified here.

(Appendix 15)
In the log management method according to appendix 13,
The reference information includes a message appearance in which the collated message that has undergone a classification process based on the collection target IP address, the operating environment, and a preset repetition period for every predetermined time is counted for each message ID. Count information,
This message appearance number information includes the current cycle count and the previous cycle count immediately before it.
The priority data storage determining unit calculates a ratio of the current cycle count to the previous cycle count,
When the calculated value exceeds a threshold set in advance from the outside, the priority data storage determination unit determines that there is the message definition information to be the priority definition data,
The priority data storage determination unit transmits information related to the determination result to the priority data update unit,
The priority data update unit receives the information related to the determination result,
The priority data update unit acquires from the log definition storage unit the message definition information corresponding to the collection reference IP address and the message ID included in the information related to the received subsequent message,
The storage area for the priority definition data in the log definition storage unit is a combination of the collection reference IP address, the operating environment, and the unit time ID received by the priority data update unit from the priority data storage determination unit. Based on
The log management method, wherein the priority data update unit stores the corresponding message definition information acquired as described above as the priority definition data in the storage area specified here.

(Appendix 16)
In the log management method according to any one of appendices 13 to 15,
Immediately after determining that there is information to be used as the priority definition data, the priority data storage determination unit adds a set time as a reference for the priority collation period preset in the setting information storage unit to the current time. By calculating the exclusion time to exclude the priority definition data from the log definition storage unit,
The priority data storage determination unit transmits the priority data exclusion information in which the exclusion time is added to the information related to the priority definition data to be excluded from the log definition storage unit to the priority data update unit,
The priority data update unit, which has a setting information storage unit that stores information related to the update of the priority definition data, stores the priority data exclusion information in the setting information storage unit,
As soon as it is confirmed that the exclusion time has been reached by referring to the priority data exclusion information stored in the setting information storage unit, the priority data update unit excludes the priority definition data from the log storage unit. ,
Immediately after the exclusion, the priority data update unit deletes the priority data exclusion information from the setting information storage unit.

(Appendix 17)
In the log management method according to any one of appendices 13 to 16,
The statistical data update processing unit that manages the update of the statistical definition data provided in the log definition storage unit classifies the collated messages based on the collection target IP address and the operating environment, and then performs the process for each message ID. Count the number of utterances,
At the end of the preset unit time, the statistical data update processing unit of the statistical data update processing means creates a count list by arranging the count numbers in descending order,
A log management method, wherein a statistical data update unit of the statistical data update processing unit updates the statistical definition data based on the count list.

(Appendix 18)
In a log management device that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the environment of each device,
A log receiving function in which a log receiving means receives a plurality of messages constituting the log data;
A log transmission function in which each of the messages is transmitted by the log receiving means to a log verification processing means;
From the log definition storage unit in which the log format definition information classified based on the operating environment and the data transmission time zone for each log collection target device is stored in advance, the verification processing unit is configured to output the log format based on each message. Log definition selection function to select definition information,
By referring to the log format definition information selected by the log definition selection function, the log verification processing unit realizes a log verification processing function for performing verification processing of each message received from the log reception unit in a computer. Log management program to let you.

(Appendix 19)
In the log management program according to appendix 18,
The log format definition information is temporarily associated with statistical definition data in which the message definition information identified by the collation process is arranged in descending order of reception, and reference information relating to preset characteristics of the message. And priority definition data that is the message definition information to be collated preferentially,
The log collation processing means selects the log format definition information that matches the combination of the collection target IP address for the log collection target device, the operating environment, and the unit time ID corresponding to the data transmission time zone. Log definition selection function,
The log verification processing means sequentially refers to the statistical definition data in the log format definition information selected here and refers to the priority definition data prior to the statistical definition data within the priority verification period. A log management program for causing a computer to execute a specific data matching function to be executed.

(Appendix 20)
In the log management program according to appendix 19,
The log collation processing unit provided with a priority data update processing unit that controls the update of the priority definition data transmits a message ID of the collated message, the collected control IP address, and the operating environment to the priority data update processing unit. Update information transmission function,
A priority data storage determination function in which the priority data update processing means determines whether or not the message definition information should be the priority definition data based on the reference information;
Based on a determination result that the message definition information to be used as the priority definition data is present, the priority data update processing unit has a priority data update function for updating the priority definition data in the log definition storage unit. Log management program to realize.

(Appendix 21)
In the log management program according to appendix 20,
The reference information is a plurality of event definition information defined based on the collection target IP address, the operating environment, and the message ID,
Each event definition information is composed of a plurality of origin messages for event determination and subsequent messages corresponding thereto.
When the combination of the collection control IP address, the operating environment, and the message ID received from the log matching processing unit matches any of the origin messages, the priority data storage determination unit sets the priority definition data as the priority definition data. An event determination function for determining that the message definition information should be present,
A determination result transmission function for transmitting information related to the subsequent message corresponding to the matched origin message to the priority data update unit;
A determination result receiving function for receiving information related to the subsequent message by the priority data update unit;
The priority data update unit acquires the message definition information corresponding to the collection reference IP address and the message ID included in the received information related to the subsequent message from the log definition master storage area in the log definition storage unit. Message definition acquisition function,
Other storage areas related to the priority definition data in the log definition storage unit are stored in the collection control IP address, the operating environment, and the unit time ID received by the priority data update unit from the priority data storage determination unit. Storage area identification function to identify based on the combination,
Log management for causing the computer to realize a priority data storage processing function in which the priority data update unit stores the corresponding message definition information acquired as described above as the priority definition data in the storage area specified here program.

(Appendix 22)
In the log management program according to appendix 20,
The reference information includes a message appearance in which the collated message that has undergone a classification process based on the collection target IP address, the operating environment, and a preset repetition period for every predetermined time is counted for each message ID. Count information,
This message appearance number information includes the current cycle count and the previous cycle count immediately before it.
A count ratio calculation function in which the priority data storage determination unit calculates a ratio of the current period count to the previous period count;
An exception state determination function for determining that the priority data storage determination unit has the message definition information to be the priority definition data when the calculated value exceeds a threshold set in advance from the outside;
A determination result transmission function in which the priority data storage determination unit transmits information related to the determination result to the priority data update unit;
A determination result receiving function for the priority data update unit to receive information related to the determination result;
A message definition acquisition function in which the priority data update unit acquires from the log definition storage unit the message definition information corresponding to the collection reference IP address and the message ID included in the information related to the received subsequent message;
The storage area for the priority definition data in the log definition storage unit is a combination of the collection reference IP address, the operating environment, and the unit time ID received by the priority data update unit from the priority data storage determination unit. Storage area identification function to identify based on,
Log management for causing the computer to realize a priority data storage processing function in which the priority data update unit stores the corresponding message definition information acquired as described above as the priority definition data in the storage area specified here program.

(Appendix 23)
In the log management program according to any one of appendices 20 to 22,
Immediately after determining that there is information to be used as the priority definition data, the priority data storage determination unit adds a set time as a reference for the priority collation period preset in the setting information storage unit to the current time. An exclusion time calculation function for calculating an exclusion time for excluding the priority definition data from the log definition storage unit,
Exclusion information transmission function in which the priority data storage determination unit transmits the priority data exclusion information including the exclusion time to the information related to the priority definition data to be excluded from the log definition storage unit to the priority data update unit ,
An exclusion information storage processing function for storing the priority data exclusion information in the setting information storage unit, the priority data update unit having a setting information storage unit for storing information related to the update of the priority definition data,
As soon as it is confirmed that the exclusion time has been reached by referring to the priority data exclusion information stored in the setting information storage unit, the priority data update unit excludes the priority definition data from the log storage unit. Message exclusion feature,
A log management program for causing the computer to realize an exclusion information deletion function in which the priority data update unit deletes the priority data exclusion information from the setting information storage unit immediately after the exclusion.

(Appendix 24)
In the log management program according to any one of appendices 20 to 23,
The statistical data update processing unit that manages the update of the statistical definition data provided in the log definition storage unit classifies the collated messages based on the collection target IP address and the operating environment, and then performs the process for each message ID. Checked count function that counts the number of utterances,
At the end of a preset unit time, the statistical data update processing unit of the statistical data update processing means creates a count list by arranging the count numbers in descending order,
A log management program for causing a computer to realize a statistical data update function in which a statistical data update unit of the statistical data update processing means updates the statistical definition data based on the count list.

  The present invention is applicable to an apparatus that collects log data from various log collection target devices connected by a network or the like because the matching process is performed using the unique characteristics of log data.

DESCRIPTION OF SYMBOLS 10 Log receiving means 20 Log collation processing means 30 Log storage part 40 Log access processing means 41 Log acquisition processing part 42 Log storage processing part 50 Priority data update processing part 51 Priority data storage determination part 52 Priority data update part 60 Statistical data update process Means 61 Statistical information calculation processing unit 62 Statistical data update unit 70 Log definition storage unit 71 Priority list 71A Priority definition data 72 Statistical list 72A Statistical definition data 80 Setting information storage unit 81 Storage determination information 81A Event definition information 81B Message appearance frequency information 81C Exception determination setting information 82 Priority data exclusion information 83 Unit time setting information 90 Log management device

Claims (10)

A log management device that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the environment of each device,
Log receiving means for receiving a plurality of messages constituting the log data;
Log verification processing means for executing verification processing for each message according to a predetermined order;
A log definition storage unit in which log format definition information classified based on the operating environment and data transmission time zone for each log collection target device is stored in advance,
The log collation processing unit executes the collation processing by referring to any one of the plurality of log format definition information stored in the log definition storage unit. . In the log management device according to claim 1,
The log format definition information is temporarily associated with statistical definition data in which the message definition information identified by the collation process is arranged in descending order of reception, and reference information relating to preset characteristics of the message. Priority definition data that is the message definition information to be collated preferentially, and
The log collation processing unit selects the log format definition information that matches a combination of a collection target IP address for the log collection target device, the operating environment, and a unit time ID corresponding to the data transmission time zone. A specific data collating unit that refers to the statistical definition data in the selected log format definition information and sequentially executes the collation processing with reference to the priority definition data prior to the statistical definition data in the priority collation period A log management apparatus comprising: In the log management device according to claim 2,
The log verification processing means includes:
An update information transmission unit that includes a priority data update processing unit that controls the update of the priority definition data, and transmits a message ID of the collated message, the collected control IP address, and the operating environment to the priority data update processing unit With
The priority data update processing means includes:
A priority data storage determination unit that determines the presence or absence of the message definition information to be the priority definition data based on the reference information;
A log management apparatus comprising: a priority data update unit that updates the priority definition data in the log definition storage unit based on a determination result indicating that the message definition information to be used as the priority definition data is present. In the log management device according to claim 3,
The reference information is a plurality of event definition information defined based on the collection target IP address, the operating environment, and the message ID,
Each event definition information is composed of a plurality of origin messages for event determination and subsequent messages corresponding thereto,
The priority data storage determination unit, when the combination of the collection control IP address, the operating environment, and the message ID sent from the log verification processing unit matches any of the origin messages, A log management apparatus characterized by determining that there is the message definition information to be transmitted and transmitting information related to the subsequent message corresponding to the matched origin message to the priority data update unit. In the log management device according to claim 3,
The reference information includes a message appearance in which the collated message that has undergone a classification process based on the collection target IP address, the operating environment, and a preset repetition period for every predetermined time is counted for each message ID. Frequency information,
This message appearance number information includes the current cycle count and the previous cycle count immediately before it.
The priority data storage determination unit calculates a ratio of the current cycle count to the previous cycle count, and the message definition to be used as the priority definition data when the calculated value exceeds a preset threshold value from the outside. A log management apparatus characterized by determining that there is information and transmitting information related to the determination result to the priority data update unit. In the log management device according to any one of claims 3 to 5,
The priority data update unit obtains the message definition information corresponding to the collection control IP address and the message ID sent from the priority data storage determination unit from a log definition master storage area in the log definition storage unit. The log management apparatus, wherein the message definition information acquired here is stored as the priority definition data in another specific storage area in the log definition storage unit. In the log management device according to any one of claims 3 to 5,
The priority data update unit stores the priority definition data in the log when a preset time as a reference of the priority collation period set in advance has elapsed since the priority definition data was stored in the log storage unit. A log management apparatus comprising a message exclusion function for exclusion from a department. In the log management device according to any one of claims 3 to 5,
The log definition storage unit is provided with statistical data update processing means for updating the statistical definition data,
The statistical data update processing means
The collated messages classified based on the collection target IP address and the operating environment are counted for each message ID, and a count list in which the counts are arranged in descending order at the end of a preset unit time is created. A statistical data update processing unit;
And a statistical data updating unit that updates the statistical definition data based on the count list. In a log management device that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the environment of each device,
The log receiving means receives a plurality of messages constituting the log data,
Next, each of the messages is sent by the log receiving means to the log matching processing means,
From the log definition storage unit in which the log format definition information classified based on the operating environment and the data transmission time zone for each log collection target device is stored in advance, the verification processing unit is configured to output the log format based on each message. Select definition information,
A log management method, wherein the log collation processing unit performs collation processing of each message received from the log reception unit by referring to the selected log format definition information. In a log management device that collects a wide variety of log data transmitted from a plurality of log collection target devices and manages the environment of each device,
A log receiving function in which a log receiving means receives a plurality of messages constituting the log data;
A log transmission function in which each of the messages is transmitted by the log receiving means to a log verification processing means;
From the log definition storage unit in which the log format definition information classified based on the operating environment and the data transmission time zone for each log collection target device is stored in advance, the verification processing unit is configured to output the log format based on each message. Log definition selection function to select definition information,
By referring to the log format definition information selected by the log definition selection function, the log verification processing unit realizes a log verification processing function for performing verification processing of each message received from the log reception unit in a computer. Log management program to let you.

Images