JP6517677B2 - Integrated operation monitoring system and calculation method of association degree of operation operation log - Google Patents

Description

  The present invention particularly relates to support technology for system operation management work.

  The operation manager of the computer system manages, as an incident, a failure or abnormality that has occurred on the computer system, and carries out a cause investigation work or an error countermeasure work for the incident as a response work to the incident.

Software supporting incident response work includes incident management software and monitoring software. Incident management software provides functions to register and manage incidents. The operation manager records the response work performed on the incident as an incident response history using the incident management software.
In the incident response history, "when, who performed what task" is recorded.

  Monitoring software is software that supports fault / error detection and cause analysis. The monitoring software communicates with the hardware and software to be managed, and collects operation data indicating the system operation status. The operation data includes numerical data such as CPU utilization, and character string data such as text messages representing the state of the application (AP). Also, when the collected operation data satisfies a predetermined condition, an alert is sent to the operation manager, and it is registered as an incident in cooperation with the incident management software. Further, the operation data is processed into a line graph or the like and displayed on the screen of the operation terminal of the administrator.

  Since monitoring software and incident management software are often used together, a product integrating both software has also been proposed. Below, such a product is called an integrated monitoring server.

  In the incident response work using the integrated monitoring server, the operation manager confirms the incident that has occurred, analyzes the cause by examining the operation data related to the incident, and manages it using other management tools etc. Implement measures for hardware and software. Then, the response work content is recorded as an incident response history.

JP 2014-115890

  In managing incidents, it is important to record incident response history in detail. If the incident response history is recorded in detail, the same evidence as incident response work and incidents that occurred in the past can be used as reference information at the time of occurrence.

  However, since the incident response history is manually recorded by the operation manager, the described contents may be omitted and a divergence from the actual work may occur. In order to reduce the difference between the incident response history and the actual work, it is conceivable to utilize an operation operation log in which the integrated monitoring server and the management tool record the incident response history. The operation operation log is an operation history of devices and management tools by the operation manager. However, conventionally, the incident response history and the operation operation log are not associated. Therefore, it has been necessary for the operation manager to manually search the plurality of operation operation logs using a plurality of key information included in the incident and its response history to extract the operation operation log.

  There is Patent Document 1 as a prior art for automatically extracting an operation log related to an incident. Patent Document 1 discloses a technique for combining input information templates and information extracted from a log file based on a correspondence history template previously defined for each correspondence procedure, and automatically inputting it as an incident correspondence history.

  According to Patent Document 1, the operation operation log related to the incident that can be clearly defined and can be predefined is automatically extracted / recorded. However, there is a problem that it can not be applied to an incident that can not be predefined and the response procedure is not clear.

  Also, as a simple automatic extraction method, a method of collecting all the operation operation logs generated in the period from the occurrence of an incident to the resolution can be considered. In this way, relevant logs can be collected even if the response procedure is not clear. However, if another operation work (another incident response work or daily operation work) is performed during that period, unrelated logs are extracted from the incident, and the accuracy of the extracted logs is not good. is there.

  An object of the present invention is to automatically and accurately extract an operation operation log related to an incident whose response procedure is not clear.

  In the present invention, as the operation operation log automatic extraction method related to the incident, the incident and incident response history and the operation operation log are input, and the degree of comprehensive association between the incident and the operation operation log from the relationship in multiple viewpoints Disclose a method for calculating the operation log and extracting the operation operation log related to the incident based on the degree of association.

  As a specific calculation method of the degree of association, the system configuration information on the managed system and the operation manager information are also used as input, and each correspondence history and operation of the incident from three viewpoints of time, system configuration, and operation manager Disclosed is a method for calculating the total degree of association by calculating the degree of association of the operation log and combining those associations.

  The operation manager can reduce the time required for collecting and searching operation operation logs related to an incident.

FIG. 1 is a view schematically showing a computer system in a first embodiment. FIG. 2 is a block diagram showing a physical configuration of an integrated monitoring server in Embodiment 1. It is a figure showing the example of the instance information which constitutes a managed system. It is a figure showing the example of the system dependence between instances. It is a figure showing an example of operation data. It is a figure which shows the example of operation management information. It is a figure showing the example of the incident. It is a figure showing the example of incident response history. It is a figure which shows the example of the operation | movement data access log which an integrated monitoring server outputs. It is a figure which shows the example of the operation | movement data access log which an integrated monitoring server outputs. FIG. 7 is a diagram showing an example of the calculation of the degree of association in the first embodiment. It is an example of the incident display in a data display part, and the operation operation log display screen relevant to an incident. It is an example of the operation | use data display screen in a data display part. It is an example of the flowchart of the degree-of-association calculation process of the related operation operation log automatic extraction part in a present Example. In calculation of relevance R1 based on the time of a present Example, an example of the functional model which quantifies the nearness of time is represented graphically. It is an example of a detailed flowchart of calculation processing of degree of association R1 based on time in this example. It is an example of a detailed flowchart of calculation processing of relevance R2 based on the system configuration in the present embodiment. It is an example of a detailed flowchart of calculation processing of relevance R3 based on the operation manager in the present embodiment. It is a detailed flowchart of calculation processing of total relevance degree Rx in a present Example. It is an example of the flowchart of a production | generation of the operation operation log display screen relevant to the incident by the data display part in a present Example. FIG. 7 schematically shows a computer system according to a second embodiment. It is a figure which shows the example of operation operation log setting. It is a figure which shows the example of the command operation log which a remote connection terminal outputs. FIG. 6 is a diagram illustrating an example of a job execution log output by a job management server. FIG. 7 is a diagram showing an example of a VMM operation event log output by a VMM management server. It is a figure which shows the example (addition to FIG. 10) of the association degree calculation result in Example 2. FIG. FIG. 16 is a diagram showing an example (addition to FIG. 9) of the operation data access log output by the integrated monitoring server in the third embodiment. It is a figure which shows the example (addition to FIG. 10) of the related degree calculation result in Example 3. FIG. FIG. 16 schematically shows a computer system according to a fourth embodiment. FIG. 18 is a diagram showing an example of operation pattern information in the fourth embodiment. FIG. 18 is an example of correcting the degree of association based on the operation pattern in the detailed flowchart of the calculation process of the total degree of association Rx in the fourth embodiment. FIG. 21 is an example of operation operation pattern automatic extraction / generation processing by an operation pattern automatic generation unit according to a fourth embodiment; FIG.

  The integrated monitoring server according to the present embodiment receives incident information, system configuration information, operation manager information, and operation operation log as input, time between the incident and the response history and the operation operation log, system configuration, operation manager It provides the function to calculate the degree of association based on the above and output a list of operation log related to the incident from the calculation result.

  In the first embodiment, the operation data access log related to the incident is output as a list.

  Hereinafter, an embodiment will be described using the drawings.

FIG. 1 schematically shows a computer system assumed in the first embodiment.
The computer system includes an integrated monitoring server 3, a management DB 4, an operation terminal 5, and a management target system 6. These devices are connected to the management network 1 through the physical communication line 2.

The management target system 6 is an example of components constituting a system to be managed. On the managed system 6, server components such as a physical machine (Physical Machine, PM) 63, a virtual machine monitor (Virtual Machine Monitor, VMM) 61, and a virtual machine (Virtual Machine, VM) 62 mounted thereon are included. There is a component such as a load balancer 67 that exists and distributes requests to multiple servers. Also, on the server, there are components such as OS 63-1 and middleware 64-1 and AP 65-1 installed on the OS. Furthermore, there is a monitoring agent 66 as a component for acquiring operation data from each component. Although each component may exist in plurality depending on the configuration of the management target system, only one component is described in the figure for simplicity.
Hereinafter, the device mounted on the managed system 6 or the functional component mounted on the device will be referred to as a component.

  The management DB (DataBase) 4 stores information necessary for the integrated monitoring server 3 to provide incident management and operation data display functions such as operation data T100, incident information T200, system configuration management information T500, and operation manager information T800. ·to manage. In addition, it stores and manages an operation operation log T 900 to be associated with an incident. Furthermore, it stores and manages information on the calculation method of the degree of association and the calculation result.

  The integrated monitoring server 3 uses a data management unit 31 that provides functions for collecting, registering, and displaying operation data, incident information, operation operation logs, etc., and incident information and operation operation logs collected and registered by the data management unit 31. And an associated operation operation log automatic extraction unit 32 which extracts operation operation logs related to the incident.

  The data management unit 31 includes a data collection unit 310, a data registration unit 311, and a data display unit 312.

  The data collection unit 310 is a program for collecting operation data and incidents from devices in the management target system. As a general collection method, Simple Network Management Protocol (SNMP) or a monitoring agent 66 specific to the monitoring system is operated on each device, and communication between the monitoring agent 66 and the data collection unit 310 enables operation data or the like. Collect Incidents The data collection unit 310 in the present embodiment also has an incident detection function of generating an incident when the collected operation data satisfies a predetermined condition.

  The data registration unit 311 is a program for registering the operation data and incident information received by the data collection unit 310 in the management DB 4. Further, the data registration unit 311 registers the incident information in the management DB 4 by an input operation from the operation manager who is a user of the data management unit 31.

  The data display unit 312 acquires the information stored on the management DB 4 and displays it on the screen in response to a request from the operation manager who is the user. For example, when the data display unit 312 receives a display request for operation data and incidents from the web browser 51 on the operation terminal 5, the data display unit 312 receives operation data T100 and incident information T200 from the management DB 4 according to the search condition included in the display request. Take out. Then, the extracted operation data T100 and incident information T200 are analyzed and processed according to the request, and the result is displayed on the Web browser 51. In addition, the data display unit 312 records the history of operation data and incident display performed via this program on the management DB 4 as the access log T1000.

The related operation operation log automatic extraction unit 32 receives the incident information acquired from the management DB 4, the system configuration information, the operation manager information, and the operation operation log as input and relates between the incident (and the response history) and the operation operation log. It is a program that calculates the degree and determines that it is relevant if the calculation result exceeds a certain threshold. The related operation operation log automatic extraction unit 32 includes an association degree calculation unit 320 and an association degree output unit 327.
The degree of association calculation unit 320 calculates the degree of association between an incident and an operation log based on three viewpoints of time, system configuration, and operation manager, and combines the results to obtain a total degree of association. Ask. The degree-of-association calculation unit 320 includes calculation units 321 to 324 in each viewpoint for calculating the above. Details of the calculation units 321 to 324 will be described later.
The degree-of-association output unit 327 provides a function of outputting the result calculated by the degree-of-association calculation unit 320. For example, in the present invention, the calculation result is stored in the management DB4.
The management DB 4 includes operation data T100, incident information T200, system configuration information T500, operation manager information T800, operation operation log T900, and association degree calculation result T1200. The various information stored in the management DB 4 is information collected by the data collection unit 311, information input by the operation manager via the data management unit 31, and the like. Information may be stored by automatic collection and manual input by other management tools. In the present embodiment, various types of information are stored in tabular form on the management DB 4. However, embodiments of the present invention are not limited to tabular data. For example, the present invention can be applied to data having a tree structure or text data as long as there is data corresponding to each information described later.
FIG. 2 is a block diagram showing the physical configuration of the integrated monitoring server 3 in the first embodiment. The integrated monitoring server 3 is a computer including an interface (I / F) 100, a processor 101, and a memory 102. The I / F 100, the processor 101, and the memory 102 are connected by a data bus 103. The integrated monitoring server 3 communicates with the management network 1 via the I / F 100. The processor 101 is an arithmetic device such as a CPU. The memory 102 is a storage area for holding programs and data. The processor 101 reads a program from the memory 102 via the data bus 34 and executes the program.
FIGS. 3 and 4 show an example of the system configuration information T500 stored in the management DB 4. FIG. The system configuration information T500 includes instance information T600 and a system dependency T700. In the present embodiment, information on current components in the managed system is managed.

In the following, the system to be managed will be simply referred to as "system" as needed.
Also, a name for uniquely identifying a management target system is called a "system name". The device on the managed system is called an "instance", and the name for uniquely identifying an instance is called an "instance name". The instance name may be a host name described in FQDN (Fully Qualified Domain Name) or an IP address.

  FIG. 3 is an example of instance information T600 stored in the management DB 4. Column T601 is the system name to which the instance belongs. Column T602 is an instance name. A column T603 is information indicating the role played by the instance in the system. The role stores information that specifies a logical role, such as “load balancer virtual host”, “AP server”, and “DB server”. The division of the column T604 is information in which roles are classified. In this example, "service" is set to an instance prepared to provide a service to an end user, and "management" is set to an instance prepared to manage other instances.

  When the managed system 6 becomes complicated / large-scaled, a plurality of similar management tools may be prepared to distribute the load of the management tools. Therefore, it may be necessary to manage system configuration information including information as to which management server performs operation management of which instance (hereinafter referred to as management system information). Therefore, in the present embodiment, instance information including a management system is managed. However, in the present embodiment, a single integrated monitoring server 3 monitors all the instances constituting the service.

  Therefore, even if the category "management" does not use instance information in particular, the relationship with the target instance is clear. Therefore, in the first embodiment in which only the operation operation log T 900 of the integrated monitoring server 3 is targeted, only the information of the instance in which the classification is set to “service” is used. The information of the instance in which the classification is set to "management" is used in the second embodiment.

  FIG. 4 shows an example of the system dependency T700 stored in the management DB 4. The dependency (example: connection) between the components of the system is managed. In this embodiment, dependencies between instances are managed. A column T701 is a system name to which the instance belongs. Column T 702 is the instance name of the dependee. Column T 703 is the instance name of the dependency source. A plurality of instance names are also set in the dependency source. A column T704 is dependency content that describes the dependency.

  For example, row T 750 indicates a dependency relationship in which the load balancer virtual host “ap1server” is “load-balanced” by connecting to the AP servers “ap1server 1” and “ap1sever”. The row T 751 shows the dependency relationship in which the AP server “ap1server1” is connected to the DB server “dbserver1” and performs “DB access”.

  In the system dependency T700, as in the instance information T600, not only the instances constituting the service but also the dependency of the management system instance is held. The dependency relationship of the management system instance (specifically, the dependency relationship in which the dependency content T 704 is set to “managed object”) is not used in the first embodiment in which only the operation log of the integrated monitoring server 3 is targeted Used in Example 2.

  FIG. 5 is an example of operation data T100 stored in the management DB 4. The operation data T100 is information obtained by analyzing the operation data itself collected from the device on the management target system or its data structure and processing it into a table format.

  In the present embodiment, the operation data is assigned information of the system name, the instance name, and the type of operation data. Here, a name for uniquely identifying the type of operation data is called "operation data type name". The operation data type includes data on hardware devices (for example, network devices), data on VMM, data on VM (and OS installed on VM), middleware commonly used by multiple applications on the OS There are data related to (DB, Web container, etc.) and data specific to AP. The data type name is character string data such as “health check of service 1”, “CPU utilization of OS”, “number of AP server simultaneous connections”, “Web AP log of service 1”, etc., which can identify these types.

  Column T101 indicates the time when the operation data was measured. Columns T102, T103, and T104 are a system name, an instance name, and a data type name that are output sources of operation data, respectively. Column T105 is detailed data other than the measurement time included in the operation data. In this example, data other than time is shown in the form of key = value.

  Here, to explain a specific example of operation data, the row T150 indicates that the result of the health check of service1 performed on the load balancer virtual host ap1server at 9:00:00 on April 10, 2015 is normal. Indicates that the response time took 200 seconds.

  FIG. 6 is an example of the operation manager information T 800 stored in the management DB 4. The operation manager name in the column T801 is a name for uniquely identifying the operation manager. The operation responsible group in the column T802 is information for specifying the operation responsible group to which the operation manager belongs. A column T 803 is an instance for which the operation manager has the operation operation authority. One or more instance names are set in T803 so as to correspond to the instance information T600. In the present embodiment, it is assumed that an instance having the operation authority is set for each operation charge group. A column T804 indicates the personal account name of the integrated monitoring server, which is information corresponding to the operation manager name T801. Each operation manager uses this account name when using each function of the integrated monitoring server 3 (eg, login to the data display unit 312, etc.). Further, information of the account name is also output to the access log T1000 for the operation data.

  In the operation management information T800, account information other than the integrated monitoring server 3 may be managed together. The common account name of the management tool in the column T 805 indicates the account name for using various management tools other than the integrated monitoring server 3. In this case, in order to simplify the example, we will use a common account set for each operation responsible group as the account of various management tools. Therefore, the information corresponds to the operation charge group T802. The common account name T 805 of the management tool is not used in the first embodiment in which only the operation log of the integrated monitoring server 3 is targeted, and is used in the second embodiment.

  Here, to describe a specific example of the operation manager information T800, lines T850 and T851 indicate information on the operation managers "user1" and "user2", respectively. Both “user1” and “user2” belong to the operation handling group “group1”, and indicate that all instances of service1 have access rights.

In the present embodiment, in order to simplify the description, an example is shown in which the operation manager manages by a simple two-tier system belonging to one operation charge group. A complex operation manager information having a structure of three or more layers, one operation manager may belong to a plurality of operation charge groups, and role information is set for each operation manager may be used.
7 and 8 show an example of the incident information T200 stored in the management DB 4. Incident information T200 includes incident T300 and incident response history T400.

  FIG. 7 is an example of the incident T 300 stored in the management DB 4. In the incident T300, basic information describing the incident and the latest status information are registered. The incident ID in the column T301 is information for uniquely identifying an incident registered in the management DB 4. In this embodiment, the incident ID is automatically numbered on the management DB 4. The occurrence date and time of the column T302 indicates the time when the incident occurred. The latest status of the column T303 indicates the latest status information of the incident, and “not supported” is set when the incident occurs, “during response” during the incident response, and “resolved” after the incident resolution. Among the information stored in the incident T300, only the latest status T303 is information to be updated after the occurrence. The system name in column T304 is the system name where the incident occurred. The degree of importance of the column T305 is information indicating the degree of importance of the incident, and “warning” is set to “failure” for an incident requiring an emergency response or “incident” for an incident not necessarily requiring an emergency response. The generated instance in the column T306 indicates the name of the instance in which the incident has occurred. The operation data type of the column T307 indicates the operation data type in which the incident occurred. The columns T304, T306, T307 correspond to the columns T102, T103, T104 of the operation data T100. The incident message in column T308 is text representing the content of the incident.

  Here, to describe a specific example of the incident T300, a row T350 is an incident to which the incident ID 100 is assigned, and it is service1 in the load balancer virtual host ap1server at 9:00:00 on April 10, 2015. It indicates that the response time of the health check of is an incident that occurred because the threshold exceeded 150 seconds. The incident T350 is an incident that occurs as a result of the threshold determination on the row T150 illustrated in the example of the operation data T100.

  FIG. 8 shows an example of the incident response history T400. In the incident response history T401, information entered by the operation manager is recorded. In this example, each row represents one response operation performed for a certain incident. The incident ID of the column T401 is the ID of the incident to be subjected to the work described in the correspondence history, and is information corresponding to the column T401 of the incident T300. The incident response ID in the column T402 is information for uniquely identifying each response history for the incident ID T401. In the present embodiment, the incident response ID is automatically numbered on the management DB 4, and a value obtained by sequentially adding from 1 to each incident ID is sequentially assigned. The corresponding date and time in the column T403 indicates the time when the corresponding work is performed and completed (usually, the time when the corresponding history is recorded). The correspondent in the column T404 is the name of the operation manager who has performed this correspondence work. The status of the column T405 indicates the status of the incident after the completion of the response work. In the present embodiment, the latest status T303 of the incident T300 is also updated simultaneously with the registration of the incident response history. The corresponding content of the column T406 is text describing the content of the corresponding work.

  Here, a specific example of the incident response history T400 will be described. Rows T450 to T454 are correspondence history indicating the process until the solution for the incident (ID 100) shown in row T350 of the incident T300. For example, a row T450 (incident response ID 1) indicates the response work performed by the operation administrator user1 at 9:30:00 on April 10, 2015. In this response work, in order to investigate the cause, the operation data is investigated using the data management unit 31 of the integrated monitoring server 3, and as a result, "it is confirmed that the load on the AP servers ap1server1 and ap1server2 is increased". Is a history of recording

  Since the incident response history is manually input by the operation staff, abstraction / simplification of work content and omission of work content may cause divergence between the description content and the actual work content. (For example, the corresponding contents of the line T453 are not filled in. In fact, the examination of the operation data is also conducted, and the like). If there is a gap between the description content of the incident response history and the actual work content, it is difficult to use only the description content of the response history as evidence or to reuse it as work know-how. Therefore, in the method for automatically extracting an operation operation log related to an incident according to the present invention, the associated operation operation log can be found even if the content is not described in the correspondence history.

  Hereinafter, the operation operation log T 900 to be associated with an incident will be described. In the first embodiment, an example in which the access log T1000 to the operation data performed by the operation manager via the data display unit 312 of the integrated monitoring server 3 is used as the operation operation log T900.

  FIG. 9 is an example of the access log T1000 to operation data. Although an access log which is not necessary in the description of the present embodiment is not described, in reality, other unrelated access logs are outputted while the access log is described as "...".

Hereinafter, the access log to the operation data is simply referred to as "access log".
The access log T1000 stores a history of the operation manager accessing the operation data T100 and the incident information T200 via the data display unit 312. The access via the data display unit 312 includes a request for display of each operation data and incident information. The access log T 1000 is generated by the data display unit 312 each time the operation manager inputs data display or the like to the data display unit 312.

  The log ID in the column T1001 is information for uniquely identifying the access log registered in the management DB 4 as the access log. In the present embodiment, log IDs are automatically assigned on the management DB 4. The operation date and time of the column T1002 indicates the time when the operation manager accesses data via the data display unit 312. The column T1003 shows the system name of the displayed operation data. Column T 1004 indicates the operation manager who made this access. The session ID in the column T1005 indicates information for identifying session information for associating accesses successively performed within a predetermined short time. When the data display unit 312 is Web AP, this column includes session information included in the HTTP cookie.

The operation operation details of the column T1006 include detailed information such as a request for display in this access. Since the contents of the operation operation details differ depending on the display screen, a plurality of key / value pairs are stored in the form of “key = value”. The contents of the operation operation details include, for example, “display screen”, “target instance”, “operation data type”, “search time condition”, and “selected incident ID”. "Display screen" indicates the type of screen displayed. The “target instance” and the “operation data type” indicate the instance name and data type name of the operation data displayed on the screen. The “search time condition” is a display period of operation data. In addition to the operation operation details, search conditions for narrowing down the operation data may be stored. In the access log described in the present embodiment, it is assumed that the target instance is always included in the operation details.
Here, to describe a specific example of the access log, a row T1050 indicates that the operation manager user1 displayed the incident with the incident ID 100 at 9:15:04 on April 10, 2015. In addition, the row T1051 performs response data display as operation data display by the operation manager user1 at 9:16:00 on April 10, 2015, and the health of service1 performed on the load balancer virtual host ap1server. It indicates that response data indicating the result of the check is displayed. Here, the row T1051 is an example of the case where the operation data is examined continuing from the incident display of the row T1050. Therefore, "100" is set as the selected incident ID. On the other hand, in the access log of row T1056, the selected incident ID does not exist despite the same operation manager and session ID as the row 1051. This shows an example in which the operation data investigation unrelated to the incident ID 100 is performed by returning to the home screen (initial screen) of the data display unit 312 or the like.

  In the data display unit 312 described in the present embodiment, while the operation data investigation is continuously performed from the incident display, the selected incident ID is recorded because the incident is explicitly selected. However, if you return to the home screen (initial screen), etc., and conduct an operation data survey that has nothing to do with incident ID 100, there is no means to determine the presence or absence of association unless it is selected from the incident display again. Therefore, the selected incident ID is not recorded.

  Even if the present invention is not used, if the incident ID selected as described above is explicitly designated, the incident can be associated with the operation operation log. However, when the operation data investigation regarding the incident is interrupted and resumed (for example, the row T1054 is applicable), it can not be dealt with if there is an implicit relationship. The operation log extraction method related to an incident according to the present invention also enables discovery of operation logs implicitly related as described above.

In this embodiment, as the operation operation log automatic extraction method related to the incident, the relationship calculated by calculating the degree of association between the incident (and the response history) and the operation operation log by using the various information described above is input It is determined that the relationship is relevant if the degree exceeds a certain threshold.
In this embodiment, the degree of association between the incident and the operation operation log is calculated based on three viewpoints of time, system configuration, and operation manager, and the total degree of association is calculated by combining the results. .

  In addition, each association degree of a present Example is scored as a value normalized between 1-100. The larger the value, the stronger the association. Further, when the degree of association exceeds the association determination threshold RT, it is determined that there is association, but the value of RT is set as “70”.

The explanation of the degree of association in the present embodiment and the calculation policy are as follows.
[Time-based relevance R1]:
-Description: An indicator that indicates the closeness of time between incidents and operation logs.
Calculation policy: The closer the operation date and time of the operation log to the response date and time of the incident response history, the higher the degree of association.
[R2 based on system configuration]:
-Description: This is an indicator that indicates the closeness of the connection between system configuration information included in the incident and the operation log (closeness of the system space).
Calculation policy: The closer the number of connection hops to the instance described in the incident occurrence instance or the response history of the target instance of the operation log, the higher the degree of association.
[Relevance R3 based on operation manager]:
-Description: This index indicates the proximity (closeness of the user space) of the responsible area of the operation manager included in the incident and the operation log.
Calculation policy: The closer the affiliation between the operation manager who executed the operation log operation operation and the response person of each incident response history, the higher the degree of association.
[Total relevance score Rx]:
-Description: An indicator that indicates that the time, system configuration, and operation manager's viewpoint are all relevant. Calculation policy: The larger the combined value of R1, R2 and R3, the larger the degree of association.

  The calculation policy described above and the logic of specific calculation processing may be bundled in the integrated monitoring server 3 or may be externally defined. The advantage of external definition is that it can easily cope with changes and additions to calculation policies and logic.

FIG. 10 is an example of the degree-of-relevancy calculation result T1200 storing the result calculated according to the degree-of-association calculation method described above. In FIG. 10, only the calculation results corresponding to the incident of the incident ID 1 shown in FIGS. 6 and 7 and the access log T1000 described in FIG. 9 are described. In practice, calculation results for combinations of incident IDs and log IDs are stored.
Furthermore, an example is described in which only the result in which the total degree of association Rx is the largest value is finally held for the combination of the log ID and the incident ID. If there is enough data storage capacity, calculation results for combinations of log IDs, incident IDs and incident response IDs may be left. Conversely, if it is desired to save data storage capacity, only the results determined to be relevant may be retained.

The log ID of the column T1201 indicates the log ID of the log for which the degree of association is to be calculated. This is information corresponding to the log ID T1001 of the access log T1000. The incident ID in the column T1202 indicates the incident ID of the incident for which the degree of association is to be calculated. Information corresponding to the incident ID T301 of the incident T300. The incident response ID in the column T1203 indicates the response history ID of the incident response history that has been subjected to the degree of association calculation. Information corresponding to the incident response ID T402 of the incident response history T400. In this example, only the result of the incident response ID with the highest relevance Rx for each incident is recorded.
The columns T1205 and T1206 are the value of the calculation result of the degree of association R1 based on time and the reason. The columns T1207 and T1208 are values of calculation results of the degree of association R2 based on the system configuration and the reason. The columns T1209 and T1210 are the value of the calculation result of the degree of association R3 based on the operation manager and the reason. The columns T1211 and T1212 are the value of the calculation result of the total degree of association Rx, and the presence or absence of association based on the determination by the association determination threshold RT.

  Here, a specific example of the association degree calculation result T1200 will be described. In the row T1253, the access log of the log ID 1003 is most relevant to the correspondence of the incident response ID 1 in relation to the incident ID 100, and the value of the total relevance Rx is “100”, and the threshold determination The result shows the result judged to be "relevant". The degree of association R1 with respect to time has a value of “100” because the access log “occurs 15 minutes before correspondence ID 1”, and the degree of association R2 based on the system configuration has the number of hops connected to the instance related to the incident The value is "100" because the "ap1sever1 ap1server2 information is close", and the relevance R3 based on the operation person in charge is because the operation person in charge of the access log "coincides with the person in the incident response history" The value is "100".

  In row T1255, the access log with log ID 1020 is most associated with the response to incident response ID 1 in relation to incident ID 100, and the value of the total relevance Rx is “21”, and the threshold determination The result shows the result judged as "not related". In this example, although the values of R1 and R2 are large, the relation reason of R3 is considered to be "-", ie, the relation is small, and the value of Rx is also small because the value is "1".

  In addition, although the example of using and recording relevance degree calculation result T1200 on management DB 4 in order to make explanation intelligible at this time was described, even if it does not record on management DB 4, it processes all in memory I do not care.

Hereinafter, the display function of the incident and operation data in the integrated monitoring server 3 and the operation operation log output will be described using an example of a display screen.
The data display unit 312 displays a login screen, receives an account name and password input from the operation manager, performs user authentication, and operates operation operation authority of operation manager information T800 according to the input account name. Refers to an instance and displays only information about authorized instances. After user authentication, the data display unit 312 displays the home screen of the operation data display, and on the home screen, a link to the operation data display screen corresponding to the data type or instance for which you have the browsing authority, a search interface, and incident information A link to the incident display screen to be displayed and a search interface are displayed. The operation data display screen and the incident display screen described later are displayed according to the selection of the operation manager, and these display screens can be moved mutually.

  FIG. 11 is an example of the incident on the data display unit 312 and the related operation operation log display screen. The incident display screen displays an incident T300 and its incident response history T400.

  The table at the top of the display screen displays information of the designated incident T300. Information on the response history T400 of the incident is displayed in a table in the center of the display screen. In this example, the contents of the incident T300 and the response history T400 of the incident ID 1 shown in FIGS. 6 and 6 are displayed.

You can use the search function by pressing the "incident search" button at the top of the display screen. Specifically, a user interface for search is displayed, and the incident can be searched by the date and time range of the incident and keyword designation, and another incident can be selected from the searched incident list. In addition, by pressing the "add response history" and "selected response history correction" buttons in the central portion, the user interface for incident response history registration is displayed, and the incident response history can be added / updated. By pressing the “Operation data investigation related to the incident” at the top, a link list to the operation data related to this incident is displayed, and by pressing that link, from then on, this incident is selected The operation data display screen can be displayed while holding it.
The table at the lower part of the display screen is a result of displaying a list of the operation operation logs related to the incident T200 currently displayed and the incident response history ID. In this example, the operation operation logs determined to be related are all displayed in the order of the associated incident response ID and the operation date and time. Although this example is a list of operation operation logs related to the incident of incident ID 1, the display is omitted except for two operation operation logs. The result of the first log ID 1061 corresponds to the row T1062 of the access log T1000 of FIG. 9 and the row T1262 of the association degree calculation result T1200 of FIG. The second case is the result of the operation log associated with the second embodiment described later.

  FIG. 12 is an example of the operation data display screen in the data display unit 312. The operation data display screen displays operation data T100 according to the search condition included in the display request. In addition, when the incident transitions in a selected state, information on the incident T200 is also displayed.

  In the table at the top of the display screen, when a transition is made in a state where an incident is selected, information on the incident T300 is displayed. If the incident is not in the selected state, this table is not displayed. The lower frame is the operation data display portion. The upper graph in the frame is a graph obtained by acquiring, analyzing, processing the operation data T100 from the management DB 4 in accordance with the designated search condition. The table at the lower part of the display screen is also a table obtained by acquiring, analyzing and processing the operation data T100 as in the graph.

A user interface “move to related operation data” in the center of the display screen is a link for moving to another operation data related to the currently viewed operation data. For example, a list of all operation data types of all the instances having a direct dependency relationship with the currently displayed instance is presented, and it is possible to transition to another operation data of the same time zone by making the links identical. Also, the "[Previous day] [Next day] [Monthly display] [annualized display]" link at the top of the operation data display part is a user interface for moving to the information of another time zone or time range of the same operation data. .
The operation manager accesses the incident and operation data display screen described above and investigates the operation data related to the incident by tracing the operation data one after another.

  Hereinafter, the automatic extraction process of the operation operation log related to the incident in the first embodiment of the present invention will be described using a flowchart.

  FIG. 13 is an example of a flowchart of association degree calculation processing of the related operation operation log automatic extraction unit 32 according to the first embodiment of the present invention. The degree-of-association calculation process includes a data management step (S101 to S104) of acquiring data necessary for calculation via the data management unit 31, and a degree-of-association calculation step (S105, S20, S30, S40, S50, S106) and result output steps (S21, S31, S41, S51) for outputting and holding the calculation result.

First, from the incident T200 and the incident response history T300 on the management DB 4, a list INC_HISTORIES of the incident response history is acquired using the incident INC for which the degree of association is to be calculated and the incident ID of INC as a key (S101). In the specific example, when the row T350 of the incident T300 in FIG. 6 is the target (INC), the incident ID is “100”, and the INC_HISTORIES is the rows T450 to T454 in the incident response history T400 in FIG.
Next, a set SYS_SET of system configuration information corresponding to the system name included in the INC is acquired from the instance information T600 on the management DB 4 and the system dependency T700 (S102). Describing the specific example, when the incident T 350 of the incident ID 100 in FIG. 6 is the target (INC), the system name is “service 1”. In the first embodiment, instance information in which the classification is set to "service" is used. When the instance information T600 in FIG. 4 is searched using the system name “service1” and the classification “service” as keys, rows T650 to 656 are acquired as instances. These are all instances that make up the service of the system "service1". With regard to the system dependency T700, when the dependency is “managed” and the system name “service1” is searched for as a key, it is possible to obtain the dependency regarding the instances that constitute the service. SYS_SET holds these pieces of information.

Furthermore, a list USERS of operation person-in-charge information corresponding to the system name included in the INC is acquired from the operation person-in-charge information T800 on the management DB 4 (S103).
Describing the specific example, when the incident T 350 of the incident ID 100 in FIG. 6 is the target (INC), the system name is “service 1”. When the operator information T800 in FIG. 6 is searched with “service 1” as a key, rows T850 to 854 are acquired as the operation manager based on the instance T803 having the operation operation authority.
Subsequently, all the logs to be related extraction targets are acquired from the operation operation log T 900 on the management DB 4 using the information acquired in S 101 to S 103. Specifically, in the first embodiment, the log generated from the access log T1000 on the management DB 4 as the operation log T900 to the response date T403 of the latest incident response history in INC_HISTORIES from the occurrence date T302 of INC to All are acquired (S104). Hereinafter, the list of acquired operation log is called LOGS.

  To explain in a specific example, in the case of incident ID 100, the occurrence date and time is "9:15:04 on April 10, 2015", and the response date and time "2015 of incident response history (incident response ID 5)" All access logs T1000 that occurred between 17:10:00 on April 12, 2007 are acquired. Since all the access logs described in FIG. 9 are included in the above period, all the rows T1050 to T1071 are acquired. In addition, in this example, since the description is omitted except for the access log necessary for the explanation, it is possible to actually acquire other access logs that are not related to the incident.

  Subsequently, the association degree calculation unit 320 acquires one uncomputed operation operation log LOG from the LOGS (S105).

  Then, using the selected LOG and the information acquired up to S103 and LOG, the relevance R1 based on time, the relevance R2 based on the system configuration, and the relevance R3 based on the operation manager are respectively calculated (S20, S30 , S40), and stores each result in the degree of association calculation result T1200 on the management DB 4 via the degree of association output unit 327 (S21, S31, S41). Furthermore, the degree of association R1 to R3 calculated above is combined to calculate the total degree of association Rx (S50), and the result is calculated as the degree of association calculation result T1200 on the management DB via the degree of association output unit 327. Store (S51). Details of the calculation processing of these S20 to S51 will be described later.

  After the above series of calculations are performed on the selected LOG, if there is an uncalculated operation log in the LOGS, the processes after S105 are repeated (S106). Further, in the above, the date and time of the latest incident response history is used as the condition for searching the operation operation log. Furthermore, the degree of association can be calculated for each incident response history as described in the description of the calculation process described later. That is, the present invention can be applied to an incident being solved.

Hereinafter, details of a method of calculating the degree of association R1 based on time will be described. In order to quantify the time closeness of the incident and the operation operation log, R1 of this embodiment has a value such that the degree of association becomes larger as the operation date and time of the operation operation log is closer to the response date and time of the incident response history. calculate.
FIG. 14 is a graphical representation of an example of a functional model for quantifying the closeness in time in the calculation of the degree of association R1 based on time. In the present embodiment, the operation date TL of the operation operation log is the current time as a period in which there is a time relationship between the response date T1 and the response date T0 between the incident response history one before in a specific incident response history. A function is used such that the value of the degree of association approaches 100 as the closer to the corresponding date and time T1 of. In the function model, the activation period A is set to prevent the slope from becoming too steep when the periods T0 and T1 are short, and the degree of association is 100 during the period from TA (= T1-A) to T1, The functional model is used such that the period from T0 to TA linearly changes from 1 to 100. Although the activation period A can be set to any value of 0 or more, it is set to 30 minutes in this embodiment.

  FIG. 15 is an example of a detailed flowchart of the calculation processing S20 and S21 of the degree of association R1 based on the time of the related operation operation log automatic extraction unit 32. FIG. 15 is an example of the case where the function model shown in FIG. 14 is implemented as calculation logic.

  The degree-of-relevance calculation unit 320 acquires the correspondence history INC_H1 with the earliest correspondence date among unverified among the INC_HISTORIES acquired up to S103 (S201).

Subsequently, the degree-of-association calculation unit 320 refers to INC_HISTORIES to determine whether or not the correspondence history INC_H0 immediately before INC_H1 exists (S202), and if it exists, the previous correspondence date T0 is used as INC_H0. The correspondence date of is set (S203). If it does not exist, the occurrence date of INC is set as the previous correspondence date T0 (S204).
Then, the previous correspondence date T0, the correspondence date T1 of this time (INC_H1), and the operation date TL of LOG are input to the function f_r1 (T0, T1, TL) indicating the closeness of the time within the correspondence time range. The association degree R1 is calculated based on the time related to the incident INC, the incident response history INC_H1, and the operation log LOG (S205).
Here, the function f_r1 (T0, T1, TL) indicating the closeness of time is as follows (wherein, TA is the date and time when the activation period A was traced back from T1).

  Subsequently, the value of the degree of association R1 regarding INC, INC_H1 and LOG calculated in S205 and the reason thereof are stored in the degree of association calculation result T1200 on the management DB 4 via the degree of association output unit 327 (S206). In "reason", f_r1 (T0, T1, TL) is not 1; that is, "occurring before {TA-TL} of correspondence ID {incident response ID of INC_H1}" when it is considered to be related Register the generated string in the format. Here, a portion enclosed by braces {} indicates a value acquired / calculated from various information.

  Thereafter, it is determined whether there is an unverified correspondence history in INC_HISTORIES, and if there is an unverified correspondence history, S201 and subsequent steps are repeated (S207).

Here, the calculation of the degree of association R1 based on time will be described using a specific example. In the case of LOG ID 1090 LOG and INC ID 100 INC and INC_H 1 incident response ID 5 as an example, T1 is "17:04:12 on April 12, 2015" and T0 is "2015 4 It will be 2:00:00 on the 11th of the month. TL is "8:05 on April 12, 2015" and the value of R1 is "77". As in this example, when T0 and T1 are vacant for one or more days, but the TL is relatively close to T1, the degree of association is calculated to be high.
In the calculation method of R1 described in the present embodiment, the incident display in the access log may be used instead of the date and time TA which traced back the activation period A from T1. This example is described in Example 3.

  Hereinafter, details of a method of calculating the degree of association R2 based on the system configuration will be described. In order to quantify the closeness of the connection relationship between the system configuration information included in the incident and the operation operation log, R2 of this embodiment, the target instance of the operation operation log is the instance described in the incident occurrence instance or the response history. As the number of connection hops is closer, the degree of association is calculated as a larger value.

FIG. 16 is an example of a detailed flowchart of the calculation processing S30 and S31 of the degree of association R2 based on the system configuration of the related operation operation log automatic extraction unit 32. In the present embodiment, the connection hop number boundary value HOP to be determined to be unrelated is defined in advance, and the connection hop between the instance included in a specific incident and incident response history and the target instance of the operation log Using a linear function model such that the value of relevance is “100” when the number HL is “0”, that is, the same instance, and the value of relevance is just “1” when HL is “HOP” Do. The value of the degree of association also becomes "1" when HL is "HOP" or more. In the present embodiment, the boundary value HOP of the connection hop number is set to "4". HOP can be set to any value of 0 or more.
The degree-of-association calculation unit 320 acquires the correspondence history INC_H1 with the earliest correspondence date among unverified among the INC_HISTORIES (S301). In the following, the degree of association R2 related to the system configuration is calculated for INC, INC_H1 and LOG.

The value of R2 to be calculated this time is initialized with the upper limit value UPPER_LIMIT (S302). In the present embodiment, UPPER_LIMIT is "100".
Next, the degree-of-association calculation unit 320 initializes the related instance list INS_LIST, and adds the generated instance T306 of INC to the INS_LIST (S303).
Furthermore, the instance name included in the text of the corresponding content T406 of INC_H1 is extracted, and the extracted instance name is added to INS_LIST (S304).

Here, as a specific method of extracting the instance name included in the text of the corresponding content T406, the correspondence may be performed using all or part of each instance name T602 of the instance information T600 related to the instances included in the service included in SYS_SET as a key. A method of searching and extracting the text of the content T406 can be mentioned. Alternatively, role information such as “AP server” may be extracted from the corresponding content T406, and an instance in which the role T603 of the instance information T600 matches may be acquired. The above-described method is used in this embodiment.
The degree of association R2 can be calculated without performing the process of S304. In that case, only the occurrence instance of INC is used in all the correspondence history, so the accuracy of the calculation of the degree of association R2 is reduced, but the value can be obtained with a smaller amount of calculation processing amount.
Subsequently, it is determined whether the target instance of LOG is included in INS_LIST (S305).

  If it is included, it stores the current value of relevance R2 and the reason in the relevance calculation result T1200 on the management DB 4 via the relevance output unit 327 as the values of INC, INC_H1 and relevance R2 regarding LOG (S306 ) And the process of S311. Here, as the reason, a character string generated in the form of “view information of instance name included in the process of S304” is registered. Here, a portion enclosed by braces {} indicates a value acquired / calculated from various information.

  If not included, the degree-of-association calculation unit 320 subtracts the subtraction value REDUCTION_BY_HOP based on the number of hops from the current value of the degree of association R2 (S307). In the present embodiment, the boundary value HOP of the number of connection hops is set to “25” based on “4” and “REDUCTION_BY_HOP”. The related operation log automatic extraction unit 32 determines whether the value of the current degree of association R2 is less than or equal to the lower limit LOWER_LIMIT (S308). In the present embodiment, LOWER_LIMIT is "1".

  If the value of R2 is larger than the lower limit value, INS_LIST and SYS_SET are compared to acquire all instances that have a dependency on each instance of INS_LIST, replace INS_LIST (S309), and process S305 and subsequent steps Do it again. In the present embodiment, the presence or absence of the dependency is determined by using the bidirectional of the dependency destination T702 of the system dependency T700 and the dependency source T703. That is, when the instance name in the INS_LIST matches any of the instances in the dependee T 702 and the instances in the depender T 703, it is considered as an instance having a dependency.

  If the value of R2 is equal to or less than the lower limit value, the value of the current association degree R2 is updated to the lower limit value LOWER_LIMIT (S310), and the calculation result of S306 is stored.

  After the degree of association R2 related to the system configuration is calculated for INC, INC_H1 and LOG by the above processing, if there is an unverified correspondence history in INC_HISTORIES, the processing after S301 is repeated (S311).

  Here, calculation of the degree of association R2 based on the system configuration will be described using a specific example.

Taking the case of LOG with log ID 1010, INC with incident ID 100, and INC_H 1 with incident response ID 1 as an example, the target instance becomes “dbserver1” from the operation operation detail T1005. Since it is not included in the initial INS_LIST “ap1server”, “ap1server1” and “ap1server2”, the value of the degree of association R2 is subtracted to be “75”. When INS_LIST and SYS_SET are compared and an instance having a dependency relationship with each instance of INS_LIST is acquired, “ap1server1”, “ap1server2”, “dbserver1”, and “vmm1” are extracted. If INS_LIST is updated in these instances and compared with the target instance of LOG, “dbserver1” is included, so the degree of association is “75”. The calculation method of R2 described in the present embodiment is an example.
For example, the system configuration information may be provided with numerical values indicating directionality or strength of dependence, and such information may be used to improve the accuracy of the calculation of the degree of association R2.

Hereinafter, details of a method of calculating the degree of association R3 based on the operation manager will be described. In R3 of this embodiment, the operation manager who performed the operation of the operation operation log and the person who responded to each incident response history, in order to quantify the closeness of the responsibility of the operation person in charge included in the incident and the operation operation log. The closer the affiliation is, the higher the degree of association is calculated.

  The functional model for quantifying the closeness of the responsible area of the operation manager in the calculation of the degree of association R3 based on the operation manager uses a linear function model based on the number of hops belonging to almost the same as the function model of R2. . In the present embodiment, as described in the operation manager information T800, the boundary value of the number of hops is “2” because an example in which management is performed by a simple two-tier management of the operation manager and the operation charge group to which it belongs is used. That is, R3 is "100" when the responder of the incident response history matches the operation manager of the operation operation log, "50" when the groups match, and "1" when both do not match.

  FIG. 17 is an example of a detailed flowchart of calculation processing S40 and S41 of the degree of association R3 based on the operation manager of the related operation operation log automatic extraction unit 32.

  The degree-of-relevance calculation unit 320 acquires the correspondence history INC_H1 with the earliest correspondence date among unverified among the INC_HISTORIES (S401).

  Next, the value of the degree of association R3 is initialized to the upper limit value UPPER_LIMIT (S402).

  Subsequently, in the management tool that has output the target LOG, it is determined whether to use a common account with another management administrator (S403). Since the integrated management server 3 uses the individual account for each operation manager, the determination result of S403 is "NO". Therefore, in the first embodiment, the determination result is always "YES". On the other hand, in various management tools other than the integrated monitoring server 3, since the common account set for each operation charge group is used, the determination result in S403 is "YES". An example using various management tools other than the integrated monitoring server 3 will be described in the second embodiment.

  If the determination result in S403 is "NO", it is determined whether the operation manager T1004 who performed the LOG operation matches the correspondent T404 of INC_H1 (S404).

  If the determination result in S404 is "YES", the value of the current relevance R3 (that is, UPPER_LIMIT = 100) and the reason thereof are calculated on the management DB 4 as a calculation result of the relevance R3 regarding INC, INC_H1, LOG. The association degree calculation result T1200 is stored (S405). Here, as a reason, the character string "match with correspondent" is registered.

If the determination result in S404 is "NO", then the responder in INC_H1 is compared with the USERS to acquire the operation manager list USERS2 of the same operation responsible group as the responder (S406). Then, it is determined whether the operation manager T 1004 who has performed the LOG operation is included in the USERS 2 (S 407).
If the determination result in S407 is "YES", the current relevance degree R3 is subtracted by a subtraction value REDUCTION_BY_USER based on the operation hierarchy (S408). In the present embodiment, REDUCTION_BY_USER is set to “50”. Then, the current degree of relevance R3 (UPPER_LIMIT-REDUCTION_BY_USER = 100-50 = 50) and the reason thereof are calculated as the degree of relevance R3 on the management DB 4 as the result of calculating the degree of relevance R3 for INC, INC_H1, LOG. Store (S405). Here, as a reason, a character string "match with person and group matches" is registered.

  If the determination result in S407 is "NO", it is considered that the operation manager is not relevant, and the value of the degree of association R3 is set to the lower limit LOWER_LIMIT (S409). Then, the value of the current degree of association R3 (that is, LOWER_LIMIT = 1) is stored in the degree of association calculation result T1200 on the management DB 4 through the degree of association output unit 327 as a calculation result of the association degree R3 regarding INC, INC_H1 and LOG. (S405). Here, a character string "-" indicating that there is no relation as a reason is registered.

  If the determination result in S403 is "YES", the common account T 805 used by the correspondent T 404 in INC_H1 is obtained from the USERS, and it is determined whether it matches the common account on which the LOG operation is performed (S410). Then, if the determination result of S410 is "YES", the processing of S408 and S405 is performed, and if the determination result of S410 is "NO", the processing of S409 and S405 is performed.

  As described above, after calculating the degree of association R3 related to INC, INC_H1 and LOG, it is determined whether INC_HISTORIES has an unverified correspondence history, and if there is an unverified correspondence history, the process returns to S401 for processing. Are repeated (S411).

  Hereinafter, details of a method of calculating the total degree of association Rx will be described. In order to quantitatively show that Rx in this embodiment is related in all aspects of time, system configuration, and operation manager, calculation is made such that the value becomes larger when all of R1, R2, and R3 are large. Do. In the present embodiment, the following geometric average is used as a synthesis function F_rx (r1, r2, r3) to satisfy the above (where r1, r2, r3 are the values of R1, R2, R3, respectively).

  The above is an example of a combination function, and another function may be used if the degree of association of each viewpoint can be combined. For example, using weighting factors W1, W2, W3, the following weighted average may be used.

  By using the weighted average, the degree of association can be calculated by weighting according to the importance of each important viewpoint.

  FIG. 18 is an example of a detailed flowchart of the calculation processing S50 and S51 of the total degree of association Rx of the related operation operation log automatic extraction unit 32. FIG. 18 is an example of the case where it is implemented using the above-mentioned geometric mean as the above-mentioned synthetic function.

  The degree-of-association calculation unit 320 acquires the correspondence history INC_H1 with the earliest correspondence date among unverified among the INC_HISTORIES (S501).

となる。
続いて、S503で計算したRxの値が関連判定しきい値RT以上かどうかを判定し（S504）、判定結果が「YES」の場合には関連有無として「関連あり」を（S505）、「NO」の場合には「関連なし」を設定する（S506）。先に述べた具体例の場合には、

となるため「関連あり」が設定される。
その後、計算した関連度Rxの値および関連有無の情報をINC, INC_H1, LOGの関連度Rxの計算結果として、関連度出力部327を介して、管理DB 4上の関連度計算結果T1200に格納する（S508）。 Next, each degree of association R1, R2, R3 corresponding to INC, INC_H1, LOG is acquired from the degree of association calculation result T1200 on the management DB 4 through the data management unit 31 (S502).
Further, the respective degrees of association R1, R2, R3 are input to the synthesis function f_rx (r1, r2, r3) to calculate the total degree of association Rx (S503).
Here, to describe a specific example of S501 to S503, in the case of LOG of log ID 1050, INC of incident ID 100, and INC_H1 of incident handling ID 1, columns T1205 and T1207 of row T1259 of relevancy degree calculation result T1200, T1209 is acquired as the values of the degrees of association R1, R2, and R3, respectively. Applying the obtained value to the composite function

It becomes.
Subsequently, it is determined whether the value of Rx calculated in S503 is equal to or more than the relation determination threshold value RT (S504), and if the determination result is "YES", "relevant" is set as the relation existence (S505), In the case of "NO", "not related" is set (S506). In the case of the example mentioned above,

“Relevant” is set because
Thereafter, the calculated value of the degree of association Rx and the information on the presence / absence of association are stored in the degree of association calculation result T1200 on the management DB 4 via the degree of association output unit 327 as a calculation result of the degree of association Rx of INC, INC_H1, LOG. (S508).

  As described above, after calculating the degree of association Rx of INC, INC_H1, LOG, the degree of association calculation unit 320 determines whether there is an unverified correspondence history in INC_HISTORIES, and there is an unverified correspondence history. In step S508, the processing after S501 is repeated (S508).

  Also, after calculation of Rx for all correspondence history in INC_HISTORIES is completed, referring to the related calculation result corresponding to all correspondence history in INC, LOG, INC_HISTORIES on the management DB 4, total relevance degree Rx Other than the one with the largest value is deleted from the management DB 4 via the degree-of-association output unit 327 (S509).

  To explain a specific example of S 509, the case of LOG with log ID 1003 and INC with incident ID 100 will be taken as an example (although the relevance calculation result T1200 shown in FIG. 10 is not described because it is the final result) The result of the degree of association Rx corresponding to the correspondence IDs 1 to 5 is “100” “58” “50” “50” “67”. In this case, only the result corresponding to the incident response ID 1 is left as one case with the highest degree of relevance Rx (row T1253), and the other results are deleted from the management DB 4.

  As described above, according to the method of calculating the total degree of association Rx described in the present embodiment, the degree of association is increased only when the association is strong simultaneously in each of the time, the system configuration, and the person in charge of operation.

  The relationship degree calculation method shown above is an example, and the present invention is not limited to the above calculation method. If it is possible to calculate the degree of association between the incident and the operation operation log from the association in a plurality of viewpoints using the incident and the incident response history and the operation operation log as an input, another calculation method may be used.

FIG. 19 is an example of a flowchart of generation of an operation log display screen related to an incident by the data display unit 312 in the present embodiment.
When the operation manager selects an incident to be displayed, the incident INC corresponding to the key and the list INC_HISTORIES corresponding to the response history of the key from the incident T300 and the incident response history T400 on the management DB 4 with the ID of the selected incident as the key Acquire (S601).

  Next, using the INC and INC_HISTORIES as keys, a set RESULTS of association degree calculation results regarding INC is acquired from the association degree calculation result T1200 on the management DB 4 (S602). As a specific example, in the case of an incident with an incident ID 100 (row 350), rows T1250 to T1271 are acquired as RESULTS.

  Subsequently, using the log ID T1201 included in each relevance calculation result in the RESULTS as a key, the corresponding log list LOGS is acquired from the operation operation log T900 on the management DB 4 (S603). As a specific example, in the case of the row T1253 which is one correspondence result of the incident ID 100 as the operation operation log T900, the log ID is 1003 and the row T1053 in the access log T1000 of FIG. 9 is acquired. Rows T1050 to T1071 of the access log T1000 are acquired as the operation operation log T900 corresponding to all the results of the incident ID 100.

Next, a temporary display table RESULT_TABLE is generated by merging the RESULTS and LOGS acquired above (S604), and a display screen is generated and output using RESULT_TABLE (S605). A specific example of the generated display screen is as shown in FIG.
The generation process of the operation operation log display screen related to the incident may be performed after the association degree calculation process of FIG.

  In the incident response history T400 in the present embodiment, information other than the columns T402 to T405 is assumed to be described as text in the corresponding content T406. However, the information used to make the association may be explicitly separated into separate lines. For example, the instance name targeted for the response work or the operation data type name to be surveyed may be a column. As a result, the time and effort of entry by the operation manager and restrictions on entry content increase, but the accuracy of association can be improved as compared to the case where these pieces of information are extracted from the text in the corresponding content.

  Further, in the incident response history T400, the work performed for one incident is divided into a plurality of lines and managed. On the other hand, even if the work history is integrated into one line, the present invention can be applied as long as information grouped in time and work content in the line can be parsed.

  In the second embodiment, an example in which an operation log (command execution log or operation event log) via another management tool is also used as an operation log to be associated with an incident in addition to the access log of operation data Show.

  FIG. 20 schematically shows a computer system assumed in the second embodiment. The configuration of the computer system is substantially the same as that of the first embodiment. The difference is that servers and terminals 7 to 9 for various management tools are added, and information on various management tools is added on the system configuration information T500 on the management DB 4 and the operation operation log T900. Hereinafter, only different parts will be described, and description of the same parts as the first embodiment will be omitted.

  As a server and terminal equipped with various management tools, a remote operation terminal 7 equipped with a remote operation tool 71, a job management server 8 equipped with a job management tool 81, and a VMM management server 9 equipped with a VMM management tool 91 are added. ing. The operation manager uses the functions of these management tools via the operation terminal 5 to perform operation operations on each instance on the managed system 6. The remote connection terminal 7 equipped with the remote operation tool 71 provides a function to remotely access and operate each instance on the managed system 6. In the present embodiment, as the remote operation tool 71, for example, an SSH (Secure SHell) client tool or the like can be mentioned. The job management server 8 equipped with the job management tool 81 provides a function of executing job execution and managing the execution state of each instance on the managed system 6. Here, a job is a unit in which a plurality of operations or processes on an instance are grouped for each work. A job is defined by a batch, a script, and a combination thereof, and the computer can automatically execute the task by executing it. As a job execution method in the job management tool 8, schedule execution, trigger execution when a preset condition is satisfied, immediate execution by a manual operation administrator, etc. can be considered. In the case of incident response, it is often used for countermeasure work, so immediate execution by manual execution of the operation manager is the main operation. The VMM management server 9 equipped with the VMM management tool 91 provides a function of operating and managing the VMM 61 and the VMM 62. By using the VMM management tool 91, operations such as start / stop of the VMM 61 and VM 62, addition / deletion of the VM 61, change of allocation resource, and the like can be performed.

Next, the system configuration information T500 on the management DB 4 will be described. In the system configuration information T500 of the second embodiment, an operation log setting T1300 is added.
FIG. 21 shows an example of the operation log setting T1300. The operation operation log setting T1300 is information describing the correspondence between the management server / terminal and its operation operation log type name. The role of the management server / terminal in the column T1301 is information on the role of the management tool, and corresponds to the role T603 of the instance information T600. The operation operation log type name in the column T1302 is information for uniquely identifying the log type name output by the management server / terminal, and the operation operation log T900 described later output by the management server / terminal shown in this embodiment is described later. Corresponding information is given.

  Information on the management instance included in this operation log setting T1300, instance information T600, and system dependency T700 links the management server / terminal (of the instance) with the instance to be managed and the type name of the operation log. Use for

  Next, the operation log T 900 on the management DB 4 will be described. In the second embodiment, a command operation log T1400, a job execution log T1500, and a VMM event operation log T1600 are added as operation operation logs for recording operation operation history using each management tool. These operation operation logs are output by each management tool and transferred and registered on the management DB 4. In the second embodiment, as a specific method thereof, the operation operation log T500, which is temporarily output on the management server / terminal instance by each management tool, is transmitted through the monitoring agent 66 and the data collection unit 310 as the operation data T100. Transfer / register on the management DB 4 (the description of the monitoring agent on each server / terminal is omitted in the figure).

  Thereafter, in the added operation operation log T900, as with the access log T1000, a log ID is held as information for uniquely identifying the log. In this embodiment, a unique ID is assigned to these log IDs throughout the operation operation log T900. For example, the log ID of the access log T1000 and the log ID of the command operation log do not overlap. As a result, even if the type of the operation operation log T 900 is increased, the process flowchart and the calculation result using the log ID of the first embodiment can be applied as it is.

FIG. 22 shows an example of the command operation log T1400. The command operation log T1400 stores a history of command operations that the operation administrator remotely accessed and executed an instance to be managed through the remote operation tool 71. The command operation log T1400 is output by the remote operation tool 71 each time the operation manager inputs a command via the remote operation tool 71.
The log ID in the column T1401 is information for uniquely identifying the command operation log registered in the management DB 4. In the present embodiment, log IDs are automatically assigned on the management DB 4. Further, it is assumed that other operation operation logs (for example, access logs) and log IDs do not overlap. The instance name of the management server / terminal in column T1402 is the instance name of the remote control terminal 7 that has output this log. The operation log type name in the column T1403 is the log type name of the command operation log. This corresponds to the column T1302 of the operation log setting T1300. The operation date and time of the column T1404 indicates the time when this log was output. A column T1405 shows a system name to be operated. The operation operation account in the column T1406 indicates the name of the account that performed this operation. The management tool added in the second embodiment uses a common account prepared for each operation charge group (the information corresponding to the column T 805 of the operation management information T 800 in FIG. 6). The session ID in the column T1407 indicates information for identifying session information for associating successive accesses within a fixed short time. When the remote operation tool 71 is an SSH client, this column contains information identifying a session from login by SSH to logout. The operation operation details in the column T1006 include detailed information such as the operation target instance and command contents in this operation. Since the contents of the operation operation details differ depending on the operation, a plurality of key / value pairs are stored in the form of "key = value". The contents of the operation details include, for example, “target instance”, “command”, and “command argument”. “Target instance” indicates an instance name to be operated. "Command" indicates a command that has been executed. Also, "command argument" indicates an argument given to the command. In this embodiment, "command" and "command argument" are expressed by replacing them with the description in Japanese, but in practice, for example, a command or script implemented as a function inside the OS is performed on the command line It is When storing in the management DB 4, the original data on the command line may be described, or the data replaced with the description may be described. Moreover, although only the operation content of the command is described in the details of the current operation, the output of the command execution result (e.g., standard output to the console) may also be described. In the command operation log described in the present embodiment, the target instance is always included in the operation operation details.
Here, a specific example of the command operation log will be described. In line T1453, at 01:00 on April 11, 2015, the operation administrator who belongs to the operation management group group1 uses a common account, It shows the history of editing the AP server configuration file by accessing the instance ap1server1 from the instance terminal1 of the remote control terminal.

  FIG. 23 shows an example of the job execution log T1500. The job execution log T1500 stores, through the job management tool 81, a history of execution of a job for an instance to be managed. The job execution log T1500 is output by the job management tool 81 each time a job is executed via the job management tool 81. The information included in the job execution log T1500 has no session ID, and the example of the operation details is changed to “job” or “job argument” instead of “command” or “command argument”. The same as the command operation log T1400, the detailed description is omitted.

  FIG. 24 shows an example of the VMM operation event log T1600. The VMM operation event log T1600 stores, through the VMM management tool 91, a history of operations performed on the VMM and VM to be managed. The VMM operation event log T1600 is output by the VMM management tool 91 via the VMM management tool 91 each time an operation is performed. Information included in VMM operation event log T1600 is command operation except that there is no session ID and the example of the contents of operation operation details is changed to “event” instead of “command” and “command argument”. The detailed description is omitted because it is similar to the log T1400.

  Here, the operation log of FIG. 22-23 has the instance name of the management server / terminal and the operation log type name, but does not have the access log T1000 of FIG. This is because it was unnecessary in the first embodiment. However, in the second embodiment, in order to enable handling in the same manner as other operation operation logs, information of the instance name of the management server / terminal and operation operation log type name for the access log T1000 of FIG. You may add it. In the case of the example of this embodiment, the instance name of the management server / terminal of all the access logs T1000 is “moni1”, and the operation log type name is “operation data access log”.

  In this example, the operation operation log T 900 is stored in a separate table for each management tool at this time, but the log type for each management tool may be identified and stored in one table if searchable .

  In the following, automatic extraction processing of the operation operation log related to the incident in the second embodiment of the present invention will be described.

  The association degree calculation processing by the operation operation log automatic extraction unit 32 according to the second embodiment uses not only the access log T1000 but also the command operation log T1400, the job execution log T1500, and the VMM event operation log T1600 as the operation operation log T900. Although different, the other process is substantially the same as the process contents in the first embodiment of the present invention shown in FIG. 13 to FIG. Hereinafter, only the other detailed differences will be described.

  In S102 of the flowchart of the degree of association calculation process shown in FIG. 13, as system configuration information T500, instance information T600 whose classification is not only “service” but also “management” is acquired as well as its system dependency T700. . Also, the operation operation log setting T1300 is acquired.

Further, as the operation operation log T900 acquired in S104, not only the access log T1000 but also other logs (specifically, the command operation log T1400, the job execution log T1500, and the VMM event operation log T1600) are acquired. By using the system configuration information T500 including the management system acquired in S102, (the instance of the management server / terminal) and the type name of the management target instance and the operation operation log can be linked, by using these as keys , And can narrow down and acquire operation operation logs related to target incidents and systems.
Furthermore, in the flowchart of the calculation processing of the degree of association R3 based on the operation management information shown in FIG. 17, in the case of the command operation log T1400, the job execution log T1500, and the VMM event operation log T1600, the account common to the operation management group is Since it is used, the determination result in S403 is "YES", and the subsequent processing is performed.
The processes other than those described above are the same as in the first embodiment of the present invention.

  FIG. 25 shows an example of the association degree calculation result T1200 storing the calculation result corresponding to the incident of the incident ID 1 shown in FIGS. 6 and 7 and the operation operation log shown in FIGS. The stored information is the same as the example of the degree of association calculation result T1200 shown in FIG.

As described in the first and second embodiments, as the operation operation log automatic extraction method related to an incident, the operation operation log automatic extraction unit 32 according to the present invention includes: incident information T200 (incident T300 and incident response history T400); Based on T900 as an input, a comprehensive degree of association between the incident and the operation operation log is calculated from the associations in a plurality of viewpoints, and the operation operation log related to the incident is extracted based on the degree of association. In the calculation of the degree of association, the system configuration information T500 concerning the management target system and the information of operation manager information T800 are also used as input, and each correspondence history and operation operation of the incident from three viewpoints of time, system configuration and operation manager The degree of association of logs is calculated, and the degree of association is synthesized to calculate the total degree of association.
In the present embodiment, it is possible to extract the operation operation log related to the incident without explicitly prescribing / specifying the operation or the operation related to each incident.
In addition, in order to extract the relationship between each correspondence history of incidents and each operation operation log based on multiple viewpoints such as time, system configuration, operation manager, etc. Accuracy is improved compared to the case where all the operation operation logs are extracted.
Furthermore, when there is a divergence between the described content of the incident response history and the actual operation content, the content not described in the response history can also be found from the associated operation operation log. For example, as shown in the row 1266 of the degree-of-relevance calculation result T1200 of FIG. 10, the operation operation log corresponding to the “operation data investigation” not described in the row T453 (correspondence ID 4) of the incident response history T400 of FIG. , Extracted as relevant.
Since the operation manager can acquire the related operation log only by specifying the incident, the time required for collecting and searching the operation operation log related to the incident can be shortened.

  Since the irrelevant information included in the acquired operation operation log is reduced, the accuracy of know-how analysis processing utilizing the operation operation log can be improved.

  In the third embodiment, an example of utilizing the access log of the incident display included in the access log of operation data in the calculation of the degree of association R1 based on time will be described as a variation of the degree of association calculation method in the first embodiment.

  It is the same as the computer system assumed in the third embodiment. The calculation processing method is also the same as that of the first embodiment except for part of the calculation processing of the degree of association R1 based on time. In the following, only different parts and parts necessary to show the effects will be described, and the description of the same parts as the first embodiment will be omitted.

  In the method of calculating the degree of association R1 based on the time of the third embodiment, when there is an access log for incident display instead of determining the activation period A described in the first embodiment in advance, the activation period is used. Determine A. This is based on the idea that, after referring to an incident, there is a high possibility of performing a response operation related to the incident.

As a specific process, the related operation operation log automatic extraction unit 32 sequentially refers to the access log in the period of T0 and T1 from the LOGS through the data management unit 31 as the preprocess of S205, and the INC incident Search one access log for which ID display has been performed. In the present embodiment, an access log in which the “display screen” in the operation operation detail T1006 is “incident display” and the “selected incident ID” matches the incident ID of the INC corresponds. If a corresponding access log is found, the period T1 is set as the activation period A from the operation date and time of the access log, and the calculation processing up to S206 is executed. If not found, the original value of A is used. The other processes are the same as in the first embodiment.
Hereinafter, description will be made using a specific example. The case where the calculation method of the first embodiment is used and the calculation method of the third embodiment will be described in comparison with each other.
In addition to the access log to the operation data shown in FIG. 9, it is assumed that there is an access log T1000 to the operation data shown in FIG. 26 as the access log to the operation data T1000. In the access log (rows T1097 and T1098) shown in FIG. 25, when the incident with incident ID 1 occurs, the operation administrator "user1" accidentally investigates the instance related to this incident for another purpose. Is an example of a log that assumes In this example, the operation data investigation regarding incident ID 1 is performed as a subsequent operation of this log. That is, the access log shown in FIG. 25 is not associated with the incident ID 1.
However, when calculated by the method described in the first embodiment, the access logs in lines T1097 and T1098 are access logs that occurred in the period from the occurrence of an incident to the registration of the first incident response history, and are determined in advance Since R1 has a large value of “90” and “79”, respectively, because it is included within the 30-minute activation period, it is determined as “relevant” from the total relevance ratio Rx as a result.

  Since these access logs are similar to the incident of ID 1 in each system configuration except time and each viewpoint of the operation manager, there is an unavoidable part that it is determined that there is a relation, but it is misjudged like this if possible Also want to reduce noise. The example described in Example 3 is one means for solving the above.

  If it calculates by the method described in Example 3, line T1050 of FIG. 9 will be extracted as an access log which performed the incident display of incident ID1. The operation date and time of this access log is "9:15:00 on April 10, 2015", which is 15 minutes before T1. That is, the activation period is temporarily changed from "30 minutes" to "15 minutes". Since the access logs of T1097 and T1098 are not included in the changed activation period, R1 becomes "33", "60" and a smaller value respectively than in Example 1, and as a result, it is determined as "not related" from Rx. Be done. An association log calculation result T1200 of the additional access log shown in the above example is as shown in FIG.

  Although the example applied to the first embodiment has been described above, the application to the second embodiment is also possible. In that case, the point that the access log of the incident display is searched / utilized referring to the access log T1000 is the same, and the log LOGS also targets other operation log T500 (example: job execution log T1500). It is different.

  As described above, in addition to the method of calculating the degree of association in the first embodiment, the related operation operation log automatic extraction unit 32 calculates the degree of association using the access log of the incident display included in the access log of operation data. Do. This enables time-based relation extraction with higher accuracy than in the first and second embodiments.

  In the fourth embodiment, as a variation of the degree of association calculation method, an example in which the degree of association is corrected by using operation pattern information input by an operation manager or automatically generated by a program is described.

  According to the fourth embodiment, it is possible to feed back to the calculation of the degree of association the input information whose presence / absence of association has been determined manually by the operation manager. Furthermore, the degree of association can be corrected based on the analysis result of the mechanical operation operation log.

  FIG. 28 schematically shows a computer system assumed in the fourth embodiment. The configuration of the computer system is substantially the same as that of the second embodiment. The difference is that pattern information of operation and operation, a processing portion for correcting the degree of association using the pattern information, and a processing portion for generating an operation pattern are added. Hereinafter, only different parts will be described, and the description of the same parts as the second embodiment will be omitted.

  The operation pattern information T1700 added to the management DB 4 is pattern information of an operation operation which is input by the operation manager or which is automatically generated by a program. In the present embodiment, the operation pattern information T1700 is input / output via the data management unit 31.

  FIG. 29 is a diagram showing an example of operation pattern information T1700 in the fourth embodiment.

  The pattern ID of the column T1701 is information for uniquely identifying this operation pattern, and is automatically numbered by the system. The system name in column T1702 is the system name targeted for this operation pattern. The operation classification of column T1703 is information identifying whether this operation pattern is an operation related to an incident, and in the case of a part of daily operation work not related to an incident, “daily operation” is an incident response operation. "Incident response" is set in some cases. The contents of the operation pattern of the column T1704 are described with one or more series of operation operation logs in order to express the operation operation pattern. In this example, the information enclosed in brackets indicates one operation log, and the arrows indicate the order. The operation operation log in the content of the operation pattern describes the complete / partial matching condition for each row of the operation operation log T 900 described above in the form of key = value. Some information is abstracted to describe partial match conditions. The degree of association correction coefficient of the column T1705 is a correction coefficient to the value of the degree of association of the operation operation log matched with the operation pattern.

  Here, to give a specific example, a row T1701 shows a pattern (pattern ID 1) of daily operation work of the system service1. The contents of the operation pattern show patterns in which the operation data of the operation data type OS of the instance dbserver 1 and the operation data of the CPU utilization rate of the operation data type OS of the instance cmdserver 1 are browsed in order in the operation data display. Here, the “before day” of the search time condition is information abstracted with respect to the specific date of the operation operation log T 900.

A row T1702 represents an operation pattern (pattern ID 10) corresponding to one incident of the system service1. The contents of the operation pattern indicate an operation pattern in which the AP server setting file of the AP server is edited by command operation via the remote operation terminal, and the number of simultaneously connected setting files is confirmed. Here, the “AP server” of the target instance name is information abstracted in the role T603 of the instance information T600 with respect to the specific instance name of the operation operation log T900.
The degree-of-association operation operation log extraction unit 32 according to the fourth embodiment performs calculation of degree of association using the degree-of-association correction unit 326 based on the operation pattern, which is a program for correcting the degree of association based on the newly added operation pattern information. Correct the total degree of association with.

  FIG. 30 shows an example of correcting the degree of association based on the operation pattern in the detailed flowchart of the calculation process of the total degree of association Rx in the fourth embodiment. This process is the same as the process of calculating the total degree of association Rx shown in FIG. 18 except for the degree of association correction portion (within the dotted line in the drawing) based on the operation pattern information. Here, only the difference will be described.

  After performing S503, the association degree correction unit 326 based on the operation pattern of the association degree operation operation log extraction unit 32 operates the operation pattern information T1700 related to the system of INC from the management DB 4 via the data management unit 31. Are all acquired (S801). Subsequently, pattern matching of the operation pattern information T1700 acquired in S801 is performed on the LOGS (S802). Matching is performed, for example, by comparing a pattern with an operation log on the same session. Then, among the matched patterns, the degree of association Rx is corrected using the degree of association correction coefficient of the pattern including LOG (S804). Here, if the degree of association exceeds the range of 1 to 100, it is corrected to the lower limit value / upper limit value. Thereafter, the degree of association determination of the total degree of association Rx shown in FIG. 18 and the result are stored in the management DB 4.

Describing the above in the case of the incident T200 of the incident ID 1 as a specific example, for example, operation pattern information T1700 of pattern ID 1 and pattern ID 10 is acquired. On the other hand, when pattern matching is performed on LOGS, the access logs T1097 and T1098 added in the third embodiment match the operation pattern of pattern ID 1. Therefore, the degree of association is corrected using the degree of association correction coefficient T1705 in the present pattern. Assuming that the degree of association thus far is calculated by the method described in Example 1 instead of Example 3, the total degree of association is “90 × correction factor 0.1 = 9”, “79 × correction factor ≒ 8”. It becomes. If the operations of the access logs T1097 and T1098 are daily operation tasks and their patterns are registered, they will not be misjudged as related even if the method of the third embodiment is not used. Further, T1453 and T1454 of the command operation log T1400 shown in FIG. 22 match the operation pattern of the pattern ID 10. When the degree of association is corrected using the degree of association correction coefficient T1705 in this pattern of this pattern, “77 × correction coefficient 1.2 = 92” is obtained. Since the degree of association of the operation operation log can be strengthened by registering the incident response pattern, it is possible to increase the accuracy of extracting the degree of association of the operation operation log in the portion close to the association determination threshold as in this example.

  As a method of generating the operation pattern information T1700, a method in which the operation manager manually registers all the information via the data management unit 31, or a method in which a part of the generated information is manually updated, the operation manager A method that records the result of excluding unnecessary logs that are not related from the related operation operation log display screen and automatically generates the program from the selected information, the program automatically generates it from the operation operation log and the calculated relevancy calculation result I can think of a method.

Here, as an example of a method of automatically generating the operation pattern information T1700, an operation pattern automatic generation unit 325, which is a program for automatically generating an operation pattern newly added to the association operation operation log extraction unit 32 of the fourth embodiment. Shows a process of automatically generating an operation operation pattern from the operation operation log and the calculated association degree calculation result. FIG. 31 is an example of the processing flowchart.
The operation pattern automatic generation unit 325 acquires all the operation operation logs T900 and the association degree calculation result T1200 from the management DB 4 via the data management unit 31 (S901). Then, using the degree of association calculation result T1200, the acquired operation operation log T900 is divided into a set R_LOGS of logs determined to be “related” and a set NR_LOGS of logs determined to be “not related” (S902).
Hereinafter, R_LOG determined to be related to an incident is an operation operation related to “incident response”, and NR_LOG determined to be not related is regarded as an operation operation related to “daily operation” and “incident response” Extract and generate operation patterns of "daily operation".

  First, processing of operation pattern extraction / generation corresponding to an incident is performed. The operation pattern automatic generation unit 325 divides the R_LOGS into groups of operation order based on the R_LOGS and the calculation result T1200 of the degree of association thereof (S903). For example, grouping can be considered for each combination of an incident ID, an incident response ID, session information, and a person in charge of an operation manager who has operated. Hereinafter, a group of operation orders is referred to as an operation sequence, and a set of extracted operation sequences is referred to as OPS_SET.

  Next, the operation pattern automatic generation unit 325 converts the operation sequences in the OPS_SET extracted in S 903 from specific information to abstract information (for example, absolute time relative time). The common operation is extracted by converting (to the previous day, the next day, etc.), converting the instance name to the role, etc. and comparing them (S904).

  Furthermore, the operation pattern automatic generation unit 325 generates the operation pattern information T1700 of the operation classification “incident response” using the content of the common operation extracted in S904 and the number of appearance thereof, and the data management unit 31 , Register in the management DB4. For example, when common operations are extracted from T1453 to T1454 and T1455 to 1456 of the command operation log T1400 shown in FIG. 22, the contents become the same as the contents of the operation pattern of the pattern ID 10 in FIG. Assuming that the appearance frequency is 2 and the 0.1 coefficient is corrected each time, the association degree correction coefficient is “1.2”. By such processing, the same operation pattern information T1700 as the pattern ID 10 can be generated.

  In this way, processing of operation pattern extraction and generation in response to an incident is performed.

  In the subsequent processes S906 to S908, daily operation pattern extraction / generation processing is performed using NR_LOGS. The process is substantially the same as the above-described S903 to S905. However, the operation sequence generation in S 907 differs in that the incident ID and the incident response ID are not used (cannot be used) for classification.

  The operation pattern automatic generation method described above is a simple example. Therefore, other general pattern matching methods may be used.

  According to the fourth embodiment, by correcting the degree of association based on the operation pattern of the operation manager based on the manual operation and the analysis of the mechanical operation operation log, the accuracy of the association extraction can be further improved.

In the embodiment of the present invention, as the best mode for carrying out the invention, the incident management in the system operation monitoring job has been described as an example. However, embodiments of the present invention are also applicable to incident response management in help desk operations such as call centers and customer support.
The embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Management network, 2 ... Physical communication line, 3 ... Integrated monitoring server, 100 ... I / F, 101 ... Processor, 102 ... Memory, 103 ... Data bus, 31 ... Data management part, 310 ... Data collection part, 311 ... data registration unit, 312 ... data display unit, 32 ... related operation operation log extraction unit, 320 ... association degree calculation unit, 321 ... association degree calculation unit based on time, 322 ... association degree calculation unit based on system configuration, 323 ... Relevance degree calculation unit based on operation manager, 324 ... Total association degree calculation unit, 325 ... Operation pattern automatic generation unit, 326 ... Relevancy degree correction unit based on operation pattern, 327 ... Relevance degree output unit, 4 ... Management DB 5 Operation terminal 51 Web browser 6 Managed system 61 VMM 62 Server 63 (VM / PM) 63 OS 64 64 middleware 65 application 66 monitoring agent 67 load 67 Balancer, 7 ... remote control terminal, 71 ... remote control tool, 8 ... Job management server 81 Job management tool 9 VMM management server 91 VMM management tool T100 Operation data T200 Incident information T300 Incident T400 Incident response history T500 System configuration information T600 Instance information, T700 ... system dependency, T800 ... operation manager information, T900 ... operation operation log, T1000 ... operation data access log, T1100 ... association degree calculation rule, T1200 ... association degree calculation result, T1300 ... operation operation log setting, T1400 ... command operation log, T1500 ... job execution log, T1600 ... VMM event operation log, T1700 ... operation pattern information

Claims (10)

An integrated operation monitoring system for monitoring the operation of a computer system to be managed comprising a device or a plurality of constituent components comprising functional components mounted on the device, comprising:
Manage incident information including the component that the incident occurred in the managed system and the incident occurrence time, the component that is the target of the operation, the operation administrator who instructed the operation, and the operation operation log information that includes the operation execution time Data management department,
A degree-of-relevance calculation unit for calculating the degree of association between the incident information and the operation log information;
The relevance degree output unit is configured to output an operation operation log that is highly likely to be associated with an incident using the relevance degree obtained by the relevance degree calculation unit .
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The degree-of-association output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The correspondence history information includes the correspondence content recorded by text,
The degree-of-association calculation unit analyzes a text included in the correspondence history information to extract a component included in a record of the correspondence history information.
The integrated operation monitoring system according to claim 1, wherein the second degree of association also includes the degree of association with a component included in a record of correspondence history information extracted by analyzing text . An integrated operation monitoring system for monitoring the operation of a computer system to be managed comprising a device or a plurality of constituent components comprising functional components mounted on the device, comprising:
Manage incident information including the component that the incident occurred in the managed system and the incident occurrence time, the component that is the target of the operation, the operation administrator who instructed the operation, and the operation operation log information that includes the operation execution time Data management department,
A degree-of-relevance calculation unit for determining the degree of association between the incident information and the operation operation log information;
The relevance degree output unit is configured to output an operation operation log that is highly likely to be associated with an incident using the relevance degree obtained by the relevance degree calculation unit .
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The degree-of-association output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The data management unit manages operation operation pattern information including pattern matching information including a condition for searching a plurality of sequences of operation operation logs, and correction value information for correcting a degree of association regarding operation operation log information matching the pattern. ,
The association degree correction unit corrects the association degree calculated by the association degree calculation unit based on the operation operation pattern information.
The integrated operation monitoring system, wherein the association degree correction unit obtains a value corrected based on the correction value information with respect to the association degree of the operation operation log information matching the operation operation pattern information . A pattern generation unit that generates operation operation pattern information;
The pattern generation unit generates first operation operation pattern information as the operation operation pattern information by using an operation operation log highly likely to be associated with the incident output by the association degree output unit as the input operation pattern information;
The integrated operation according to claim 2 , wherein the degree-of-association calculation unit calculates a value corrected such that the degree of association of the operation operation log information matching the first operation operation pattern information becomes larger. Monitoring system. The pattern generation unit receives an operation operation log that is highly likely to be related to the incident output by the association degree output unit and the operation operation log information, and an operation operation log that is unlikely to be related to any incident And second operation operation pattern information is generated as the operation operation pattern information from the extracted operation operation log,
4. The integrated operation according to claim 3 , wherein the degree-of-association calculation unit calculates a value corrected such that the degree of association of the operation operation log information matching the second operation operation pattern information becomes smaller. Monitoring system. An integrated operation monitoring system for monitoring the operation of a computer system to be managed comprising a device or a plurality of constituent components comprising functional components mounted on the device, comprising:
Manage incident information including the component that the incident occurred in the managed system and the incident occurrence time, the component that is the target of the operation, the operation administrator who instructed the operation, and the operation operation log information that includes the operation execution time Data management department,
A degree-of-relevance calculation unit for calculating the degree of association between the incident information and the operation log information;
The relevance degree output unit is configured to output an operation operation log that is highly likely to be associated with an incident using the relevance degree obtained by the relevance degree calculation unit .
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The degree-of-association output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The integrated operation monitoring system, wherein the association degree output unit determines whether the operation operation log is likely to be associated with an incident depending on whether the association degree exceeds a predetermined association determination threshold. . An integrated operation monitoring system for monitoring the operation of a computer system to be managed comprising a device or a plurality of constituent components comprising functional components mounted on the device, comprising:
Manage incident information including the component that the incident occurred in the managed system and the incident occurrence time, the component that is the target of the operation, the operation administrator who instructed the operation, and the operation operation log information that includes the operation execution time Data management department,
A degree-of-relevance calculation unit for calculating the degree of association between the incident information and the operation log information;
The relevance degree output unit is configured to output an operation operation log that is highly likely to be associated with an incident using the relevance degree obtained by the relevance degree calculation unit .
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The degree-of-association output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The integrated operation monitoring system, wherein the degree-of-association output unit outputs, when outputting the degree of association, a reason that the association between the incident and the operation operation log is determined to be high. A method for calculating the degree of association between an incident and an operation log of a managed computer system comprising a plurality of components consisting of a device or functional components mounted on the device, comprising:
The data management unit includes incident information including the component where the incident occurred in the managed computer system and incident information including the incident occurrence time, the component to be subjected to the operation operation, the operation manager who instructed the operation, and the operation including the operation execution time Get operation log information,
The degree of association calculating unit obtains the degree of association between the incident information and the operation log information,
The degree-of-relevance output unit outputs an operation operation log that is highly likely to be related to an incident using the degree of association obtained by the degree-of-relevance calculation unit,
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The association degree output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The correspondence history information includes the correspondence content recorded by text,
The degree-of-association calculation unit analyzes a text included in the correspondence history information to extract a component included in a record of the correspondence history information.
The second degree of association includes a degree of association with a component included in a record of correspondence history information extracted by analyzing a text . A method for calculating the degree of association between an incident and an operation log of a managed computer system comprising a plurality of components consisting of a device or functional components mounted on the device, comprising:
The data management unit includes incident information including the component where the incident occurred in the managed computer system and incident information including the incident occurrence time, the component to be subjected to the operation operation, the operation manager who instructed the operation, and the operation including the operation execution time Get operation log information,
The degree of association calculating unit obtains the degree of association between the incident information and the operation log information,
The degree-of-relevance output unit outputs an operation operation log that is highly likely to be related to an incident using the degree of association obtained by the degree-of-relevance calculation unit,
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The association degree output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The data management unit manages operation operation pattern information including pattern matching information including a condition for searching a plurality of sequences of operation operation logs, and correction value information for correcting a degree of association regarding operation operation log information matching the pattern. ,
The association degree correction unit corrects the association degree calculated by the association degree calculation unit based on the operation operation pattern information.
The association degree calculation method, wherein the association degree correction unit obtains a value corrected based on the correction value information on the association degree of the operation operation log information matching the operation operation pattern information as the association degree. A method for calculating the degree of association between an incident and an operation log of a managed computer system comprising a plurality of components consisting of a device or functional components mounted on the device, comprising:
The data management unit includes incident information including the component where the incident occurred in the managed computer system and incident information including the incident occurrence time, the component to be subjected to the operation operation, the operation manager who instructed the operation, and the operation including the operation execution time Get operation log information,
The degree of association calculating unit obtains the degree of association between the incident information and the operation log information,
The degree-of-relevance output unit outputs an operation operation log that is highly likely to be related to an incident using the degree of association obtained by the degree-of-relevance calculation unit,
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The association degree output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
The relevance degree calculation method, wherein the relevance degree output unit determines whether the operation log is likely to be associated with an incident depending on whether the relevance degree exceeds a predetermined relevance determination threshold. . A method for calculating the degree of association between an incident and an operation log of a managed computer system comprising a plurality of components consisting of a device or functional components mounted on the device, comprising:
The data management unit includes incident information including the component where the incident occurred in the managed computer system and incident information including the incident occurrence time, the component to be subjected to the operation operation, the operation manager who instructed the operation, and the operation including the operation execution time Get operation log information,
The degree of association calculating unit obtains the degree of association between the incident information and the operation log information,
The degree-of-relevance output unit outputs an operation operation log that is highly likely to be related to an incident using the degree of association obtained by the degree-of-relevance calculation unit,
The incident information includes a plurality of response history information corresponding to the incident,
The degree-of-association calculation unit obtains the degree of association between the incident and the operation operation log for each correspondence history information,
The association degree output unit associates operation operation log information having high relevance with each correspondence history information and outputs it.
The degree of association includes a first degree of association, a second degree of association, and a third degree of association,
The first degree of association is the degree of association between one of the time when an incident occurs and the time when the correspondence history information is recorded, and the time when an operation is performed.
The second degree of association is the degree of association between the component in which the incident occurred or the component included in the record of the correspondence history information, and the component that performed the operation,
The third degree of association is the degree of association between the operation manager who registered the incident response history and the operation manager who performed the operation,
The degree-of-association calculation unit determines the degree of association between the incident and the operation operation log based on at least two degrees of association of the first, second, and third degrees of association;
The data management unit includes system configuration information including components constituting the management target system and a dependency between components, a set of components constituting the management target system, and an operation person in charge of operation thereof. Manage operation manager information including the correspondence relationship of the set of
The second degree of association further includes the degree of association with the system configuration information.
The third degree of association further includes the degree of association with the operation manager information,
A method of calculating a degree of association, wherein the degree-of-association output unit outputs, when outputting the degree of association, a reason why the association between the incident and the operation operation log is determined to be high .

Images