JP5847497B2 - System management support apparatus and system management support method - Google Patents

Description

  The present invention relates to a data processing technique, and more particularly to a technique for supporting management of components of an information processing system.

  In recent years, services using a network, such as an online shopping service, have spread widely in society. Information processing systems that operate these services tend to be large-scale and complex systems having a plurality of blade servers, storage, network switches, and the like as constituent elements.

  For each component that makes up such an information processing system, the degree of influence on the service is calculated in consideration of the relevance between the components, and the component is calculated based on the degree of influence and the importance of the service. A technique for calculating the priority of the above has been proposed (see Patent Document 1).

JP 2008-217285 A

  When an incident occurs during the operation of a service and a component is changed as a countermeasure, the order of countermeasures can be determined according to the above-described priority. However, although the above-mentioned priority suggests an important component for maintaining a service in operation, it does not indicate a component that may cause a problem before the problem occurs. A large-scale information processing system may take time to deal with a problem, and it is required to prevent the problem from occurring before the problem occurs.

  The present invention has been made in view of these problems, and a main object of the present invention is to provide a technique for supporting identification of components that can become a bottleneck in an information processing system.

  In order to solve the above problems, a system management support apparatus according to an aspect of the present invention is configured by hardware or software of the information processing system involved in an incident that occurs in response to an incident that has occurred in the information processing system Refers to the system management database that stores the change history when an element is changed, and refers to the system management database, counts the number of times each component has been changed in response to an incident, and notifies the predetermined number of times. It includes a change record analysis unit that identifies a component that is equal to or greater than the reference count, and a notification unit that notifies the component specified by the change record analysis unit.

  In order to solve the above problems, another aspect of the present invention is a system management support method. This method counts the number of times that a component composed of hardware or software of the information processing system involved in the incident has been changed in response to an incident that occurred in the information processing system, and the number of times is a predetermined notification reference number The processor is caused to identify the constituent elements as described above and detect the bottleneck of the information processing system.

  It should be noted that any combination of the above-described components, and a representation of the present invention converted between a method, a system, a program, a recording medium storing the program, and the like are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the technique for assisting specification of the candidate of the component which can become a bottleneck in an information processing system can be provided.

It is a figure which shows typically the function structure of the information processing system which concerns on embodiment. It is a figure which shows typically the internal structure of the blade which performs a virtual machine. It is a figure which shows typically the data structure of the structure information master which manages the structure information of the information processing system which concerns on embodiment. It is a figure which shows typically the data structure of the change history master which manages the change history of the component of the information processing system which concerns on embodiment. It is a figure which shows typically the data structure of the relationship management master which manages the connection between the components of the information processing system which concerns on embodiment, and the relationship between a component and an incident. It is a figure which shows typically the data structure of the bottleneck candidate master which manages the candidate of the component which can become a bottleneck among the components of the information processing system which concerns on embodiment. It is a figure which shows an example of the screen which shows the candidate of the component which can become a bottleneck. It is a flowchart which shows the flow of a process in the system management assistance apparatus which concerns on embodiment.

  The information processing system 500 according to the embodiment is a computer system built in an IT service company (for example, an SI company). The system management support apparatus 100 according to the embodiment manages an incident that has occurred in the information processing system 500 and a change in each component of the information processing system 500 that is performed in response to the incident. Here, “components” of the information processing system 500 refer to hardware and software including firmware necessary for operating the information processing system. Hereinafter, the configuration of incident management will be mainly described. However, those skilled in the art will understand that the same configuration can be applied to problem management.

  FIG. 1 is a diagram schematically illustrating a functional configuration of an information processing system 500 according to an embodiment. The information processing system 500 includes a blade group 200, a network switch group 300, a storage group 400, and a system management support apparatus 100.

  FIG. 1 shows a functional configuration for realizing an information processing system 500 according to the embodiment, and other configurations are omitted. In FIG. 1, each element described as a functional block for performing various processes can be configured by a CPU, a main memory, and other LSI (Large Scale Integration) in terms of hardware, and in terms of software, This is realized by a program loaded in the main memory. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one.

  Each component of the blade group 200 is a blade server (hereinafter simply referred to as “blade”) that executes various server programs such as a database server and a web server. Therefore, the blade group 200 includes the first blade 210 and the second blade 220, and includes L blades up to the Lth blade 230. Here, L is the number of blade servers included in the blade group 200. The value of L may be determined by experiment in consideration of the scale of service provided by the information processing system 500, for example.

  FIG. 2 is a diagram schematically illustrating an internal configuration of a blade that executes a virtual machine. Each blade included in the blade group 200 according to the embodiment has a virtualization function, and executes various server programs such as the database server 252 and the web server 254 by executing the virtual machine 250. The virtual machine 250 includes a virtual first network interface 256 that connects the virtual machine 250 and a real blade, and is connected to the network switch group 300 via a second network interface 260 that is a real network interface of the blade. To do. As a connection method between the virtual machine 250 and the network switch group 300, for example, several methods such as a direct connection from the virtual machine 250 to the second network interface 260 can be considered. To do.

  Returning to the description of FIG. The storage group 400 stores various information necessary for the information processing system 500 to provide services, such as customer information, product information, and advertisements. Therefore, the storage group 400 includes M network switches from the first storage 410 and the second storage 420 to the M-th storage 430. Here, M is the number of storages included in the storage group 400. Similar to the value of L, the value of M may be determined by experiment in consideration of the scale of service provided by the information processing system 500, the degree of redundancy, and the like.

  The network switch group 300 is a network switch group that relays between each blade server included in the blade group 200 and each storage included in the storage group 400. The network switch group 300 includes the first switch 310 and the second switch 320, and includes N network switches up to the Nth switch 330. Here, N is the number of network switches included in the network switch group 300. As with the values of L and M, the value of N may be determined by experiment in consideration of the scale of service provided by the information processing system 500, the degree of redundancy, and the like.

  The system management support apparatus 100 manages an association between an incident that occurs in the information processing system 500 and a component that is involved in the incident. Here, the constituent elements in the information processing system 500 are each blade included in the blade group 200, a server program and network interface executed there, each network switch included in the network switch group 300, and each storage included in the storage group 400. Etc. The system management support apparatus 100 includes an incident management unit 10, a system management database 20, a change record analysis unit 30, and a notification unit 40.

  The system management database 20 is a variety of tables used to manage changes in each component of the information processing system 500. The configuration information master 22, the related management master 24, the change history master 26, and the bottleneck candidate master 28 including.

  FIG. 3 is a diagram schematically illustrating a data structure of the configuration information master 22 that manages configuration information of the information processing system 500 according to the embodiment. The configuration information master 22 associates the names (for example, the first web server, the third blade, etc.) of each component of the information processing system 500 with the configuration information ID (IDentification) uniquely determined for these components. And store. The configuration information master 22 also includes a configuration information ID and a “total number of changes” indicating how many times the configuration element corresponding to the configuration information ID has been changed since it was installed in the information processing system 500. Store in association. The configuration information master 22 further stores the configuration information ID and the “number of changes within the period” in association with each other. Details of the “number of changes within a period” and the “total number of changes” will be described later.

  FIG. 4 is a diagram schematically illustrating the data structure of the change history master 26 that manages the change history of the components of the information processing system 500 according to the embodiment. The change history master 26 stores the configuration information ID of the component and the modification date in association with each other when a certain incident occurs and the component is changed to deal with the incident. For example, in the example of the change history master 26 shown in FIG. 4, the history information ID 0001 indicates that the first switch 310 related to the configuration information ID 0006 involved in the incident that occurred has been changed on October 11, 2010. ing.

  FIG. 5 is a diagram schematically illustrating a data structure of the association management master 24 that uniformly manages the connection between the components of the information processing system 500 according to the embodiment and the relationship between the components and the incident. In the association management master 24, the line in which the items “association source type” and “relation destination type” are both “configuration management” indicates the connection between the components. For example, in the example of the relation management master 24 shown in FIG. 5, in the row with the relation management ID 0001, both “relation source type” and “relation destination type” are “configuration management”. This indicates that the configuration information ID 0001 (first web server) and the configuration information ID 0006 (first switch) in the example of the configuration information master 22 shown in FIG.

  In the related management master 24, the line in which one of the items “related source type” and “related destination type” is “configuration management” and the other is “incident management” indicates an incident and its Describe the components involved in the incident. For example, in the example of the related management master 24 shown in FIG. 5, the line with the related management ID 0002 has “related source type” “configuration management” and “related destination type” “incident management”. This indicates that the configuration information ID 0006 (first switch) in the example of the configuration information master 22 shown in FIG. 3 is involved in the incident related to the history information ID 0001 in the example of the change history master 26 shown in FIG.

  As described above, the relationship management master 24 can be used to uniformly manage the connection between the components of the information processing system 500 and the relationship between the component and the incident.

  Hereinafter, the system management support apparatus 100 according to the embodiment will be specifically described with reference to an example in which the information processing system 500 is a system that provides online shopping. Of course, it is understood by those skilled in the art that the information processing system 500 can be applied to a case where other services such as a network auction service and social network service (SNS) are provided.

  It is assumed that an incident occurs in which a user who uses nettioning provided by the information processing system 500 via a network such as the Internet cannot access the nettioning site. As a result of investigating the cause by a person in charge of managing or maintaining the information processing system 500 (hereinafter, simply referred to as “person in charge”), the link is down between the third blade and the third port of the first switch 310. There was found. Therefore, as a countermeasure, it is assumed that the network cable connecting the third blade and the first switch 310 is replaced and the first switch 310 is restarted. At this time, the incident management unit 10 stores the change history of the first switch 310 in the change history master 26. The incident management unit 10 further stores in the related management master 24 that the first switch 310 is involved in the incident and that the third blade is involved in the incident.

  Later, another incident that the response of the settlement process at the online shopping site was very slow occurred, and as a result of the investigation, the first port connecting the first storage 410 and the first switch 310 has high load access. Suppose that it turns out. Accordingly, it is assumed that the response speed of the settlement process is restored when the connection of the first port in the first switch 310 is changed to the connection of the fourth port. At this time, the incident management unit 10 stores the change history of the first switch 310 in the change history master 26. The incident management unit 10 further stores in the related management master 24 that the first switch 310 is involved in the incident and that the first storage 410 is involved in the incident.

  The change record analysis unit 30 refers to the related management master 24 in the system management database 20 and counts the number of times each component constituting the information processing system 500 has been changed in response to an incident. In the case of the above example, the first switch 310 is changed twice, and the third blade and the first storage 410 are changed once.

  The above two examples of incidents are a problem of connection to a shopping site and a problem of response of settlement processing, and there is no direct connection at first glance. However, the first switch 310 has been changed to deal with these incidents. As described above, a component that has been changed for dealing with different incidents may have a fundamental problem in the component, and may become a bottleneck in the operation of the information processing system 500. There is. Hereinafter, in this specification, a component that may become a bottleneck in operating the information processing system 500 may be simply referred to as a “bottleneck candidate”.

  The change record analysis unit 30 identifies a component whose number of changes is equal to or greater than a predetermined notification reference number. Here, the “notification reference count” is used to determine whether or not to notify the person in charge as a candidate that may become a bottleneck in operating the information processing system 500 among the constituent elements of the information processing system 500. The standard number of times set in The notification reference frequency may be determined by experiments in consideration of the usage frequency of the service to be operated, the robustness of the constituent elements, and the like, but it is two as an example of the notification reference frequency. Although not shown, the notification reference count is stored in the system management database 20.

  In this way, the change record analysis unit 30 analyzes the incidents that have occurred in the past, the components involved in the incidents, and the number of times they have been changed for coping, so that the bottleneck in operating the information processing system 500 It becomes possible to reveal components that have fundamental problems that can be.

  Here, the change record analysis unit 30 determines only the changes that have been within a predetermined change counting period from the time of performing the specification when specifying a component whose number of configuration changes is equal to or greater than a predetermined notification reference number. May be counted, and changes that have been past in the change counting period may not be counted. This prevents a fundamental problem from being solved at some point even if there were components that required many changes in the past, and prevents components that would not cause problems from being reported as bottleneck candidates. it can. Therefore, the “change counting period” is a reference period for determining whether or not a change should be added as the number of changes when specifying a component as a bottleneck candidate. The change counting period may be determined by experiments in consideration of the service contents provided by the information processing system 500, the robustness of the components, and the like, but as an example, it is three months.

  The change record analysis unit 30 stores the number of times of change within the change count period in the “number of changes within period” in the configuration information master 22 described above. The change record analysis unit 30 also stores the number of changes that have occurred between the time when the component is installed in the information processing system 500 and the present time in the “total number of changes” in the configuration information master 22.

  The incident management unit 10 stores the bottleneck candidates identified by the change record analysis unit 30 in the bottleneck candidate master 28. FIG. 6 is a diagram schematically illustrating the data structure of the bottleneck candidate master 28 that manages bottleneck candidates. The bottleneck candidate master 28 responds when the configuration information ID corresponding to the component that becomes the bottleneck candidate, the status indicating whether or not the bottleneck candidate has been investigated, etc., and the correspondence has been completed. A bottleneck candidate ID that uniquely represents a bottleneck candidate is associated with the date and stored.

  The notification unit 40 notifies the component specified by the change result analysis unit 30 as a bottleneck candidate of the information processing system 500. As a notification method, for example, an e-mail is transmitted to a person in charge of the information processing system 500. Alternatively, a dedicated site showing bottleneck candidates may be prepared and presented as a screen by the person in charge accessing it. The date when the notification unit 40 first notifies the bottleneck candidate is stored in the bottleneck candidate master 28 as “notification date” in association with the component.

  FIG. 7 is a diagram illustrating an example of a screen showing candidate component elements that can become bottlenecks. In the example of the screen shown in FIG. 7, the information processing system names including the bottleneck candidates and the names thereof are displayed as a top page together with IDs uniquely indicating them. A link is established in the name, and a dedicated screen (not shown) for displaying the contents of the bottleneck candidate master 28 corresponding to the information processing system 500 is displayed when a viewer of the screen clicks the link. The

  Assume that the person in charge of the information processing system 500 recognizes that the first switch 310 is a bottleneck candidate by confirming the notification of the bottleneck candidate from the notification unit 40. Assume that the person in charge of the information processing system 500 examines the first switch 310 in detail and finds a problem in the firmware of the first switch 310. Therefore, when a problem is solved in the firmware by applying a patch to the firmware of the first switch 310, the person in charge inputs the fact to the incident management unit 10. The incident management unit 10 stores that the firmware of the first switch 310 has been changed in the related management master 24 and the change history master 26, and updates the status and response date of the bottleneck candidate master 28. This prevents another person in charge who has seen the screen showing the bottleneck candidates from investigating the bottleneck candidates already in charge.

  FIG. 8 is a flowchart showing a processing flow in the system management support apparatus 100 according to the embodiment. The processing in this flowchart starts when the system management support apparatus 100 is turned on, for example. The flowchart shown below represents a processing flow during operation of the information processing system 500. For example, it is assumed that the input of the configuration information of the information processing system 500 to the configuration information master 22 and the related management master 24 has been completed. And

  When an incident occurs during the operation of the information processing system 500 (S10), the incident management unit 10 stores the incident and the components involved in the incident in association management master 24 in association with each other (S12).

  When the person in charge of the information processing system 500 changes the component involved in the incident in order to deal with the incident (Y in S14), the person in charge inputs the change information to the incident management unit 10, and the incident management unit 10 updates the change history master 26 (S16). When the person in charge of the information processing system 500 does not change the component involved in the incident in order to deal with the incident (N in S14), the incident management unit 10 does not update the change history master 26.

  The change record analysis unit 30 refers to the related management master 24 for a certain component constituting the information processing system 500 and identifies a case involved in the incident. When the component is involved in the incident, the change record analysis unit 30 refers to the change history master 26 and specifies whether or not the component has been changed in response to the incident. The change record analysis unit 30 repeats the above-described processing for all the constituent elements constituting the information processing system 500, thereby counting the number of changes P of the constituent elements (S18).

  Subsequently, the change record analysis unit 30 acquires the notification reference count T from the system management database 20 (S20). The change record analysis unit 30 compares the counted number of changes P with the acquired notification reference number T to obtain a magnitude relationship (S22).

  When the counted number of changes P is equal to or greater than the acquired notification reference number T (Y in S24), the change record analysis unit 30 updates the bottleneck candidate master 28 (S26). The notification unit 40 notifies the component whose change count P is equal to or greater than the notification reference count T as a bottleneck candidate (S28).

  When the counted change count P is less than the acquired notification reference count T (N in S24), or after the notification section 40 notifies the bottleneck candidate, the change performance analysis section 30 stores the “within period” in the configuration information master 22 The “number of changes” and the “total number of changes” are updated (S30).

  By repeating the above processing, the system management support apparatus 100 according to the embodiment manages changes in the constituent elements of the information processing system 500 and identifies the bottleneck candidates.

  The operation according to the above configuration is as follows. When the information processing system 500 is constructed, the person in charge inputs the configuration information of the information processing system 500 to the configuration information master 22 and the related management master 24. The incident management unit 10 manages the incident that occurred during the operation of the information processing system 500 and the change of the constituent elements accompanying the handling of the incident using the related management master 24 and the change history master 26. Thereby, the change performance analysis part 30 specifies the component which may become a bottleneck of the information processing system 500 by counting the frequency | count of a change. The person in charge of the information processing system 500 can take countermeasures in advance before a problem occurs by analyzing the bottleneck candidates.

  As described above, according to the system management support apparatus 100 of the embodiment, it is possible to support identification of components that can become a bottleneck in the information processing system 500. Thereby, the bottleneck candidate can be investigated before a problem occurs during the operation of the information processing system 500, and if there is a problem, it can be dealt with in advance.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  A first modification will be described. As described above, the bottleneck candidate master 28 in the system management database 20 stores the date when the person in charge is first notified of the bottleneck candidate as the notification date. The notification unit 40 may not notify the component as a bottleneck candidate when a predetermined period determined as the bottleneck candidate period has elapsed from the date of first notification as a bottleneck candidate. Here, the “bottleneck candidate period” is a period defined as a period in which the component specified by the change record analysis unit 30 is regarded as a bottleneck candidate, for example, three months.

  Since the component identified by the change record analysis unit 30 is a bottleneck candidate, if the component does not interfere with the operation of the information processing system 500 for a predetermined period, the component may be a bottleneck. It is considered low. Therefore, by introducing a bottleneck candidate period and not informing the person in charge of the bottleneck candidate that has passed that period, the person in charge does not need to analyze the components that are unlikely to be a bottleneck. Can do.

  A second modification will be described. In the above description, a case has been described in which a common notification reference count is set for all components constituting the information processing system 500, but a different value may be set for each component as the notification reference count. In this case, the change record analysis unit 30 acquires the notification reference number determined for each component from the system management database 20, and the counted number of component changes is equal to or greater than the notification reference number determined for the component. Compare whether or not there is.

  For example, since the operating system that is executed locally on the blade and the virtual machine 250 that is executed on the operating system are software, the number of changes compared to the second network interface 260 that is hardware built in the blade. Tend to increase. In the case of hardware, it is changed when it is physically deleted or replaced, whereas in the case of software, not only when the software itself is replaced, but also correction by online update is a change record. .

  Therefore, for each component constituting the information processing system 500, including whether the component is hardware or software, considering the nature of the component itself, the robustness of the component, the frequency of use, etc. The notification standard number suitable for the component is determined by experiment. Compared with the case where a common notification reference count is set for all the components, the identification accuracy of the bottleneck candidate by the change record analysis unit 30 can be further increased.

  It should also be understood by those skilled in the art that the functions to be fulfilled by the constituent elements recited in the claims are realized by the individual constituent elements shown in the embodiments and the modified examples or by their linkage.

  DESCRIPTION OF SYMBOLS 10 Incident management part, 20 System management database, 22 Configuration information master, 24 Related management master, 26 Change history master, 28 Bottleneck candidate master, 30 Change performance analysis part, 40 Notification part, 100 System management support apparatus, 200 Blade group 250 virtual machine, 252 database server, 254 web server, 256 first network interface, 260 second network interface, 300 network switch group, 400 storage group, 500 information processing system, 500 period information processing system.

Claims (5)

A system management database for storing a change history when a component composed of hardware or software of the information processing system involved in the incident is changed in response to an incident occurring in the information processing system;
Referencing the system management database, counting the number of times that each component has been changed in response to the incident, and identifying a component that is equal to or greater than a predetermined notification reference number, the change record analysis unit,
A notification unit for notifying the component identified by the change record analysis unit,
The system management database further stores a component identified by the change history analysis unit in association with a date when the notification unit notifies the component,
The said management part does not notify the said structural element, when the predetermined period passes from the date which notified the structural element which the said change performance analysis part specified, The system management assistance apparatus characterized by the above-mentioned.   The change record analysis unit counts changes that have occurred within a predetermined change counting period from the time when the identification is performed, when identifying a component whose number of changes is equal to or greater than a predetermined notification reference number. The system management support apparatus according to claim 1. The notification reference number is determined for each component constituting the information processing system,
The said change performance analysis part compares whether the frequency | count of a change of the counted component is more than the said notification reference frequency defined for the said component, It is characterized by the above-mentioned. System management support device. A system management support method executed by a processor,
In response to the incident that occurred in the information processing system, refer to the system management database that stores the change history when the component consisting of hardware or software of the information processing system involved in the incident is changed, Counting the number of times that the component constituting the information processing system has been changed in response to an incident;
Identifying a component whose number of changes is equal to or greater than a predetermined notification reference number;
Notifying the identified component,
The system management database further stores the identified component in association with the date of notification of the component,
The notification steps, when the lapse of a predetermined period of time from the date of notification of the identified components, system management support method characterized in that said components are not notified. A program that causes a computer to implement a system management support function.
In response to the incident that occurred in the information processing system, refer to the system management database that stores the change history when the component consisting of hardware or software of the information processing system involved in the incident is changed, A function that counts the number of times that the information processing system has been changed in response to an incident with respect to a component that constitutes the information processing system;
A function for identifying a component whose number of changes is equal to or greater than a predetermined notification reference number;
Including the function of notifying the identified component,
The system management database further stores the identified component in association with the date of notification of the component,
The notification function is a program in which, when a predetermined period has elapsed from a date when the specified component is notified, the component is not notified .

Images