JP5459472B2 - Failure recovery apparatus, failure recovery method, and program - Google Patents

Description

  The present invention relates to a failure recovery apparatus, a failure recovery method, and a program for performing recovery when a failure occurs in a computer system.

  In recent years, in the event of a failure in a computer system, a mechanism has been proposed in which recovery from a failure is automatically performed in order to quickly recover and reduce the burden on the user (for example, Patent Documents). 1 and 2).

  Specifically, Patent Document 1 includes, as a mechanism for detecting and automatically recovering from a failure that has occurred in a computer system, a database that stores information relating to the cause of the failure and a database that stores information relating to a recovery method. System (automatic failure recovery system) is proposed.

  In the system disclosed in Patent Document 1, when a failure occurs in a target computer system, the failure is detected by the failure detection means, and a recovery method corresponding to the detected failure is inquired of the database. Then, an appropriate recovery method is specified, and failure recovery is performed by the recovery execution means.

  However, in principle, the system disclosed in Patent Document 1 automatically executes a recovery process immediately after detecting a failure that has occurred. Therefore, in this system, after recovery, it is difficult to collect information that can be collected only when a failure occurs, such as OS dump, memory information, performance information, and log information rotated by restoration processing. For this reason, in the automatic failure recovery system disclosed in Patent Document 1, information that can be collected for failure analysis is limited, and after recovery, the true cause of the failure cannot be investigated, and there is a risk that the failure cannot be dealt with in earnest. There is.

  On the other hand, Patent Document 2 proposes a system that recovers after collecting necessary information when a failure occurs. In the system disclosed in Patent Document 2, when a failure occurs in a target computer system, information necessary for investigating the cause is collected and then the computer system is restarted.

Japanese Patent Laid-Open No. 11-073336 Japanese Patent Laid-Open No. 11-120032

  However, in the system disclosed in Patent Document 2, even though the problem of the system disclosed in Patent Document 1 can be solved, the time required for collecting information necessary for investigating the cause is not taken into consideration. It can take a lot of time. For this reason, when the system disclosed in Patent Document 2 is used, there is a possibility that the stop time of the computer system becomes very long.

  An object of the present invention is to solve the above-mentioned problems and execute a collection of information that can be collected only when a failure occurs from a computer system, while suppressing an increase in the time taken to restore the computer system. And providing a failure recovery method and program.

In order to achieve the above object, a failure recovery apparatus according to the present invention detects a failure that occurs in a target computer system, and a failure detection unit;
A recovery method determination unit for determining a recovery method corresponding to the detected failure;
Obtaining one or more collection information required to be collected for the analysis of the failure, collection time required for collecting each collection information, and analysis information specifying the priority given to each collection information; An analysis information acquisition unit;
The time that can be used for collecting the collection information is specified, and the collection information that can be collected within the specified usable time in order of priority based on the collection time included in the analysis information. A collection information determination unit for determining
An information collection unit that performs collection of the collection information determined by the collection information determination unit;
A recovery execution unit configured to recover the computer system in accordance with the recovery method determined by the recovery method determination unit after execution of the collection by the information collection unit;

In order to achieve the above object, the failure recovery method according to the present invention includes:
(A) detecting a failure occurring in the target computer system; and
(B) determining a recovery method corresponding to the failure detected in the step (a); and
(C) One or two or more pieces of collection information that are required to be collected for analysis of the failure, collection time required for collection of each piece of collection information, and analysis information that identifies the priority given to each piece of collection information Get, step, and
(D) A time that can be used for collecting the collection information is specified, and can be collected within the specified usable time in order of priority based on the collection time included in the analysis information. Determining the collection information; and
(E) performing the collection of the collection information determined by the step of (d); and
(F) After the collection in the step (e) is executed, the computer system is restored in accordance with the restoration method determined in the step (b).

Furthermore, in order to achieve the above object, a program according to the present invention is a program for recovering a computer system in which a failure has occurred by a computer,
In the computer,
(A) detecting a failure occurring in the computer system; and
(B) determining a recovery method corresponding to the failure detected in the step (a); and
(C) One or two or more pieces of collection information that are required to be collected for analysis of the failure, collection time required for collection of each piece of collection information, and analysis information that identifies the priority given to each piece of collection information Get, step, and
(D) A time that can be used for collecting the collection information is specified, and can be collected within the specified usable time in order of priority based on the collection time included in the analysis information. Determining the collection information; and
(E) performing the collection of the collection information determined by the step of (d); and
(F) After the collection in the step (e) is executed, the computer system is restored according to the restoration method determined in the step (b).

  With the above features, according to the failure recovery apparatus, the failure recovery method, and the program of the present invention, while collecting information that can only be collected from a computer system when a failure occurs, the time required for recovery of the computer system can be reduced. It becomes possible to suppress the prolongation.

FIG. 1 is a block diagram showing a configuration of a failure recovery apparatus according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of failure information registered in the failure information database shown in FIG. FIG. 3A is a diagram illustrating an example of recovery method information registered in the recovery method database illustrated in FIG. 1, and FIG. 3B is a diagram illustrating an example of the determined recovery method. FIG. 4A is a diagram illustrating an example of analysis information registered in the failure information database illustrated in FIG. 1, and FIG. 4B is a diagram illustrating an example of acquired analysis information. FIG. 5 is a diagram illustrating an example of a collection information list created by the collection information determination unit illustrated in FIG. FIG. 6 is a flowchart showing the operation of the failure recovery apparatus in the embodiment of the present invention. FIG. 7 is a flowchart specifically showing the failure detection process shown in FIG. FIG. 8 is a flowchart specifically showing the recovery method determination process shown in FIG. FIG. 9 is a flowchart specifically showing the analysis information acquisition process shown in FIG. FIG. 10 is a flowchart specifically showing the collection information determination process shown in FIG. FIG. 11 is a flowchart more specifically showing steps D4 and D5 shown in FIG. FIG. 12 is a flowchart specifically showing the information collection process shown in FIG. FIG. 13 is a flowchart specifically showing the recovery process shown in FIG.

(Embodiment 1)
Hereinafter, a failure recovery apparatus, a failure recovery method, and a program according to an embodiment of the present invention will be described with reference to FIGS. First, the configuration of the failure recovery apparatus according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of a failure recovery apparatus according to an embodiment of the present invention.

  The failure recovery apparatus 1 in the present embodiment shown in FIG. 1 is an apparatus that executes an appropriate recovery process corresponding to a failure on the computer system 20 when a failure occurs in the computer system 20. As illustrated in FIG. 1, the failure recovery apparatus 1 includes a failure detection unit 11, a recovery method determination unit 12, an analysis information acquisition unit 13, a collection information determination unit 14, an information collection unit 15, and a recovery execution unit 16. And.

  The failure detection unit 11 detects a failure that occurs in the target computer system 20. The recovery method determination unit 12 determines a recovery method corresponding to the detected failure. The analysis information acquisition unit 13 acquires analysis information. The analysis information specifies one or more collection information required to be collected for failure analysis, a collection time required for collecting each collection information, and a priority given to each collection information.

  In addition, the collection information determination unit 14 identifies a time that can be used for collecting the collection information, and can be collected within the specified usable time in order of priority based on the collection time included in the analysis information. Appropriate collection information. Then, the information collection unit 15 performs collection of the collection information determined by the collection information determination unit 14. Further, the recovery executing unit 16 recovers the computer system 20 according to the recovery method determined by the recovery method determining unit 12 after the information collecting unit 15 executes the collection.

  As described above, in the failure recovery apparatus 1, first, from the information (collection information) necessary for analyzing the failure that has occurred, the time allowed to be collected, the collection time of each collection information, and the priority order Considering the above, collection information that can actually be collected is selected. Then, the selected collection information is collected, and then a failure recovery process is performed.

  That is, the failure recovery apparatus 1 collects as much information useful for analysis as possible in descending order of priority within a limited time, and immediately executes recovery processing. Therefore, according to the failure recovery apparatus 1, it is possible to suppress an increase in time required for recovery of the computer system 20 while executing collection of information that can be collected only when a failure occurs from the computer system 20.

  Here, the configuration of the failure recovery apparatus 1 will be described more specifically with reference to FIGS. 2 to 5 in addition to FIG. 1. In the present embodiment, as shown in FIG. 1, the failure recovery apparatus 1 is connected to a failure information database 111, a recovery method database 122, an analysis information database 132, and an operation policy database 142.

  In this embodiment, when detecting a failure that has occurred in the computer system 20, the failure detection unit 11 accesses the failure information database 111 and identifies an identifier (failure information ID) of the failure that has occurred.

  Specifically, the failure information database 111 stores the failure information shown in FIG. FIG. 2 is a diagram showing an example of failure information registered in the failure information database shown in FIG. As shown in FIG. 2, the failure information includes a failure content and a failure information ID assigned to each failure content. In the example of FIG. 2, a specific failure occurrence location is used as the failure information ID, but the present embodiment is not limited to this.

  Therefore, the failure detection unit 11 specifies the content of the detected failure, compares this with the failure information shown in FIG. 2, and specifies the corresponding failure information ID. Further, the failure detection unit 11 transmits the specified failure information ID to the recovery method determination unit 12 and the analysis information acquisition unit 13.

  In the present embodiment, the recovery method determination unit 12 accesses the recovery method database 122 when receiving the failure information ID from the failure detection unit 11. Then, the recovery method determination unit 12 determines a recovery method using the failure information ID received from the failure detection unit 11 as a key.

  Specifically, the recovery method database 122 stores the recovery method information shown in FIG. FIG. 3A is a diagram illustrating an example of the recovery method information registered in the recovery method database illustrated in FIG. As shown in FIG. 3A, the recovery method information includes a specific recovery process set in advance for each failure information ID and a time (recovery time) necessary for executing each recovery process. . Each restoration processing content and each restoration time correspond to each failure.

  Therefore, when receiving the failure information ID from the failure detection unit 11, the recovery method determination unit 12 specifies the recovery processing content and the recovery time corresponding to the failure information ID, and determines these as recovery methods. FIG. 3B is a diagram illustrating an example of the determined recovery method. The example of FIG. 3B shows a recovery method when the failure information ID of the failure that has occurred is “memory 1”. Further, the recovery method determination unit 12 transmits information (decision recovery method information) 121 for specifying the determined recovery method to the collection information determination unit 14 and the recovery execution unit 16.

  In the present embodiment, the analysis information acquisition unit 13 accesses the analysis information database 132 when receiving the failure information ID from the failure detection unit 11. Then, the analysis information acquisition unit 13 acquires analysis information corresponding to the detected failure using the failure information ID received from the failure detection unit 11 as a key.

  Specifically, the analysis information database 132 stores the analysis information shown in FIG. FIG. 4A is a diagram showing an example of analysis information registered in the failure information database shown in FIG. As shown in FIG. 4A, the analysis information includes one or more collection information set in advance for each failure information ID, a collection time of each collection information, and a priority order. Note that the priority is given to the collection information having the same failure ID.

  Accordingly, when the analysis information acquisition unit 13 receives the failure information ID from the failure detection unit 11, the analysis information acquisition unit 13 identifies one or more collection information corresponding to the failure information ID, the collection time of each collection information, and the priority order. And obtained as analysis information corresponding to the detected failure. FIG. 4B is a diagram illustrating an example of the acquired analysis information. In the example of FIG. 4B, analysis information when the failure information ID of the failure that has occurred is “memory 1” is shown. Further, the analysis information acquisition unit 13 transmits the acquired analysis information (acquired analysis information) to the collection information determination unit 14.

  In the present embodiment, the collection information determination unit 14 first accesses the operation policy database 142. The operation policy database stores an operation policy for recovery processing. In addition, the operation policy defines the time that can be used for the recovery process, that is, the time that can be allowed until the recovery is completed after the failure occurs (recovery allowable time). Therefore, the collection information determination unit 14 specifies the allowable recovery time from the operation policy database 142.

  Then, when the recovery information determining unit 14 specifies the recovery allowable time, the recovery information included in the determination recovery method information 121 (FIG. 3B (see)) is compared with the recovery information for analysis. Specify the time available for sampling (allowable time for analysis). Subsequently, the collection information determination unit 14 performs the collection within the analysis allowable time in the order of priority of each collection information based on the collection time of each collection information included in the selection analysis information (see FIG. 4B). Determine the collection information that can be collected.

  Further, as shown in FIG. 5, the collection information determination unit 14 creates a list (collection information list) 141 that specifies the determined collection information, and transmits this to the information collection unit 15. FIG. 5 is a diagram illustrating an example of a collection information list created by the collection information determination unit illustrated in FIG.

  In this embodiment, the information collection unit 15 accesses the computer system 20 when it receives the collection information list 141 from the collection information determination unit 14. Then, the information collection unit 15 acquires the collection information included in the collection information list 141 (see FIG. 5) from the computer system 20. When the acquisition of the collection information is completed, the information collection unit 15 notifies the recovery execution unit 16 of that.

  In the present embodiment, when the recovery execution unit 16 receives a notification from the information collection unit 15, the recovery execution unit 16 specifies the recovery process from the determined recovery method information 121 (see FIG. 3B) received from the recovery method determination unit 12. Then, the specified recovery process is executed.

  However, in this embodiment, the collection information determination unit 14 collects collection information when there is no collection information that can be collected within the analysis allowable time (for example, when the recovery allowable time is 0 (zero)). Is notified to the information collecting unit 14. In this case, the recovery execution unit 16 immediately executes the recovery process of the computer system 20.

  Next, operation | movement of the failure recovery apparatus 1 in embodiment of this invention is demonstrated using FIGS. First, the overall operation of the failure recovery apparatus 1 will be described with reference to FIG. FIG. 6 is a flowchart showing the operation of the failure recovery apparatus in the embodiment of the present invention. In the following description, FIG. 1 is taken into consideration as appropriate. In the present embodiment, the failure recovery method according to the present embodiment is implemented by operating the failure recovery apparatus 1. Therefore, the description of the failure recovery method in the present embodiment is replaced with the following description of the failure recovery apparatus 1.

  As shown in FIG. 6, first, the failure detection unit 11 detects a failure that has occurred in the computer system 20 (step S1). Next, the recovery method determination unit 12 determines a recovery method corresponding to the detected failure (step S2). Subsequently, the analysis information acquisition unit 13 acquires analysis information corresponding to the detected failure (step S3).

  Next, the collection information determination unit 14 specifies a time (analysis allowable time) that can be used for collecting the collection information, and sampling is performed within the analysis allowable time from the collection information necessary for analyzing the detected failure. Collection information to be performed is determined (step S4).

  Next, the information collection unit 15 collects the collection information determined in step S4 from the computer system 20 (step S5). Thereafter, the recovery execution unit 16 executes the recovery method determined in step S2, and the computer system 20 is recovered (step S6). By executing step S6, the process in the failure recovery apparatus 1 is completed.

  Here, steps S1 to S6 shown in FIG. 6 will be described more specifically with reference to FIGS. First, step S1 (failure detection processing) shown in FIG. 6 will be described with reference to FIG. FIG. 7 is a flowchart specifically showing the failure detection process shown in FIG.

  As shown in FIG. 7, first, the failure detection unit 11 detects a failure that has occurred in the computer system 20 (step A1). Then, the failure detection unit 11 makes an inquiry to the failure information database 111 and specifies the failure information ID (see FIG. 2) of the detected failure (step A2). For example, when the detected failure is “memory unauthorized access”, the failure detection unit 11 refers to the failure information stored in the failure information database 111 and sets “Memory 1” as the corresponding failure information ID. Is identified.

  Next, the failure detection unit 11 transmits the specified failure information ID, for example, “memory 1” to the recovery method determination unit 12 (step A3), and further transmits this to the analysis information acquisition unit 13 (step S3). A4). Execution of step A4 completes step S1 shown in FIG. In the present embodiment, step A3 and step A4 may be executed simultaneously, or step A3 may be executed after step A4.

  Next, step S2 (recovery method determination process) shown in FIG. 6 will be described with reference to FIG. FIG. 8 is a flowchart specifically showing the recovery method determination process shown in FIG.

  As shown in FIG. 8, first, the recovery method determination unit 12 receives a failure information ID from the failure detection unit 11 (step B1). Next, the recovery method determination unit 12 makes an inquiry to the recovery method database 122 and determines a recovery method (recovery process, recovery time) corresponding to the received failure information ID (step B2).

  For example, in step B1, it is assumed that the recovery method determination unit 12 has received “memory 1” as the failure information ID. In this case, the recovery method determination unit 12 uses “memory 1” as a key, and compares this with the recovery method information stored in the recovery method database 122 (see FIG. 3A). Then, the recovery method determination unit 12 specifies the recovery process “server restart” corresponding to “memory 1” and the recovery time “600” corresponding to “memory 1” from the recovery method information, These are determined recovery methods (see FIG. 3B).

  Next, the recovery method determination unit 12 transmits information (decision recovery method information) 121 for specifying the determination recovery method obtained in step B2 to the collection information determination unit 14 (step B3), and further recovers this. It transmits also to the execution part 16 (step B4). By executing step B4, step S2 shown in FIG. 6 ends. In the present embodiment, step B3 and step B4 may be executed simultaneously, or step A3 may be executed after step B4.

  Next, step S3 (analysis information acquisition process) shown in FIG. 6 will be described with reference to FIG. FIG. 9 is a flowchart specifically showing the analysis information acquisition process shown in FIG.

  As shown in FIG. 9, first, the analysis information acquisition unit 13 receives a failure information ID from the failure detection unit 11 (step C1). Next, the analysis information acquisition unit 13 makes an inquiry to the analysis information database 132 and acquires analysis information corresponding to the received failure information ID (step C2).

  For example, in step C1, the analysis information acquisition unit 13 receives “memory 1” as the failure information ID. In this case, the analysis information acquisition unit 13 specifies the collection information corresponding to “memory 1”, the collection time and priority corresponding to each collection information. Specifically, as analysis information, memory dump (collection time: 120 seconds, priority: 1), memory performance information (collection time: 30 seconds, priority: 2), OS dump (collection time: 600 seconds, priority) : 3), a process activation log (collection time: 30 seconds, priority: 4) is acquired (see FIG. 4B).

  Next, the analysis information acquisition unit 13 transmits the analysis information (collection information, collection time, priority) acquired in step C2 to the collection information determination unit 14 (step C3). Execution of step C3 completes step S3 shown in FIG.

  Next, step S4 (collection information determination process) shown in FIG. 6 will be described with reference to FIG. FIG. 10 is a flowchart specifically showing the collection information determination process shown in FIG.

  As shown in FIG. 10, first, the collection information determination unit 14 receives the determination recovery method information 121 (see FIG. 3B) from the recovery method determination unit 12 (step D <b> 1), and further, the analysis information acquisition unit 13. Acquisition analysis information 131 (see FIG. 4B) is received (step D2).

  Next, the collection information determination unit 14 makes an inquiry to the operation policy database 142 and specifies the allowable recovery time specified in the operation policy (step D3). Next, the collection information determination unit 14 specifies the analysis allowable time by comparing the recovery allowable time with the recovery time included in the determined recovery method information 121 (step D4).

  Next, the collection information determination unit 14 can collect samples within the analysis allowable time based on the analysis allowable time specified in step D4 and the collection time and priority of each piece of collection information included in the acquired analysis information 131. Information is determined, and the collection information list 141 (see FIG. 5) is created (step D5).

  Thereafter, the collection information determination unit 14 transmits the collection information list 141 created in step D5 to the information collection unit 15 (step D6). Execution of step D6 completes step S4 shown in FIG.

  Here, steps D4 and D5 will be described more specifically with reference to FIG. FIG. 11 is a flowchart more specifically showing steps D4 and D5 shown in FIG. Of steps E1 to E8 shown in FIG. 11, step E1 corresponds to step D4, and steps E2 to E8 correspond to step D5.

  As illustrated in FIG. 11, the collection information determination unit 14 specifies the analysis allowable time (step E1). Specifically, the collection information determination unit 14 uses the following formula 1, and calculates the recovery time (FIG. 3A and FIG. 3A) included in the determined recovery method information 121 from the allowable recovery time registered in the operation policy database 142. (B) is subtracted to calculate the allowable analysis time.

(Equation 1)
Allowable analysis time = Allowable recovery time-Restore time

  Next, the collection information determination unit 14 determines whether or not the calculated analysis allowable time is greater than 0 (zero) (step E2). If the analysis allowable time is not greater than 0 (zero) as a result of the determination in step E2, that is, if it is 0 (zero) or less, the collection information determination unit 14 executes step E8. The contents of step E8 will be described later.

  On the other hand, if the analysis allowable time is greater than 0 (zero) as a result of the determination in step E2, the collection information determination unit 14 determines whether or not the acquired analysis information 131 (see FIG. 4B) is empty. (Step E3). Specifically, it is determined whether or not a list including one or two or more pieces of collection information to be collected and collection times and priorities corresponding to the pieces of collection information exists as the acquisition analysis information 131.

  If the acquisition analysis information 131 is empty as a result of the determination in step E3, that is, if there is no collection information to be collected, the collection information determination unit 14 executes step E8. On the other hand, if the result of determination in step E3 is that the acquisition analysis information 131 is not empty, the collection information determination unit 14 executes step E4.

  In step E4, the collection information determination unit 14 selects the collection information having the highest priority from the acquired analysis information 131, and calculates the analysis remaining time using the following formula 2. In the following formula 2, “corresponding collection time” means the collection time for the collection information selected in step E4 (see FIGS. 4A and 4B). The remaining analysis time is a temporary numerical value and is a value used only in the processing shown in FIG.

(Equation 2)
Analysis remaining time = Analysis allowable time-Applicable sampling time

  Next, when step E4 ends, the collection information determination unit 14 deletes the collection information for which the calculation in step E4 has ended from the list of collection information in the acquisition analysis information 131 (step E5). Then, the collection information determination unit 14 determines whether or not the remaining analysis time calculated in Step E4 is greater than 0 (zero) (Step E6).

  As a result of the determination in step E6, if the remaining analysis time is not greater than 0 (zero), the collection information selected in step E4 is not appropriate because of the relationship with the analysis allowable time. Step E3 and subsequent steps are executed.

  On the other hand, if the result of determination in step E6 is that the remaining analysis time is greater than 0 (zero), the collection information determination unit 14 adds the collection information selected in step E4 to the collection information list 141, and further the analysis remaining The time is set to the analysis allowable time (analysis allowable time = analysis remaining time) (step E7).

  And after execution of step E7, the collection information determination part 14 implements step E2 and subsequent steps again. By executing steps E2 to E7, collection information that can be collected within the analysis allowable time is specified in order of priority.

  In step E8, the collection information determination unit 14 determines the collection information list 141. The confirmed collection information list 141 is transmitted to the information collection unit 15.

  Here, steps E1 to E8 will be described based on specific examples. First, a case where the allowable recovery time is “2000 seconds”, the decision recovery method information 121 is an example of FIG. 3B, and the acquisition analysis information 131 is an example of FIG. 4B will be described.

  In the above case, in step E1, the value of the analysis allowable time is a value “1400” obtained by subtracting the recovery time “600” (see FIG. 3B) from the recovery allowable time “2000”. Next, when the value of the analysis allowable time is determined in step E2, since “1400> 0”, step E3 is executed.

  In step E3, the collection information determination unit 14 includes the acquisition analysis information 131 including “memory dump”, “memory performance information”, “OS dump”, and “process activation log” (see FIG. 4B). If it is not “empty”, step E4 is executed.

  In step E4, the collection information determination unit 14 selects the collection information “memory dump” having the highest priority (priority = “1”) and calculates the remaining analysis time. In this case, the recovery allowable time is “1400” and the collection time of “memory dump” is “120”, so the remaining analysis time is “1280”.

  After calculating the remaining analysis time, in step E5, the collection information determination unit 14 deletes the “memory dump” line from the collection information list in the acquired analysis information 131. Thereafter, in step E6, the collection information determination unit 14 determines whether or not the value of the analysis remaining time is greater than 0 (zero).

  Since the remaining analysis time is “1280” and is greater than 0 (zero), the collection information determination unit 14 adds “memory dump” to the collection information list 141 (see FIG. 5) in step E7, and allows the analysis allowable time. Is updated to the value of the remaining analysis time, that is, “1280”. Thereafter, the collection information determination unit 14 executes Step E2 and subsequent steps again using the updated analysis allowable time.

  By executing again after step E2, the “memory performance information” shown in FIG. 4B is added to the collection information list 141, and the analysis allowable time is updated to “1250”. Furthermore, “OS dump” is added to the collection information list 141, and the analysis allowable time is updated to “650”. Subsequently, the “process activation log” is also added to the collection information list 141, and finally the analysis allowable time becomes “620”.

  Further, since the acquisition analysis information 131 becomes “empty” due to the addition of the collection information list 141 of the process activation log, “Yes” is determined in step E3 after the addition of the process activation log. Therefore, step E8 is executed and the final collection information list 141 is confirmed. In this case, unlike the example of FIG. 5, the collection information list 141 includes all of “memory dump”, “memory performance information”, “OS dump”, and “process activation log”. As described above, when the recovery allowable time set in the operation policy is sufficient, it is possible to collect all information necessary for the analysis.

  Next, a case where the allowable recovery time is “1000 seconds”, the decision recovery method information 121 is an example of FIG. 3B, and the acquisition analysis information 131 is an example of FIG. 4B will be described. In this case, after the memory dump and the memory performance information are selected, the value of the updated analysis allowable time is “250”. Therefore, when “OS dump” is selected in step E4, the collection time of “OS dump” is as long as “600”, so the remaining analysis time is “−350”.

  Therefore, in step E6, “−350 <0” is established, “No” is determined, and the process returns to step E3. Therefore, the “OS dump” is not added to the collection information list 141, and the value of the analysis allowable time is not updated and remains “250”.

  On the other hand, since the collection time of the “process activation log” whose priority is set next to the OS dump is as short as “30”, the analysis remaining time in step E4 is “220”. Therefore, the collection information determination unit 14 adds “process activation log” to the collection information list 141. In this example, finally, in step E8, the collection information list 141 includes only “memory dump”, “memory performance information”, and “process activation log”.

  As described above, when the allowable recovery time set in the operation policy is insufficient, the failure recovery apparatus 1 is capable of sequentially starting from the information with the highest priority in order to prevent the system from being stopped more than the allowable recovery time. As much information is collected as possible.

  Next, the case where the allowable recovery time is “0 second”, the decision recovery method information 121 is an example of FIG. 3B, and the acquisition analysis information 131 is an example of FIG. 4B will be described. In this case, by the execution of step E1, the analysis allowable time becomes “−600”, which is a value obtained by subtracting the recovery time “600” from the recovery allowable time “0”.

  Therefore, since it is determined as “No” in Step E2, Step E8 is executed thereafter. In this case, collection information is not added to the collection information list 141, and the collection information list 141 is in an “empty” state.

  As a result, in the post-processing, the information collection unit 15 does not collect any information, and the recovery execution unit 16 immediately performs the recovery process. As described above, when there is no allowable recovery time set in the operation policy, the failure recovery apparatus 1 prioritizes recovery over information collection and immediately performs recovery.

  Next, step S5 (collection process) shown in FIG. 6 will be described with reference to FIG. FIG. 12 is a flowchart specifically showing the information collection process shown in FIG.

  As shown in FIG. 12, first, the information collection unit 15 receives the collection information list 141 (see FIG. 5) from the collection information determination unit 14 (step E1). Next, the information collection unit 15 collects collection information included in the collection information list 141 (step E2). For example, when the collection information list 141 is the list shown in FIG. 5, the information collection unit 15 collects a memory dump and memory performance information from the computer system 20 (see FIG. 1). Thereafter, the information collection unit 15 instructs the recovery execution unit 16 to recover (step E3).

  Next, step S5 (recovery processing) shown in FIG. 6 will be described with reference to FIG. FIG. 13 is a flowchart specifically showing the recovery process shown in FIG.

  As shown in FIG. 13, first, the recovery executing unit 16 receives the determined recovery method information 121 from the recovery method determining unit 12 (step G1), and further instructs the recovery (execution of recovery) from the information collecting unit 15 (Instruction) is received (step G2). Note that the order of steps G1 and G2 may be interchanged, or both may be executed simultaneously.

  Thereafter, the recovery execution unit 16 executes the recovery process (see FIGS. 3A and 3B) defined in the decision recovery method information 121 received in step G1 (step G3). As described above, in the present embodiment, the recovery process is performed after the collection of the collection information is completed. In addition, when there is no collection information within which the collection time is within the analysis allowable time, the recovery execution unit 16 executes Steps G1 to G3 without waiting for collection of the collection information.

  As described above, according to the present embodiment, it is possible to collect as much information as can be collected only when a failure occurs within a preset recovery allowable time range. The collected information is selected based on a preset priority from the viewpoint of whether it is useful for analysis or the like, and is information effective for analysis. Further, according to the present embodiment, it is possible to automatically recover from a failure without extending the system stop time more than allowable.

The program in the embodiment of the present invention may be a program that causes a computer to execute steps S1 to S6 shown in FIG. By installing and executing this program on a computer, the failure recovery apparatus 1 and the failure recovery method in the present embodiment can be realized. In this case, the CPU (Central Processing) of the computer
Unit) functions as a failure detection unit 11, a recovery method determination unit 12, an analysis information acquisition unit 13, a collection information determination unit 14, an information collection unit 15, and a recovery execution unit 16, and performs processing.

  In the present embodiment, the failure information database 111, the recovery method database 122, the analysis information database 132, and the operation policy database 142 store data files constituting them in a storage device such as a hard disk provided in the computer. It can be realized by doing. Note that the storage device may be connected to a computer via a network.

  In the present embodiment, the computer on which this program is installed may be a computer constituting the target computer system 20 (see FIG. 1). In this case, the failure recovery apparatus 1 is built inside the computer system 20.

  As described above, according to the present invention, it is possible to suppress the lengthening of the time required for recovery of the target system while collecting information from the target system that can be collected only when a failure occurs. Therefore, the present invention is useful for recovery from a failure in a computer system.

DESCRIPTION OF SYMBOLS 1 Failure recovery apparatus 11 Failure outstanding part 12 Recovery method determination part 13 Analysis information acquisition part 14 Collection information determination part 15 Information collection part 16 Recovery execution part 20 Computer system 111 Failure information database 121 Decision recovery method information 122 Recovery method database 131 Acquisition analysis Information 132 Analysis information database 141 Collection information list 142 Operation policy database

Claims (9)

A failure detection unit for detecting a failure that occurs in the target computer system;
A recovery method determination unit for determining a recovery method corresponding to the detected failure;
Obtaining one or more collection information required to be collected for the analysis of the failure, collection time required for collecting each collection information, and analysis information specifying the priority given to each collection information; An analysis information acquisition unit;
From the preset time that can be used for the restoration process and the time that is necessary for the execution of the restoration method, the time that can be used for collecting the collection information is specified, and the analysis information is A collection information determination unit that determines the collection information that can be collected within the specified usable time in the order of priority based on the included collection time;
An information collection unit that performs collection of the collection information determined by the collection information determination unit;
A failure recovery apparatus comprising: a recovery execution unit that recovers the computer system in accordance with the recovery method determined by the recovery method determination unit after execution of collection by the information collection unit. The recovery method is preset for each possible failure, together with the time required to implement the recovery method,
The failure recovery apparatus according to claim 1, wherein the recovery method determination unit determines a recovery method corresponding to the detected failure from among the recovery methods set in advance. The collection information determination unit notifies the information collection unit that the collection of the collection information is not executed when there is no collection information that can be collected within the specified usable time, and the recovery execution is performed. wherein to recover the computer system parts, disaster recovery apparatus according to claim 1 or 2. (A) detecting a failure occurring in the target computer system; and
(B) determining a recovery method corresponding to the failure detected in the step (a); and
(C) One or two or more pieces of collection information that are required to be collected for analysis of the failure, collection time required for collection of each piece of collection information, and analysis information that identifies the priority given to each piece of collection information Get, step, and
(D) identifying a time that can be used for collecting the collection information from a preset time that can be used for the restoration process and a time that is necessary for performing the determined restoration method ; and Determining the collection information that can be collected within the specified usable time in the order of priority based on the collection time included in the analysis information; and
(E) performing the collection of the collection information determined by the step of (d); and
(F) After executing the collection in the step (e), the step of restoring the computer system according to the restoration method determined in the step (b) is provided. . The recovery method is preset for each possible failure, together with the time required to implement the recovery method,
5. The failure recovery method according to claim 4 , wherein in the step (b), a recovery method corresponding to the detected failure is determined from the recovery methods set in advance. In the step (d), when the collection information that can be collected within the specified usable time does not exist, the step (f) is executed without executing the step (e). The failure recovery method according to claim 4 or 5 . A program for recovering a failed computer system by a computer,
In the computer,
(A) detecting a failure occurring in the computer system; and
(B) determining a recovery method corresponding to the failure detected in the step (a); and
(C) One or two or more pieces of collection information that are required to be collected for analysis of the failure, collection time required for collection of each piece of collection information, and analysis information that identifies the priority given to each piece of collection information Get, step, and
(D) identifying a time that can be used for collecting the collection information from a preset time that can be used for the restoration process and a time that is necessary for performing the determined restoration method ; and Determining the collection information that can be collected within the specified usable time in the order of priority based on the collection time included in the analysis information; and
(E) performing the collection of the collection information determined by the step of (d); and
(F) After executing the collection in the step (e), the program is executed to restore the computer system according to the restoration method determined in the step (b). The recovery method is preset for each possible failure, together with the time required to implement the recovery method,
The program according to claim 7 , wherein in the step (b), a recovery method corresponding to the detected failure is determined from the recovery methods set in advance. In the step (d), when the collection information that can be collected within the specified usable time does not exist, the step (f) is executed without executing the step (e). The program according to claim 7 or 8 .

Images