JP2015064806A - Fault recovery support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To lessen operator's work burden and shorten operation system recovery time if a work system stops and an operator switches operation to an alternative server.SOLUTION: A work-continuation-status confirmation unit 211 confirms whether it is possible to continue work executed by each agent operating before occurrence of a fault, before the occurrence of the fault when the fault occurs, and generates work continuation information indicating whether each agent can continue the work. A recovery support unit 212 displays the generated work continuation information and information indicating an agent allocated as an alternative server of each agent for each agent operating before the occurrence of the fault, receives selection of an agent as a re-execution target and an agent as an execution stop target input by an operator, re-executes the agent selected as the re-execution target, and stops executing the agent selected as the execution stop target.

Description

  In the present invention, when a failure occurs due to a disaster or the like, and a job management system operating on a specific server stops unexpectedly, the disaster recovery system copies data to another server installed at a remote location. The present invention relates to a failure recovery support system for supporting an operation switching operation by an operator in a system for continuing or resuming operation based on the operation data stored.

  The disaster recovery system uses software and hardware replication and mirroring systems to protect the job management system data in operation by copying it to a remote location. As a result, in the case of a disaster recovery system, if the job management system in operation stops unexpectedly due to a disaster or the like, the operator switches from the operation switching source (hereinafter referred to as the main site) based on the copied data. The operation of the job management system once stopped can be continued or resumed by performing a switching operation from to the operation switching destination (hereinafter referred to as a remote site).

  On the other hand, in a job management system comprising a job management manager server (hereinafter referred to as a manager) and a plurality of job execution agent servers (hereinafter referred to as agents) connected to the job management agent server, the agent is sent from the manager. The job is executed in response to the job execution request, and the execution result is reported to the job management manager, whereby the status is managed and the job is performed. In a job management system, it is common to continue operation using a disaster recovery system.

  As a restoration support method for a business system in which a manager and its subordinate agents are connected, for example, a method described in Patent Document 1 is known. The technology described in Patent Document 1 assigns a server of a separate system for alternative operation at a remote location to each operating server in advance, enables automatic operation continuation, and easily resumes operations. It is aimed at.

JP 2007-265333 A

  When a business system stops when a failure occurs due to a disaster, when an operator performs an operation switching operation to restore the system and continue the operation, information is collected and a troublesome procedure is involved. After switching the operation, the operator resumes operation with job execution suppressed, checks the job status when the system stops at the main site, changes the job status not reflected on the remote site, and re-executes Or it is necessary to cancel the execution.

  In addition, in the job management system, if there are managers in each prefecture and agents are scattered in sales offices in each municipality, the equipment costs of disaster recovery servers for switching agents can be reduced. In order to suppress this, a disaster recovery server may not be prepared for all agents, but a disaster recovery server may be prepared only for the main agent. When operation is continued using the disaster recovery system in such a form, it is necessary to determine the operation continuation method according to the disaster situation of each agent. If there are several agents, it is possible to check one by one. However, if there are tens or hundreds of agents, it takes a lot of time to check one agent at a time. It becomes a factor to be late.

  An object of the present invention is to provide a failure recovery support system capable of reducing the work load on an operator and shortening the recovery time of the business system when the business system is stopped and the operator performs an operation switching operation to an alternative server. It is to be.

In order to achieve the above object, the failure recovery support system of the present invention provides:
A manager and a plurality of agents arranged at the main site, a manager and a plurality of agents arranged at a remote site as an alternative server for the manager and each agent of the main site, and a plurality arranged at an area other than the main site and the remote site Execution job information on each job being executed by each agent of the main site managed by the manager and each of the agents other than the main site and the remote site during operation of the manager of the main site Among these, a job having an execution job information copy means for copying information necessary for taking over the job at the time of failure from the manager at the main site to the manager at the remote site In the stem, a failure recovery support system that operates the manager of the remote site,
Agent operation information storage means for storing agent operation information including information indicating each agent at the remote site assigned as an alternative server for each agent at the main site;
When a failure occurs, each agent managed by the manager at the main site checks whether it is possible to continue the work that was being executed before the failure, and indicates whether the agent can continue the work. Business continuity confirmation means for creating availability information;
For each agent managed by the manager of the main site at the time of failure, the operation continuity information and the agent operation information stored in the agent operation information storage means are displayed, and the re-execution target input by the operator And agent recovery support means for accepting the selection of the agent to be canceled and re-executing the agent selected as the target for re-execution and canceling the execution of the agent selected as the target for execution cancellation.
It is characterized by providing.

Preferably, in the failure recovery support system of the present invention, for each job being executed by each agent managed by the manager of the main site, the execution job information and the agent executing the job and the job Job status information indicating the execution status of
For each job executed by each agent managed by the manager of the main site at the time of failure, the business continuity information of the agent executing the job and each copy copied by the execution job information copying means The job status information of the job is displayed, the selection of the job to be re-executed and the target of execution cancellation input by the operator is accepted, the job selected for re-execution is re-executed, and the job is selected for execution cancellation Job recovery support means for canceling job execution,
It is characterized by providing.

  ADVANTAGE OF THE INVENTION According to this invention, when a business system stops and an operator performs operation switching operation to an alternative server, while reducing an operator's work burden, the recovery time of a business system can be shortened.

It is a figure which shows an example of a structure of the whole business system in which the failure recovery assistance system which concerns on embodiment of this invention operate | moves. It is a figure which shows an example of the allocation as an alternative server between a main site and a remote site. It is a figure which shows an example of a structure of the manager of a remote site. It is a figure which shows an example of a structure of an agent operation information management table. It is a figure which shows an example of a structure of a continuation status management table. It is a figure which shows an example of a structure of a job status information management table. It is a figure which shows an example of the flow of a business continuation status confirmation process. It is a figure which shows an example of the flow of a recovery assistance process. It is a figure which shows an example of the operation target list selection window displayed by step S201. It is a figure which shows an example of the agent list window displayed by step S203. It is a figure which shows an example of the job list window displayed by step S208. It is a figure which shows an example of the recovery process confirmation message box displayed by step S205 and step S210.

  Hereinafter, a failure recovery support system according to an embodiment of the present invention will be described with reference to the drawings. In all the drawings for explaining the embodiments, common constituent elements are denoted by the same reference numerals, and repeated explanation is omitted.

FIG. 1 shows an example of the configuration of the entire business system 100 in which the failure recovery support system according to the embodiment of the present invention operates.
The business system 100 includes a main site 101, a wide area 121 in which agents are scattered, and a remote site 111, and is configured as a disaster recovery system.
The business system 100 includes a job management system that manages job execution and a job for executing business. Note that the job management system cuts out the work (program, batch processing, etc.) constituting the work, defines the execution order of the work, and determines the work flow. The work that is the smallest unit of business processing is called a job.

The main site 101 includes a manager 102 that manages jobs, a job management DB (Data Base) 103 that stores job management information, and an agent 104 and an agent 105 that execute jobs in response to requests from the manager 102.
The wide area 121 includes an agent 122 and an agent 123 that execute a job in response to a request from the manager 102.
The remote site 111 has a fault in the manager 112 that takes over the business and manages the job when the manager 102 in the main site 101 fails, the job management DB 113 that stores job management information, and the agent 104 and the agent 105 in the main site 101. When an error occurs, an agent 114 and an agent 115 that take over a job and execute a job, and an operator terminal 116 that includes a GUI (Graphical User Interface) that displays a status and a status and accepts an operation switching operation. The operator terminal 116 displays an agent list window 800 described later.
The manager 102 of the main site 101, the agents 104 and 105 of the main site 101, the manager 112 of the remote site 111, the agents 114 and 115 of the remote site 111, and the agents 122 and 123 of the wide area 121 are connected to the network. , Can communicate with each other.

FIG. 2 shows an example of assignment as an alternative server between the main site 101 and the remote site 111.
A manager 112 of the remote site 111 is assigned as an alternative server for the manager 102 of the main site 101. When a failure occurs and the manager 102 stops, the manager 112 takes over the business and continues. Similarly, an agent 114 and an agent 115 are assigned as substitute servers for the agent 104 and the agent 105, respectively. There is no substitute server for the agent 122 and agent 123 in the wide area 121.
Note that the manager 102 and the agents 104 and 105 of the main site 101 may be located at remote locations, and the manager 112 and the agents 114 and 115 of the remote site 111 may be located at remote locations. The agents 122 and 123 in the wide area 121 may be arranged near the manager 102 of the main site 101 and the manager 112 of the remote site 111.

The managers 102 and 112 and the agents 104, 105, 114, 115, 122, and 123 are all computers, and a main memory including a CPU (Central Processing Unit), a RAM (Random Access Memory), a hard disk, and the like And a storage device.
When the manager 112 of the remote site 111 reads a predetermined program stored in the storage device into the main memory and executes it, as shown in FIG. 3, the job management unit 200, the business continuity confirmation unit 211, and Each function with the failure recovery support system 210 having the recovery support unit 212 is realized.
Note that both the manager and agent functions can be realized on a single computer. The job management DB 103 of the main site 101 may have a configuration in which the data is stored in the storage device of the manager 102, and the CPU of the manager 102 executes the database management program. The job management DB 113 of the remote site 111 also has the data May be stored in the storage device of the manager 112, and the CPU of the manager 112 may execute a database management program.

The manager 102 of the main site 101 manages the agents 104, 105, 122, and 123 and performs predetermined tasks. The business system 100 is a disaster recovery system. Data of the manager 102 of the main site 101 is protected by being copied (replicated) to the manager 112 of the remote site 111 during its operation. The manager 102 stores data related to the agents 104 and 105 such as scheduling of jobs executed by the agents 104 and 105 during the operation and execution results of the agents 104 and 105 in the storage device of the manager 102 itself. . Data that is protected by being copied to the manager 112 of the remote site 111 also includes data regarding such agents 104 and 105.
The failure recovery support system 210 operates with the manager 112 of the remote site 111. The failure recovery support system 210 supports recovery of the job management system by the manager 112 of the remote site 111 when a failure occurs in the manager 102 of the main site 101.
When the main site 101 goes down and a failure occurs in the job management system, the operator of the business system 100 uses the failure recovery support system 210 to restore the job management system and operate the job management unit 200 of the manager 112. The job management unit 200 manages the agents 104 and 105 or the agents 114 and 115 and the agents 122 and 123, and resumes the job.
At this time, the business continuation status check unit 211 of the failure recovery support system 210 performs the business that each agent in the main site 101 and the wide area 121 that was operating before the failure was executed before the failure occurred. To confirm whether or not it is possible to continue, and create business continuity information indicating whether or not each agent can continue business.
The recovery support unit 212 displays an agent list window 700 in FIG. 10 or a job list window 800 in FIG. 11 to be described later on the operator terminal 116, and selects an agent or job to be reexecuted and to be canceled by the operator. Accept and re-execute the agent or job selected for re-execution, and cancel the execution of the agent or job selected for re-execution.
The business continuity confirmation unit 211 is an example of a business continuation status confirmation unit, and the recovery support unit 212 is an example of an agent recovery support unit and job recovery support unit of the present invention.

The storage device of the manager 112 at the remote site 111 stores the agent operation information management table 300 in FIG. 4, the continuation status management table 400 in FIG. 5, and the job status information management table 500 in FIG. .
The agent operation information management table 300 in FIG. 4 stores agent operation information for each agent in operation. The agent operation information includes an agent host name 301, an agent IP (Internet Protocol) address 302, an alternative server host name 303, an alternative server IP address 304, a business execution time zone 305, and a recovery priority 306.
The agent host name 301 and the agent IP address 302 indicate the host names and IP addresses of the agents in the main site 101 and the wide area 121.
The substitute server host name 303 and the substitute server IP address 304 indicate the host name and IP address of the agent of the remote site 111 assigned as the substitute server of the agent of the main site 101.
The business execution time zone 305 indicates a time zone during which the agent executes business. In the example of FIG. 4, 0 indicates night and 1 indicates other than that.
The restoration priority 306 indicates the restoration priority. For example, the lower the numerical value, the higher the restoration priority.
The agent operation information management table 300 is created in advance by an operator of the job management system 100 and registered in the manager 112 of the remote site 111. An operator of the job management system 100 can update the agent operation information management table 300 at an arbitrary timing.
The agent operation information management table 300 is an example of the agent operation information storage unit of the present invention.

The continuation status management table 400 in FIG. 5 stores business continuation permission / inhibition information. The business continuation permission / inhibition information includes a host name 401 and a business continuation permission / inhibition 402.
The host name 401 indicates the host names of agents in the main site 101 and the wide area 121.
The business continuation permission / inhibition 402 indicates whether or not it is possible to continue the business performed by each agent in the main site 101 and the wide area 121 when a disaster or the like occurs in the main site 101 and the business system is stopped due to a failure. Indicates. Here, “continuable” includes a case where switching to an alternative server can be continued. The business continuation permission / inhibition 402 is set by the business continuation status confirmation unit 211 as described later.
The continuation status management table 400 manages which agents in the main site 101 and the wide area 121 can continue the work that is being executed when a disaster occurs. For each agent for which an alternative server is prepared, after switching to the manager 112 at the remote site 111, information (job execution host name) indicating whether the agent at the main site 101 or the alternative server will continue to execute the job is provided. Managed by a job management system.

The job status information management table 500 in FIG. 6 stores job status information. The job status information includes a job ID 501, a job ID 502, a job name 503, an execution host name 504, and a status 505.
The job ID 501 is job identification information that uniquely identifies all jobs.
The business ID 502 is business identification information that is set for each business and uniquely identifies all businesses.
A job name 503 indicates the name of each job.
The execution host name 504 indicates the host name of the agent that executes each job. Note that job ID 501 = “1000” and job name 503 = “business group 1” are job nets in which jobs A to D are collectively defined and the order of operations is defined, so the execution host name 504 is blank.
A status 505 indicates the execution status of each job. The execution state includes normal end, abnormal end, and non-execution.
The job status information management table 500 is managed by a job management system. The job management system to which the disaster recovery system is applied is a part of the execution job information for each job being executed by each agent managed by the manager 102 during operation of the manager 102 of the main site 101. Information necessary for business handover is copied (duplicated) from the manager 102 of the main site 101 to the manager 112 of the remote site 111. The execution job information includes job status information. In other words, the job management system to which the disaster recovery system is applied transmits information necessary for taking over the job at the time of failure, including job status information, from the manager 102 of the main site 101 to the manager 112 of the remote site 111. The received information is stored in the storage device of the manager 112 of 111.
The job management system is an example of an execution job information copy unit according to the present invention.

FIG. 7 shows an example of the flow of business continuity status confirmation processing in the business continuity status confirmation unit 211.
The business continuation status confirmation process is started in the manager 102 of the remote site 101 when a disaster or the like occurs in the main site 101 and the business system 100 is stopped due to a failure.
The failure recovery support system 210 needs to recover while checking the status and status of each agent and job, so the manager 112 of the remote site 111 is activated in a job execution inhibited state so that the job is not executed arbitrarily (S101). ). Then, the business continuity confirmation unit 211 determines whether or not the agent can continue thereafter for each target agent (S102).
The business continuity confirmation unit 211 attempts to connect to the agent (S103) and determines whether or not connection is possible (S104). If the connection is possible (S104: Yes), the business continuity status checking unit 211 updates the continuation status management table 400 in FIG. On the other hand, when the connection is impossible (S104: No), the business continuity confirmation unit 211 determines whether there is an alternative server for this agent (S105). When there is an alternative server (S105: Yes), the business continuation status confirmation unit 211 updates the continuation status management table 400 with this agent being allowed to continue (S107). When the alternative server does not exist (S105: No), the business continuity status confirmation unit 211 updates the continuation status management table 400, assuming that this agent cannot be continued (S106). After the business continuity confirmation unit 211 completes the business continuity confirmation process for all the target agents, the recovery support unit 212 displays a GUI screen on the operator terminal 116 (S108).

FIG. 8 shows an example of the flow of recovery support processing in the recovery support unit 212.
The restoration support unit 212 displays an operation target list selection window 601 shown in FIG. 9 to be described later on the operator terminal 116 (S201), and the GUI screen of either the agent list window 700 in FIG. 10 to be described later or the job list window 800 in FIG. The operator is made to determine whether to operate with (S202).
When the operator selects the agent list window 700 (S202: Yes), the recovery support unit 212 displays the agent list window 700 of FIG. 10 on the operator terminal 116 (S203). The operator selects processing items (re-execution processing selection 701 and execution cancellation processing selection 702) while confirming information such as the business continuation permission 707 and the restoration priority 706 of each agent, and presses the confirm button 708 (S204). ). It is possible to process them individually, but you can select multiple agents that you want to handle in the same way and process them in a batch. When the confirmation button 708 is pressed, the recovery support unit 212 displays a recovery process confirmation message box 901 shown in FIG. 12 to be described later on the operator terminal 116 (S205), and allows the operator to determine whether to execute the recovery process (S206). . When the execution of the recovery process is not permitted (S206: No), the recovery support unit 212 returns to step S204. When the execution of the recovery process is permitted (S207: Yes), the recovery support unit 212 executes the selected process for all jobs executed by this agent (S207), and the job status is determined as a result. Update (S213).

  When the operator selects a job list in step S202 (S202: No), the recovery support unit 212 displays the job list window 800 of FIG. 11 on the operator terminal 116 (S208). While confirming information such as the status 806 of each job, the operator selects a processing item (re-execution process selection 801, execution cancellation process selection 802), and presses the confirm button 807 (S209). Although it is possible to process individually, it is also possible to select a plurality of jobs to be dealt with in the same way and process them in a batch. When the confirm button 807 is pressed, the recovery support unit 212 displays the recovery process confirmation message box 901 of FIG. 12 on the operator terminal 116 (S210), and causes the operator to determine whether to execute the recovery process (S211). When the execution of the recovery process is not permitted (S211: No), the recovery support unit 212 returns to step S209. When execution of the recovery process is permitted (S211: Yes), the recovery support unit 212 executes the selected process for this job (S212), and updates the status of the job with the result (S213). .

  FIG. 9 shows an example of the operation target list selection window 601 displayed in step S201. The operation target list selection window 601 allows the operator to select an agent list or a job list as a target to be restored. The operator presses the agent list button 602 when displaying the agent list window 700 in step S203, and presses the job list button 603 when displaying the job list window 800 in step S208.

FIG. 10 shows an example of the agent list window 700 displayed in step S203.
The recovery support unit 212 creates an agent list window 700 based on the agent operation information stored in the agent operation management table 300 in FIG. 4 and the business continuity information stored in the continuation status management table 400 in FIG. To do.
The agent list window 700 is a selection column for processing items (re-execution processing selection 701, execution cancellation processing selection 702), agent host name 703, alternative server host name 704, business execution time zone 705, recovery priority 706, and business continuity availability. A display field 707 and a confirmation button 708 are included.
The re-execution process selection 701 is a column for selecting an agent when the job is re-executed for all jobs having the execution destination. When the operator selects the re-execution process selection 701, a check mark is displayed.
The execution cancellation process selection 702 is a column for selecting an agent when the execution is canceled for all jobs having the agent as an execution destination. When the operator selects the execution cancellation process selection 702, a check mark is displayed.
The agent host name 703 indicates the host name of the agent that executes the job.
The substitute server host name 704 indicates the host name of the agent of the remote site 111 assigned as the substitute server.
The business execution time zone 705 indicates a time zone during which the agent executes business. In the example of FIG. 10, 0 indicates nighttime and 1 indicates the other.
The recovery priority 706 indicates the recovery priority. For example, the lower the numerical value, the higher the recovery priority.
The business continuation permission / inhibition 707 indicates whether or not the business performed by the agents in the main site 101 and the wide area 121 can be continued. Here, “continuable” includes a case where switching to an alternative server can be continued.
The confirm button 708 is pressed when the operator finishes selecting the processing items (re-execution process selection 701 and execution cancellation process selection 702).
The agent list window 700 guides work for recovery support. The operator can perform individual or collective operation switching operations for jobs that are executed by each agent while confirming information for determining the operation continuation method for each agent using the GUI in the agent list window 700. Can be done.

FIG. 11 shows an example of the job list window 800 displayed in step S208.
The recovery support unit 212 displays the job list window 800 based on the job continuation permission information stored in the continuation status management table 400 in FIG. 5 and the job status information stored in the job status information management table 500 in FIG. create.
The job list window 800 includes a selection column for processing items (re-execution processing selection 801 and execution cancellation processing selection 802), a job name 803, an agent host name 804, a business continuation propriety 805, a status 806 display column, and a confirmation button 807. Including.
The re-execution process selection 801 is a column for selecting a job when the specified job is re-executed. When the operator selects the re-execution process selection 801, a check mark is displayed.
The execution cancellation process selection 802 is a column for selecting a job when the execution of the designated job is to be canceled. When the operator selects the execution cancellation process selection 802, a check mark is displayed.
A job name 803 indicates the name of the job.
The agent host name 804 indicates the host name of the agent that executes the job.
The business continuation permission / inhibition 805 indicates whether or not the business performed by each agent in the main site 101 and the wide area 121 that has executed the job can be continued. Here, “continuable” includes a case where switching to an alternative server can be continued.
A status 806 indicates the job execution status when a failure occurs.
The confirm button 807 is pressed when the operator finishes selecting the processing items (re-execution process selection 801 and execution cancellation process selection 802).
The job list window 800 guides work for recovery support. The operator can perform an operation switching operation individually or collectively while checking the status of each job using the GUI.

  FIG. 12 shows an example of the recovery process confirmation message box 901 displayed in steps S205 and S210. The operator presses the YES button 902 when the execution of the recovery process is permitted in step S205 and step S210, and the NO button 903 when the execution of the recovery process is not permitted.

  As described above, according to the present invention, when an operator performs an operation switching operation, the restoration work can be performed by displaying a list of information necessary for taking over the operation in the job management system in a GUI or performing a batch operation. Can be guided. For this reason, when the business system is stopped and the operator performs the operation switching operation, it is possible to reduce the work load on the operator and shorten the recovery time of the business system.

DESCRIPTION OF SYMBOLS 100 ... Business system, 101 ... Main site, 102 ... Main site manager, 103 ... Main site job management DB, 104, 105 ... Main site agent, 111 ... Remote site, 112 ... Remote site manager, 113 ... Remote site job management DB, 114, 115 ... Remote site agent, 116 ... Operator terminal, 121 ... Wide area, 122, 123 ... Wide area agent, 200 ... Job management unit, 210 ... Failure recovery support system, 211 ... Business continuity confirmation unit 212 ... Recovery support unit, 300 ... Agent operation information management table, 400 ... Continuation status management table, 500 ... Job status information management table, 601 ... Operation target list selection window, 700 ... Agent list Indou, 800 ... job list window, 901 ... recovery process confirmation message box

Claims (2)

A manager and a plurality of agents arranged at the main site, a manager and a plurality of agents arranged at a remote site as an alternative server for the manager and each agent of the main site, and a plurality arranged at an area other than the main site and the remote site Execution job information on each job being executed by each agent of the main site managed by the manager and each of the agents other than the main site and the remote site during operation of the manager of the main site Among these, a job having an execution job information copy means for copying information necessary for taking over the job at the time of failure from the manager at the main site to the manager at the remote site In the stem, a failure recovery support system that operates the manager of the remote site,
Agent operation information storage means for storing agent operation information including information indicating each agent at the remote site assigned as an alternative server for each agent at the main site;
When a failure occurs, each agent managed by the manager at the main site checks whether it is possible to continue the work that was being executed before the failure, and indicates whether the agent can continue the work. Business continuity confirmation means for creating availability information;
For each agent managed by the manager of the main site at the time of failure, the operation continuity information and the agent operation information stored in the agent operation information storage means are displayed, and the re-execution target input by the operator And agent recovery support means for accepting the selection of the agent to be canceled and re-executing the agent selected as the target for re-execution and canceling the execution of the agent selected as the target for execution cancellation.
A failure recovery support system comprising: The execution job information includes, for each job being executed by each agent managed by the manager of the main site, job status information indicating the agent executing the job and the execution status of each job,
For each job executed by each agent managed by the manager of the main site at the time of failure, the business continuity information of the agent executing the job and each copy copied by the execution job information copying means The job status information of the job is displayed, the selection of the job to be re-executed and the target of execution cancellation input by the operator is accepted, the job selected for re-execution is re-executed, and the job is selected for execution cancellation Job recovery support means for canceling job execution,
The failure recovery support system according to claim 1, further comprising:

Images