JPH04241037A - Automatic error recovery system - Google Patents

Abstract

PURPOSE:To enable an automatic recovery process in a system which must not stop throughout 24 hours even when plural kinds of device are connected to one adapter and there are plural device control parts for each of the devices. CONSTITUTION:The system wherein the various devices 3 are connected through the adapters 2 has adapter control parts 4 and adapter control parts, which control the states of the adapters, placed between a device control part 1 which controls the devices 3 and the adapters is provided with a means which holds information on the trouble state of an adapter 2 by the adapter control part 4 and informs the adapter control part 5 of the trouble information, a means which resets the adapter corresponding to the trouble state and initially sets the adapter, and a means which reports the trouble recovery of the adapter to all device control parts connected to the adapter.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention provides that various devices have a common part,
For example, in a computer system that is connected to multiple device control units via adapters and cannot be stopped for 24 hours, even if a failure occurs in the adapter, even if the failure is minor, is an automatic error recovery method that automatically recovers from failures without stopping the application (system), and even in the event of a severe failure, it is possible to stop only the failed part and replace the failed hardware. and device management methods.

[0002] In recent computer systems,
As they become larger and more complex, even in the event of a failure,
In order to recover from a fault, it is required to repair the fault without stopping the entire system, and also to repair the fault without stopping the entire system even when a serious hardware fault that requires a shutdown occurs.

[0003] For this reason, in recent computer systems, even if the adapter becomes inoperable, measures are taken to automatically recover it as much as possible, and if recovery is not possible, the necessary minimum It is necessary to be able to take measures such as temporarily stopping only the limited parts and replacing them.

0004

2. Description of the Related Art FIG. 4 is a diagram illustrating problems in a conventional computer system. As shown in this figure, various devices 3 are connected to multiple device control units via an adapter 2.
In the computer system connected to 1, the device A control section 1a and the device B control section 1b control the device A 3a and the device B 3b independently of each other. Device A 3a
If the adapter 2 detects an inoperable state during a control operation for the computer, the conventional system would temporarily stop the computer system and call the administrator (operator) who manages the computer system to take action. .

However, the problem with adapter 2 is as follows.
Relatively light adapter firmware (R inside the adapter
These errors can be classified into two types: errors in the hard control program (hardware control program stored in the adapter) or microprogram (hardware control program stored in the ROM inside the adapter), and serious hardware failures in the adapter itself. An operator-mediated recovery method would shut down the entire system, regardless of the severity of the fault.

[0006]

[Problem to be Solved by the Invention] Therefore, even if the fault is a minor one that is recoverable, the entire computer system must be stopped to perform recovery work, and it takes a long time to recover. was occurring.

[0007] Therefore, even if a failure occurs in the adapter, it is necessary to take measures to automatically recover as much as possible, for example, by performing various recovery processes corresponding to the failure situation, and in case recovery is not possible. is a mechanism that allows you to take measures such as temporarily stopping and replacing only the minimum necessary parts.

However, when attempting to provide such a mechanism in the device control section, the following problems occur. As shown in Figure 4 above, one adapter 2 has multiple types of devices A.
3a, device B 3b is connected, and the device A
3a, and a plurality of device control units 1 for device B 3b, for example, as shown in the figure, device A control unit 1a
, if there is a device B control unit 1b and a failure related to the adapter 2 is detected in the device A control unit 1a,
The device A control unit 1a can automatically recover from a fault in the adapter 2 by resetting the adapter 2 and performing initial settings for the adapter 2 and initial settings for the device A 3a.

However, after resetting the adapter 2, it is necessary not only to initialize the adapter but also to initialize all connected devices A, B 3a, and 3b. , device A control section 1
a and other device control units, in this example, device B control unit 1
Since B operates independently, the other device B control unit 1b has no means of recognizing that such recovery processing has been performed, and is unable to initialize device B 3b. A problem arises in that the device B 3b cannot be controlled.

In view of the above-mentioned drawbacks of the conventional art, the present invention automatically recovers from failures in the adapter as much as possible, and if recovery is not possible, takes measures such as temporarily stopping and replacing only the minimum necessary parts. The purpose is to provide a mechanism that makes it possible.

[0011]

[Means for Solving the Problems] FIG. 1 is a diagram illustrating the principle of the present invention. The above problems are solved by an automatic error recovery method configured as follows.

In a computer system in which various devices 3 are connected to a plurality of device control units 1 via one of a plurality of adapters 2, the device control unit 1 and the plurality of adapters 2 that control the various devices 3 are connected to each other. An adapter control unit 4 and an adapter management unit 5 for uniformly managing the status of each adapter 2 are placed in between, and the adapter control unit 4 retains at least information on various failure states of the adapters 2. , means for notifying failure information of the adapter 2 to the adapter management unit 5 according to instructions from the adapter management unit 5, and control instructions corresponding to various failure states of the adapter 2 from the adapter management unit 5, at least: A means for resetting the adapter 2 and re-initializing the adapter 2; and a means for resetting the adapter 2;
When the fault condition of the adapter is recovered, the adapter 2
means for notifying all of the device control units 1 that control the devices 3 connected to the adapter 2 of a failure recovery notification, and the device control unit 1
When controlling the corresponding device 3 via the adapter 2, the adapter control unit indicates that a failure has occurred in the adapter 2.
When detected by 4, it notifies the adapter management unit 5 of the adapter type information and failure information, and the adapter management unit 5
5 determines a countermeasure corresponding to the notified adapter type information and failure information, instructs the adapter control unit 4 to take the countermeasure, and the instructed adapter control unit 4 executes the corresponding recovery process. At the same time, before the execution of the recovery process is started, all devices connected to the adapter 2 are notified that the adapter 2 has been reset, and after the execution is completed, that the recovery process has been completed. After receiving the completion notification of the recovery process, the device control unit 1 performs the necessary recovery process for each device 3 to recover from the failure of the adapter 2 without stopping the computer system. Configure it to do so.

[0013]

[Operation] In the present invention, in a computer system in which various devices 3 are connected to a plurality of device control units 1 via one of a plurality of adapters 2, various devices 3
The device control unit 1 that controls the above and the plurality of adapters
2, an adapter control unit 4 and, at the same time, an adapter management unit that uniformly manages the status of each adapter 2.
5 will be provided.

The adapter 2 is a composite adapter, and a plurality of types of devices 3a and 3b are connected to one adapter 2, and a device control unit 1 for the device 3 is connected to the adapter 2.
There are also multiple a and 1b.

When the present invention is implemented, the device control section 1
does not directly control the adapter 2, but does everything via the adapter control section 4. That is, the adapter control section 4 is located between the device control section 1 and the adapter 2, and all controls for the adapter 2 are performed here. It also has an interface with the device control units 1a, 1b of all the devices 3a, 3b connected to the adapter 2. It also has an interface with the adapter management section 5.

The adapter management section 5 notifies the adapter control section 4 of control information regarding the adapter 2 and instructs the adapter control section 4 to control the adapter 2 . Also, refer to the data of adapter management information 6.

The adapter management information 6 determines what action the adapter management section 5 should take based on the type of the adapter 2 notified by the adapter control section 4 and failure information on the adapter 2. This is information for

Normally, each device control section 1a, 1b uses the adapter 2 via the adapter control section 4. If a failure occurs in the adapter 2, it is detected by the adapter control unit 4, and the adapter control unit 4 notifies the adapter management unit 5 of the occurrence of the failure. Applicable adapter 2
The adapter management unit 5, which has been notified of the failure occurrence, refers to the adapter management information 6 and performs necessary recovery processing based on the type of adapter and the failure information notified from the adapter control unit 4. instruct.

When all the recovery processing is completed and the adapter 2 becomes usable again, the adapter control section 4 sends a message to all the device control sections 1a and 1b of the devices 3a and 3b connected to the adapter 2. Notifies that recovery processing for adapter 2 has been performed.

[0020] With this notification, each device control unit 1a, 1
b can recognize that automatic recovery processing has been performed for a failure that occurred outside of his own operation, and can perform necessary initial settings for each device 3a and 3b.

In this way, even when multiple types of devices 3a and 3b are connected to one adapter 2 and there are multiple device control units 1a and 1b for the devices 3a and 3b, automatic recovery is possible. The effect of making it possible to perform processing is obtained.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Examples of the present invention will be described below in detail with reference to the drawings. The above-mentioned FIG. 1 is a diagram explaining the principle of the present invention, and FIGS. 2 and 3
2 is a diagram showing an embodiment of the present invention, FIG. 2 shows the operation during normal operation, and FIG. 3 shows the operation when a failure is detected.

In the present invention, various devices 3a, 3b
is connected to multiple device controllers via one of multiple adapters 2.
1a and 1b, an adapter control section 4 is provided between the device control section 1 that controls various devices 3a and 3b and the plurality of adapters 2, and at the same time, the state of each adapter 2 is controlled. An adapter management section 5 is provided to manage the adapter in a unified manner.
If a failure occurs in the adapter 2, it is detected by the adapter control unit 4, and the adapter control unit 4 notifies the adapter management unit 5 of the failure occurrence, and the adapter management unit 5 is notified of the failure occurrence of the adapter 2. refers to the adapter management information 6 and instructs the adapter control unit 4 to perform necessary recovery processing based on the type of adapter and fault information notified from the adapter control unit 4. When all the recovery processing is completed and the adapter 2 becomes usable again, the adapter controller 4 sends the adapter to all the device controllers 1a and 1b of the devices 3a and 3b connected to the adapter 2. The means for notifying that the recovery process in step 2 has been performed is necessary for carrying out the present invention. Note that the same reference numerals indicate the same objects throughout the figures.

The automatic error recovery system of the present invention will be described below with reference to FIG. 1 and FIGS. 2 and 3. First, the functions of the adapter control section 4 and the adapter management section 5 will be explained.

The adapter control unit 4 is realized as a common adapter driver (program). The common adapter driver is a driver for controlling adapters in accordance with the common adapter specifications, and various requests from the upper device control units 1a and 1b to all adapters 2 are sent to the adapter control in the form of function calls, for example. The adapter control unit 4 interprets the request, converts it into a common adapter specification format, and issues it to the adapter 2. In addition, each device control unit 1a, 1b is an adapter control unit 4.
It has an entry for information notification. Adapter management section 5
The interface between the two is realized, for example, as a character interface.

The adapter control section 4 has the following specific functions. - Adapter initial setting function: Reset adapter 2. After the reset is complete, the system's operating system (O
Set the interrupt level obtained from S) to adapter 2.・Controlled device (device) Registration function: The device 3 to be controlled by the device control unit 1 is connected to the adapter control unit 4
Register for.・Function for issuing commands to adapters: Converts device control requests from device control unit 1 into command format of common adapter specifications, and when issuing commands, monitors the status of output destination adapter 2 and outputs them. Issues an adapter operation when enabled.・Adapter failure detection function: The adapter control unit 4 has an adapter failure detection function.
For example, if the error bit of adapter 2 is set,
It has a function to detect when the adapter's busy bit does not fall for a certain period of time or when an error occurs on the access bus to adapter 2.・Interface function with adapter management department: adapter
It has a character interface with an adapter management unit 5 that manages various information of 2, for example, status information, failure information, etc., and performs processing according to firmware loading instructions, dump downloading instructions, status requests, etc. from the adapter management unit 5. Notify processing results. - Adapter/device attach/detach function: In response to a request from the adapter management program and adapter management command of the adapter management unit 5, the specified adapter 2 can be placed in a detached state (use of the device control unit is not permitted), Provides a function to attach a detached adapter (make it available for use by the device control unit). It also provides a function to notify the device control unit 1 of detach and attach requests to each device 3 from the adapter management program and adapter management commands.

[0027] Adapter management unit 5 and adapter control unit
4, for example, as described above.
This is realized using a character interface. Adapter management information 6 is realized as one file,
It is located in user space. This allows the user to easily change the adapter management information 6.

The operation of this system is performed in the following sequence. (1) Initial Processing ■ Initial Setting of Adapter The adapter control unit 4 performs initial setting processing of the adapter 2, such as resetting the adapter 2, loading firmware, and setting the interrupt level.

■ Registration of controlled devices Each device control unit
Devices 3a and 3b to be controlled by 1a and 1b are registered in the adapter control unit (more specifically, the common adapter driver, hereinafter omitted) 4. At this time, an entry for information notification from the adapter control unit 4 is also registered.

① Initial Setting of Controlled Devices Each device control section 1a, 1b performs initial setting of the devices 3a, 3b to be controlled. (2) Processing in normal state (see FIG. 2) a: The application program (application) issues various requests to the system.

b: If the request requires input/output to the device A 3a, the device A controller 1a issues a device control request to the adapter controller 4. c: Adapter control unit 4 converts the request from the device A control unit 1a into an adapter command, and sends the request to the adapter C 2c.
After confirming that the device A is ready for operation,
3a issues a command to the adapter C 2c connected to it. At this time, if the adapter C 2c is in an inoperable state, this is detected, and processing is started when an adapter failure is detected. If the command is successfully issued, the process returns to the device A control unit 1a as a normal termination.

d: When the operation of the command is completed normally,
As described above, it returns to the device A control unit 1a. e: Device A control unit 1a notifies the application program that the request has been successfully completed.

f: During normal processing, the above processes a to e are repeated. (3) Processing when an adapter failure is detected (see FIG. 3) a: The application program issues various requests to the system.

b: If the request requires input/output to the device A 3a, the device A controller 1a issues a device control request to the adapter controller 4. c: Adapter control unit 4 converts the request from the device A control unit 1a into an adapter command, and sends the request to the adapter C 2c.
After confirming that the device A is ready for operation,
3a issues a command to the adapter C 2c connected to it. At this time, if the adapter C 2c is in an inoperable state, this is detected, and processing is started when an adapter failure is detected.

The adapter control unit 4 detects adapter failures such as a state in which an error bit of the adapter is set, a state in which a busy bit of the adapter does not fall for a certain period of time, a state in which an error has occurred in the access bus to the adapter, etc. .

g: The adapter control unit 4 notifies the adapter management unit 5 that the adapter C 2c is inoperable. h: The adapter management unit 5 refers to the data of the adapter management information 6 based on the notified failure information, and
Determine what to do with adapter C 2c .

i: The adapter management unit 5 instructs the adapter control unit 4 to take measures such as resetting the hardware of the adapter C 2c using an IO control command. j: Adapter control unit 4 performs instructed processing, ie, processing such as hardware reset of adapter C 2c.

k: adapter control unit 4 is adapter C
For each device 3a, 3b using adapter C
2c The fact that the reset has been performed is notified using the registered information communication entry. The device control units 1a, 1b that have received the notification suspend device control requests for the devices 3a, 3b.

l: The adapter control unit 4 notifies the adapter management unit 5 that the instructed process has been completed. m: Adapter management unit 5 is an adapter control unit.
Instructs the initial setting procedure.

n: Adapter control unit 4 performs the instructed processing. o: The adapter control unit 4 notifies all devices 3a using the adapter C 2c that the instructed processing has been completed normally and the adapter C 2c has become operational again.
, 3b, the notification is made using the entry registered in the device control unit 1a, 1b.

p: Device control units 1a and 1b perform necessary recovery processing for each device 3a and 3b. After the recovery processing from g to p is performed for the adapter failure detected in q:c, a device control request is sent to the adapter control unit 4 again, and the flow returns to the normal processing flow.

By performing the recovery process as described above,
If recovery is not possible, it is possible to take measures such as temporarily stopping and replacing only the minimum necessary parts. in this way,
In the present invention, various devices 3a, 3b are connected to a plurality of device control units 1a, 1b via one of a plurality of adapters 2.
In the computer system connected to the device control unit 1a, 1 that controls various devices 3a, 3b,
Between b and the plurality of adapters 2, an adapter control section is installed.
4, and at the same time, an adapter management section 5 that uniformly manages the status of each adapter 2 is provided.
If a failure occurs in the adapter, it is detected by the adapter control unit 4, and the adapter control unit 4 notifies the adapter management unit 5 of the occurrence of the failure, and the adapter management unit 5 that has been notified of the failure occurs in the adapter. Based on the type of adapter and fault information notified from the control section 4, the adapter control section 4 refers to the adapter management information 6 and instructs the adapter control section 4 to perform necessary recovery processing. Then, all recovery processing is completed,
Once the adapter 2 is available again, the adapter control 4 will notify the device connected to the adapter 2.
The device controllers 1 and 3 are notified that the recovery process for adapter 2 has been performed.
1a and 1b recognize the failure of adapter 2, perform the necessary recovery processing on the corresponding devices 3a and 3b, and issue a device control request again to the adapter via the adapter control unit 4.
The feature is that it can be sent to 2.

[0043]

As explained above, according to the present invention, in a computer system in which various devices are connected to a plurality of device control units via one of a plurality of adapters, device control for controlling various devices can be performed. An adapter control unit and an adapter management unit that uniformly manage the status of each adapter are placed between the adapter and the adapter, and the adapter control unit holds information such as failure status of the adapter and transmits the failure information. means for notifying the adapter management unit; means for resetting the adapter in response to the failure state and re-initializing the adapter according to instructions from the adapter management unit; If a failure occurs in the adapter, the adapter control unit detects the occurrence of a failure, and the adapter control unit notifies the adapter management unit of the occurrence of the failure. The adapter management unit, which has been notified of the failure of the adapter, refers to the adapter management information and performs necessary recovery processing on the adapter, based on the type of adapter and failure information notified by the adapter control unit. Instructs the control unit. When all the recovery processing is completed and the adapter becomes usable again, the adapter control unit notifies all device control units of the devices connected to the adapter that the adapter recovery processing has been performed. In this way, each device control unit can recognize a fault in an adapter, perform necessary recovery processing for the corresponding device, and send a device control request to the adapter again via the adapter control unit. As a result, there are device control units that operate independently of each other, and even if a failure occurs in the common part, recovery processing can be performed automatically, and if a failure occurs in the adapter. Even in the case of minor problems, there is no need for operator intervention, and the application (
Since recovery processing can be performed automatically without stopping the system, it is possible to shorten the downtime when a system failure occurs and reduce the number of system outages due to system failure.

[Brief explanation of the drawing]

[Fig. 1] Diagram explaining the principle of the present invention

[Fig. 2] Diagram showing one embodiment of the present invention (Part 1)

[Figure 3
] Diagram showing an embodiment of the present invention (Part 2)

[Figure 4] Diagram explaining problems with conventional computer systems

[Explanation of symbols]

1 Device control section
1a Device A control section 1b Device B control section
2 Adapter 2c Adapter C
3 Device 3a Device A
3b Device B4 Adapter control unit 5 Adapter management unit 6 Adapter management information a to p Each operation

Claims (1)

[Claims] [Claim 1] Various devices (3) include a plurality of adapters (2)
In a computer system connected to a plurality of device control units (1) via one of the various devices (3)
Between the device control unit (1) that controls the adapters (1) and the plurality of adapters (2), there is an adapter control unit (4) and an adapter management unit (4) that uniformly manages the status of each adapter (2). 5) The adapter control unit (4) retains at least information on various failure states of the adapter (2), and the adapter management unit (5) stores the information on the adapter (2) according to instructions from the adapter management unit (5). Part (5
) and control instructions corresponding to various failure states of the adapter (2) from the adapter management unit (5), at least reset the adapter (2) and change the initial settings of the adapter (2). A means of repair,
When the failure state of the adapter is recovered by the above means, the failure recovery notification of the adapter (2) is sent to all the device control units (1) that control the devices (3) connected to the adapter (2). ) when a failure occurs in the adapter (2) while the device control unit (1) is controlling the corresponding device (3) via the adapter (2). The adapter control part (
4) When detected, the adapter management unit (5) is notified of the adapter type information and fault information, and the adapter management unit (5) receives the notified adapter type information,
It determines a countermeasure corresponding to the failure information and instructs the adapter control unit (4) about the countermeasure, and the instructed adapter control unit (4) executes the corresponding recovery process and also performs the recovery process. Before starting execution, the adapter (2)
After the execution is completed, it notifies all device control units (1) connected to the adapter (2) that the recovery process has been completed, and sends a notification that the recovery process has ended. The device control unit (1) that received the
An automatic error recovery method characterized by performing necessary recovery processing to recover from a fault in the adapter (2) without stopping the computer system.