JP2968484B2 - Multiprocessor computer and fault recovery method in multiprocessor computer - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multiprocessor computer, and more particularly to an improvement in reliability and availability in a computer in which components are not multiplexed.

[0002]

2. Description of the Related Art There is a fault-tolerant computer as a computer that does not hinder the operation of the entire system even if a hardware failure or a software failure occurs in a part of the system. This fault-tolerant computer is useful in constructing a highly reliable system that is resistant to failures, and provides a particularly desirable system form to a core part of the computer system.

[0003] High reliability and high availability of a conventional fault-tolerant computer are realized by multiplexing hardware components. Each component is provided as an on-line exchange module which can be separated during operation of the system. When a failure occurs in a computer, the system detects the failure location and isolates the failure location on a component basis. At that time, the fault location can be repaired while the operation of the system is continued by other components. That is, it is possible to repair a faulty part without logically bringing down the system.
Of these, the processors mounted on the fault-tolerant computer are naturally multiplexed for high reliability, but they are so-called “pair-and-spare” having two sets of two processors. The configuration or the configuration in which three or more processors are provided to take a majority decision are executed while comparing the results output by the processors.

[0004] A server system constructed by a general personal computer (PC) may have a plurality of processors like a fault-tolerant computer, but in the server system, in order to improve performance. Is provided with a plurality of processors. That is, since a multiplexed configuration such as a fault-tolerant computer is not used, if any failure occurs in the server system, the system must be stopped regardless of whether a failure location can be detected.

[0005]

In order to pursue high reliability and high availability of the above-mentioned PC server system, it is conceivable to construct such that a plurality of processors are multiplexed similarly to a fault-tolerant computer.

However, when processors are multiplexed, a large number of components are required not only for the processor but also for its peripheral circuits, which increases the circuit scale. This leads not only to an increase in the size of the CPU board on which the processor is mounted, but also to an increase in price and an increase in power consumption.
Further, since the multiplexing of the processors requires a comparison circuit at the time of majority decision, the frequency of the processor cannot be increased, and the performance cannot be improved.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a multiprocessor computer which improves reliability and availability without multiplexing components such as processors. It is in.

[0008]

In order to achieve the above object, a multiprocessor computer according to the present invention relates to a method for determining whether data has been modified or not.
Built-in cache memory that holds status information
At least one processor having
Includes and holds the contents of the flash memory for each line
Status information on whether the modified data has been modified
And the owner of each line where data is stored
An external cache that holds the identification information of the processor and each line
Memory, the processor and the external cache
Processor that connects memory and performs control management for each
With a bridge and a self-failure detection function
And one CPU board, main memory,
Based on the release condition, the failure location can be
Isolation determination means for determining whether isolation is possible;
Failure point when separation determination means determines that isolation is possible
Isolation executing means for isolating the isolation during system operation;
If the determination means determines that isolation is not possible,
Restarting means for stopping and restarting,
The processor bridge is a processor on the same CPU board.
External reference flag information assigned to each process
Cleared at the time of switch, and inside the same CPU board
Modification data owned by the processor only held
Is reflected on the outside of the same CPU board.
The isolation determining means refers to the external reference flag information.
From the processor that failed due to
Judge whether the influence of is transmitted to the outside,
The quarantine execution means is configured so that the quarantine determination means transmits the influence to the outside.
If not, the identification information of the processor
Equipped with the faulty processor obtained based on the information
Of the failure held in the CPU board
After invalidating all of the data modified by the processor,
The faulty processor is separated.

Further, the processor bridge is provided for each processor.
Write correction data in the built-in cache memory of Sessa
Means for receiving a return processing start request for each processor
And before being processed by the received start request
The processor stores the external cache memo of the correction data.
Issue a write-back processing request to perform write-back processing to the
Means for performing the process when the process switch is performed.
Stored in the internal cache memory of the target processor
Write the modified data back to the main memory
It is.

[0010] Further, the isolation determination means may be configured to generate the fault.
The external reference flag information corresponding to the processor is clear.
The influence from the failed processor is transmitted to the outside.
It is determined that it has not changed.

The processor bridges may have the same C
Other processes assigned to each processor on the PU board
Clear the process reference flag information at the time of process switch.
Before the correction data owned by the processor is stored
The owner of the external cache memory line
Other processors on the same CPU board
Is set when the other processor
Failure occurred by referring to the reference flag information
The influence from the processor is on the same CPU board
Determines whether it has been transmitted to another processor, and
The quarantine execution means may be such that the isolation determination means
The effect is transmitted to other processors on one CPU board
If it is determined that there is no
Equipped with the failed processor obtained based on the
The fault occurrence program held in the CPU board
After invalidating all the data corrected by Sessa,
It separates the harm-causing processor.

[0012] Further, the isolation judging means may be configured to execute the fault occurrence.
Other processor reference flag information corresponding to the processor is
Rear, the effect from the failed processor is
Transmitted to other processors on the same CPU board.
It is determined that there is not.

[0013] Further, the isolation determination means may include a fault generated by the fault occurrence processor by a self fault detection function.
Upon receiving the detection signal, the faulty processor is
It is separated from the processor bridge.

[0014] Further, the isolation execution means may be configured to execute each of the processes.
Whether the processor has been disconnected from the processor bridge
Is stored.

[0015] The processor bridge may include the remote controller.
It has separation determining means.

Further, the processor bridge is provided with the
It has separation execution means.

According to another aspect of the present invention, there is provided a multiprocessor computer.
Failure recovery method is to transfer data to the data
Retained along with status information on the presence or absence of correction
At least one processor having a storage cache memory
And the data held in the internal cache memory.
Modify data that is included and retained for each line
Status information and data on the presence or absence of positive
Identification information of the processor that is the owner of each line
An external cache memory for each line,
Connecting the processor and the external cache memory,
Each control and management, and on the same CPU board
External reference flag information assigned to each processor
Cleared when process switch, same CPU board
Processor-owned fixes that are only kept inside the
When data is reflected outside the same CPU board
Equipped with a processor bridge to set
At least one CPU board with harm detection function
External reference flag in a multiprocessor computer
Failure to refer to the information during system operation
The effect of the processor that was generated is not transmitted to the outside
An isolation determination step for determining whether or not the
Influence from the failed processor in the fixed step
System is determined not to be transmitted to the outside
Said faulty processor mounted without down
The faulty processor held in the CPU board
After invalidating all the data corrected by the
An isolation execution step of isolating the raw processor;
In the determination step, the shadow from the faulty processor
System is determined to be transmitted to the outside
Including a restarting step of stopping and restarting.
It is.

Further, the processor bridges have the same C
Other processes assigned to each processor on the PU board
Clear the process reference flag information at the time of process switch.
Before the correction data owned by the processor is stored
The owner of the external cache memory line
Other processors on the same CPU board
When set when the
Failure by referring to processor reference flag information
The same CPU is affected by the processor
To see if it has propagated to other processors on the board.
Determining, and the isolation execution step includes the isolation determination step.
Of the failed processor
Also affects other processors on the same CPU board.
If it is determined that the
The CPU board on which the failed processor is mounted.
Fixed the failed processor held in the code
After disabling all of the failed data,
It is to separate the Sassa.

The external cache memory may be
It has a structure that can be divided into a
The stage is not affected by the faulted way.
External to the CPU board with the external cache memory
The quarantine execution means
Means that the isolation determination means is affected by the faulty way.
Has not been transmitted to the outside of the CPU board
In the case, all repairs stored in the external cache memory
After invalidating the positive data, degenerate the faulty way
Things.

[0020] Further, the isolation judging means is configured to generate the fault.
Of the processor that owns each line in the way
Assigned to the processor identified by the identification information
By referring to the external reference flag information,
The correction data stored in the raw way is externally reflected.
This is to determine whether or not it is in operation.

Further, a multiprocessor meter according to another invention is provided.
The failure recovery method in the computer is to transfer the data to the data.
Retained along with status information on whether or not there has been a modification
At least one processor having a built-in cache memory
Can be divided into processors and ways that include multiple lines
Having a simple structure and stored in the internal cache memory.
Data that contains and holds data for each line
Status information and data on whether data has been modified
Of the processor that is the owner of each line for which
External cache memory that holds identification information for each line
And the processor and the external cache memory
Connect, perform control and management of each, and use the same CP
External reference file assigned to each processor on the U board
Lag information is cleared at the time of process switch and is the same
The processor held only inside the CPU board
The modified data owned by the server to the outside of the same CPU board.
Install the processor bridge that is set when reflected.
At least one CP having a self-failure detection function
In a multiprocessor computer having a U board,
System operation by referring to section reference flag information
Correction data stored in the way that failed during
The CPU having the external cache memory mounted thereon;
Quarantine to determine if it is traveling outside the board
And C in the determination step and the isolation determination step.
When it is determined that it is not transmitted outside the PU board
The external cache memory can be
After invalidating all stored correction data,
An isolation execution step of degenerating a raw way, and the isolation determination
Transmitted to the outside of the CPU board in the step
If it is determined that the
Re-startup step.

[0022]

[0023]

[0024]

[0025]

Preferred embodiments of the present invention will be described below with reference to the drawings.

Embodiment 1 FIG. 1 is a diagram showing an embodiment of a multiprocessor computer according to the present invention.
In the present embodiment, a PC server system will be described as an example of a multiprocessor computer. The server system includes two CPU boards 2 and 20 on a system bus 1.
, An I / O bridge 4 and a main storage device (main memory) 6. Each CPU board 2 has the same configuration, and includes processors 10, 11, a CPU bridge 12, and an external cache 14, respectively. I /
An external storage device 8 such as a disk device is connected to the O-bridge 4 via an I / O controller 7. The CPU board 2 according to the present embodiment has a function of being able to self-detect a failure that has occurred therein and to identify the location of the failure. Further, the server system has a failure detection function for identifying the external storage device in which the failure has occurred. Some of the failure detection functions are performed by hardware, while others are performed by software.

Next, a basic operation when a failure occurs in this embodiment will be described with reference to the flowchart shown in FIG.

When a failure occurs in the server system (step 100), the location of the failure is detected (step 200). As a result, if a failure point cannot be detected, or even if it can be detected, if the data handled internally has an inconsistency, the operation cannot be continued and the system is stopped (step 500). In the case where data inconsistency occurs, data consistency cannot be maintained.
For example, there is a case where another processor continues the operation using the operation result of the failed processor. In such a case, the system must be brought down because it is necessary to undo and redo the processing.

When a fault location can be detected, it is determined whether the fault location can be isolated (step 300). Being separable means that the component containing the target fault can be separated while the operation is continued without bringing down the system and without causing inconsistency in the internal data. If it can be isolated, the fault location is separated from the system in units of separable components (step 400). As a result, if the failure location is the input / output device, it is possible to prevent access from others, and if the failure location is a processor, it is possible to prevent processing from being performed.

If it is determined that a faulty location cannot be isolated even if a faulty location can be detected, the system is explicitly stopped because it is impossible to continue the operation of the system. If the fault location is related to hardware, the component is replaced if the component can be removed, and then the system is restarted (step 600).

As described above, according to the present embodiment, when a failure occurs, if the consistency of the internal state of the system can be maintained, the failure can be isolated without shutting down the system and operating the system as it is. To continue. If the consistency of the internal state of the system cannot be maintained, the system is temporarily shut down to isolate the fault location, and then the system is started. As described above, it is determined whether or not it is necessary to bring the system down to isolate the fault location before the isolation, so that there is no need to bring down the system unnecessarily. As a result, the reliability and availability of the system can be improved without multiplexing components.

Incidentally, a system down can occur not only due to a hardware failure, but also due to a bug in software (OS, middleware, application program, etc.) or an erroneous operation by an operator. System down due to this software or the like cannot be avoided even in a fault-tolerant computer. Here, assuming that the system is more likely to be down due to hardware failure than software or the like, using a fault-tolerant computer is very useful, but the hardware failure rate is very small. Does not fully demonstrate the effect of using a fault-tolerant computer. For example, assume that the detection rate of a hardware failure is X, the isolation rate at which the detected hardware can be isolated is Y, and the failure rate of hardware components is λ. The detection rate X corresponds to the probability of the process of shifting from step 200 to step 300 in FIG. In FIG. 2, the isolation possibility rate Y is calculated from step 300 to step 4
This corresponds to the probability of the process shifting to 00. Here, in FIG. 2, processing other than steps 100 to 400,
In other words, the probability Z that the process proceeds to step 500 or step 600 is Z = (Σ (1−X * Y) * λ)
However, this probability Z is a probability that the operation of the system cannot be continued and the system is forced to go down. Therefore,
The value of the probability Z is very small, and Z <(down rate due to software, for example, (software bug rate)
+ (Operation error rate)), the present embodiment is very useful without multiplexing components.

The configuration of the server system shown in FIG. 1 is merely an example, and it is within the scope of the present invention to provide other components or to provide three or more CPU boards. Also,
Although the server system has been described above as an example, it is needless to say that the present invention can be applied to a system having another multiprocessor computer.

Embodiment 2 In the second embodiment, an example will be described in which a processor is a component that can be separated in the basic operation shown in the first embodiment.

Normally, in a non-multiplexed multiprocessor computer, if at least one processor fails and fails, the data consistency of the entire system is lost,
The system must be shut down. Since the failure rate in the system increases in proportion to the number of processors, a computer having a large number of processors alone has low reliability. On the other hand, it is conceivable to multiplex the processors, but as described above, this method has significant disadvantages in cost and the like.

Therefore, the present embodiment shows a specific example in which the above-mentioned object can be achieved by separating the processor on the CPU board in which a failure has occurred.

Configuration of this Embodiment A system configuration diagram of this embodiment is the same as that of FIG. FIG. 3 shows the CP according to the present embodiment shown in FIG.
FIG. 3 is a detailed configuration diagram of a U board 2. CPU board 2
Processors 10, 11 each having an internal cache 18, 19, a CPU bridge 12 connecting all the processors 10, 11; an external cache 14 connected to the CPU bridge 12 and including the internal caches 18, 19; , And a CPU bus 16 for connecting the CPU bridge 12.

Processor 1 used in the present embodiment
0 and 11 are assumed to be the latest commercially available products such as Pentium processors manufactured by Intel Corporation. That is,
Processors 10 and 11 according to the present embodiment have a built-in cache memory, have a self-failure detection function, and have relatively high coverage. In the CPU bus 16, the data line has an error detecting / correcting means such as parity / ECC, and the control line has a means for performing a protocol check and detecting an error.
A fault occurring on the CPU bus 16 can be detected. As described above, the CPU board 2 has a self-failure detection function that can detect a failure occurring in each mounted component.

The CPU bridge 12 has a CPU bus request issuing circuit 24 having a function of issuing a request to each of the processors 10 and 11 via the CPU bus 16, and a CPU bus receiving circuit 38 for receiving data and the like from each of the processors 10 and 11. A system bus request issuing circuit 30 having a function of issuing a request to the system bus 1, a system bus receiving circuit 40 for receiving data and the like from other components via the system bus 1, and an address management function of the external cache 14. A cache control circuit 28 having
An error detection unit 36 having a function of detecting an error that has occurred in the external cache 14, and has a control management function of the processors 10 and 11 and the external cache 14. The other configuration will be described later.

FIG. 4 shows that the CPU bridge 12 is the same CPU.
3 is a setting example of a management table 32 held for managing processors 10 and 11 mounted on a board. In the management table 32, a processor number, which is identification information of a processor, a data output flag, another processor reference flag, and a failure flag are set for each processor. The contents of each flag information will be described later.

The external cache 14 is composed of a plurality of ways including a plurality of lines, can be divided for each way, and can be set to be unusable. The data in the internal caches 18 and 19 of the processors 10 and 11 are always held in the external cache 14. The external cache 14 and the internal caches 18 and 19 hold the held data together with status information for the data. The status information according to the present embodiment is held for each line, and is prepared with “Modify”, “Exclusive”, “Shared”, and “Invalid”. However, in order to exhibit a function that is a feature of the present embodiment, at least a modification is required. In the following description, the status information is referred to as a cache state, and a line of the internal caches 18 and 19 or the external cache 14 that stores the modified data is referred to as a “Modify line”.
FIG. 5 is a conceptual diagram of the external cache 14 according to the present embodiment, and shows only a structural example of the way 0 as a representative. Of course, other ways have the same structure as way 0. Each way assigns an index, a tag address, a cache state, and a Modify owned processor number, which is one of the features of the present embodiment, to each data and stores the data. Mod
The ify owned processor number holds the identification information of the processor that becomes the owner of the line whose cache state is “Modify”, that is, the processor number. That is, the external cache 14 stores which processor each Modify line corresponds to.

Note that depending on the number of processors, the CPU
Although some CPU boards do not include a bridge or an external cache, in the present embodiment, a CPU having a CPU bridge 12 and an external cache 14 is provided so as to be compatible with a multiprocessor computer system having a certain scale. I decided to use board 2 and that CP
The U board 2 is assumed to be capable of self-detection of a failure without multiplexing the processors 10 and 11.

The operation in the normal time Next, the specific operation in the present embodiment based on the basic operation will be described in the event of a failure shown in the first embodiment, but before that, during normal operation of the server system The processing performed in will be described. First, how the corrected data of each line included in the internal caches 18 and 19 is written back will be described.

The CPU bridge 12 in the present embodiment
Are the internal caches 18 and 19 in addition to the configuration described above.
Has a port 22 provided for each processor as a means for receiving a request to start a write-back process of the correction data stored in the processor. Unless otherwise specified, processing for the processor 10 will be described as an example.

When the internal cache 18 of the processor 10 to which a new process is to be allocated at the time of a process switch holds data of a line whose cache state is "Modify", the operating system (OS) externally stores the data. Write back to cache 14. This is performed as follows. That is, OS
Requests a write-back process by accessing the port 22 corresponding to the processor 10 at the time of the process switch. With this access, the CPU bridge 12 issues an invalidation request to the processor 10 corresponding to the port 22 from the CPU bus request issuing circuit 24 via the write-back circuit 26. The invalidation request is a write-back processing request for causing the processor to write-back modified data to the external cache 14. As a result, the data of all the Modify lines held by the processor 10 are written back to the external cache 14, and the cache state of the written-back line is changed from Modify to Invalid.
(Invalid). Further, the CPU bridge 12
In response to a command from the OS, a write-back request is issued from the system bus request issuing circuit 30 via the write-back circuit 26, and all data written back to the external cache 14 is written back to the main memory 6. At this time, the cache state of the rewritten line is stored in the external cache control unit 2
8 is updated from Modify to Invalid.

As described above, at the time of the process switch, all the Modify lines are eliminated from the internal cache included in the processor to be processed. Further, there is no data owned by the processor from the external cache 14. Thus, during normal operation, the data in each cache is written back each time a process switch is performed. In addition, the Modif mentioned above.
In the write-back (write-back) processing of the y-line, the cache state is changed from “Modify” to “Invalid”.
The cache state may be updated to a state other than id.

By the way, in the present embodiment, FIG.
As shown in the figure, the external cache 14
It stores which processor the line corresponds to. Therefore, at the time of the process switch, the write-back processing of the correction data corresponding to the processor is performed, and when all the Modify lines owned by the processor are exhausted,
The cache state and the Modify owned processor number are initialized. Thereafter, when the data of the internal cache 18 is updated by executing a new process, the content of the external cache 14 is also updated.
External cache control unit 28 in CPU bridge 12
Updates the cache state to “Modify” and sets the processor number owned by “Modify” to the external cache 1
4 will be set. In this manner, the external cache 14 is updated during the normal operation. In this embodiment, the management table 32 is updated as described below.

As shown in FIG. 4, the CPU bridge 12 has a data output flag in the management table 32 corresponding to each processor as external reference flag information.
As described above, the external cache 14 stores all the data held by the built-in caches 18 and 19 of the processors 10 and 11 and holds the processor number of the owner of the Modify line for each line. It is as follows. The data output flag is external cache 1
Processor 10 and 11 with reference to the processor number 4
Has a Modify line in the external cache 14.

In this embodiment, a new process is assigned to the processor 10 at the time of the process switch. The correction data held by the processor 10 is as follows:
Since the external cache 14 is further written back to the main memory 6, the Modify line owned by the processor 10 from the external cache 14 disappears. Accordingly, the data output flag corresponding to the processor 10 is cleared. Then, the data output flag is set in the following cases when the process is executed. (1) When the processor 10 issues an I / O request to the CPU bus 16 and the system bus 1, (2) When the processor 10 issues a write-through write request to the CPU bus 16 and the system bus 1, 3) When the Modify line owned by the processor 10 in the external cache 14 is to be evicted and written back to the main memory 6, (4) the processor 1 in the external cache 14
This is when the Modify line owned by 0 hits a request from another CPU board via the system bus 1 and is output to the system bus 1 and read. That is, data (modification data) of the Modify line owned by the processor 10 in the external cache 14 is accessed from a component other than the CPU board 2 on which the processor 10 is mounted, and a data output flag is set when the contents of the modification data are reflected. Is set.

As described above, the data output flag is
Although set / cleared during operation of the system, according to the present embodiment, by providing the data output flag,
It is possible to detect that the Modify line has been accessed from outside, such as a processor on another CPU board 20.

Further, as shown in FIG. 4, the CPU bridge 12 has a management table 32 in which a processor reference flag is associated with each processor. The other processor reference flag indicates whether the Modify line owned by a certain processor in the external cache 14 has been accessed by another processor on the same CPU board as that processor.

In the present embodiment, a new process is assigned to the processor 10 at the time of the process switch. The correction data held by the processor 10 is as follows:
Since the external cache 14 is further written back to the main memory 6, the Modify line owned by the processor 10 from the external cache 14 disappears. Accordingly, the other processor reference flag corresponding to the processor 10 is cleared. Then, when a new process is executed on the processor 10 and the data is updated, the Modify line owned by the processor 10 is held in the external cache 14 as described above. At this time, a Modify owned processor number is set in the external cache 14.

The CPU bridge 12 is connected to the external cache 1
4, when a processor other than the processor 10 on the same CPU board 2 (the processor 11 in the example of FIG. 3) accesses the Modify line owned by the processor 10, the other processor reference flag of the referenced processor 10 is set. You. That is, the external cache 1
4, the data (modification data) of the Modify line owned by the processor 10 is the same as the processor 10
The other processor reference flag is set when it is accessed from another component on the board 2 and the content of the correction data is reflected. The Mod owned by the processor 10
Whether or not another processor 11 has accessed the ify line can be easily determined by referring to the Modify owned processor number.

According to the present embodiment, by providing the other processor reference flag, it is possible to detect that the Modify line has been accessed by another processor on the same CPU board.

Operation When Failure Occurs In this embodiment, the external cache 14 and the management table 32 described above during normal operation of the server system are used.
Is updated. Next, an operation performed when a failure occurs in the processor in the server system will be described with reference to FIG.
This will be described with reference to the flowchart shown in FIG. even here,
The case where a failure occurs in the processor 10 mounted on the CPU board 2 will be described as an example. In the processing corresponding to the flowchart of the basic operation shown in FIG.

When the processor 10 detects the occurrence of a failure during the execution of the process by the self-failure detection function (step 110), the processor 10 generates a failure interrupt to generate a CPU interrupt.
2 sends a failure detection signal via an error line. CPU
When the disconnection determination circuit 34 in the bridge 12 recognizes that the failure detection signal has been received from the processor 10,
A process of determining whether or not the influence from the failed processor 10 has been transmitted to another is performed based on a predetermined disconnection condition (step 210). This is performed by the CPU bridge 12 referring to the setting contents of the data output flag and the other processor reference flag. If the data output flag is cleared, the Modify line owned by the processor 10 in the external cache 14 is changed to the CPU board 2
Can not be accessed from outside. If the other processor reference flag is cleared, it can be determined that the Modify line owned by the processor 10 in the external cache 14 is not accessed by the processor 11 on the same CPU board 2.
Therefore, if the disconnection condition that both flags are cleared is satisfied, the data corrected by the failed processor 10 is not referred to from any processor or the like, and the influence is not transmitted to the other processors. You can judge. On the other hand, if the data output flag or the other processor reference flag corresponding to the processor 10 is set, it can be determined that the influence has been transmitted to the other processor.

As a result of the above determination processing, when it is determined that the predetermined disconnection condition is satisfied and that the processor 10 can be isolated (step 310), all the Modify lines owned by the processor 10 in the external cache 14 are determined. The cache state is changed from "Modify" to "Invalid" to invalidate the data of the line (step 410). At this time, the contents of the Modify owned processor number are also cleared. Then, the disconnection determination circuit 34 disconnects the processor 10 by sending a reset signal to the processor 10 via a reset line (step 411). Then, a failure flag corresponding to the processor 10 in the management table 32 is set. The failure flag is set for each processor, and is initialized (cleared) when the processor is normal.
When a failure occurs and the CPU bridge 12 is disconnected, a flag corresponding to the failed processor is set. As a result, the OS can operate so as not to allocate a process to the processor for which the failure flag is set.

On the other hand, when it is determined that the predetermined disconnection condition is not satisfied and the processor 10 cannot be isolated, it is determined that data consistency cannot be maintained, the system is stopped, and the system may be restarted in some cases. Start-up is performed (step 610). And if necessary, OS
Allocates the process executed by the processor 10 to another processor and executes the process again.

As described above, according to the present embodiment, the data output flag and the other processor reference flag are provided to easily detect whether or not another processor has access to the Modify line owned by the failed processor. I can do it. If there is no access, even if the failed processor 10 is disconnected, data consistency will not be lost, so that the system can be isolated without bringing down the system.

In the present embodiment, an example of a server system in which a plurality of CPU boards each having a plurality of processors are connected has been described. Therefore, step 21 shown in FIG.
In the process of step 0, if the server system has a plurality of CPU boards each having a single processor, only the presence or absence of access from a processor on another CPU board may be determined as a disconnection condition. When only one CPU board on which a processor is mounted is connected, only the presence or absence of access from another processor on the same CPU board may be determined as a disconnection condition.

In this embodiment, means for judging whether a failed processor is detachable and means for detaching the failed processor are mounted on the CPU bridge 12. It may be provided as.

Embodiment 3 In the second embodiment, an example in which a failure occurs in the processor has been described. However, in the present embodiment, a specific example in which the above object is achieved when a failure occurs in the external cache 14 is shown. Note that the external cache 14 in the present embodiment has a self-failure detection function by checking parity or ECC. Here, occurrence of a failure can be specified for each way, and degeneration is performed for each way (unusable state). ).

Next, the operation when a failure occurs in the external cache 14 in the server system will be described with reference to the flowchart shown in FIG. The processing corresponding to the flowchart of the basic operation shown in FIG. 2 is denoted by the same reference numeral in the hundreds. Further, the processing in the normal operation in the server system is the same as that in the second embodiment, such as the write-back processing at the time of the process switch and the updating of each flag included in the management table.

When the external cache 14 detects the occurrence of a failure in any of the ways by the self-failure detection function (step 150), it generates a failure interrupt and sends a data parity signal, for example, to the CPU bridge 12 via the error line. Send out. When recognizing that the data parity signal has been received, the error detecting unit 36 in the CPU bridge 12 detects whether the data on the Modify line in the faulty way has been accessed from a processor on another CPU board 20 or not. A process of determining whether or not the influence is transmitted to another is performed based on a predetermined disconnection condition (step 250). This is because if the line is a modify line, data consistency cannot be maintained. In addition,
If the cache state is a line other than “Modify”, since there is data in the main memory 6, no problem occurs if the line is invalidated, so that there is no need to make it a determination target. The determination process in the present embodiment is performed by the CPU bridge 12 referring to the setting content of the data output flag.
If the predetermined disconnection condition that the data output flags of all the processors 10 and 11 in the faulty way are cleared is satisfied, the correction data in the CPU board is not accessed from the outside and affects other devices. You can judge that it is not. On the other hand, if any of the data output flags is set in the faulty way, it can be determined that the correction data held in the faulty way may have been accessed from outside.

Therefore, as a result of the determination processing, if it is determined that the predetermined disconnection condition is satisfied and the faulty way can be degenerated (step 350), all data in the external cache 14 is invalidated in order to invalidate all data. Modify
The cache state of the line is changed to Invalid (step 450). This processing is performed in the external cache 1
4 means to discard all data. This is because a processor having a Modify line in the way in which a failure has occurred may have a Modify line in another normal way, and data integrity can be maintained by discarding all data. It is.
At this time, the contents of the Modify owned processor number are also cleared. Then, the error detection unit 36 degenerates the failure occurrence way (Step 451). The cache control unit 28 internally manages a degeneration flag for managing data write permission to each way for each way. By setting the degeneration flag, the cache control unit 28 controls and manages so that data will not be loaded into the degenerated way in the future. . Then, if necessary, the OS allocates the process being executed to any of the processors and causes the processor to execute the process again.

In this way, even if a failure occurs in any of the ways in the external cache 14, only the way is degraded without shutting down the system, thereby continuing the operation of the system while maintaining data consistency. be able to.

In this embodiment, the presence or absence of external access is determined by using the data output flag set in the second embodiment.
For the determination processing in the present embodiment, dedicated external reference flag information may be provided for each way. The dedicated external reference flag information is cleared at the time of the process switch similarly to the data output flag, and is set when the correction data held in the way is reflected outside the CPU board 2.

The system configuration shown in FIG. 1 is merely an example, and the number of CPU boards, the number of processors, the configuration of devices connected to the system bus, and the like are not limited thereto.

[0069]

According to the present invention, when a failure occurs,
Determines whether the fault location can be isolated without bringing down the system.If it is determined that the fault location can be isolated, the fault location can be separated while the system is operating and it is determined that isolation cannot be performed. The system was brought down once, the faulty part was isolated, and then the system was started. In other words, whether or not it is necessary to bring the system down to isolate the fault location is determined before the isolation, so that it is possible to recover from the failure without unnecessarily bringing down the system and to process
System reliability and availability without multiplexing
Can be improved. In particular, in the present invention,
By providing the unit reference flag information, the fault
External cache that stores data modified by the server
Install the faulty processor on the memory line.
That access from outside the CPU board
It becomes possible to detect. As a result, the CPU button
Access to the line from outside
If it is determined that data consistency can be maintained,
Isolate failed processors without bringing down the system
can do.

[0070]

[0071]

Further, by providing the other processor reference flag information, it is possible to detect that another processor on the same CPU board has accessed the line of the external cache memory storing the corrected data. It becomes possible. As a result, if it is determined that another processor on the same CPU board has not accessed the line owned by the failed processor and that data consistency can be maintained, the failure occurs without bringing down the system. The processor can be isolated.

Further, by providing the failure flag information, it is possible not to assign a process to the separated failed processor.

The provision of the external reference flag information makes it possible to detect that the correction data held in each way is accessed from outside the CPU board on which the way is mounted. . As a result, when it is determined that the corrected data held by the faulty way is not accessed from outside the CPU board and it is determined that data consistency can be maintained, the faulty way can be determined without bringing down the system. Isolation is possible, and the remaining ways allow the system to continue operating.

[Brief description of the drawings]

FIG. 1 is a diagram showing an embodiment of a multiprocessor computer according to the present invention.

FIG. 2 is a flowchart showing a basic operation in the first embodiment.

FIG. 3 is a block diagram of a CPU board according to a second embodiment.

FIG. 4 is a diagram showing an example of the contents of a management table held by a CPU bridge according to the second embodiment.

FIG. 5 is a conceptual diagram of an external cache according to a second embodiment.

FIG. 6 is a flowchart illustrating an operation when a failure occurs in a processor according to the second embodiment;

FIG. 7 is a flowchart illustrating an operation when a failure occurs in an external cache in the third embodiment.

[Explanation of symbols]

1 system bus, 2,20 CPU board, 4 I /
O bridge, 6 main storage (main memory), 8 external storage, 10, 11 processors, 12 CPU bridge, 14 external cache, 16 CPU bus, 1
8, 19 Built-in cache, 22 ports, 24 CP
U bus request issuing circuit, 26 write-back circuit, 28
Cache control circuit, 30 system bus request issuing circuit, 32 management table, 34 disconnection determination circuit, 36 error detection unit, 38 CPU bus reception circuit,
40 System bus receiving circuit.

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G06F 15/177 678 G06F 15/177 672 G06F 11/20 310

Claims (14)

(57) [Claims] (1)The data must be modified with respect to the data.
Keep with status information about nothingBuilt-in cap
At least one processor having a cache memory, and including the contents of the internal cache memory for each line
On whether the data held with
Ownership of each line where status information and data are stored
And the identification information of the processor to be used for each line
Connecting an external cache memory, the processor and the external cache memory
And at least one processor bridge having a self-failure detection function
CPU board and main memory,  Failure to shut down the system based on the specified disconnection conditions
Isolation judging means for judging whether the harmful area can be isolated
When,When the isolation determination means determines that isolation is possible Obstacle
Quarantine execution means for separating a harmful point during system operation;When the isolation determination means determines that isolation is not possible Shi
Restarting means for stopping and restarting the stem;The processor bridge is a processor on the same CPU board.
The external reference flag information assigned to each processor is
Access switch, clear the same CPU board.
Modification data owned by the processor only held by the
Data is reflected outside the same CPU board.
Set The isolation determination unit refers to the external reference flag information.
The effect from the failed processor is
Determine whether it is transmitted to the outside, The quarantine execution means may be configured such that the quarantine determination means does not
If it is determined that the information has not been transmitted,
The failed processor obtained based on the separate information
The fault held in the mounted CPU board
Invalidated all data modified by the originating processor
After that, disconnect the failed processor Characterized by
Multiprocessor calculator. 2. The processor bridge according to claim 1, wherein the processor bridge receives, for each processor, a start request for a write-back process of the correction data in the built-in cache memory of each processor; Means for issuing a write-back processing request for performing a write-back processing of the correction data to the external cache memory, wherein the processing is held in a built-in cache memory of the processor which is the processing target at the time of the process switch. multiprocessor computer of claim 1, wherein the writing back modified data to the main memory. 3. The isolation determining means, when the external reference flag information corresponding to the faulty processor is clear, determines that the influence from the faulty processor is not transmitted to the outside. Item 4. The multiprocessor computer according to item 1 . 4. The processor bridge according to claim 1, wherein the processor bridges
Other processors assigned to each processor on the board
Clears reference flag information at process switch
And the correction data owned by the processor is stored.
For lines in the external cache memory,
An external processor on the same CPU board accessed
Set, and the isolation determination means refers to the other processor reference flag information.
From the processor that failed due to
Influence is transmitted to other processors on the same CPU board.
It is determined whether or not the separation has been performed.
Affects other processors on the same CPU board
Identification information of the processor when it is determined that the
The failed processor acquired based on the
The fault occurrence program held in the CPU board
After invalidating all the data corrected by the processor,
Claims wherein the faulty processor is isolated.
2. The multiprocessor computer according to 1 . Wherein said isolation determining means, when the other processors see flag information corresponding to the failure processor is clear, the influence of the failure processor the
5. The multiprocessor computer according to claim 4, wherein it is determined that the signal is not transmitted to another processor on the same CPU board . Wherein said isolation execution unit receives the failure detection signal emitted by the self-fault detection from the failure processor, according to claim 1, characterized in that separating the fault occurrence processor from the processor bridge
A multiprocessor computer as described. 7. The isolation execution means indicates whether each of the processors has been disconnected from the processor bridge.
7. The apparatus according to claim 6, wherein the failure flag information is held.
A multiprocessor computer as described. Wherein said processor bridge multiprocessor computer according to claim 1, characterized by having the isolation determination unit. Wherein said processor bridge multiprocessor computer according to claim 1, characterized by having the isolation execution unit. 10. The method according to claim 1 , wherein the data is a modification of the data.
At least one processor having a built-in cache memory for holding together with status information on presence / absence, and having a correction to the held data for each line including the data held in the built-in cache memory.
Status information and data about
Identification information of the processor that is the owner of the line
An external cache memory for holding each said processor and connect the external cache memory, performs respective control management, the same CPU board
External reference flags assigned to each processor on the
Information is cleared at the time of process switch, and the same CP
The processor location held only inside the U board
Reflect the existing correction data outside the same CPU board
Mounting a processor bridge, a setting when brought into, in a multiprocessor computer having at least one CPU board having a self-fault detection, fault occurs during the system operation by reference to the external reference flag information An isolation determination step for determining whether or not the influence from the processor is transmitted to the outside; and the faulty processor in the isolation determination step
If it is determined that the influence from the faulty processor has not been transmitted to the outside, the system disables all the data corrected by the faulty processor held in the CPU board on which the faulty processor is mounted without shutting down the system. After doing
And isolation execution step of separating the failure processor, the failure processor in the isolation determination step
A restarting step of stopping and restarting the system when it is determined that the influence from the outside is transmitted to the outside, and a method of recovering a failure in the multiprocessor computer. 11. The processor bridges having the same CP
Other processors assigned to each processor on the U board
Clear the reference flag information at the time of process switch
And the correction data owned by the processor is stored.
For lines in the external cache memory,
An external processor on the same CPU board accessed
When set, the isolation determination step includes the other processor reference flag information.
The processor having failed by referring to the
From other processors on the same CPU board
The quarantine execution step includes determining whether the quarantine is transmitted to the quarantine.
The effect from the failed processor is also
The effect is transmitted to other processors on one CPU board
If it is determined that there is no
In the CPU board on which the faulty processor is mounted
The data modified by the failed processor being retained.
After disabling all of the
11. The multiprocessor according to claim 10, wherein
How to recover from a failure on a computer. 12. The external cache memory has a structure that can be divided in units of ways, and the isolation determination unit determines that the influence of the way in which a failure has occurred is caused by the CP having the external cache memory mounted thereon.
The isolation execution means determines whether or not the influence from the faulty way has not been transmitted to the outside of the CPU board. after the cache memory has disabled all correction data to be stored, the multiprocessor computer of claim 1, wherein the degenerate the failure way. 13. Before SL isolation determining means identifies the processors to which the owner of each line included in the failure-way
By referring to the external reference flag information assigned to the processor specified by the separate information, it is possible to determine whether or not the correction data stored in the faulty way is externally reflected. 2. The method according to claim 1, wherein
2. The multiprocessor computer according to 2 . 14. The method according to claim 11 , wherein said data is a modification of said data.
At least one processor having a built-in cache memory that holds status information on presence / absence of presence / absence, and a structure that can be divided into ways containing a plurality of lines
Data stored in the internal cache memory.
For each line
Status information and data on the presence or absence of corrections are stored
Of the processor that is the owner of each line
An external cache memory which holds for each line, said processor and connect the external cache memory, performs respective control management, the same CPU board
External reference flags assigned to each processor on the
Information is cleared at the time of process switch, and the same CP
The processor location held only inside the U board
Reflect the existing correction data outside the same CPU board
Mounting a processor bridge, a setting when brought into, in a multiprocessor computer having at least one CPU board having a self-fault detection, the failure occurs during the system operation by reference to the external reference flag information The correction data stored in the way is stored in the CP having the external cache memory.
An isolation determination step for determining whether the information is not transmitted outside the U board; and the external cache memory is stored without shutting down the system when it is determined in the isolation determination step that the information is not transmitted outside the CPU board. After all the correction data to be invalidated, the isolation execution step of degenerating the faulty way, and the system is stopped and restarted when it is determined in the isolation determination step that the data is transmitted outside the CPU board. And a re-starting step for performing a fault recovery method in a multiprocessor computer.