EP0164414A4 - Computer processor controller. - Google Patents
Computer processor controller.Info
- Publication number
- EP0164414A4 EP0164414A4 EP19850900389 EP85900389A EP0164414A4 EP 0164414 A4 EP0164414 A4 EP 0164414A4 EP 19850900389 EP19850900389 EP 19850900389 EP 85900389 A EP85900389 A EP 85900389A EP 0164414 A4 EP0164414 A4 EP 0164414A4
- Authority
- EP
- European Patent Office
- Prior art keywords
- processors
- processor
- controller
- xxx
- interrupt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/1675—Temporal synchronisation or re-synchronisation of redundant processing components
- G06F11/1687—Temporal synchronisation or re-synchronisation of redundant processing components at event level, e.g. by interrupt or result of polling
Definitions
- the present invention relates generally to control apparatus for a computer central processing unit, and more particularly to a processor controller for faulttolerant computers having at least two central processors operating simultaneously.
- Fault-tolerant computers typically utilize two or more separate central processing units. In the event one processing unit or processor fails, the remaining processor(s) is (are) relied upon to maintain the computer in a functioning mode.
- Various techniques have been used to detect processor failures.
- One such technique commonly referred to as the lockstep method, utilizes multiple processors executing identical code. The processors are all clocked by a common clock source so that every bus cycle of the processors will be identical to one another, provided the processors are operating properly. Comparison of the data, addresses or control output of the processors are periodically made to verify proper operation. If the comparison is negative, steps are taken to determine which of the processors has failed so that the processor may be disabled.
- Another technique for verifying proper processor operation is sometimes referred to as the checkpoint approach.
- Each of the processors are processing different code. Periodically, each of the processors will transmit a signal to the other processor that the transmitting processor is operating properly. Steps are taken to ensure that a defective processor will not issue an OK signal. If the signal is not periodically received, it is assumed that a fault in the processor has occurred. In addition, data from each of the processors are periodically made available to another processor (i.e., a checkpoint) so that an operating processor will be able to assume the operations of a failed processor, at a reduced rate, without loss of data.
- a checkpoint another processor
- the above-described techniques for verifying proper processor operation possess various shortcomings.
- the lockstep approach cannot be implemented using commercially available hardware and operating systems without major modifications.
- the lockstep approach utilizes a common clock, therefore a failure of the clock source will result in a total failure of the system.
- the checkpoint approach requires that excess processor capacity be utilized so that the system response time will be maintained following a processor failure. If excess processor capacity is not used, it is necessary to reduce the number of functions carried out by the system in order to maintain the same response time.
- the checkpoint approach cannot be applied using most existing hardware and operating systems without relatively major modifications.
- existing application programs typically must be extensively modified or rewritten to be fault tolerant.
- the present system overcomes the shortcomings of the prior art approaches.
- a processor controller for a computer system having a plurality of processors which execute programmed instructions is disclosed.
- An exemplary computer system would be a fault-tolerant computer having redundant processors which execute substantially identical instructions.
- the subject controller receives asynchronous external signals, primarily interrupt signals, and forwards the asynchronous signals to the processors in a predetermined manner.
- the controller includes apparatus for determining the position of each of the processors in executing the code associated with the processor.
- the position is referred to herein as virtual time.
- the position can be determined, for example, by monitoring the number of bus cycles which have been executed by each of the processors. This may be accomplished by counting the number of address strobes generated by each processor, although other signals may be used for this purpose.
- the number of bus cycles that a particular processor has executed is used to determine the location of the processor in virtual time.
- the processors will typically execute their respective identical programs at different rates. Thus, at any given point in actual time, the processors are likely to be at different positions in virtual time.
- the subject controller When an asynchronous signal, such as interrupt, is received, the subject controller examines the virtual time of each processor. If the virtual times are identical, the controller notifies each of the processors of the signal. If the processors are at different virtual times, one embodiment of the subject controller refrains from forwarding the asynchronous signal to either processor. The leading processor in virtual time is halted and the trailing processor is permitted to execute code until it has reached the leader in virtual time. Once the virtual times are the same, the asynchronous signal is presented to both processors at the same actual time.
- an asynchronous signal such as interrupt
- the asynchronous signal is forwarded to the lead processor when the signal is received.
- the virtual time of the leading processor is then stored.
- the lagging processor is then permitted to execute code until its virtual time matches the stored virtual time. At that point, the lagging processor is notified of the asynchronous signal.
- the processors are notified of the asynchronous signal at the same virtual time. However, the actual times of the notification will not usually be the same.
- the subject controller preferably also includes means for verifying proper processor operation.
- the two processors will have been programmed to periodically request for synchronization by the controller. When each such request is made, the requesting processor is immediately halted.
- the lead processor will be the first to issue a request to the controller and thus will be halted first.
- the following processor will continue to execute code until it also makes a request for synchronization. At this time both processors will be halted. Since the processors are supposed to be executing identical code, the processors should be halted at the same virtual time.
- the subject controller verifies that the virtual times are the same, and then releases the processors. In the event the virtual times are not identical, an error is reported.
- Figure 1 is a simplified block diagram of a fault-tolerant computer utilizing the subject processor controller.
- FIG. 2 is a simplified flow chart which illustrates the manner in which external events such as interrupts are processed by the subject processor controller.
- Figure 3 is a flow chart which illustrates the manner in which a first embodiment of the subject controller processes external events.
- Figure 4 is a timing diagram which depicts an exemplary operation of the Figure 3 embodiment of the subject invention.
- Figure 5 is a flow chart which illustrates the manner in which a second embodiment of the subject controller processes external events.
- Figure 6 is a timing diagram which depicts an exemplary operation of the Figure 5 embodiment of the subject invention.
- Figure 7 is a flow chart which illustrates the manner in which proper operation of the processors is verified by the subject controller.
- Figure 8 is a timing diagram which depicts an exemplary verification operation of the subject controller.
- Figure 9 is a functional block diagram of the first embodiment of the subject controller.
- Figure 10 is a functional block diagram of the state machine of the subject controller.
- Figure 11 is a flow chart illustrating the overall operation of the subject controller state machine.
- Figure 12 is a flow chart showing the operation of the Counters Running Main Loop block of the subject controller state machine.
- Figure 13 is a flow chart showing the operation of the Interrupt Handler block of the subject controller state machine.
- Figure 14 is a flow chart illustrating the operation of the Autohalt Handler block of the subject controller state machine.
- Figure 15 is a flow chart illustrating the operation of the Present Interrupts block of the subject controller state machine.
- Figure 16 is a flow chart illustrating the operation of the Wait For Counters On Request block of the subject controller state machine.
- Figure 17 is a detailed schematic drawing of a portion of the subject controller, including the virtual time counters and comparator circuits.
- Figure 18 is a detailed schematic drawing of a portion of the subject controller, including the timeout timer and halt control circuitry.
- Figure 19 is a detailed schematic drawing of a portion of 16 subject controller, including various interface circuits.
- Figure 20 is a detailed schematic drawing of a portion of the subject controller, including interrupt control circuitry.
- Figure 21 is a detailed schematic drawing of a portion of the subject controller, including the state machine microcode memory.
- Figure 22 is a detailed schematic diagram of a portion of the subject controller, including bus interface circuits.
- Figure 23 is a detailed schematic diagram of a portion of the subject controller, including the connector pin assignments.
- the computer includes a pair of substantially identical processors, including a processor A which is designated by the numeral 30 and a processor B, which is designated by the numeral 32.
- Each processor includes an internal microprocessor (not shown), such as the 16-bit microprocessor manufactured by Motorola, Inc., under the designation MC 68000.
- Each processor further includes a local memory (not shown), a peripheral device in the form of a terminal controller (not shown) connected to a local bus (not shown).
- the computer system further includes a common memory, represented by block 34, which may be accessed by either processor 30 or 32 by way of an interprocessor bus 36.
- Peripheral devices such as terminal controllers and disc controllers, are also coupled to the interprocessor bus, as represented by block 38.
- the subject processor controller generally designated by the numeral 40, is also coupled to bus 36.
- Processor controller 40 performs various functions to be described later in greater detail. One such function is to verify proper operation of the two processors 30 and 32. Another exemplary function is to control the flow of asynchronous external events, primarily interrupts, to the two processors.
- Processors A and B together with processor controller 40 can be viewed as a single logical processor.
- processor controller 40 receives interrupts and other asynchronous signals from external sources which are intended for the two processors. These signals will be collectively referred to herein as interrupts.
- Processors A and B are identically programmed and, under normal operating conditions, execute identical code. For various reasons, however, the processors typically do not execute the same code at the same time. For example, if the two processors request interprocessor bus 36 simultaneously, only one processor will be given access. The remaining processor will be requested to wait until the first processor has released the bus. As a consequence, the second processor to access the bus will fall behind the first processor in executing the common code.
- the two processors have separate internal clocks which are not identical in frequency. The processor with the higher frequency clock will tend to execute code at a higher rate.
- Virtual time can be defined as the measure of how far a particular processor has progressed in its execution of its associated code. The smallest measurable change in virtual t ime can b e expressed in terms of virtual time ticks.
- processor events There are several possible choices of processor events which can be used as a virtual time tick for the purpose of measuring virtual time. Depending upon the particular processor used, instructions, data references or bus cycles could be used as virtual time ticks. MOVE instructions, procedure calls and the like could also be used for this purpose.
- the ideal virtual time tick is chosen such that the processors will be at the same virtual time when they are at the same point in the execution of their respective code.
- the capability must exist to count each tick of virtual time. For example, instruction counts should not be used as virtual time ticks for processors such as the Motorola model MC68010 which utilize an internal cache memory. This is because there are no external indications of instruction cycles for this type of processor. However, instruction fetches or bus cycles could be used as virtual time ticks.
- the virtual time tick should occur at least once during a predetermined maximum period of time. It is preferable that the ticks occur at least once every few microseconds or even more frequently. Ticks which occur at a lower frequency do not provide sufficient resolution for verifying proper processor operation.
- the subject controller monitors the positions of each of the processors in virtual time and presents interrupts to the processors only at the same point in virtual time.
- This aspect of the invention is illustrated in the flow chart of Figure 2.
- the controller monitors for asynchronous external events (interrupts) which are intended for the two processors. If no events are detected, the controller remains in a loop condition. After receipt of an interrupt, the two processors are notified of the interrupt at the same virtual time, as indicated by block 46. The controller then sequences back to state 44 and waits for further interrupts.
- Figure 3 illustrates a first embodiment of the subject controller. At state 44, the controller waits for an interrupt. After an interrupt is received, the lead processor in virtual time is halted as indicated by block 46.
- This processor will no longer advance in virtual time. Meanwhile, the lagging processor is permitted to execute code. As indicated by element 48, when the lagging processor has finally reached the halted processor in virtual time, both processors are notified of the interrupt as indicated by block 50. Thus, the notification is given at the same virtual and actual time.
- the operation of the first embodiment controller is further illustrated in the timing diagram of Figure 4.
- the vertical axis of the graph represents virtual time and the horizontal axis represents actual time.
- the solid line A and broken line B represent the states of processors A and B, respectively. At virtual and actual times V o and T o , lines A and B indicate that the processors are at the same virtual and actual times.
- processor A is at time T 2 .
- processor B starts to lag in virtual time.
- the two processors advance in both actual and virtual time until actual time T 14 , when an interrupt is detected by the controller.
- processor A is at virtual time V 10 and processor B is at virtual time V 6 .
- the controller then commands the advanced processor, processor A, to halt.
- the processor will not proceed further in virtual t ime until it is un-halted.
- processor B is permitted to advance in virtual time.
- processor B also reaches virtual time V 10 , at actual time T 19 , the two processor are at the same virtual and actual times. At this time both processors are un-halted and notified of the interrupt.
- the processors then service the interrupt and proceed to execute code.
- FIG. 5 Operation of a second embodiment of the subject invention is illustrated in the flow chart of Figure 5.
- the controller waits for an interrupt.
- the lead processor in virtual time is notified of the interrupt as indicated by block 52.
- the virtual time at which the notification is given is stored.
- the leading processor then services the interrupt and proceeds to process code.
- the controller compares the stored virtual time with the virtual time of the lagging processor as indicated by element 54. When the virtual times are equal, the lagging processor is notified of the interrupt, as represented by block 56.
- the second processor then services the interrupt and proceeds to process code.
- FIG 6 a timing diagram similar to that depicted in Figure 4 is used to further illustrate the operation of the second embodiment controller.
- processor A is leading processor in virtual time.
- the controler receives an interrupt.
- Lagging processor B was only at virtual time V 6 when the lead processor was notified of the interrupt. Processor B continues to process code until it reaches virtual time V 8 and real time T 17 . At this time, the controller detects that the virtual time of the lagging processor matches the stored virtual time V 8 and processor B is notified of the interrupt. Processor B then services the interrupt and continues to process code. Thus, both processors are notified of the interrupt at the same virtual time, but not necessarily the same actual time.
- Controller 40 derives the virtual time of the two processors by counting processor local bus cycles. A bus cycle, or virtual time tick, occurs whenever the processor moves data via the local bus. The local bus cycles of the Motorola MC 68000 microprocessor are indicated by occurrences of address strobes. Address strobe signals from processor A, referred to as Clock A, are coupled to controller 40, as indicated by line 52. Similarly, address strobe signals from processor B, referred to as Clock B, are also fed to controller 40, as indicated by line 52'. The controller provides processor halt signals to processors A and B as represented by lines 54 and 54', respectively. The interrupts received by the controller on line 42 are forwarded to processors A and B at the same virtual time on lines 56 and 56', respectively.
- processors A and B have a means for communicating with one another by, for example, exchanging messages. Unless semaphore techniques are used, messages cannot be reliably exchanged unless the processors are synchronized.
- the processors are synchronized by definition when they are simultaneously at the same virtual and actual times. To enable an exchange of messages and for other reasons, the subject invention is capable of synchronizing the processors when requested by the processors to do so. Once the processors are synchronized, the subject controller also verifies proper processor operation.
- the processors will be programmed to request synchronization at the same virtual times.
- the processor most advanced in virtual time will be the first to request synchronization.
- the processors are halted by the subject controller. Since both processors are supposed to be executing identical code, the lagging processor will eventually also request a synchronization. Again, the processor will have halted just after the issuance of the request.
- the controller then compares the virtual times of the two processors as indicated by element 62. If both processors are operating properly, the virtual times should be equal.
- processors are released or un-halted as shown by block 64.
- the processors are thereby synchronized. If the virtual times do not correspond, the processors are notified that there is an error and that they are not synchronized as indicated by block 66.
- the operation of the processor synchronization process is further illustrated by the timing diagram of
- FIG 8 which is similar to the diagrams of Figures 4 and 6.
- the two processors are synchronized. As time passes, the processors go out of synchronization, with processor A leading.
- processor A requests that the processors be synchronized and is halted.
- processor B requests that the processors be synchronized and is also halted.
- the virtual times of the two processors are then compared. Proper operation is verified since both processors are at virtual time V 8 . The processors are then un-halted and permitted to continue executing code. If the virtual times did not correspond, the processors would have been notified of the anomaly.
- each processor would be suitably programmed to request verification of proper operation at the same point in virtual time. If, in fact, the requests for verification are received by the subject controller at the same virtual times, proper operation is confirmed. If the requests occur at different virtual times, a fault has occurred and the processors are notified of the anomaly.
- the two processors are identical in hardware configuration, that the processors execute identical code, and that the data present within the processors are identical.
- the processors then are capable of executing non-identical code with the controller off. During this time, the controller will not forward interrupts to the two independently-operating processors.
- the processor programs also include instruction for turning the subject controller back on for synchronized operation. Steps are taken to ensure that both processors turn the subject controller off at the same virtual time and that the means for tracking virtual time on the controller is reinitialized when the controller is turned back on.
- the controller receives the clock A and clock B signals from the processors on lines 52 and 52', respectively.
- the clocks are synchronized with the internal processor controller clock (not shown) utilizing conventional synchronization circuits 70 and 70'.
- a pair of virtual time counters, 72 and 72', are provided for counting the two clocks outputed by the synchronizers.
- the subject controller further includes a state machine 68 which provides various control outputs in response to microcoded instructions and selected condition inputs.
- the major components of the state machine are shown in Figure 10.
- the machine includes a memory 76. loaded with approximately 256 32-bit microcode instructions.
- Memory 76 is a Programmable Read Only Memory (PROM) although a Random Access Memory (RAM) may be used provided a non-volatile memory is included for permanently storing the microcode.
- a program counter 78 is included for providing an 8-bit address for memory 76.
- the state machine further includes a branch condition selector multiplexer having 24 inputs which represent 24 different conditions of the subject processor controller.
- Input 1 may be a logic 1 when the virtual time counter 72 for processor A is greater than the counter 72' for processor B and a logic 0 at all other times.
- the particular condition to be selected is determined by 5 bits of the 32 bits of data read from memory 76, as represented by line 82.
- the output of multiplexer 80 is coupled to the load input of program counter 78. If the selected condition is not present, program counter 78 increments to the next higher memory 76 address to read the next instruction. If the condition is present, 8 bits of the 32-bits of data read from the memory are loaded into counter 78 and used as the address to read the next instruction out of memory. Unless another branch condition occurs, the counter will increment from the new branch address. The remaining 19 bits of the 32 bits of data from the memory are used as control outputs for the subject processor controller.
- state machine 68 generates count enable signals on lines 84 and 84' which are coupled to logic elements 74 and 74', respectively.
- virtual time counters 72 and 72' count clocks from the processors A and B, respectively.
- the outputs of the virtual time counters 72 and 72' are compared by the subject controller as represented by block 86. If processor A is advanced further in virtual time than processor B, a signal indicative of this condition is generated on line 88 and forwarded to state machine 68. If processor A is lagging in virtual time, a signal indicative of this condition is generated on line 90 and forwarded to to the machine. If processor A is neither leading nor lagging, the state machine concludes that the virtual time counters are equal.
- Halt control circuitry 88 and 88' is provided which, among other things, synchronize the Halt A and Halt B signal with the bus cycles of the respective processors.
- the processors can request the subject controller to synchronize the processors. Also, the processors can instruct the controller to either turn on or turn off the virtual time counter 12 and 12'. This may be done if the processors are programmed to operate independently of one another for a period of time. These functions are accomplished utilizing a pair of control registers 94 and 94' which are associated with processor A and B, respectively. Control registers 94 and 94' are coupled in a timemultiplexed fashion to the local data bases of processors A and B, respectively, by way of interprocessor bus 36 ( Figure 1). An interprocessor bus interface circuit 97 is provided for interfacing between the interprocessor bus 36 and the local bus of the subject controller.
- Interface circuit 97 includes address recognition apparatus which detects commands addressed to control registers 94 and 94' and provides enable signals to the registers on lines 102 and 102', respectively.
- the enable signals cause command data addressed to registers 94 and 94' to be loaded.
- the address recognition circuit causes a signal to be sent back to the processor which originated the command, acknowledging receipt of the command.
- the commands which may be loaded into registers 94 and 94' includes virtual time counter on commands, counter off commands and synchronization request commands.
- an autohalt signal is forwarded to the halt control circuitry as represented by lines 95 and 95'.
- the autohalt signals cause the halt control circuitry to halt the particular processor which transmitted the command. The halt is achieved very quickly since state machine 68 is not utilized to generate the autohalt signals.
- control registers 94 and 94' are then transferred to state machine 68 as represented by lines 104, 104', 106 and 106'. After the commands have been acted upon by state machine 68, the registers are cleared as indicated by lines 110 and 110'.
- the processors also provide lock requests, reset requests and (by implication) unreset requests, which are received by control registers 94 and 94' and transferred to state machine 68. These latter three request commands will be subsequently described.
- error data identifying the type of error is generated on line 114.
- the error status data will then be loaded into an error status register 116.
- the processors will periodically request error status data from the controller by generating an appropriate address which wil be detected by the address recognition circuitry of interprocessor bus interface 97.
- An enable signal will be produced on line 120 causing error status data to be forwarded to the requesting processor, as indicated by line 117, via interprocessor bus interface 97.
- the address recognition circuit of interface 97 will transmit an acknowledgment signal back to the processor requesting the error status data.
- the subject controller further includes interrupt control circuitry 120.
- interrupt control circuitry 120 When an interrupt for the processors is received on line 42, control 120 forwards an interrupt request signal to state machine 68 by way of line 122.
- comparator circuit 86 determines that the virtual times of the two processors are matched, an inhibit signal on line 124 to control 120 is taken away thereby enabling the control to notify processors A and B of the interrupt by way of lines 56 and 56', respectively.
- interrupt levels 0 - 7, are defined.
- Four interrupt levels, 0 - 3, are reserved for devices to assert directly to the processors, by-passing the subject controller.
- Interrupt levels 1 - 3 are reprogrammed to be forwarded by the controller as levels 4 - 6, respectively.
- Level 0 is reprogrammed as level 7.
- Interrupt level 0 is the software refresh timer which must be received by the processors at least every two milliseconds for proper operation of the processor's internal dynamic memory.
- the state machine program counter 78 is in a reset state at address 00 as represented by block 128. This is a latched state wherein virtual time counters 72 and 72' ( Figure 9) are cleared, but not running, and control registers 94 and 94' and halt control circuits 38 and 88' are cleared. The controller stays in the reset state until one or both of the processors unresets either or both of control registers 94, 94'. This unreset request is the only processor command which need not be given by both processors in order to be acted upon.
- the program counter advances to address 04, as indicated by block 131, and the state machine waits for processors A and B to issue a request to turn the virtual time counters on. Since the virtual counters are already off, the processors should not issue a counters off request or a synchronization request. If either of such requests occurs from either of the processors, the program will report an error as represented by Error Reporting block 135. The particular type of error which has occurred can be determined from the error status data of the
- the subject controller stops further operation thereby permitting the processors to examine the outputs of virtual time counters 72 and 72', control registers 94 and 94' and other circuits to facilitate locating the source of the error.
- the subject controller will issue a status word to the processors indicating that the counters have been turned on.
- the program will then advance to address 57 of a Present Interrupts block 130.
- a comparison is made of the count of the virtual timers. If the counts are identical, any interrupts presently on line 42 ( Figures 1 and 9) will be presented to both processors by the Present Interrupts block. Once all interrupts have been presented, the virtual counters 72 and 72' are enabled.
- the program then makes an unconditional jump from the Present Interrupts block and enters the Main Counters Running Loop at address 17 as indicated by block 133.
- the program will remain in the loop until either an interrupt is received or the processors issues an autohalt signal.
- An autohalt will be issued anytime the processors make a synchronization request, a counters on request or a counters off request.
- the program will jump to address 20.
- the interrupt will then be processed by an Interrupt Handler, as represented by block 132.
- the processor which is most advanced in virtual time will, be halted.
- the program proceeds to address 57 of the Present Interrupt block 130, at which time interrupts are presented to the two processors. If errors are detected by Interrupt Handler 132, the program will proceed to Error Reporting block 135 to report the error to the processors. An exemplary error would indicate that the lagging processor has failed to respond to a halt request.
- Interrupt Handler 132 is not implemented to process autohalts.
- Autohalt Handler 134 responds to autohalt signals which are produced when the processors request that the virtual time counters be turned on, that the counters be turned off or that the processors be synchronized.
- the autohalt signal cause the processor which issued the request to half, independent of the state machine of the subject controller. If processors request that they be synchronized, Autohalt Handler 134 will verify that the processors have been halted at the same virtual time. If the virtual time counters are not equal, then the program proceeds to Error Reporting block 132.
- the processors should not issue a counters on request while the counters are running. Thus, if this request is made, the program will also proceed from the Autohalt Handler to block 135 so that an error will be reported. If the processors make a counters off request, the program will proceed from the Autohalt Handler to Wait For Counters on Request block 131 and wait until a counters on request issues.
- the counters off request allows the processors to diverge and accomplish different tasks in a protected manner.
- the subject controller will not forward interrupts while the counters are off. This ensures that there will be no critical points while the processors are diverged.
- a critical point occurs when data are either being changed and/or tested in the normal execution of the code when an asynchronous event, such as an interrupt, takes place which also changes and/or tests the same data.
- the one interrupt which is forwarded while the counters are off is the software refresh interrupt, a level 7 interrupt which occurs typically every two milliseconds. Steps are taken in the software such that critical points do not occur during a software refresh routine.
- the program will proceed to error reporting block 135.
- the program will then remain in Wait For Counters On Request block 131 until the processors requests that the counters be turned back on. Certain steps must be taken to configure the processors so that they will again execute similar code after having been diverged.
- one of the processors will issue the previously-noted lock request which is received by control registers 94 and 94' ( Figure 9).
- the processor then loops, waiting for the subject controller to issue a refresh-locked-out status bit. When this status bit is active, the subject controller will refrain from forwarding refresh interrupts to the processor.
- the synchronization requests which are processed by Autohalt Handler block 134, are issued by the processors primarily to verify proper processor operation.
- the requests which should be issued fay the processors at the same virtual times, cause the counters to first be turned off and the virtual time counters to be compared. The counters are then turned back on. The counters are turned on at the same virtual time so that the processors will be synchronized in both actual and virtual times. If the processors wish to exchange data, the processors can cause the data to be written in a predetermined location, issue a synchronization request, and then read the data of the other processor.
- the synchronization request ensures that each processor will have written the data before it is read by the other processor.
- processors must utilize a synchronization request in order to synchronize themselves if the virtual time counters are running. For example, if a polling loop were to be used while the processors are waiting for certain data to appear, there is no way to ensure that the processors will execute the loop the same number of times since the processors are not running at identical speeds. Thus, the virtual times of the processors will not be the same and the subject controller will conclude that an error has occurred.
- the subject controller may also be reset by the processors, in which case the sequence will return to Reset block 128.
- a reset request may be made by either or both processors. In the event only one processor makes a reset request, the request command must be forwarded to both control registers 94 and 94'.
- Figure 12 shows a more detailed flow chart further illustrating the operation of Counter Running Main Loop block 133.
- the program will jump to address 20 of Interrupt Handler 132. Otherwise, the counter will increment to address 18 at which time a determination is made as to whether processor A has issued an autohalt signal AAHLT, as indicated by block 138. If the signal has issued, the program will jump to address 63 of Autohalt Handler 134. If the signal has not issued, the counter will advance to address 19 at which time a determination will be made as to whether processor B has issued an autohalt signal BAHLT, as indicated by block 140. If the signal has issued, the program will proceed to the Autohalt Handler, otherwise the program will branch back to address 17 as indicated by block 142.
- Interrupt Handler 132 is shown in the flow chart of Figure 13. As previously described, when an interrupt is received while the program is in the Counters Running Main Loop block, the program branches to address 20 of Interrupt Handler 132 as indicated by element 142. At this time, a time-out timer (not shown) is cleared. The time-out timer is used to determine whether an anticipated event occurs within a predetermined time period. The timer of the present embodiment will time-out after 360 microseconds. A determination is then made as to whether the comparator 86 output on line 88 ( Figure 9) indicates whether the virtual time counter 72 for processor A is greater than the counter for processor B. If so, the program will jump to address 3C as represented by block 144.
- a time-out timer (not shown) is cleared. The time-out timer is used to determine whether an anticipated event occurs within a predetermined time period. The timer of the present embodiment will time-out after 360 microseconds. A determination is then made as to whether the comparator 86 output on line 88 ( Figure 9) indicates whether
- processor A is not leading processor B
- the program will proceed to address 21.
- a determination is then made as to whether comparator 86 indicates that processor B is the leading processor. If this is the case, the program will proceed to address 3A as indicated by block 148. If processor B is not leading, the virtual time of the processor happened to be equal when the interrupt was received. In this unlikely event, the program will proceed to address 22.
- both processors are instructed to halt at the first opportunity utilizing halt controls 88 and 88'. This is signified, by the terms A PGM HALT and B PGM HALT. The processors are designed to actually halt at the end of the processor bus cycle after receipt of these signals.
- block 144 indicates that processor A is instructed to halt at the first opportunity.
- Expression B EQ HALT of block 144 signifies that control register 88' is set such that processor B will halt when the processor B virtual time counter catches up with the counter of halted processor A.
- block 148 indicates that processor B is instructed to halt at the next opportunity and processor A is instructed to halt at the same virtual time as that of halted processor B.
- the program will advance to address 23 as indicated by block 152. At this time, one or both of the processors will have been instructed to halt. However, it is possible that the processors have not yet reached the end of the current bus cycle where they are designed to actually halt. At block 152, the term
- BHLTQQQQ signifies that a determination is made as to whether processor B has received the halt instruction provided by the state machine of subject controller. This is in contradistinction to halt which results from the occurrence of an autohalt signal which is produced when a processor has issued a command to the subject controller. Assuming that the answer is no, the program advances to address 24 where element 154 signifies that a determination is made as to whether processor B has been halted as the result of an autohalt signal. This is represented by the term BAHLT. If this event has occurred, the sequence will be controlled by Autohalt Handler 134.
- the program will then jump to address 3E, as signified by block 156, at which time halt control circuitry 88 and 88' are cleared so that no halt originating from the state machine of the subject controller will be acted upon by the processors.
- the program will then proceed to the Autohalt Handler. Assuming that an autohalt has not occurred, the program will proceed to address 25 as indicated by block 158.
- a determination is then made as to whether the time-out timer has timed out. The 360 microsecond period is sufficiently long such that, unless an anomaly has occurred, processor B will have acknowledged that it received a halt command from the subject controller. Note that the halt command could have originated from any of three sources represented by blocks 144, 148 or 150.
- the program will continue in the loop defined by blocks 152, 154 and 158 until either a time out occurs or the halt command is received by processor B. If a time out occurs, an error is reported to Error Reporting block 132, otherwise, the program will proceed to block 160.
- a determination is made as to whether processor B has actually responded to the controller halt commands. If not, a determination is made at block 162 as to whether a time out has occurred. The program will remain in the loop defined by blocks 160 and 162 until a time out occurs, in which case an error will be reported, or an indication is received that processor B has actually halted.
- processor B has halted, the program will proceed to address 29 as represented by block 164. At this time, a similar sequence for processor A is carried out.
- a determination is made as to whether processor A has received the halt command which originated from the state machine of the controller.
- a test is made as to whether an autohalt has been caused to be issued by processor A.
- a time-out condition is tested at block 168.
- a determination is made as to whether processor A has actually halted.
- both processors When the program has advanced to address 2F, both processors will actually have been halted.
- the subject controller is implemented such that, at this point, the two virtual time counters will be close to one another, but usually not the same.
- the lagging processor will overshoot the leader by one or two counts. In that case, the leading processor, which becomes the lagging processor, will be single stepped by the subject controller until the counts are identical, as described as follows.
- halt control registers 88 and 88' are instructed to halt both processors in the event the processors proceed to run at their normal rate. This precaution is taken because, under certain circumstances, a processor may respond to single step instruction by running rather than single stepping.
- processor B being single-stepped until a determination is made at blocks 180 and 186 that the counters are identical.
- the program will then proceed to address BF at block 192 at which time both processors will be single stepped an identical number of times as a precaution to avoid certain anomalies.
- the program will then advance to Present Interrupts block 130. In the event processor B had overshot processor A, the program would have advanced from block 186 to blocks 188 and 190 so that processor A would be single-stepped in the same manner.
- Autohalt Handler 134 is depicted in the flow chart of Figure 14.
- processors A and B cause autohalt signals to be issued in the event the processors request synchronization, request that the virtual time counters be turned on or request that the virtual time counters be turned off. If an autohalt signal is produced when the program is in Counters Running Main Loop 130, the program will jump to address 63 as indicated by block 192. At this time, the 360 microsecond time-out counter will be cleared. The program will then advance to address 64. In the event an autohalt signal is produced while the program is in the Interrupt Handler, the program will also jump to address 64.
- the program will proceed to address 70 where, as represented by element 198, a determination will be made as to whether the time-out timer has timed out. Assuming that it has not timed out, the program will jump back to block 196. The program will remain in this loop until either a time out has occurred or processor B has caused an autohalt. If a time out occurs, an error message is forwarded to Error Reporting block 135, otherwise the program will branch to address 68. In the event the autohalt was caused by processor B, a similar wait sequence for a processor A autohalt will be performed according to blocks 200 and 202.
- the processors cause themselves to be halted upon their issuance of any of the three previously-noted requests to the subject controller.
- the two virtual time counters should be exactly the same when an autohalt signal has occurred.
- the processor A counter is greater than that of B, an error will be forwarded to Error Reporting block 135.
- the program will proceed to address 69 where, as indicated by block 206, a determination is made as to whether the A counter value is less than the B value. If not, the counters are identical and the program will advance to address 6A. If A is smaller, an error will be forwarded to Error Reporting block 132.
- the virtual time counters are running either because the processors have just requested synchronization or have just requested that the counters be turned off. Accordingly, a request that the counters be turned on is not proper at this time.
- a determination is made as to whether processor A had requested counters on. If so the program will jump to address 76, as indicated by block 210, and a determination will be made if processor B also issued a counter on request. In either case, an error will be forwarded to Error Reporting block 135. Two different error signal are produced to facilitate locating the source of the anomaly by the processors.
- the program will proceed to address 6B as shown by element 212. A determination will then be made as to whether processor A had requested that the counters be turned off. If this request has been made, the program will proceed to address 74, as shown by element 214. At this address, it is determined whether processor B has issued the same request. If not, an error will be forwarded to Error Reporting block 135. If both processors requested that the counters be turned off, the counters are stopped and the program will proceed to address 03, which is Wait For Counter On Request block 131.
- the program will proceed to address 6C as indicated by block 216 and a determination will be made as to whether processor A requested a synchronization. If this request was made, the program will proceed to address 78, as shown in block 218. A determination will then be made as to whether processor B made a similar request. If not, an error signal will be forwarded to Error Reporting block 135. If a similar request was made, the program will jump to address 57 of Present Interrupts block 130. If a synchronization was not requested, an anomaly has occurred since it had already been determined that the processors had not requested that the counters be turned off. These being the only two valid processor requests executed by the Autohalt Handler, an error will be reported.
- FIG. 15 A flow chart further illustrating the operation of Present Interrupts block 130 is shown in Figure 15. If the Present Interrupt block is entered from either Autohalt Handler block 134 or Wait For Counters On Request block 131, the program will proceed to address 57. As indicated by block 222 and 224, if the virtual time counters are not equal at this point, an error is forwarded to Error Reporting block 135. If the counters are equal, the program will advance to address 5A. Also, any outputs from Interrupt Handler 132 enter the Present Interrupts block at this point. As indicated by block 226, a delay timer is reset at this time. The delay timer typically provides a 12 microsecond delay before presenting interrupts to the two processors.
- the program will then proceed to address B5 where a determination is made as to whether the 12 microsecond delay provided by the delay timer has lapsed. The program will remain in this loop until the delay is over and then proceed to address 36. As indicated by block 230, all pending interrupts are then presented to the two processors. In addition, both virtual time counters are cleared. The program then will advance to address 5E as shown by block 232 at which time the halt control circuits 88 and 88' will be cleared, this being required before the processors can be released or un-halted.
- the virtual time counters are off, either because the computer was just powered-up and the processors have not yet requested that the counters be turned on or because the processors had previously requested that the counters be turned off. All halts are cleared from halt control circuits 88 and 88' and the control registers 94 and 94' are cleared.
- the program then proceeds to address 04, as indicated by element 240, and it is determined whether processor A has requested that the virtual time counters be turned on. If such a request has been made, the program will proceed to address OE shown in block 252 at which t.ime the 360 microsecond time-out timer is reset.
- the program will then advance to address OF where a determination is made as to whether processor B has also requested that the counters turned on. Unless the two processors are closely synchronized at this time, the lagging processor, processor B, probably will not have yet made the request, therefore the program will proceed to address 10 as indicated by block 256. If the time-out timer has not timed out, the program will jump back to address OF of block 254. The program will remain in this loop until either 360 microseconds have lapsed, in which case an error signal is transferred to Error Reporting block 135, or processor B has also requested that the counters be turned on. If processor B has made the request within 360 microseconds, the program will proceed to the Present Interrupts block at which time the counters will be turned on.
- processor A has not yet made a turn on request, the program will proceed to address 05, as indicated by block 242. If a request has been made by processor B, the program will then wait for a request by processor A, as represented by blocks 258, 260 and 262. If A does not make a timely request, an error will be reported. If such a request is made, the program will proceed to the Present Interrupts block. Assuming that neither processor has requested a counter turn on, the subject controller will then verify that no improper processor requests have been made. As indicated by blocks 244 and 246, if either processor requests that the counters be turned off, an error will be reported since the counters are already off. Similarly, if either processor should request a synchronization, an error will be reported since such a request is not permitted when the counters are off. If no errors are reported, the program will jump back to address 04 and repeat the sequence.
- microcode listing is for use with a particular implementation of the state machine.
- the details of such particular implementation are shown in Figures 17 through 24 of the drawings.
- Figures 17 through 24 of the drawings show detailed schematic diagrams of one implementation of the subject processor controller which incorporates a state machine which utilizes the submitted microcode instructions.
- the schematic diagrams use conventional circuit symbols which can be readily understood by persons having even less than average skill in the applicable art. Accordingly, further description of the schematic diagrams will not be given, except for the following listing of descriptions of certain ones of the integrated circuits called out in the diagrams:
- the interrupt handler is only executed when the counters are turned on.
- Mcode sets the hard err bit (which can be strapped to NMI) , freezes the counters while in this loop to maybe help with diagnostics. and removes all halts from A&B so that, they are free to diagnose the problem.
- This routine sets two bits in the bit register to indicate that cntrs were off when procs requested counters off.
- First bit is set here,second bit is set by jumping to error routine that sets what was called the TIMEOUT bit but is now used as a modifier bit in combination with the other bits.
- timing pad prior to presenting interrupts, mcode stays at address B5 until timer has timed out, timer is jumper programmable-see page 7 of schematics for timing TIMPADQQ.
- UNEXPECTED SUPERBIT - Super bit is the OR of either proc setting his unused bit in the control reg.
- This bit is not really unused, it is used to tell the microcode to go back to waiting for "CNTR ON RQST" after microcode has detected an error and has sent NMI. However the processors may mistakingly set this bit, in that case the microcode sets the "UNexpectd Superbit” status bit.
- the code then un-halts both procs if they are halted so that they can diagnose the NMI, and the mcode waits for SUPER Bit to first go low, getting rid of the unexpected Super bit and then go hi again telling the microcode to resume normal operation by Timeout ERRORS - Timeout errors are reported by setting the timeout status bit and waiting for the super bit to tell the mcooe to resume normal process son software prefers, normal operation can be resumed by Reseting and then unresetting the board. On the prototype board there is a reset bit, on BETA if this bit is not part of the design then reset will be accomplished by activating the INIT line, probably via the status board. Which brings up the next subject....
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Hardware Redundancy (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US56075983A | 1983-12-12 | 1983-12-12 | |
US560759 | 1983-12-12 |
Publications (2)
Publication Number | Publication Date |
---|---|
EP0164414A1 EP0164414A1 (en) | 1985-12-18 |
EP0164414A4 true EP0164414A4 (en) | 1986-06-05 |
Family
ID=24239250
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP19850900389 Withdrawn EP0164414A4 (en) | 1983-12-12 | 1984-12-10 | Computer processor controller. |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP0164414A4 (en) |
AU (1) | AU3746585A (en) |
WO (1) | WO1985002698A1 (en) |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE3412049A1 (en) * | 1984-03-30 | 1985-10-17 | Licentia Patent-Verwaltungs-Gmbh, 6000 Frankfurt | SIGNAL-SAFE DATA PROCESSING DEVICE |
AU568977B2 (en) * | 1985-05-10 | 1988-01-14 | Tandem Computers Inc. | Dual processor error detection system |
CA1293819C (en) * | 1986-08-29 | 1991-12-31 | Thinking Machines Corporation | Very large scale computer |
CA2003338A1 (en) * | 1987-11-09 | 1990-06-09 | Richard W. Cutts, Jr. | Synchronization of fault-tolerant computer system having multiple processors |
AU616213B2 (en) * | 1987-11-09 | 1991-10-24 | Tandem Computers Incorporated | Method and apparatus for synchronizing a plurality of processors |
US4908502A (en) * | 1988-02-08 | 1990-03-13 | Pitney Bowes Inc. | Fault tolerant smart card |
JPH0271644A (en) * | 1988-09-07 | 1990-03-12 | Toshiba Corp | Master slave type control system |
US4965717A (en) * | 1988-12-09 | 1990-10-23 | Tandem Computers Incorporated | Multiple processor system having shared memory with private-write capability |
AU625293B2 (en) * | 1988-12-09 | 1992-07-09 | Tandem Computers Incorporated | Synchronization of fault-tolerant computer system having multiple processors |
US5203004A (en) * | 1990-01-08 | 1993-04-13 | Tandem Computers Incorporated | Multi-board system having electronic keying and preventing power to improperly connected plug-in board with improperly configured diode connections |
JPH05128080A (en) * | 1991-10-14 | 1993-05-25 | Mitsubishi Electric Corp | Information processor |
US5613127A (en) * | 1992-08-17 | 1997-03-18 | Honeywell Inc. | Separately clocked processor synchronization improvement |
JPH0773059A (en) * | 1993-03-02 | 1995-03-17 | Tandem Comput Inc | Fault-tolerant computer system |
WO1995006277A2 (en) * | 1993-08-18 | 1995-03-02 | Honeywell Inc. | Separately clocked processor synchronization improvement |
FR2742015B1 (en) * | 1995-12-01 | 1998-01-09 | Sextant Avionique | METHOD FOR SECURING AN ACTION AND DEVICE FOR IMPLEMENTING IT |
GB2399190B (en) * | 2003-03-07 | 2005-11-16 | * Zarlink Semiconductor Limited | Parallel processing architecture |
US9732483B2 (en) * | 2015-09-11 | 2017-08-15 | Westfield Retail Solutions, Inc. | Vehicle barrier system |
US20210107512A1 (en) * | 2020-03-27 | 2021-04-15 | Intel Corporation | Computing system for mitigating execution drift |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4196470A (en) * | 1976-12-17 | 1980-04-01 | Telefonaktiebolaget L M Ericsson | Method and arrangement for transfer of data information to two parallelly working computer means |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3810119A (en) * | 1971-05-04 | 1974-05-07 | Us Navy | Processor synchronization scheme |
CH556576A (en) * | 1973-03-28 | 1974-11-29 | Hasler Ag | DEVICE FOR SYNCHRONIZATION OF THREE COMPUTERS. |
US3866184A (en) * | 1973-08-31 | 1975-02-11 | Gte Automatic Electric Lab Inc | Timing monitor circuit for central data processor of digital communication system |
US3909795A (en) * | 1973-08-31 | 1975-09-30 | Gte Automatic Electric Lab Inc | Program timing circuitry for central data processor of digital communications system |
US4456952A (en) * | 1977-03-17 | 1984-06-26 | Honeywell Information Systems Inc. | Data processing system having redundant control processors for fault detection |
US4358823A (en) * | 1977-03-25 | 1982-11-09 | Trw, Inc. | Double redundant processor |
IT1111606B (en) * | 1978-03-03 | 1986-01-13 | Cselt Centro Studi Lab Telecom | MULTI-CONFIGURABLE MODULAR PROCESSING SYSTEM INTEGRATED WITH A PRE-PROCESSING SYSTEM |
US4428044A (en) * | 1979-09-20 | 1984-01-24 | Bell Telephone Laboratories, Incorporated | Peripheral unit controller |
US4453215A (en) * | 1981-10-01 | 1984-06-05 | Stratus Computer, Inc. | Central processing apparatus for fault-tolerant computing |
-
1984
- 1984-12-10 AU AU37465/85A patent/AU3746585A/en not_active Abandoned
- 1984-12-10 WO PCT/US1984/002011 patent/WO1985002698A1/en not_active Application Discontinuation
- 1984-12-10 EP EP19850900389 patent/EP0164414A4/en not_active Withdrawn
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4196470A (en) * | 1976-12-17 | 1980-04-01 | Telefonaktiebolaget L M Ericsson | Method and arrangement for transfer of data information to two parallelly working computer means |
Non-Patent Citations (2)
Title |
---|
COMPUTER DESIGN, vol. 21, no. 11, November 1982, pages 211,212,215,216,218,220, Winchester, US; J.H. WENSLEY: "Fault tolerant systems can prevent timing problems" * |
See also references of WO8502698A1 * |
Also Published As
Publication number | Publication date |
---|---|
EP0164414A1 (en) | 1985-12-18 |
AU3746585A (en) | 1985-06-26 |
WO1985002698A1 (en) | 1985-06-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO1985002698A1 (en) | Computer processor controller | |
US5068851A (en) | Apparatus and method for documenting faults in computing modules | |
US4695946A (en) | Maintenance subsystem for computer network including power control and remote diagnostic center | |
CA1235524A (en) | User interface processor for computer network | |
CA1310129C (en) | Interface of non-fault tolerant components to fault tolerant system | |
JPH01154240A (en) | Double-rail processor with error check function added to single-rail interface | |
CA1306546C (en) | Dual zone, fault tolerant computer system with error checking on i/o writes | |
US5345566A (en) | Method and apparatus for controlling dual bus system | |
JPH02118872A (en) | Dual rail processor having error checking function for reading of i/o | |
US5185877A (en) | Protocol for transfer of DMA data | |
US5153881A (en) | Method of handling errors in software | |
US4358823A (en) | Double redundant processor | |
US5249187A (en) | Dual rail processors with error checking on I/O reads | |
US5251227A (en) | Targeted resets in a data processor including a trace memory to store transactions | |
US5291494A (en) | Method of handling errors in software | |
EP0496506A2 (en) | A processing unit for a computer and a computer system incorporating such a processing unit | |
JPH0833874B2 (en) | Device for synchronizing multiple processors | |
US5163138A (en) | Protocol for read write transfers via switching logic by transmitting and retransmitting an address | |
EP0415546A2 (en) | Memory device | |
US20040193735A1 (en) | Method and circuit arrangement for synchronization of synchronously or asynchronously clocked processor units | |
US20050229035A1 (en) | Method for event synchronisation, especially for processors of fault-tolerant systems | |
EP0411805A2 (en) | Bulk memory transfer during resync | |
EP0416732B1 (en) | Targeted resets in a data processor | |
JPH04102935A (en) | Device and method for monitoring power on sequence of recording device | |
JPH06274469A (en) | Constitution control system and test system for computer system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 19850806 |
|
AK | Designated contracting states |
Designated state(s): AT BE CH DE FR GB LI LU NL SE |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 19860605 |
|
17Q | First examination report despatched |
Effective date: 19881028 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 19890308 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: KATIN, NEIL, A. Inventor name: KOLB, WILLIAM, W. Inventor name: MCMURRAY, RICHARD, D. |