JPH10228458A - Multiprocessor computer having configurable hardware system domain - Google Patents

Abstract

PROBLEM TO BE SOLVED: To separate an entire computer into many independent units from the viewpoint of both software and hardwares by allowing a global address router and a global data router to interconnect different system unit groups. SOLUTION: Each LAS 55L3 on a system unit is connected to one of four different GABs 55G1. An arbiter 55L4 physically consists of four same parts, and each of the parts responds to an access request from lines 811 and communicates with the GAA 55G2 and one of the different LAS 55L3. That is, entire functions of a local part where address router 450 is connected and a global part establishes schedule of the four GABs 55G1 among competitive requests from six ports of all of system unit group 410. The decision to all of the four GABs 55G1 is mutually and simultaneously processed in LAA 55L4 of each system unit.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic computer, and more particularly to an electronic computer, in which a large number of processors can be dynamically separated into a variable set, that is, a group of domains, to obtain processing independence and to reduce hardware errors. Nevertheless, it relates to a multi-processor architecture that allows it to continue to operate.

[0002]

2. Description of the Related Art Many centralized mainframe computers that drive many simple terminals have been replaced by networks of personal computers.
Most of these networks incorporate one or more server computers, which store data and programs for individual users. In fact, the server is a high-performance superserver that has inherited many of the attributes of the mainframe.
It is developing into. However, the super server
Functionality is different in networked systems, and therefore the architecture of the superserver must be different from the architecture of the mainframe or personal computer.

[0003]

What sets superservers apart from other architectures is that they contain one or more operating systems-or one or more versions of one type of operating system,
Because it must be able to run simultaneously for different jobs or for different users.

[0004] Also, the super server must have very high availability and reliability. They must be highly resistant to hardware and software errors, and should be able to provide services while the computer is running. Unlike the single (or many closely coupled) processor architectures of personal computers, and unlike the massively (super) parallel design of supercomputers, superservers have a wide range of unpredictable resource requirements. Need the flexibility to perform a large number and wide variety of tasks.

[0005] Many methods require that the superserver run as a very large computer and as a small computer. This places a number of conflicting (conflicting) requirements on those architectures.

[0006]

SUMMARY OF THE INVENTION The present invention addresses the foregoing and other related problems by means of software configurable "hardware domains," which are used in terms of both software and hardware. Provides a comprehensive computer architecture that separates the whole into a number of independent units. That is, different domains not only run different operating systems and applications independently of each other, but also operate independently of fatal (serious) hardware errors that have occurred in other domains. "Cluster (clu
ster) "allows a plurality of domains to share a common range of memory addresses, thereby speeding up data transfer. Privileged configuration-the control software partitions computer resources into multiple domains and domain clusters without physically modifying the computer, and reconfigures multiple domains and clusters at any one time Is enabled by an operator or software process. Computers using this architecture are readily available, commercially available component components, such as microprocessors commonly used in personal and workstation computers.
Can be built with

[0007] The present invention is directed to testing new versions of software in a completely isolated environment.
While continuing normal tasks on the rest of the computer. One part of the computer can perform extended diagnostics or preventive maintenance, while the other part can perform normal user tasks at the same time. Different parts of the same computer can run under different operating system software (or different versions of the same software, different tunings or parameter settings), time sharing and database queries Or a plurality of different types of workloads, such as online transaction processing and decision support systems.

Each part of the computer is insensitive to hardware failures, such as hard memory errors and malfunctioning address request lines, as well as software errors in other parts. The computer according to the present invention
Hardware faults prevent address or data signals from being accidentally transferred to any processor or memory not in the same hardware domain, and many system-wide control signals differ. Physically prevent the hardware in the domain from being affected.

[0009] Further advantages will be apparent to those skilled in the art.
For example, interactive (interactive) jobs can be separated from batch jobs by running in different domains. Production tasks can run uninterrupted in one domain and development or problem isolation can occur simultaneously in other domains. The compatibility of the newly released software can be checked on the same system running the old version at the same time. Sometimes, multiple organizations (organiz
ation) share the same system and use separate domains, each of which can guarantee some level of resource occupancy for their own tasks,
Can be easily scheduled and short notices by simply reconfiguring domains and clusters under software control without physically replacing components or manually switching signal lines It is also possible to change with.

[0010] Briefly, a computer according to the present invention has a number of individual system units, each having a processor, a memory segment, and / or an input / output adapter. A central interconnect transports addresses and data between system units. A domain controller dynamically configures the domain filters and functions independently of one another to form domains that are independent of even significant hardware errors in other domains. One domain's processors, memory and I / O function as a single, centralized computer system, the physical locations of which may be on the same system unit or on different system units It is. In addition, multiple domains can dynamically interconnect with the cluster and share some or all of their memory space. The domains and clusters are defined by register contents set under software control.

[0011] All communications between the various system units occur as "transactions" on the interconnect. Transactions can include memory addresses, but some do not. Normal memory transactions are performed on potentially cacheable main memory, such as by non-privileged application programs. Other transactions are to system control registers (non-cacheable), I / O
Some are for the portion of the address space used by the adapter, the latter being only accessible by privileged mode code, such as system boot, OS kernel, and I / O drivers. Still other transactions can be interrupted.

The domains are separated from each other in both software and hardware. Each subsystem consists of multiple system cards, boards,
Alternatively, other units can be provided that can potentially incorporate hardware for processing memory and / or I / O functions. The individual system units do not all have to contain all the functions of a complete processor, but the set of units forming the domain must, among other things, contain all the functions of a complete data processing system. Must. A single system unit may form one domain. Any system unit can belong to 1
Only one domain. A domain functions as a single data processing system, whose individual system units are not secret from each other.

[0013] "Software isolation" means that software running in one domain cannot affect software running in another domain unless there is a hardware failure in the subsystem. I do.
This requires that each domain has its own physical processor (s), multiple memory units, and I / O adapters that are not shared with other domains. The domain filter hardware between each system unit and the common address interconnect hardware has a mask register that contains a separate bit for each unit potentially included in the complete system. The state of these bits indicates which of the other systems are members of the same domain. The unit's interface responds to the transaction only if the transaction from the interconnect originated in a system unit in the same domain. The hardware distributed between subsystems in this manner is not sufficient to guarantee software separation, as long as the hardware can only be controlled by an agency outside the subsystem, such as a separate service processor. is there.

[0014] In addition, "hardware isolation" indicates that hardware errors that occur in one domain do not affect the operation of different domains in the computer. Hardware separation cannot be implemented if a common bus architecture is used between the individual subsystems. This is because the failed subsystem can shut down all buses. Thus, switching interconnects, such as crossbars and / or routers, are used between the subsystems. 1
Hardware failures within one subsystem can be disguised as belonging to different subsystems, or can cause fatal interface signals throughout the system, such as control signal parity errors. , Subsystem hardware isolation requires some central control logic other than the subsystem itself, and couples at least some of the control signals one-to-one (point-to-point) between this central logic and each subsystem. It is necessary. If the interconnect hardware also has a domain mask register, each system in the originating domain has a "valid transaction"
A signal can be generated. This prevents any unit from impersonating like other units. Since all units other than the source domain ignore the transaction, they cannot generate an error condition on hardware error signals originating from other domains. Failures in the interconnect hardware itself can also affect all domains,
In practice, the interconnects are small and robust when compared to the hardware in the subsystem.

In some applications, certain domains may require high bandwidth communication to communicate with each other by sharing one or more segments of their individually addressable memory space. The present invention can provide a cluster of domains having properties similar to those of individual domains. One individual system unit can be its own cluster, any single unit can be a member of only one cluster,
Cluster relationships change. Also, the domains are in exactly one cluster, and a cluster includes one or more domains. The need for the cluster relationship to be variable stems from its use in sharing memory between domains. If A exports memory to domains B and C, B and C must respond to each other's transactions on the interconnect and must therefore be in the same cluster. This requirement is that in the system described above, if the current value of the data from the shared memory in A may actually be located in the cache in B or C, and the processor in B writes a new value to this address, The copy in C's cache must be invalidated, and to achieve this, C
Arises from the possibility that all transactions from B must be seen.

A system unit in a cluster need not share memory with each other unit in the same cluster, but can share memory only with units in the same cluster. If system unit A exports a range of shared addresses to unit B, B is necessarily in the same cluster, but units C in the same cluster as A and B do not need to share this address range. . Within the same cluster, any system unit can be a transaction originating unit
While receiving the transaction, it is not always necessary to respond to the transaction. That is,
The receiving unit can also filter it. In practice, the domains are combined in a cluster only to share memory, and all system units in all member domains of the cluster are It is configured to respond to normal memory transactions for a specific range of memory addresses corresponding to the shared memory. The shared memory itself is the one in the cluster.
Located on one system unit, this range is said to be "exported" to domains in the cluster other than itself. (A unit always responds to all transactions from source units in its own domain. This is not an "export" in the sense used here. The term It only means adding a response to the memory transaction.) Therefore, the system unit
It may include a cacheable memory that is inaccessible from units exporting to different memory address ranges. The system control registers and I / O devices in a unit have no access to units in different domains. Even if these units belong to the same cluster, they cannot be accessed.

The clustering adds a shared memory register to the domain registers on each system unit, which addresses are shared, ie, at least one system unit in another domain in the cluster. Range registers that indicate if they are to be exported. The shared memory register indicates which of the other units can transfer addresses with that unit. Therefore,
The system unit responds to addresses in transactions from other units because (a) the source unit is a member of the same domain, or (b) it is a member of the same cluster. Specified in a memory register and only if the address is within the range (if any) specified to be shared, and for normal memory transactions as defined above. Only do it. The domain registers in the interconnect are used by other systems in the same cluster as the unit that originated the transaction.
A validity confirmation signal can be sent to the unit group,
It becomes a domain cluster register.

[0018]

FIG. 1 shows a prior art computer 100 having an architecture typical of a server or medium computer. The computer 100 includes a processor 1 on a mother board or a plug-in board.
10, a separate board 120 for memory and a separate board 130 for I / O adapters. A data bus 140 and an address bus 150 couple different functional blocks together. Control distribution bus 160 sends control signals, including error signals, to various boards. If the system is large, it may have dedicated control and service units 170 for startup, diagnostics, and similar functions.

A bar 101 schematically represents the entire address space of the computer 100. The processor or processors (s) send out all addresses on the bus 150 for the board or unit 120 containing the memory. Each board has memory responsive to a range of addresses. Each board contains a different address range, usually set by mechanical switches or registers on the memory board. The processor 110 also communicates with all of the I / O adapters on all the boards 130.

FIG. 2 illustrates a number of system units 210.
Each include memory within the processor itself, and I / O adapters coupled to each other, allowing the unit to potentially function as a complete computer by itself. 2
00 is shown. (However, depending on the system unit, actually only the processor, only the memory, the I / O
Some may include only some or some of the sub-combinations of all of their potential functions. 2.) Individual system units 210 may transmit addressed data within the same unit, or the centerplane interconnect structure (centerpla) into which the system units are plugged.
The addressed data is transmitted over high speed routers 240, 250, which are constructed as ne interconnect structures, to other system units in the same complex. Control distribution bus 260 sends control and error signals to all system units 210. Such computers are not limited to the typical personal or medium computer bus organization. For example, data and address router 24
0,250 may be implemented as regular point-to-point wiring, cross-point switches, or multiple arbitrated buses. The entire system 200 can be characterized as a shared memory symmetric multiprocessor system. Preferably, a coherent cache function is also used. This mechanism may be implemented in a number of conventional ways. The commercially available CS6400 available from Sun Microsystems is an example of this type of machine.

A bar 201 indicates the address space of the computer 200. Although each system unit is potentially a complete computer by itself, the interconnect provided by the address router 250 places all system units throughout a common address space. That is, the fully authorized address of every memory location on each system unit 210 must be different from that of every memory location on every other unit.

The error and status signals on distribution bus 260 affect every unit 210 in the system. For example, the error correction code is
It corrects the bit errors on 50 and generates a fatal error signal for other errors where the code is detectable but not correctable. Such a catastrophic error signal usually indicates the fault that caused the error in a single system.
Even if trapped in a unit or router location, it will affect the entire system. The failed system
The unit can continuously assert an error signal, shutting down the entire system. CANCEL (sometimes AB
ORT) signal presents a different situation. Some high-performance systems speculatively start multi-cycle processing and cancel processing if those assumptions are incorrect. CA in such a single domain system
Asserting NCEL will stop any unit in the entire system.

FIG. 3 shows one hypothetical computer 300.
Wherein the various system boards 310 corresponding to the unit 210 of FIG. 2 are physically divided into a number of domains, each with its own physically separate data router or bus 340, It may have its own address router or bus 350 and its own control distribution means or bus 360, as well as its own system controller 370. In practice, computer 300 will be a number of separate computers or domains S1, S2, S3. In addition, multiple domains can share some or all of their memory addresses to form clusters, as indicated by area 351 of address router 350. In FIG. 3, domain S1 is itself a (denatured) cluster CA, while domains S2 and S3 together form a cluster CB. The address spaces of these separate clusters may overlap with each other, each may run its own operating system independently of the others, and may have memory failures or other hardware in one domain cluster Errors do not affect the operation of other domain clusters.

Bars 301, 302, 303 indicate that the memory address space of computer 300 can be treated as three separate spaces, some or all of whose addresses may overlap with each other. In addition, some memory addresses can be physically shared between multiple domains, as shown at 304 and 305. An area 351 bridging the lower two address routers 350 symbolizes a memory address shared between different domains.

In computer 300, many of the control signals can be separated from domains to which they cannot be applied. Thus, a fatal error in domain S1 (such as an uncorrectable error in the address or control bus) will generate an ARBSTOP signal on bus 360 only within the domain cluster in question and the domain S2, S3 Continue operation. However, the system 300 must be manually configured permanently or at least semi-permanently. That is, reconfiguration requires complete shutdown of the system and rewiring or manual adjustment for board relocation or switch reset. The system cannot be dynamically or easily reconfigured into different domains or clusters with variable amounts of resources.

FIG. 4 is a block diagram of the computer of FIG. 2 and FIG.
Building on the background of the systems 200, 300, an overview of the preferred form of the invention in an example environment is presented. Many of the details set forth below are not themselves directly related to the concepts of the present invention, but are helpful in understanding how the present invention functions in this environment.

The computer 400 has a system unit group 410 corresponding to the unit group 310 in FIG. Data router 440 physically interconnects all system units 410. Address router 450 and control bus 460 physically couple all system units, just as in FIG.
In computer 400, however, an additional domain filter 480 allows computer 400 to
It is electronically divided into domains and clusters that can operate independently of each other. Router 450 and unit group 4
The placement of the ten domain filters symbolizes that it acts on address and control signals to achieve domain separation. In a preferred embodiment, the filter 480 is physically located in a chip forming part of the address router 450, and the router 450 itself
Physically located partially within system unit 410 and partially within a common centerplane structure. Filter 480 may also include components located within data router 440. Domain configurator 420 is filter 4
80 and arbitrarily and dynamically set domains and clusters. The example in FIG. 4 has the same memory address maps 401-403 as the corresponding maps 301-303 in FIG.

As will be described in detail later, the numbers next to the map indicate addresses at various points. The numbers are hexadecimal and, in the exemplary embodiment, range from '00 0000 0000 'to' 0F FFFF FFFF '.
(FIG. 4 shows only eight of the 16 possible system units, so the first half of this space, address '08 00
Includes only up to 000000 '. Also, the system employs addresses from '10 0000 0000 'to' 1F FFFF FFFF 'as another space for accessing system registers and I / O devices. With some details not relevant to the present invention, the exemplary system architecture assigns each system unit 410 an address range of 4 gigabytes (GB). Each range starts with the assigned unit number multiplied by 4 GB, but any memory actually mounted on the unit may begin and end anywhere within the assigned range.
This almost always results in a puncture in the address range of the implemented memory, but the system 400
Addresses this situation easily. Other systems, however, can readily implement the present invention with completely different memory architectures.

FIG. 5 shows the system unit 410 of FIG.
To the data router 440, the address router 450,
And a portion of the domain filter 480. These are part of the system unit and are connected to it. FIG. 5 shows little of the individual control lines of the distributor 460 managed by the domain filter of the present invention. 8-10, representative control signals will be shown and discussed in more detail.
In this example, computer 400 has eight of sixteen system units 410 that can be implemented.

System unit 410 includes space and wiring on one physical structure, such as a circuit board, for all of the major components 110-130 of computer 100, all of which are specific units. It is not necessary to provide them completely or even partially.
The processor subsystem 510 can have up to four microprocessors 511, each with its own cache 512. I / O subsystem 530
Includes two system I / O buses 531, each controlling a variety of conventional I / O adapters 532. on the other hand,
I / O adapter 532 couples line 533 to external I / O devices such as disk drives, terminal controllers, and communication ports. Memory subsystem 520
Contains up to four banks of memory 521, each pair comprising a conventional pack / unpack module (pack / unpack m).
odule) 522. Specialized boards can be used as an alternative to completely general-purpose system units. For example, a first type of system unit may have wiring and locations only for the processor and memory subsystem, and a second type of one or more I / Os.
It may include only the / O subsystem.

The data router 440 is connected to the subsystem 5
Transaction data is exchanged between 10 and 530. In this embodiment, the data router is physically divided between a local portion 54L0 on each system unit 410 and a global portion 54G0 located on the center plane. In FIG. 5, the label 54L0
Are the components 54L1-5 of the data router 440.
Label L shows the entire local part with 4L3
0 indicates the entire global part having the components 54G1 to 54G2.

Each buffer 5 of the local router 54L0
4L1 has, for example, a small amount of high-speed static RAM capable of holding 256 transactions. Its usual purpose is to provide a holding queue for separating and smoothing the flow of data against possible bursts of activity. Local data switch 54L2 is a full-duplex bidirectional 1x4 crossbar. Local data arbiter 54L3
Receives permission from global arbiter 54G2 and stores at most one buffer with the corresponding transaction
Instructs to store the packet. At the same time, the local arbiter 54L3 uses the conventional fair algorithm (f
Using the airness algorithm), a waiting packet is selected from one of the buffers, and a transmission request to the global arbiter 54G2 is generated instead of this packet. The global data router 54G0 transfers data from the LDR 54L0 of one system unit to the LDR 54L0 of the same unit or a different unit. At this time, the 16 × 16 crossbar array 54G1
Which receives 16 sets of 4-bit steering logic from arbiter 54G2. At the lower level, this is one for each system unit,
Implemented as 16 16-input multiplexers.

The address router 450 is connected to each system
Addresses are transferred between the subsystems 510 to 530 on the unit 410, and addresses are transferred from one system unit to another system unit. Similar to the data router, it has a local part indicated by 55L0 and a global part 55G0. In this embodiment, address routing follows the same path for both local (intrasystem) and global (intersystem) transactions.
Port controller group 55L1 and memory controller 55L2 provide a conventional interface between subsystems 510-530 and individual routing switches 55L3. In this case, the individual processors 511, the I / O bus 531 and the memory unit 52
1 is effectively a local address switch (LAS:
(local address switch) may be regarded as being directly connected to 55L3. The LAS group 55L3 is
Performs a number of conventional functions, such as cache coherency. For the purposes of the present invention, their functions are: a processor group 511, an I / O bus group 531 and a memory 5 in the system unit.
21 is to transfer the address from the global address router 21 to the global address router 55G0.

The global portion 55G0 of the address router 450 of this embodiment has four address buses 55G1 shared among 16 system units. Separate global address arbiter 55G2
Is the local address arbiter 55 in each unit.
Each address bus is assigned to a different system unit 410 in response to a transaction request from L4.

In the present embodiment, each LAS 55L3 on the system unit connects to a different one of the four GABs 55G1, as symbolized by the open circles on lines 915 and 922. Arbiter 55L4 physically consists of four identical parts, each of which responds to an access request from line 811 with GAA 55G2 and L
It communicates with a different one of the AS55L3s. That is, the overall function of the combined local and global portions of the address router 450 is the system unit group 4
10 conflicting requests from all six ports (two from each of the controllers 55L1)
G1 schedule. 4 GABs
The decisions for all 55G1 are processed simultaneously for each other in the LAA 55L4 of each system unit.

FIG. 6 illustrates the relevant address routing within the conventional port controller 55L1 of FIG. Each controller chip includes address lines and control lines, and includes two processors or two I / O lines.
The O bus is interfaced with any of the four address buses. The bidirectional driver / receiver 610
An outbound transaction is sent out on line 611 to a first in / first out (FIFO) buffer 620. The switch group 621 outputs the FIFO output to the bidirectional driver / receiver 630, and outputs the FIFO output to the line 911.
Through the local address router 55L3 of FIG. An inbound transaction from line 911 is sent from driver / receiver 630 to FIFO 640.
Proceed to. Multiplexers 641 make a selection from the stored transactions and send them to driver / receiver 610 for transmission over line 611. Line 811 is a switch group 621, a multiplexer group 641,
And controls other components (not shown) in the port controller 55L1. Also, the components of the controller send the status method and other information to the local address arbiter 55L4 in FIG.

FIG. 7 shows relevant parts of the conventional memory controller 55L2 of FIG. This chip
The controller 55L1 is a DRAM memory chip 521
Performs the same functions as those performed for the four banks. The line 911 from the local address router 55L3 in FIG.
To supply. Crossbar switch 720 passes the addresses from these transactions to line 4 via line 721.
To the number of memory banks. That is, as long as the data is located in different subranges of the memory addresses of the memory segments located within the unit, multiple banks on the same system unit 410 can simultaneously read and write data. it can. Normal arbitration logic 722 assigns individual FIFOs 710 to various outputs 721. Line 948 from FIG. 9 cancels (cancels) memory accesses from transactions that are not made visible to this system unit.

FIG. 8 shows details of the local address arbiter 55L4 of FIG. Each system unit 41
0 includes one LAA chip 55L4. Each port
Controller 55L1 raises the available global address bus 55G1 by raising a GAB request signal to a queue in FIFO buffer 810.
In one, a shot may be requested. These lines are the normal port control lines 8 shown in FIG.
11 is formed. The arbitration logic 820 makes a selection between these requests using any of a number of conventional fair algorithms, and then raises the GAB request and direction control line 821. The request line provides an indication as to whether the logic 820 wants to access a particular global address bus. If the global address arbiter 55G2 grants a request for a particular address bus on line 822, the arbiter chip 55L4 sends the appropriate LAS chip 5 via line 823.
Notify 5L3.

The LAA 55L4 can interrupt the operation of the computer 400 in a number of ways. A fatal error, such as a parity error on the system unit, can generate an ARBSTOP control signal on line 824. That is, the LAA acts as a generator of the ARBSTOP control signal. In a typical computer, this signal would be broadcast directly through the control distributor 460 to the ARBSTOP detect line 827 in the LAA of each of the other system units.
Therefore, a fatal error in one unit is
Typically, shutting down each system unit of the entire computer 400 results in contamination of user data.
(corruption) as well as allowing an immediate dump for failure analysis. As will be described with reference to FIG. 10, however, the computer filters this signal (f
ilter), and only the system units in the same domain cluster are one of the units in the domain cluster.
To receive the ARBSTOP signal coming out of one.

The system units can also assert HOLD control signals 825 for all other units on their corresponding detection lines. Usually,
Outbound HOLD from any system unit
The signal also goes directly to the corresponding inbound HOLD line 826 of each other unit, so whenever the system unit's input queue is saturated with pending processing,
Prohibits the entire computer from requesting further transactions. In addition, the faulty system unit 410 may shut down the entire computer by asserting HOLD continuously. However, in FIG. 10, this signal is also filtered, so line 8
The outbound HOLD on 25 only affects the incoming HOLD 826 on the system units in the same domain cluster.

As described above, the local address arbiter 55L4 has a possibility of affecting the operation of another system unit, such as GAB REQ, ARBSTOP-out, HOLD-o.
Acts as a generator of control signals such as ut. This is also because these control signals GA from other system units
Also acts as a recipient of B GRANT, ARBSTOP-in and HOLD-in. For conventional systems, all systems
The exit control signals from the units are simply tied together and sent to the receivers of all other units. However, the system does not
Because of the filtering, the signal generated at one LAA only affects the LAA of units within the same domain or domain cluster only. As will become apparent below, other operating devices may also be passed by a domain filter according to different domain definitions,
Acts as a generator and receiver of control signals that can be blocked.

FIG. 9 illustrates the domain filter 480 of FIG.
5, the local address router chip 55L3 of FIG. 5 is shown in detail. The overall operation is complex and requires a large number of cycles per transaction, but the purpose in this case is that of FIG.
Each address bus 55G1 carries address bits and some control signals for a given transaction.

Outbound address control section 910
Represents each port controller 55L1 on the line group 911.
It receives the transaction addresses from (FIG. 5) and sends them through the error correction code generator 912 to the FIFO buffer 913. Multiplexer 914 selectively couples a waiting address through driver 916 onto outgoing global address line 915 depending on those characteristics established by local address arbiter 55L4 and communicated over line 823.

Whenever the VALID signal on line 1023 indicates to a particular system unit 410 that a transaction is valid, a conventional incoming address switch 920 may be connected to the receiving unit via incoming address line 922. At 921, a transaction address from the global address bus is received. If this line remains inactive, LAS 55L3 will treat the corresponding bus cycle as an idle cycle and take no action. The address from a valid transaction goes directly to memory controller 55L2 via line 923. Addresses from other system units are passed through the ECC decoder 924 and the cache coherency unit 930 to receive incoming address
Proceed to switch 925. Some of the addresses are read /
Some go to the switch 925 through the write control unit 926 or the response control unit 927. Finally, the switch unit 925 sets the incoming address on the appropriate line 911,
Pass through one of the port controller groups 55L1.

Block 930 maintains coherency between the caches of FIG. 5 in the usual manner. Line 931 is
When the cache control unit 930 determines that the operation should be interrupted, it generates a CANCEL control signal from its own system unit. High performance systems can perform a process speculatively over a number of clock cycles, while concurrently making a determination as to whether or not the process should really be performed. If the condition for executing this process is not satisfied, the outgoing line 931 is sent through the control distributor 460 to the incoming CANCEL line 93 of all the other system units.
2 to broadcast a CANCEL signal. As a result, the cache control unit 930 sends a request to the memory controller 55L2.
Assert the MEM CANCEL line 948 to inhibit the completion of any memory operations before data can be changed. For example, the memory reads from RAM while the system rather determines whether the current value is located in a cache of one of the processors. In this case as well, the domain filter 480 determines that the CANCEL-out signal 931 from one system unit is not connected to the CANCEL-in line 932 of a unit not in the same domain cluster.
, Each cluster can operate independently of the other clusters with respect to this and other control signals. Line 933 also
Cancel any on-board memory activity over line 948. This will be described below.

The system 400 does not distinguish between transactions occurring and ending in different system units 410 and transactions originating and ending in the same unit. All transactions are handled by the global address
Pass the bus. This is because each cache controller in a domain or cluster must know about transactions on cache lines of all other caches in the same group.

FIG. 4 for each arbiter chip 55L3
The local portion 940 of the domain filter 480 of
Other chips 55L3 in the same system unit 410
, And carry the same data at all times. However, each copy of block 940 receives an incoming address line 921 from a different one of bus groups 55G1 via line 921.

Comparator 941 detects a match between the address from line 922 and each of the four registers 942 and 944-946.

The domain mask register (DMR: dom
ain mask register) 942 contains 16 bits, one bit for each possible system unit in computer 400.
With bits. The bit position for each domain register (each copy) in each system unit in a given domain includes a "1" bit in all other registers of the system unit in the same domain. FIG.
Using the example of, the first four system units (4
10-0 to 410-3) are defined as the first domain, the next two (410-4 and 410-5) form the second domain, and the next two (410-6 and -7) Assume that the third domain is configured and that there are only 8 out of 16 possible system units. Then, the domain mask registers 942 of the eight mounted system units 410 contain the following values:

[0050]

[Table 1]

Again, the same system unit 410
All four copies of register 942 contain the same value.

Line 922 includes a signal representing the number of the particular system unit 410 that issued the current transaction. If the corresponding bit of the DMR 942 of the receiving unit is not turned on, the comparator 941 sets the NON-DOMAI
An inhibit signal is generated on the N line 943 and the incoming switch 925
Prohibits the transaction from passing to the port group 55L1 of FIG. 5 through the line 911. The comparator also has a line 94
9 and OR gate 901, MEMO on line 948
Generate RY CANCEL inhibit signal. This signal instructs the memory controller 55L2 to ignore the address on line 923 if the current transaction originated outside the domain. This effectively separates the domains and makes them insensitive to transactions originating in other domains.

As described above, the system units in the different domains communicate with each other only through external I / O devices, such as serial communication lines interconnected by dedicated wiring, such as 533 in FIG. Can be exchanged. Cooperating different domains in a greatly accelerated manner would extend computer 400 to many uses. To this end, the domain filter 480 of FIG. 4 can also aggregate multiple domains together into a single cluster. Domains within a cluster can share some or all of their memory with each other. If a processor in one domain writes data to a pre-defined range of address space, a processor in another domain of the same cluster can read this data. That is, different domains within a cluster have a range of shared memory. This memory can be physically located in any system unit in the cluster, accessed through the global address router 55G0 of FIG. 5 and any other system unit in any domain in the cluster. The transfer of data to the unit can also be performed through the global data router 54G0.

Shared memory mask register 94 located in each copy of local domain filter 940
4 defines which system units include the physical RAM 521 to be exported to other units as shared memory in the cluster defined by the cluster register 1020 in FIG. The contents of each SMMR 944 within the same system unit is identical.

Each SMMR 944 has a computer 400
1 bit for each possible system unit within
6 bits, each system unit 410 has a global address bus 55G
It has four copies of its own SMMR, one copy for each. Bit position j for SMMR in system unit 410-i in a given cluster
Means that the unit 401-i is the system unit 410-
Include a "1" value if any memory transaction from j should be responded to. Returning to the example shown in FIG. 4, two units 410-4 and 410-5 of the second domain are replaced with two units 410-6 and 410-7 of the third domain.
And form a cluster with unit 410-4, assuming that unit 410-4 exports the shared memory to a domain consisting of units 410-6 and 410-7. That is, unit 4
At least some of the memory of the address numbers physically implemented on the unit 10-4, as if the memory were implemented on the latter unit, the unit 410-
Reading and writing are possible under the same address number by the processor on 6,410-7.
(Again, the values for bit positions 8-F are not significant, as only 8 of the 16 possible system units are available.) Therefore, system unit 4
The ten SMMRs include the following values:

[0056]

[Table 2]

Bit positions 8 to F in all registers
Is “0”. Because the corresponding system
This is because the unit does not exist. Unit 410-0
To 410-3 have no value of "1". This is because they are in the same domain, and none of the units in that domain export memory to other domains. Units 410-4 through 410
A "1" value up to bit 7 for -7 indicates that these units must respond to normal memory transactions from all of units 410-4 through 410-7 and implement shared memory. The memory is located on one of these units (e.g., 410-4), but the specific location cannot be inferred from SMMR 944. Since cache coherency is required on shared memory, all units that use this shared memory will receive
You must see all transactions within this address range.

Register 944 alone will be sufficient to indicate whether all of the memory of the system unit is shared or not at all. However, in almost all cases, it is desirable to share only a specified portion of the memory between the domain groups of the cluster.
Registers 945 and 946 specify the boundaries of the address range to be shared in a particular cluster. Each shared memory base register (SMBR: shar) on each system unit that has access to the shared memory in the cluster
The ed-memory base register 945 includes the lowest address in the entire address space of the computer 400 to be shared. In the example of FIG. 4, the unit 410-4 has the address' 0
4 0000 0000 'to '04 FFFF FFFF' memory is physically accommodated, but exported is the upper 1 GB, that is,
Only the memory from address '04 C0000 0000 'to' 04FFFF FFFF '. Since only the upper 25 bits of the 41-bit address are actually stored in the register 945, the granularity of the shared memory is 64 Kbytes. Therefore, the units 410-4 through 41
An SMBR of 0-7 contains the value '004 C000'. There are various ways to specify an SMBR that does not hold any base address value. In this example, such a register would have the value '000 0
Hold 000 '. (The '0' added to the upper side of these addresses is an address space bit, '0' indicates a memory address, and '1' indicates a system address such as the address of the register 940 itself).

Similarly, each shared memory limit register (SMLR: shared-memory limit register) in the same cluster
r) 946 includes the upper 25 bits of the highest address of the shared address range. In this example, the SMLR of system units 410-4 through 410-7 has the value '004 FFFF'
And that the highest shared address is the same as the highest address of the physical memory on the unit, that is, '004 FFFF FFFF'. SMLR of all other units
Holds the specified invalid value '000 0000'.

[0060]

[Table 3]

The register control section 947 controls the control line 1143
And different values in registers 942, 944, 945, 94
Load 6 This includes dynamic reconfiguration of system units 410 in domains and clusters,
And allows dynamic reconfiguration of the shared memory location of each cluster. FIG. 11 and FIG. 12 are for explaining how to perform this function. Each register set 9
Placing additional copies of the base and boundary registers 945, 946 within 40 allows the single domain
Multiple ranges of shared addresses are possible within a cluster. Alternatively, the base address and shared segment size, or other parameters, may be stored in registers.

It is also possible to use the NON-DOMAIN line 943 to prohibit or prevent transactions from non-shared memory, just as prohibit other transactions from outside the domain. This configuration allows for quick control of non-memory transactions, but increases the time required for memory filtering in comparator 941. Since the latency in the memory subsystem 520 is more critical than in the other subsystems of FIG. 5, the comparator 941 also preferably receives a conventional signal from line 922 indicating the type of the current transaction. If line 923 specifies a non-memory transaction, line 943 will be connected to line 9 as described above.
11, but normal memory transactions are not filtered at this point and the memory subsystem 52
0, where preparation for its execution is started.
However, the comparator 941 does not allow the system outside the domain of this unit (defined by DMR 942).
The MEMORY CANCEL line 948 is activated for any normal memory transaction originating from the unit. For this memory transaction, registers 945 and 946 indicate that it is out of range of memory shared with other domains. That is, this memory transaction is a memory transaction originating from a system unit not shown in SMMR 944. next,
This line 948 directly blocks the transaction at switch 720 of FIG.
1 is prevented from actually affecting data stored in any one of them.

Thus, the computer 400 achieves “software separation” between the domain group and the cluster group. Different domains may, for example, run entirely different operating systems and do not interfere with each other. Further, if the error only affects the operation of system units in other domain clusters, the hardware error signal from control bus 460 of FIG. To provide "hardware separation". For example, the EC in the system unit 410-0
The error detected by C block 924 is 410
Does not affect system units like -5. This is because these hardware units operate rather independently of one another, and hardware failures in one unit have no effect on the processing being performed on the other.

FIG. 10 shows the domain filter 48 of FIG.
5, one of the four identical global parts of 0
One global address arbiter 55G2 is shown in detail. It is assumed that the arbiter 55G2 in FIG. 10 controls the first bus 55G1-0 of the four global address buses (GAB) 55G1. This arbiter is located on each system unit 410 in the computer 400 and is located on the local address
It receives one of the four GAB-request lines 821 from the arbiter (LAA) 820. Whenever the LAA55L4 determines which port on the system unit deserves the next access to each of the four global buses, its line 821 is asserted by the arbiter
Through GAB controlled by logic 1010,
Assert a request to broadcast a transaction.
Since computer 400 has four GABs 55G1, there are four separate lines 821 running from each local arbiter 55L4 to four global arbiters 55G2.

The arbitration logic 1010 employs any of a number of conventional algorithms to control the 16 grant lines 1
By launching one of the 013, the GAB
The transfer cycle of 55G1 (FIG. 5) is assigned to one LAA 55L4 of the 16 system units 410. As in the conventional system, the permission signal
Through the line 822 of the system unit LAA 55
Return directly to each of the L4s. In this case, the filter logic 1022 is ignored and address transactions originating from the selected LAS 55L3 are
Propagate through G1 to the corresponding LAS on all 16 system units. In the next transfer operation of the global address router 450, the global address arbiter 55G2 stores the address in its corresponding GAB.
Selected LAA to signal local address switch 55L3 to pass through
Command 55L4. GRAN for successful transactions
T-line 1013 indicates to all system units which of them has issued a transaction on that GAB 55G1. When the receiving system unit receives the transaction, it identifies the source unit from information in the transaction itself. The local data router 54L2 determines which of the global data paths 54G1 will carry any data required by a successful transaction.
Negotiation is performed with the data arbiters 54L3 and 54G2 in FIG.

In a multi-domain computer according to the present invention, the global portion of the domain filter 480 is physically located at each global address arbiter 55
With G2. 16 possible system units 410
Each of the banks 1020 of the cluster register 1021 receives one of the 16 enable signal lines 1013.
Each individual cluster register 1021-i has one bit position 1021-ij for every 16 system units 410-j. For example, the first register 1
A value of '1' at the position of “unit-3” of 021-0 indicates that the system unit 410-3 is in the same cluster as the system unit 410-0. The following table shows the registers 1021 for the example configuration described above.
Indicates the contents of

[0067]

[Table 4]

All 16 possible system units are always equipped with registers. Register 1021-8
The values in 1021 -F correspond to system units not implemented in FIG. 4 and are not significant. However, by assigning a “1” to all diagnostic bit positions (ie, position i of register i) and assigning a “0” to other positions, it does not interfere with any other units already in the system. In addition, the system unit can be plugged in during the operation of the computer 400, and a single diagnosis can be immediately performed.

Filter logic 1022 couples enable line 1013 to line 1023 according to the cluster definition in register 1021. Each line 1023 arrives at its corresponding system unit 410 as a “global address valid” (VALID) signal 822. In a typical system, such as 300 in FIG. 3, the VALID signal simply indicates that the current transaction on the bus is valid,
It is merely a timing signal indicating that it is being broadcast to all system units. On the other hand, the system 40
0, multiple system units in different clusters can have the same address, and receivers in the same cluster as the origin must receive the transaction, and system units in other clusters The unit must continue to completely ignore what is happening, no matter which transaction holds the address corresponding to the system unit.

In a typical single domain computer,
HOLD signal 825 from any LAA 55L4 in FIG.
Also other system units 410 throughout the computer
It is only propagated on line 826 towards the GAB in each LAA 55L4. However, computer 4
At 00, the HOLD signal 825, as defined by the cluster register 1020, by another set of filter logic 1026 on each GAA chip 55G2.
Can only reach line 826 belonging to another system unit in the same hardware group. The ARBSTOP signal 824 operates similarly. A STOP asserted by one system unit only reaches the other group of units specified by register 1020, rather than simply being connected to the incoming ARBSTOP line 826 going to all other LAAs. The global portion of the domain filter 480 includes each set of filter logic for other control signals as well. For example, the CANCEL signal 931 asserted by the cache controller 930 of FIG.
Incoming CANCEL line 92 only if logic 1028 allows.
3, the transaction can be canceled. Filter logic such as 1022 and 1026-1028 are all clustered by line 1025.
They are connected in parallel to the register group 1021.

The control unit 1024 sets the line 1144 to the register group 10 to dynamically reconfigure the cluster definition.
Allows 21 to be loaded with different values. As an option in the embodiment, each global arbiter 55G2 occupies the same integrated circuit, each of which contains the same set of cluster registers and filter logic in duplicate. All sets of cluster registers are loaded with the same set of stored values.

FIG.
28 shows details of a circuit 1400 that implements a set of domain filter logic, such as 28. FIG. 14 shows logic 1.
022 is used as a paradigm, and a signal designation to the circuit 1400 in that case is shown. For ease of illustration, FIG. 14 shows not only the bit line 1025, but also the cluster register 1021 itself.

Line 1023-0 indicates that a system unit has started a transaction whenever any system unit in the hardware domain cluster has begun a transaction.
Assert the VALID signal to unit 410-0. GRANT signal 1013-0 associated with unit 410-0
Indicates that if bit 1021-0-0 of register 1021-0 contains a "1" value, indicating that unit 410-0 is in its own cluster, AND gate 14
01-00 is satisfied. Next, a logical OR gate 1402
-0 asserts output 1023-1 and returns it to system unit 410-0. System unit 41
When GRANT line 1013-1 from 0-1 is asserted, if these two units are in the same cluster, line 1023 going to unit 410-0 also rises. If they are in the same cluster, when 1013-1 rises, the value “1” of bit 1 of register 1021-0 (referred to as bit 1021-0-1 for simplicity) is AND gate 1401-01 and OR 1402 −0 is satisfied. The remaining 14 AND gates in this bank are also connected to register bit group 1 through 1021-0-F.
The same operation is performed with respect to 021-0-2.

Gates 1401-10 to 1401-1
F and 1402-1 function similarly, and whenever a system unit in the same domain cluster offers a transaction, the VALID signal 1023-1
Is generated for the system unit 410-1. 1
Four additional gate banks process the remaining lines through 1023-F. Normally, the contents of registers 1021 form a diagonal matrix, so that bits 1021-ij always have the same value as bits 1021-ji. Also, since each unit is usually a member of its own cluster, all the main diagonal bits 102
1-ii is always "1".

FIG. 11 shows the configurator 42 of FIG.
0 indicates a mode in which a domain group and a cluster group are dynamically set in the computer 400.

The normal control and service unit 470 of FIG. 4 is a dedicated service processor 11
It takes the form of 10 and functions as a remote console. These two units are connected by a cable or other link 1122, a standard adapter 1
They communicate with each other through 111 and 1121. Console 1120 may be connected to its own input / output device (not shown) by adapter 1123. service·
Processor I / O adapters 1112 sense and control a number of functions within computer 400. For example, line 1113 interfaces to the power and cooling subsystem 1130 for the entire computer. Line 1
114 connects to a number of lines in the control distribution means 440 of FIG.

One of the usual functions implemented in the computer 400 is through a set of lines 1115,
By sending a stored test pattern to various elements as indicated by reference numeral 116, there is a function capable of performing a test on the logic circuit. The traditional function of these lines is the KP Parker, THE BOUNDARY-SCAN HANDBOOK (Klu
wer Academic Publishers, 1992), a boundary-scan test
There is to implement. Those skilled in the art usually refer to this protocol as the "JTAG standard".

The configurator 420 selects (coopts) an existing JTAG line 1115 for the additional function. Typically, these lines provide conventional address and data lines to many chips throughout the computer 400 and are intended to test the function of these chips. In the chip for the LAS55L3 and GAA55G2, the control logic 947 of FIG. 9 and the control logic 1024 of FIG.
A predetermined signal combination on 44 is detected. These lines carry the domain and cluster specifications on line 1143 and, as shown in FIGS. 5 and 9, the contents of filter registers 940 in local address routers 55L3 of selected system unit 410. To load. Also, the line 1142 indicates the cluster specification by the line 114.
4 and load the filter register 1020 associated with the global address arbiter group 55G0 as shown in FIGS. Systems 400 that do not already have JTAG or other such lines in place may use dedicated lines from service processor 1110 to easily use as control lines 1143, 1144 or perform configuration functions. You can switch to another line. These lines simply happen to be readily usable in this particular embodiment. Another alternative is to treat the registers 940, 1020 as small blocks of memory in the system memory space. As noted above, the computer 400 has a '10
Such a space from 0000 0000 'to' 1F FFFF FFFF 'is included in the entire range.

There is no critical point in the form of service processor 470. In some cases, it may even be possible to use part of the normal system itself for this function without having a physically separate entity. In fact, the preferred computer 400 allows the system unit itself to provide some of the functions of the domain configurator, if necessary. Privileged software in the operating system (privi
leged software), to re-designate the current shared memory block, the shared memory register 945, FIG.
946. The service processor also allows the system units to selectively write to the register 944 by setting a configuration bit in the status word in the I / O controller 1112, which can control one of the control lines 1114. Appears in books.

FIG. 12 describes a method 1200 for dynamically configuring domains and clusters in a computer 400. In FIG. 12, the right-pointing blocks are executed in the remote console 1120, and the left-pointing blocks are executed in the service processor 1110 of FIG. 11 of the previously described embodiment. Block group 1210
Sets up the configuration process. Block 1211 initiates configuration mode in response to an operator command. Block 1212 initializes registers 940 and 1020 to default values. Preferably, all registers 942
Receives a "1" bit at the location where they are placed in its own domain and a "0" at other locations. All registers 944-946 receive a value indicating that the shared memory is not exported. Register group 1020 preferably includes "0" except for one bit of a diagonal stripe. This indicates that each system unit is itself in a cluster.

The block group 1220 corresponds to the domain in FIG.
The configuration of the filter 480 is set. At block 1221, the operator at the remote console selects a particular domain to configure and enters the number of the system unit 410 belonging to that domain at block 1222. Next, service processor 1110 sends a signal on lines 1115 and 1142 and block 1223
, The appropriate value is loaded into the domain mask register 942 of FIG. Block 1224 sets the appropriate registers 1020 to make each domain its own cluster. This step may be performed at any time, but in this embodiment, it is necessary to set the cluster register even when the domain group is incorporated into the cluster. Block 1225 returns control to block 1221 once the operator has specified additional domains to configure. Otherwise, block 1226 queries whether there are any more multi-domain clusters to configure.

If so, block 1230 sets up any desired shared memory. At block 1231, the operator selects one of the system units 410 to export memory and returns to block 1232
In, a domain for importing the memory is selected. (The system unit "exports" the memory if its physically implemented memory is available to other system units, and the other system unit is located on the importing unit. As such, the memory is "imported.") At block 1233, the appropriate registers 944 are loaded, as described in connection with FIG. Block 1
At 234, the appropriate bits in registers 1020 are set, as described in connection with FIG. At block 1235, the value of the base address of the shared memory range is received from the operator. In block 1236, this is input to the appropriate SMB register group 945. At block 1237, the corresponding restricted address value is received,
This is loaded into SMLRs 946 at block 1238. If the operator wishes to define additional clusters, block 1226 returns control to block 1231. Otherwise, procedure 1200 ends. Most modifications can be made to the series of steps shown in FIG. Similarly, routine 1 for other tasks on the computer
The timing of 200 is not important either. Further, privileged software within computer 400 may execute routine 1200 on behalf of the operator. Broken line 1201
Symbolically indicates that the reconfiguration can be performed repeatedly, either by the operator or by software, without any manual changes to the computer hardware.

The system units 410 can be arbitrarily incorporated within a domain, but obviously all domains and clusters are comprised of at least one system unit, at least one processor implemented,
And a system unit that includes memory. Domains or clusters are almost always
Some I / O on the one or more system boards
Includes O equipment. How these resources are distributed among the various system boards in a domain or cluster, however, is arbitrary. Method 12
00 can configure domains and clusters during normal operation of the entire system 400 and its operating system (s). To avoid complexity and the possibility of trivial mistakes, it is advisable to allow reconfiguration only if the system is in a special state before any of the operating systems are booted.

FIG. 13 shows a typical transaction 13
12 is a simplified diagram of FIG. 10, highlighting the effects of the domains and clusters on computer 400 during normal operation, ie, after block 1242 of FIG. Transaction 1300 assumes that computer 400 contains the largest complement of 16 system units. The transaction starts at line 1301.

Block 1310 represents all system units 4 as symbolized by the multiple columns of FIG.
10 is executed. They initiate a transaction request either between two different system units or within the same unit. Requests proceed asynchronously and simultaneously whenever any one or more ports of one or more system units request a transaction in any of blocks 1311. In the block group 1312, the local arbiter 55L4 selects one requesting port on the system unit based on any of a number of traffic equalization priority algorithms, and proceeds.

The block group 1320 sends addresses for all transactions. As indicated by lines 1321-0 through 1321-F, each block 1322
Are all system units 410-0 through 410
-F receives a transaction request from the
Permission is given to one of the bus groups 55G1 to the unit. 4
Each of the global address arbiters 55G2 executes block 1322 in parallel and allocates its particular bus 55G1 among competing transactions using standard fairness methods. Next, block 1323, as indicated by line 1324,
Broadcasts the address from the selected system unit to all 16 system units.
Also in this case, each of the four buses 55G1 in this embodiment can simultaneously broadcast a separate address simultaneously with all of the other buses.

At step 1330, each bus 55G
1 so that only the appropriate system units 410-0 through 410-F are allowed to act on the transaction. Since there is a separate block group 1330 for each global address bus for each system unit, there are 4 × 16 = 64 blocks 1330 in this embodiment. Each block 1330 determines from the register group 1020 simultaneously whether the system unit is in the same cluster as the transaction sending unit on the bus. (Recall that a single domain is itself defined as a cluster in registers 1020.) If not, the system unit ignores the transaction and control passes to output 1302. Otherwise, control passes to 1340.

A separate set of blocks 1340 appears for each global address bus in each system unit. That is, 4 × 16 = 64 sets of blocks appear. The block group 1341 has a transaction of line 92
When moving along comparator 2 to the comparator 941 of FIG.
Read the source unit number from the transaction itself on 55G1. If the domain mask register 942 determines that the source unit is not in the same domain as the unit in which it is located, block 1341 passes control to block 1342. If the shared memory register 944 detects that the system unit is sharing memory with the source unit, it moves from block 1342 to block 1343. If comparator 941 indicates that the address of the transaction being carried on line 922 exceeds the base address stored in register 945, block 1344 indicates that the address is stored in register 946. A check is made to see if the shared memory is below the upper limit. Each block 13 indicating that the system unit is not involved in the current transaction
For each set of 30, at transaction 1345, the transaction at that location is terminated. However, any of the filter block chains 1330-1340
If it senses the same domain, or the same cluster and appropriate address range, line 1346 passes control to block 1350 for that system unit.

Block 1350 performs the actual transaction from the requesting system unit to the proper destination in the target unit. This includes the transfer of any requested data through the data router 440. (There are many different types of transactions, as noted above). Point 1302 marks the completion of the transaction. At any given time, in the same or different blocks of flowchart 1300, several different transactions may be in progress, each of which proceeds independently of the others.

FIG. 15 shows, in addition to the preferred embodiment, an additional type of hardware failure that can be prevented from affecting system units outside its own domain cluster. Indicates a domain filter.
The domain filter 480, as described above,
Limiting the effects of errors in system unit 410 or address router 55G0 of FIG. 5 to other system units that are not in the same domain cluster.

As described in connection with FIG. 5 and elsewhere, a transaction may involve a transfer of data from one system unit to another system unit on global data router 54G0. The global data arbiter receives conventional signals 1510 from all of the system units. For example,
The line 1510-0 from the local data arbiter 54L3 of the system unit 410-0 is connected from the unit 410-0 to the system unit 410-0 of FIG.
To request a transfer to a particular one of the STAs 410-F. Line 1501-1 is connected to system unit 410-
0-410-F specifies which ones will receive the transfer from unit 410-0, and so on. Arbitration output 1520
Indicates that data from one of the data lines 1530 is
By allowing it to flow to the other one of the 0
Establish a data path. For example, the logic 54G2 is
If the request on line 1510-0, which carries data from unit 410-0 to unit 410-1, is granted, FROM-0 line 1
521 couples data bus 54G1 to line 1530-0, TO-1 line 1540-1 is directly coupled to TO-1 line 1580-1, and line 1530-1 couples data to unit 410-1.
Can be sent to

Under normal conditions, this configuration is transparent to the computer 400 domain structure. However, erroneously transferring data to a different system unit (a system unit that is not in the same domain cluster)
Failures to send to other clusters can disrupt the operation of system units in other clusters. For example, FIG.
Suppose that unit 410-0 has attempted to send data to unit 410-3 in the same domain S1, but an error signal has instead (or additionally) sent it to unit 410-7. Such failures can be
Bypassing the separation implemented by filter 480;
The domain S1 may affect the operation of the domains S2 and S3. This is called a "transgression error".

Another filter logic 1550 eliminates this possibility by signaling an attempted out-of-cluster data transfer. Register group 10 in FIG.
20, another set of cluster registers 1560 holds a copy of the cluster definition of computer 400;
Pass them to logic 1550 via line 1565.
Logic 1550 is the filter logic 140 of FIG.
Like the case of 0, it is constructed by an AND / OR circuit. Logic 1
550 produces two sets of outputs. Output 1570 is an ARBSTO of the same type as signal 824 shown in FIGS.
Generate a P signal. These stop the source system unit that initiated the improper data transfer. Output 1580 is sent to any system unit that is not in the same cluster as the source unit that made the improper request.
Prevent this transfer from affecting. Continuing the above example, a failure in system units 410, request line 1510, etc. may cause data path 54G1 to activate the incorrect set of lines 1530. However, the data router filter logic 1550
Is the correct destination from unit 410-0, as defined by the bits in register 1560, from units 410-1, 410- in the same domain cluster.
It is detected that there is only 2,410-3. 410-7
154, such as TO-7, which specifies the destination as a destination
A 0 activates the ARBSTOP-0 line 1570-0, indicates that this unit 410-0 has attempted an illegal transfer, and stops this unit. That is, since the ARBSTOP signal reaches the source unit and other units in the same domain cluster, an error in the domain cluster CA only affects the system unit in the domain cluster CA. .

The logic 1550 uses the definition in the cluster register group 1560 to prohibit any TO signal 1540 from reaching a destination unit that is not in the same cluster as the unit that issued the FROM signal. In this example, when any of the TO lines 1540-0 to 1540-3 is asserted, the corresponding TO line 1580-0 to 1580-1 is output.
580-3 and the corresponding system unit 41
Enable 0-0 through 410-3 and enable line 1530-
Receive the data on 0-1530-3. On the other hand, the coincidence of the FROM signal and TO signal 1550-7 on line 1520-0, that is, for units in different clusters, is blocked by logic 1550. Accordingly, the corresponding TO line 1580-7 remains dormant and data path 54G1 does not pass data to system unit 410-7. Thus, when a transgression error occurs, the filter logic shuts down this unit by sending an ARBSTOP to the unit that initiated the data transfer, and inhibits its TO line, thereby allowing the transfer to proceed to the destination unit. The unit is not actually allowed to transfer anything, as it prevents any effect on the unit.

FIG. 15 only shows one data path of the crosspoint data router 440 of FIG. The additional data path functions similarly and simply requires an additional set of filter logic 1550. Further, each data path may be divided into a number of parts for further fault isolation and redundancy, without affecting the operation of FIG.

[Brief description of the drawings]

FIG. 1 is a conceptual schematic diagram of a prior art bus-oriented multiprocessor digital computer.

FIG. 2 is a similar schematic diagram of a computer having multiple system units.

FIG. 3 is a block diagram showing a case where the computer of FIG. 2 is divided into a system domain and a cluster according to the concept of the present invention.

FIG. 4 illustrates how the present invention allows the computer of FIG.
FIG. 7 is a diagram showing whether the domain is divided into domains and clusters.

FIG. 5 is a block diagram of the fully configured system unit of FIG. 4, including relevant portions of other computer system units.

FIG. 6 is a detailed view of the port controller of FIG. 5;

FIG. 7 is a detailed view of the memory controller in FIG. 5;

FIG. 8 is a detailed view of a local address arbiter of FIG. 5;

FIG. 9 is a detailed view of the local address router of FIG. 5, including the local part of the domain filter according to the present invention.

FIG. 10 is a detailed view of the global address arbiter of FIG. 5, including the global portion of the domain filter.

FIG. 11 is a diagram showing a domain configurator of FIG. 4;

FIG. 12 is a flow chart illustrating a method of configuring a computer in a clustered system domain according to the present invention.

FIG. 13 is a flow chart of a transaction process emphasizing domain filtering of the present invention.

FIG. 14 is a detailed diagram of a logic circuit used in the domain filter.

FIG. 15 is a detailed diagram illustrating the global data arbiter of FIG. 5 including any further global portions of the domain filter.

[Explanation of symbols]

 Reference Signs List 100 computer 110 processor 120 memory board 130 I / O board 140 data bus 150 address bus 160 control distribution bus 170 service unit 200 architecture 210 system unit 240, 250 high-speed router 260 control distribution bus 300 computer 310 system board 340 data router 350 address router 360 control distribution means 370 system controller 400 computer 410 system unit 440 data router 450 address router 460 control bus 480 domain filter 510 processor subsystem 511 microprocessor 512 cache 520 memory Subsystem 521 Memory 522 Pack / Unpack module Lumpur 530 O subsystem 531 system I / O bus 532 I / O adapter

 ────────────────────────────────────────────────── ─── Continuation of the front page (71) Applicant 597004720 2550 Garcia Avenue, MS PAL1-521, Mountain View, California 94043-1100, United States of America, United States of America 72. San Diego, Walker Drive 2831 (72) Inventor Nicholas e Anne Shansley 11382 Pennova Street, San Diego, California 92129, USA

Claims (35)

[Claims] 1. A multi-processor computer having hardware domains variably configurable by instructions from an operator, comprising: a plurality of separate system units for executing a series of transactions, each system comprising: A processor unit which is physically removable and replaceable individually within the computer, each of which generates an address within a predetermined global range; data at a set of addresses within the predetermined global range. And a memory unit for storing and / or receiving at least one I / O adapter for generating and / or receiving a set of addresses within the predetermined global range, the system unit being coupled to the system unit. From any of the above system units A global address router for transferring the assigned address to another unit of the system unit; and a global data router for transferring data from any of the system units to another unit of the system unit. A router and the system unit from any of the system units.
A control signal distributor for communicating a plurality of control signals to other ones of the units and for affecting all operations of the system units in response to a condition occurring in any of the system units; and the computer Electronically divided into a plurality of software configurable hardware domains, each domain comprising any subset of the system units independent of any physical reconnection of the system units in the computer A configurator; a computer controller responsive to the instructions that specifies to the domain configurator which of the system units belongs to each of the hardware domains; and all of the system units Of the system unit A domain filter for electronically inhibiting at least some of the control signals emitted in one of the mains from affecting one of the system units outside the one domain. Multiprocessor computer equipped. 2. The computer according to claim 1,
The domain filter is coupled to at least one of the global routers, and a transaction on the one global router occurring in one of the domains of the system unit is:
A computer that prohibits being received on some of the system units outside the one domain. 3. The computer according to claim 2,
The computer wherein the one global router is the global address router. 4. The computer according to claim 3,
The global address router is connected to the system
A computer coupled to all of the units and having a plurality of paths for carrying a plurality of transactions simultaneously between different subsets of the system units. 5. The computer according to claim 2, wherein
The computer wherein the one global router is the global data router. 6. The computer according to claim 5,
The computer wherein the global data router is coupled to all of the system units and has a plurality of paths for carrying data relating to a plurality of transactions between different subsets of the system units simultaneously. 7. The computer according to claim 2,
The domain filter is coupled to both the global address router and the global data router so that both addresses and data generated in one of the domains of the system unit are transmitted to the system unit. A computer that prohibits being received on some of the units outside of the one domain. 8. The computer of claim 1, wherein the domain configurator further couples a plurality of the hardware domains and removes from any physical reconnection of the system units in the computer. Forming a domain cluster consisting of any subset of the independent domains; the computer controller responding to another one of the commands, and telling the domain configurator which of the system units is the domain cluster; And wherein the domain filter determines that the at least some control signals generated in one of the system units in one of the system units are out of the one of the system units. But the domain
A computer that allows you to affect what is in a cluster. 9. The computer according to claim 8, wherein
Computer wherein one of the domains in the domain cluster includes physical memory accessible by a different domain in the domain cluster within the same predetermined shared address range. 10. The computer of claim 1, wherein the domain filter comprises: a connection identifying which of the system units generated the current address in the address router; and the system unit. Responsive to at least one domain mask register for each of the system units, specifying which of the domains belongs to which of the domains; the source identification connection; and the domain mask register; Gate logic for disconnecting the system unit from all of the system units that are not in the same domain where the system unit generated the current address. 11. The computer of claim 10, wherein the domain filters each identify which of the system units belong to a domain cluster and respond to a current one of the transactions. A register, and a connection for sending a valid transaction signal to each of the system units in the common cluster for any of the transactions originating from one of the system units belonging to the domain cluster; Including computer. 12. The computer of claim 11, wherein the domain filter includes a shared address register indicating a shared address range between different system units in the domain cluster. 13. A computer having a plurality of system units, a global address router, a global data router, a control signal distributor, and a domain filter, comprising: A method for partitioning into independent hardware domains, comprising: (a) initiating a configuration mode; and (b) designation data defining a subset of the system units to be included in one of the hardware domains. (C) loading the specified data into a domain filter, causing one of the system units in the domain to respond to a control signal in the distributor, and Causing the distributor to be unresponsive to another of the units; and (d) the system. For another specified data defining a different subset of units, the method comprising the steps of repeating steps (b) and (c), the. 14. The method according to claim 13, wherein said step (c) is also responsive to said designated data loading said domain filter, wherein said step (c) includes: , The system
Responding to an address on the global address router originating from one of the units within the one domain, and providing the system unit in the first domain to the first domain of the system unit Not responding to addresses on the global address router originating from at least some that are not. 15. The method of claim 14, further comprising: (f) receiving second specified data defining a cluster of domains of the domain; and (g) storing the second data in the domain. Loading into a filter and responding to an address on the global address router emanating from one of the system units in the cluster of the domain and from one of the system units in the cluster of the domain Causing the data to be transmitted. 16. The method of claim 15, further comprising: (h) address sharing physically present in one of the domains and accessible to other domains in a cluster of the domains. Receiving third specified data defining a range; and (i) loading the third data into the domain filter and assigning the third data to one of the system units in the cluster of the domain. Responding to addresses on the global address router originating from within the cluster of domains but within the sharing range. 17. The method of claim 16, wherein the shared range is less than an entire address range of a memory physically located in at least one of the system units in the domain cluster. 18. The method of claim 13, wherein the method is performed after step (d), and (j) transmitting a transaction from one of the system units in the first domain to the global address. Broadcasting all of the system units, both inside and outside the first domain, via a router; and (k) filtering the transaction at each of the system units, Enabling response of the transaction to those of the units that are in the first domain and disabling the response to the transaction to other of the system units that are outside of the first domain; Disabling. 19. The method according to claim 18, wherein steps (j) and (k) are performed after step (e). 20. The method of claim 18, wherein step (k) does not disable all of the system units outside the first domain. 21. The method according to claim 18, wherein the plurality of systems are located in different domains of the domain.
The method wherein the unit physically comprises a memory having addresses in each range, wherein the addresses in each range at least partially overlap. 22. A system unit for a multiprocessor computer having a global address router, a global data router, and a control signal distributor, wherein the control signal distributor comprises a plurality of the system units. Interconnecting the other, said computer also having a computer controller, said system
A unit coupled to both of the global routers for receiving at least one processor unit for generating an address within a predetermined global range; and a unit coupled to the global router and configured to receive an address within the predetermined global range. Means for receiving at least one memory unit for storing data at a set of addresses; and at least one coupled to the global router for generating and / or receiving a set of addresses within the predetermined global range. Means for receiving an input / output adapter, and means for generating a control signal at the distributor, coupled to at least one of the aforementioned means, wherein the control signal is indicative of an error condition within the system unit; Displays the error status in the other system unit. Means for receiving control signals; means for filtering the control signals so that only control signals from selectable ones of the other units can affect the operation of the system unit; Means connectable to a computer controller and selecting said one from said other units. 23. The system unit according to claim 22, wherein said filter means comprises: a domain mask register holding data specifying said selectable one of said other units; and said selectable one. Gate means for passing certain signals from and blocking said certain signals from others of said system unit. 24. The system unit according to claim 23, further comprising the step of storing variable data in the domain mask.
A system unit that has means for loading registers. 25. The system unit according to claim 22, wherein said filter means stores data specifying a memory physically implemented in any of said system units in said computer. A shared memory register that is accessible to and maintained in a portion of said global range. 26. The system unit of claim 25, further comprising means for loading variable data into said shared memory register. 27. The system unit according to claim 25, wherein said filter means specifies an address range consisting only of a part of said memory physically mounted on any one of said system units. A system unit including at least one other shared memory register for holding 28. A system unit among a plurality of system units for a multiprocessor computer, wherein the multiprocessor computer sends an address issued in any of the system units to the system unit. A global address router for forwarding to all other units, wherein each address of said address has a source identifier indicating which of said plurality of system units issued said respective address. A router and a global router for transferring data from any of the system units to all other of the system units.
A data router, transmitting a plurality of control signals from any of the system units to all other ones of the system units, and responding to a condition occurring in any of the system units; A control signal distributor that affects the overall operation of the unit; and electronically divides the computer into a plurality of software configurable hardware domains, each one of the system units in the computer. A domain configurator composed of any subset of the system units, independent of physical reconnection of the system units, and in response to the command, which of the system units belong to each of the hardware domains Is the computer code that specifies the domain configurator. A controller, coupled to all of the system units, wherein at least some of the control signals emitted in one of the domains of the system units are outside of the one domain. A domain filter for electronically inhibiting any of the following: wherein said one system unit is at least one subsystem connected to said global address router, said subsystem comprising: Combining the address between a system and any other of the system units, coupled to the global data router, and transferring transaction data between the subsystem and any other of the system units. Subsystem that executes a transaction A processor subsystem selected from the group consisting of a processor subsystem that stores data within the global range, an input / output subsystem that communicates with an input / output adapter, and a processor subsystem coupled to the distributor. , At least one of the control signals
A generator, at least one receiver for the control signal, and a domain for receiving from the computer controller a value indicating which of the plurality of system units belongs to the same domain as the one system unit. A writable mask register; coupled to the domain mask register, generating an inhibit signal if the source identifier indicates that each address did not originate from within the same domain;
A comparator coupling the inhibit signal to the at least one subsystem and rendering it unresponsive to each of the addresses. 29. The system unit of claim 28, wherein the domain configurator further combines a plurality of the hardware domains and is independent of any physical reconnection of the system unit in the computer. At least one writable, indicating which of the plurality of system units belongs to the same domain cluster as the one system unit, forming a domain cluster comprising an arbitrary subset of the domains. A shared memory mask register, wherein the comparator is further coupled to the domain mask register, wherein each of the addresses originated at some of the system units outside the same domain cluster. The source identifier indicates that Generating said inhibit signal, the system unit. 30. The system unit of claim 31, wherein said at least one subsystem includes said memory subsystem storing data within a portion of said global address range.
The unit further comprises at least one shared memory address register for receiving from the computer controller a value defining a shared range of memory addresses within the portion of the global range, the comparator comprising: A system unit responsive to the shared memory address register for inhibiting the memory subsystem if the respective address is outside the shared range. 31. The system unit of claim 28, wherein said at least one generator of said control signal forms part of a local address arbiter requesting and receiving access to said global address router. System unit to do. 32. The system unit of claim 33, wherein said at least one receiver of said control signal further forms part of said local address arbiter. 33. The system unit of claim 28, wherein said one system unit comprises at least two different ones of said subsystems.
unit. 34. The system unit of claim 33, wherein said one system unit includes all three of said subsystems. 35. The system unit of claim 28, wherein said processor subsystem includes a plurality of separate microprocessors.