JP3416159B2 - Dynamic hierarchical associative memory - Google Patents

Abstract

An associative memory having an associativity of 2q, where (q) is an integer greater than or equal to one, is provided for storing information relating to data. The memory includes (n) tables, each having a plurality of entries for storing signals associated with data descriptors having a common set portion and common other portions. The entries of table (k), where (k) represents successive integers between (1) and (n-1), store pointers to respective entries of table (k+1). The entries of table (1) are arranged for access as a function of the common set portion and the common portion (1) with which they are respectively associated. The entries of the other tables are arranged for access as a function of (i) a value of the common set portion, (ii) a value of a pointer-representative signal of the respective table (m-1) entry means, and (iii) the value of the common portion(m) with which such table(m) entry means is respectively associated. The entries of table(n) store information relating to one or more data having a common portion(n).

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION This invention relates to digital data communication circuitry and, more particularly, to packet switching circuitry for use with, for example, high speed distributed multiprocessing systems.

[0002]

Multiple instruction / multiple data (MIMD) parallel processing computers can be classified according to address space, memory system and memory hierarchy. Regarding the former,
The system will be characterized as utilizing a single or multiple address spaces. The single address space is commonly referred to as shared memory and includes implicit communication as part of any memory access. Dedicated address space multiplexing includes explicit communication eg by menu Tsu message delivery.

Memory systems are characterized as centralized or decentralized. In a system based on centralized memory, a common memory element is positioned in the center, the access time to any physical memory location is to be the same for the processor of all hands. On the other hand, in a distributed memory system, the system memory is divided into modules, some located near each processor.

The memory hierarchy is static tee click type, characterized as a mixed or dynamic. Although one hierarchy is classified as <br/> static tee click, in this type,
System data is divided between local and / or broadband memory units, each data having an address explicitly assigned based at least in part on the location of the unit in which it is stored. On the other hand, in a fully dynamic system, individual data are not assigned an address based on physical memory location to access the data. Mixed memory hierarchy, local and / or broadband manner memory including a static tee Tsu <br/> click is in this type, a memory hierarchy is dynamic (e.g., cache memory), the other part is static tee It is.

The prior art provides a number of architectures in which a single address space or shared memory is organized in a centralized manner. The processing units of such systems communicate via a high bandwidth shared bus or switching network. One problem with this type of architecture is that shared memory forms a bottleneck that hinders system performance except in cases where there are relatively few processors.

To avoid this problem, Frank et al., US Pat. No. 4,622,631, discloses a plurality of processors, each with an associated private memory or cache.
A multi-processing system for sharing data contained in a main memory element is disclosed. The data in the common memory is partitioned into blocks, each of which can be owned by main memory and any one of a plurality of processors. The current owner of a data block is said to have the correct data for that block.

The solution suggested in the above patent allows an increase in the number of processors that can be supported without a bottleneck, but a centralized memory system cannot scale.

In order to achieve scalability, a distributed memory system must be used. This is because parallel high bandwidth access to memory can theoretically be increased in proportion to the number of processors and memory modules. The prior art provides two alternative programming models for this type of system . A multiple address space format and a single address space model. Both forms present a dilemma for computer system designers and programmers.

From the programmer's point of view, the single address architecture is a simple programming model because the movement of data is implicit in memory operations. On the other hand, in a multiple address architecture, 1
Explicit message passing is required for one multiple address architecture. In addition, the multi-address architecture allows the correct parallel program to be generated by
Explicit data localization, explicit memory allocation and deallocation, require explicit replication and explicit coherency over. These aspects are theoretically implicitly addressed by the single address architecture.

Simultaneous access to a single address space or logically shared memory, as found by designers in the prior art from a hardware perspective, is extremely expensive. This is due to the lack and potential complexity of the solution of a general high performance against complexity of the switching network, and the memory coherency over. Therefore, most scalable distributed memory systems such as Intel / IPSC have historically implemented with multiple address architecture.

[0011] of a single address architecture performance, main
Depends on Mori hierarchy. Static memory hierarchy, pro <br/> grayed Lama, requires the explicitly manage the movement of data to obtain optimum performance in a manner similar to the multiple address architecture. Distributed memory that implements a single address architecture using a static memory hierarchy
Two examples of systems are BBN Butterfly and IBM
It is RP3. This type of implementation, a programmer is required to explicitly manage coherence <br/> Nshi over.

[0012] Mixed hierarchy, communication bottlenecks that limit inevitably decrease possibilities including still programmer needs to be partially manage data movement. One such hierarchical approach is described by Wilson Jr. U.S. Pat. No. 2,178,205, et al., In which the multiprocessing system is said to comprise distributed cache memory elements coupled to each other via a first bus. A second higher level cache memory, attached to either the first bus and higher level cache or main system memory, maintains a copy of each memory location in the cache, if any, in the system main memory. Maintains a copy of each memory location below the cache. Wilson Jr. Processor, etc., transmits the modified copy of the data from its own dedicated cache to the associated higher level cache and system main memory, while at the same time invalidating its own copy of the newly modified data. Notify other caches so that they will change.

Despite the solutions provided by the prior art, none have achieved a high performance, fully dynamic, consistent shared memory programming environment with unlimited scalability.

[0014] A general object of the present invention is to provide such a system were further using a shared memory addressing model and a distributed system and to identify and <br/> coherency over improvement multiplexing It is to provide an instruction / multiple data parallel processing system.

Another particular object of the present invention is to provide a fully dynamic memory hierarchy that achieves high performance in a distributed system utilizing a shared memory address model.

Yet another particular object of the present invention is to provide improved digital data communication circuitry.

Yet another object of the present invention is to provide a packet switch network for use in, for example, a high speed distributed multiprocessing system.

Yet another particular object of the present invention is to provide an improved switching mechanism for use in routing data and data requests through digital communication circuitry.

[0019]

SUMMARY OF THE INVENTION These and other objects of the invention are accomplished by providing improved digital data communication circuitry, packet switches and associative directories, for use with, for example, multiprocessing computer systems.

According to one aspect of the invention, the system of the invention comprises two groups of processing cells, and the data referred to in a packet and the data required by the first of said processing groups or A routing cell that transfers digital data information packets between the groups based on the identification of the data assigned to the first one or the data routed between the processing groups.

Each processing group comprises a plurality of processing cells, for example a central processing unit and an associated local cache memory, which communicate by transferring data packets and request packets via an associated group bus. Each cell has associated with it data or data copies identifiable by a unique descriptor. The cache control unit in the processing cell generates data request packets as well as packets that provide data or data copies in response to such requests. Each packet contains a descriptor corresponding to the requested or supplied data.

The route setting element, and a second input and an output unit connected to the bus of the first and second input and output unit and the second processing group are connected to the bus of the first processing group However, these inputs and outputs send and receive packets to and from cells in their processing groups. The request and response packets received on the first and second inputs are
It is routed to one or both outputs depending on whether the data identified by the descriptor in the packet is assigned to a cell of the first processing group.

To facilitate the determination of where to route the packet received by the routing cell,
An associative memory-based directory stores a list of descriptors corresponding to the data assigned to the cells of the first processing cell group. The directory is
Furthermore, a list of descriptors required by those cells and routed to the second processing group for processing, as well as descriptors required by cells of the second processing group and routed to the first processing group for processing. Remember the list. Together with these descriptors, the directory maintains information about the access status of data.
This includes, for example, invalid, read-only, non-exclusive ownership, exclusive ownership, and atomic status in addition to pending status.

According to a related aspect of the invention, the above-mentioned second processing group itself comprises a plurality of other processing groups, which groups themselves are interconnected via a routing cell and a network bus. They are connected and each comprises a plurality of processing cells interconnected by an intra-group bus. As an example,
In a two-level system, each other group is associated with a routing cell that selectively sends and receives information packets to and from the individual group buses. A higher level bus, referred to below as the Level 1 bus,
These other routing cells are connected with the routing cells associated with the first processing group.

In accordance with another aspect of the present invention, there is provided a hierarchical digital data communication network having a plurality of packet transfer levels each including a plurality of transfer segments of interconnect processing cells. Data or data copy identifiable by a unique descriptor is assigned to each cell. The cell also comprises a cache control unit that can generate request and response packets referencing the data by their respective descriptors. The number of transfer segments is reduced at each higher transfer level, with only one segment at the highest level.

Within the network, the routing cells enable communication between each level of processing cells and higher level processing cells. The routing cell that transfers the information packet between the transfer level segments selectively routes the request packet based on the association between the data requested in the packet and the cells of the lower transfer segment. In particular, these routing cells route packets requesting data that is not associated with a derived segment to higher level transport segments, while packets requesting data associated with lower segments to lower segments. Route.

According to another aspect of the invention, the request packet may include a requester (requester) ID field that identifies the processing cell that made the request. The cell that generates the response to the request copies the requester ID into the response packet. The routing cell is the requester ID
According to the value of the field, the response packet for the requester is set as the return route.

In addition, the routing sets the response packet containing the requested data in the preceding unrelated packet, ie the request packet generated by another cell.
You can route to one requesting processing cell.
Thus, for example, a response packet containing a read-only copy of the data independently requested by two processing cells may be routed to both regardless of the requester ID of only one of them being included in the response packet. Can be done.

These and other objects and advantages of the present invention will be apparent from the description given below with reference to the drawings.

[0030]

EXAMPLES Preferred digital data processing system utilizing a switching network constructed in accordance with the synopsis invention, one or more Routing interconnect is coupled to 0, 1 or more processing cells by ring bus It comprises a plurality of processing cells arranged in a respective ring hierarchy. Routing cells that link the interconnects provide a transmission path that selectively transfers information packets between the rings. A directory in the routing cell keeps track of the data (and requests) transferred through the associated cell,
Thereby, a routing path is determined for subsequent routing of subsequent packets.

[0031] In one preferred not examples, processing cell, each physical data and control signals store,
It comprises a central processing unit coupled with a local memory element having a directory and control elements, i.e. a cache. Cells are joined along a unidirectional inter-cell bus ring to form units called segments. These segments together form a larger unit called "Information Transfer Level: 0".

Communication between cells of different level: 0 segments is carried out via higher information transfer levels, eg information transfer level: 1 and information transfer level: 2. These higher levels themselves consist of one or more segments, each consisting of a plurality of routing cells coupled via a unidirectional bus ring. Each routing cell is associated with an associated segment of the next lower information transfer level. These combined lower segments are referred to as "lower".

Each information transfer level contains fewer segments than the next lower level . Aside from a single segment at the highest level of the system, the signal is transferred through the higher level segments between each information transfer level segment.

The route settings each include a directory of descriptors corresponding to the data assigned to the memory elements of the lower segment. As an example, level: 0 segments Level: 1 route connecting the segment set cell, the level of: include 0 segment directory data of all the hand assigned to the cells of the other levels:
1 segment level: Routing connecting two segments, that level: including 0 directory of data allocated to the cells of the entire hand segments: one segment connected to the level.

The directory stores, together with the descriptor, the access state of the data assigned to the lower segment. These states include, for example, an ownership state, a read-only state, and an invalid state. The ownership state, like the atomic state, is associated with data that can only be modified by the local processor. On the other hand, the ownership state, unlike the atomic state data, can be accessed by other processing cells-for example, to generate a read-only copy. The read-only state is associated with data that cannot be changed but can be read by the local central processing unit. The invalid state is associated with the invalid data copy.

In this regard, the preferred digital data processing system constructed in accordance with the present invention includes a main memory element,
That is, it will be appreciated that it does not utilize memory elements that are coupled to and shared by multiple processing cells. Rather, data is distributed on the exclusive and shared buses between the local memory elements associated with each of these processors. Changes to data stored exclusively in any one processing cell need not be communicated to other processing cells via circuitry. As a result, only the data that the cells dynamically share is transmitted on the network, ie along the ring bus and routing cells, thereby reducing bus contention and bottlenecks.

The routing cell directory also stores descriptors corresponding to the data required in the packets routed by the associated routing cell. As mentioned above, the directory, along with these descriptors, stores ongoing information data that indicates the requested access state of the data.

In view of the above, the contents of the routing cell directory--ie, the allocated data and the ongoing request list--are the routes of the data and data requests routed between the processing groups. Reflect the conventional history.

A routing cell uses descriptors and a directory of states to allow a particular packet of information traveling along a level segment to be forwarded to other processing cells in the segment from which the packet was issued. It should be routed upwards or downwards towards other segments instead.

To further understand this routing function, it will be appreciated that in the preferred multiprocessing system constructed in accordance with the present invention, data access requests are processed in the processing cell which originated them whenever possible. See. Specifically, the controller associated with each monitor responds to local processor requests by monitoring the cell's internal bus and comparing the requests with descriptors listed in the local cache directory. If matching data is found from the cache itself, the data is sent back along the internal bus to the requesting processor.

If the data request cannot be resolved locally,
Processing cell generates an information request packet, and this is forwarded to the local ring bus through a local cache control unit. As the packet travels along the ring, the control element of the processing unit adjacent to the requester checks its own directory,
Pass the requested data (if any) in the response packet.

When an outstanding request passes through the routing element of the segment to which the requesting processing cell belongs, that element also checks its directory. If the directory indicates that the requested data is on the local ring with the proper access conditions, the routing cell causes the request packet to continue along the local ring bus. If not present, the routing cell extracts the packet and passes it on the associated Level: 1 segment. Unresolved request packets travel along the Level: 1 segment in a similar manner, ie, compared to the directory of the associated Level: 1 routing cell. One of these directories is at a lower level:
If you have listed the requested data in the correct access state in the 0 segment, the request is passed to that segment. On the other hand. The request may be passed to a higher level of the hierarchy (if any) or returned to the outstanding requestor.

Packets containing the requested data are routed back to the requesting cell by a different mechanism. The first mechanism is the address or ID of the requesting cell.
Depends on. Here, each requesting cell includes an ID uniquely identifying the cell in its request packet. When the packet arrives at the response cell, the cell copies the requester ID along with the data and corresponding descriptor into the response packet. As the reply packet travels along the segment ring, the routing cell tests the requester ID to determine if the requesting cell is in a lower or parent segment and routes the packet accordingly.

A second mechanism is used in connection with response packets containing data required by processing cells but not specifically generated in response to those requests.
As an example, this mechanism is applied when two or more requesting cells issue read-only copy requests for particular data held in remote cells.

In accordance with the preferred embodiment of the present invention, assuming that the circuitry prevents at least some, but not all, of these requirements from reaching the processing cell that has a copy of the data, the responding cell is A response packet carrying only the requester ID of the request packet that reached it is generated.

[0046] When a response packet is routed set along the network, routing cells compare the descriptor and access information status of the packet and the request entry ongoing in those directories. By comparison, Routing cell and receives the format data request in the packet to already have the kite becomes apparent, that packet (or a copy thereof) may be Routing to one of the cells in the request.
In this way, a single packet, including, for example, a read-only copy of the data, can effectively act as a response to multiple outstanding read-only requests.

System Architecture FIG. 1 depicts a preferred multiprocessing system utilizing communication networks constructed in accordance with the present invention. The exemplary system 10 has three levels of information transfer:
Includes level: 0, level: 1 and level: 2. Each information transfer level comprises one or more segments, characterized by a bus element and a plurality of interface elements. Specifically, the level of the exemplary system 10: 0
Includes 6 12A, 12B, 12C, six segments indicated respectively 12D, 12 E and 12F.
Similarly, level: 1 comprises segments 14A and 14B, while level: 2 comprises segment 16.

Level: Each segment of 0, that is, 12
Each of A, 12B, and 12F includes a plurality of processing cells. For example, segment 12A has cells 18A, 1
8B and 18C with segment 12B being cell 18
D, 18E and 18F. Each of these cells comprises a central processing unit and a memory element interconnected by an in-cell processor bus (not shown). In the preferred embodiment of the invention, the memory element contained within each cell stores all control and data signals used by its associated central processing unit.

[0049] As further illustrated, each level: 0 segment in the art will characterized as having a bus element providing a communication path for <br/> transferring information packets between the cells of the segment. Thus, the exemplary segment 12A has a bus 20A as its feature, segment 12B has 20B, segment 12C has 20C (and so on ) . As described in detail below, the digital information packet is passed through the memory elements associated with each of these cells to the exemplary segment 12A.
Cells 18A, 18B, and 18C. The particular interface between these memory elements and the bus 20A is the cell interface unit 22 as shown.
Provided by. Similar direct channels are set up by the cell interfaces 22D, 22E, --22R between the respective cells 18D, 18E, --- 18R in the segments 12B, 12C and 12D as illustrated.

As illustrated and referred to above, the remaining information transfer levels, namely level: 1 and level: 2, are:
Each includes one or more corresponding levels: segments.
The number of segments in each sequential segment is less than the number of segments of the previous one. That is, level: 1 of 2
One segment 14A and 14B has less than six segments 12A, 12B--12F of level: 0, while level: 2 having only one segment 16 has the smallest number of all. Level: 1 and level: 2, that is, more segments of the higher level, a bus element for transferring packets within the respective segments. In the example, the level: 1 segments 14A and 14B are respectively buses 24A and 24B.
, On the other level : 2 segments 16 are bus elements 26
including.

Routing itself provides a mechanism to transfer information between successive levels of related segments. For example, the routing cells 28A, 28B and 28
C provides a means for transferring information between each of the level: 1 segment 14A and level: 0 segments 12A, 12B and 12C. Similarly, route settings
Cell 28D, 28 E and 28F, the level: 1 segment 14B and level: 0 segments 12D, provides a means for transmitting 12E, and 12F of information between each. Further, the routing cells 30A and 30B are
As shown, an information transfer path is provided between the level: 2 segment and the level: 1 segment 14A and 14B. As will be discussed in detail below, the routing cells each include two processing units.

The routing cells interface those segments through the interconnections on the bus elements. That is, the routing cell 28A includes bus elements 20A at ring interconnects 32A and 34A, respectively.
And 24A, and element 28B is
The bus elements 20B and (24) B are interfaced with ring interconnects 32B and 34B, respectively, and so on. Similarly, the routing cells 30A and 30
B are respectively ring interconnects 36 as shown.
A, 36B, 38A and 38B interface with respective buses, ie, 24A, 24B and 26.

FIG. 1 further illustrates a preferred mechanism for interconnecting remote levels and cells in a digital data processing system constructed in accordance with the present invention. Cell 1 at a point physically remote from bus segment 20F
8R is the bus of its segment and its associated cell (18P) via the fiber optic transmission line indicated by the dotted line.
And 18Q). The remote interface unit 19 provides a physical interface between the cell interface 22R and the remote cell 18R. Remote cell 18R is configured and operates similarly to other exemplary cells and includes a remote interface unit for coupling fiber optic links.

Similarly, level segments 12F and 14
Bs are interconnected from their parent segments via fiber optic links. As indicated by the dotted lines, the respective portions of the routing cells 28F and 30B are physically separated but joined by another fiber optic link. For example, looking at the routing unit 28F, the first part is the routing interface 3 of the segment 14B via the standard bus interconnect.
4F, while the second portion is directly linked to the routing interface 32F of segment 12F. As mentioned above, the physical interface between the routing unit portion and the fiber optic medium is provided by a remote interface unit (not shown).

[0055] Figure 2A illustrate the good or format has to <br/> for maintaining coherency over data in a multi-processing system of the type described above. The illustrated system comprises a plurality of central processing units 40A, 40B and 40C connected to associated memory elements 42A, 42B and 42C, respectively. Communication between each pair of processing and memory units is carried out along buses 44A, 44B and 44C as shown. The circuitry 46 representing the level segments and routing described above is an exemplary processing cell 42A.
Information packet (bus 48A, 48B)
And passed through network 46) via 48C).

In the illustrated embodiment, the central processing units 40A, 40B and 40C each include an access request element labeled 50A, 50B and 50C, respectively. These access request elements generate access requests to the data stored in memory elements 42A, 42B and 42C. Elements 50A, 50B and 50
Among the access request signals generated by C is the ownership request signal, which represents an exclusive modification access request to the data stored in the memory element. In the preferred embodiment, the access request elements 50A, 50B and 50C include a subset of instructions that are executed on the central processing units 40A, 40B and 40C. This instruction subset will be described later.

The memory elements 42A, 42B and 42C are
Each has control elements 52A, 52B and 52C. These control elements are each connected to data storage areas 54A, 54B and 54C via corresponding directory elements 56A, 56B and 56C as shown. Stores 54A, 54B and 54C are utilized by the illustrated system to provide physical storage space for data and command signals required by their respective central processing units. Thus,
Store 54A maintains the data and control information used by CPU 40A, and stores 54A and 54C maintain the information used by central processing units 40B and 40C, respectively. Data maintained in each store is identified by its own to Disk <br/> descriptor corresponding to the system address. These descriptors are
It is stored in the address storage location of the corresponding directory. Descriptors are considered unique, but some descriptors
Multiple copies of I descriptors, the memory elements 42A, 4
It may be between 2B and 42C. In this case, these multiple copies themselves identify copies of the same data element.

Central processing units 40A, 40
The access request signal generated by B and 40C includes, along with other control information, a descriptor or SVA request portion that matches the descriptor or SVA (system virtual address) of the requested data. Control element 52
A, 52B and 52C determine whether the requested data is stored in the corresponding storage elements 54A, 54B and 54C in response to an access request generated by the respective central processing units 40A, 40B and 40C. decide. If If so, the item of information is transferred for requesting use that by the processor. If not, the control units 52A, 52
B and 52C send the packet containing the request to the bus 48A, 4
Transmission to network 46 along 8B and 48C.

The control units 52A-52C also monitor the circuitry 46 to determine if they can satisfy remote access requests (ie, requests received from other processing cells). As described above, compared with the descriptors of the descriptor and state of the access request received on circuits network 46 stored in their directories. When the required data is found in the required state, it is transferred back to the network 46 in a response packet for routing to the requesting unit. When required data is not present in any of the processing cell system, operating system can search the peripheral devices in the system.

[0060] Data coherency over is maintained by cooperative働動operation of the processing cells in response to the data request and transfer. More specifically, upon generation of the ownership-access request packet by the first processing cell, the associated memory allocates physical space for its store and holds the requested data. Similarly, upon transfer of the requested data from the previously stored processing cell, the associated memory deallocates the previously allocated physical storage space for storage of the requested data. .

These collaborative operations are illustrated in FIGS. 2A and 2A.
This is illustrated in Figure B. In the first of these figures, namely Figure 2A, DATUM (0), DATUM (1) and DATUM (2) are
It is held in the store of the memory element 42A paired with the CPU 40A. The descriptors foo, bar and bas held in the directory 56A correspond to those data stored in the store 54A. The descriptors each include a pointer that supports the location of the associated information signal in store 42A.

The memory element 42B paired with the CPU 40B is
Store DATUM (3) and DATUM (2). The descriptors car and bas held in the directory 56B correspond to each of these data elements. DATUM (2) and its descriptor bas have been copied from store 42A and thus retain the same label.

The system illustrated in FIG. 2A has a CPU
No data is stored in the memory element 54C paired with 40C.

[0064] Figure 2B is for data by processing cell which does not already accessed to the data illustrate how it will move in conjunction with the ownership for it. In particular, the illustration depicts the movement of a DATUM (0) following the issuance of an outstanding ownership request to DATUM (0) by a processing cell consisting of CPU 40C and memory 42C. First, the control unit 52C
Simultaneously with the request, allocate physical storage space to store 54C of memory element 42C. The response packet generated by the memory 42A is the requested data, ie DA
Transfer TUM (0) from store 54A (where the data was previously stored here) to requester store 54C. At the same time, the control unit 52A deallocates the space of the store 54A that previously held the requested data.
At the same time, the control unit 52A invalidates the descriptor foo in the directory 56A (which was previously used to identify DATUM (0) in store 54A),
Control unit 52C, on the other hand, reassigns that same descriptor to directory 56C. Thus, the descriptor is then used to identify the signal in store 54C.

In addition to descriptors, memory elements 42A-42C assign access state information for data and control signals to their respective stores. Disable, read-only, these access conditions including ownership and atomic states, govern the manner that give access to the data by a particular processor. Data stored in a memory element whose associated CPU maintains modified access for that data is assigned an ownership state. On the other hand, data stored in a memory element whose associated CPU does not maintain priority access for that data is assigned a read-only state. In addition, data associated with "bad" data is assigned an invalid state.

[0066] The preferred structure for maintaining coherency over memory configured digital data processing system according to the present invention, the aforementioned related patents e.g.
It will be understood by reference to USSN 136,930 and USSN 370287.

FIG. 3 depicts a preferred form for the exemplary level: 0 segment 12 of FIG.
Segment 12A comprises processing cells 18, 18B and 18C interconnected by cell interconnects 22A, 22B and 22C along bus segment 20A. The routing unit 28A includes a level-0 segment 12A and a level-1 segment 14A in FIG.
It provides an interconnection between (if a parent). This routing unit 28A, as shown,
It is coupled to bus 20A via cell interconnect 32A.
The structure of the illustrated bus segment 20A and its relationship to the cell interconnects 22A, 22B, 22C and 32A is better understood by reference to the above-referenced patents, eg, U.S. patent application Ser. See.

[0068] Figure 4 is intended to draw a preferred have structure for an exemplary processing units 18A embodiment.
The exemplary processing cell 18A is a processor bus 6
Central processing unit 5 coupled to external device interface 60, data sub-cache 62 and instruction sub-cache 64 via 6 and instruction bus 68.
Eight. A more complete understanding of the circuit illustrated in this figure can be found in the related patents mentioned above, such as U.S. Patent Application No. 136,930 dated December 22, 1987 and U.S. Patent Application No. 509,480 dated April 13, 1990, June 1989. US Patent Application No. 370,3 dated 22
No. 25, and U.S. Patent Application No. 499,182 dated 26 March 1990.

Memory System As discussed above, the multiprocessing system 10 constructed in accordance with the present invention references the unique descriptor associated with each datum based on the system virtual address (SVA) to process cell 18A. , 18
Allows access to individual data elements assigned to B, ---, 18R. In the exemplary system 10,
The implementation of this capability is provided by the processing cell memory units and the operation of transferring request and response packets through the circuitry 46 between these memory units. In the discussion that follows, this is collectively referred to as the memory system.

A complete understanding of the structure and operation of a memory system may be achieved through a review of its architecture, listed below. (1) Data storage--The memory of each cache is divided into pages, and each page can be dynamically allocated to a page having SVA space. The memory system maintains application and state information about the data in each cache, facilitating efficient movement to and from secondary memory. (2) Data Locality--The memory system keeps data recently referenced by a processor in a sub-cache or cache of the same cell of that processor. (3) Data movement--The memory system moves data to the cache of the processor that refers to it. (4) Data Sharing--The memory system holds a copy of the SVA data in one or more caches, facilitating efficient data sharing by parallel programs. (5) Data coherency over - the memory system implements the strongly ordered Kohi over rent memory model and the transaction model. If those skilled in the art, an if the result of any execution as if it was performed in the sequential order is the operation of the processor of all hand the same system "sequentially consistent" each individual It will be appreciated that the behavior of the processors in this order will appear in this order, in the order specified by the program.

Further, if access to data by any one processor is initiated, issued, and performed in program order, and if store by processor I is observed by processor K, before issue of store. to if all hand access to data that is carried out with respect to I is not must be carried out with respect to K, memory access is considered "strongly was ordered." Comparison, is if the instruction accesses strongly to variables that are synchronized, if before the data access selection of all the hand from being performed, not issued access to variables that are synchronized in the processor, If no access to the data is issued by the processor before the previous access to the synchronized variable has been performed, then the memory access is weakly ordered.

[0072] Kohi over rent system with a strong command of the event is a sequentially consistent.

The organization of SVA space within the exemplary system is a major departure from the usual virtual memory scheme. Conventional architectures include software controlled page level translations that map system addresses to physical memory addressers and generate missing page exceptions. In these systems, software is responsible for multiplexing the page table among all the segments in use. In the exemplary system architecture, there is no software controlled page level translation mechanism. The memory system can handle a significant portion of the address space management normally performed by software in conventional architectures. Responsibilities for managing these include:

(1) Maintain page usage and status information. (2) Reuse old pages. (3) ensuring coherency over to synchronize the shared data access between multiple processors. (4) a copy of the data and data in the system to move from the location in the sub-page basis to the location, to hold the most frequently used and data near the processor Ru Tei it possible. (5) Directly implement sparse display of the entire SVA space by moving the SVA of the data together with the data. (6) Move the page between reference cells (there is no fixed origin for the page).

The processors of the exemplary system, eg, processors 40A, 40B, 40C, communicate with the memory system via two main logical interfaces. The first interface is the data access interface, which is implemented by load and store instructions. In data access mode, the processor
Giving SVA and access mode information in the memory system, the memory system by returning it finds the including subpage data, an attempt to satisfy that access.

The second logical interface mode is control access, and this is implemented by memory system control instructions. In a control access, the processor commands the memory system to perform some side effect and return some information other than the actual data from a page. System software, in addition to the main interface,
Use control locations in SPA space for configuration, maintenance, fault recovery and diagnostics.

A hierarchical memory system implements hierarchical storage.
In the preferred embodiment illustrated, each processing cell has a central processing unit (ie, CE) with a sub-cache storing 0.5 megabytes of data.
U) is included. These processing cells also include caches that store 32 megabytes of data. Furthermore,
A preferred level: 0 with, for example, 15 processing cells stores a total of 480 megabytes. On the other hand, the preferred level: 1 with, for example, 32 levels: 0 is
It has a total of 15360 megabytes. The cache structure memory system stores data in units of pages and subpages. Each individual cache describes the SVA space subdivided 32 bytes to 2048 pages. A page contains 2 ¹⁴ (16348) bytes that are sub-page divided into 2 ⁷ (128) bytes. The memory system allocates memory in cache on a page basis,
Each page of SVA space is either completely represented in the system or not at all. The memory system shares data between caches in units of sub-cache.

[0078] If a page of SVA space is present in the system, the following can be said. (1) One or more caches have one page of memory
Allocate pages. Each sub-page of the page, but is 1 or more in the storage of the cache with the assigned space for each cache, all hands subpages of a page with a space allotted (2) with respect to one page May or may not include a copy of.

The association between the cache page and the SVA page is recorded in each cache in that cache directory. Each cache directory is composed of a de I subscription <br/> descriptor. 1 Tsunode I descriptor exists for each page of memory in the cache. At a particular point in time, the de I descriptor is said to be valid or invalid. If de I descriptor is valid, the corresponding cache memory page is made to associated with a page of SVA space, de I descriptor stores SVA page address and state information related to. If de I descriptor is invalid, the corresponding cache memory page is not logically use state. No explicit validity flag associated with de I descriptor, if clear the A <br/> mooring and held fields are both de I descriptor can be considered as invalid, and effective subpages for the SVA page not exist.

Each cache directory acts as a content addressable memory. This cache is able to find the descriptor for a particular page of SVA space without forming the interactive search throughout the hand of the descriptor. Each cache directory is a 16- way set with 128 sets
It is implemented as an associative memory . All pages of SVA space are divided into 128 equivalence classes, each associated with a cache directory set. Only one descriptor per page is stored in the set of cache directories corresponding to the page's equivalence class. The equivalence class is selected by SVA [20:14]. At any given time, the cache can describe no more than 16 pages with the same value for SVA [20:14]. This is because there are 16 elements in each set.

Subpages and Data Sharing When a page exists in the memory system, each of the subpages resides in one or more caches. The containing SVA if the subpage is in cache
The descriptor for a page (in its cache) records the existence of that subpage in several states. The state of subpages in the cache determines two things. : (1) whether the local processor's cache may perform any action relates to a data that exists in the sub-page. (2) If so, what response does the cache make to requests for that subpage received from another cache via the level?

The state of the sub-cache in the cache is
Changes over time when a program requires an action that requires a particular state. A set of diversion rules specifies changes in subpage state that result from processor requests and inter-cache level communication.

In order for the processor to complete the instruction or data reference, several conditions must be met at the same time. That is, in the case of (1) instruction reference, the sub-block containing data must exist in the instruction sub-cache. For most data operations, the subblock containing the data is
It must be present in the data subcache in its proper state. (2) The subpage containing the data must be in the local cache. (3) The local cache must hold the subpage in an appropriate state.

If the data is not in the required state in the subcache, but is in the local cache in the correct state, the CPU gets the data from the cache.
If the local cache does not have the data in the correct state, the cache will communicate through the levels to get a copy of the subpage, get the required state for that subpage, or both. To do. If the cache does not satisfy the request, the cache returns an error signal signaling the processor of the appropriate exception.

[0085] The set of instructions, including several different types of load and store instructions, this instruction, the program, the future data reference pattern expected for the current control thread suitable subpage state, and Forces protocols between different threads of control in parallel applications. This section first describes the states and their transformations in terms of processor instructions and their impact on caches.

Subpage States Subpage states and their transformation rules provide two general mechanisms for programs. (1) The mechanism transparently implements a strongly ordered, sequentially consistent memory access model for normal load and store accesses by the system's processors. (2) mechanism provides a set of transaction Prin primitive used by the program to synchronize the parallel computation. These primitives can be applied to various conventional and non-traditional synchronization mechanisms.

The basic model of data sharing is defined by three classes of subpage state: invalid, read-only and owner. These three classes are dictated by the strength according to the access they allow. The disabled state does not allow access. The read only state allows load and instruction fetch access. Several owner state is present, but they allow full and load access, allowed in response to a data request from the interconnect by the cache, or is intended to permit store access.
Only one cache may hold a particular subpage in the owner state at any given time. The cache that holds the subpage in the owner state is called the subpage owner. Ownership of each subpage moves from cache to cache when the processor requests ownership through store and special load instructions that request ownership.

The following sections describe state classes and how they interact to implement a strongly ordered sequential and consistent memory access model.

Invalid State If a subpage is not in the cache, it is said to be invalid for that cache. When a processor requests a load or store to a subpage that is in an invalid state in its local cache, that cache must request a copy of that subpage in some other state to satisfy the data access. There are two invalid states: invalid descriptor and invalid.

[0090] If no <br/> de I descriptor for a particular cache specific page, all subpages of that page are said to be in invalid de I descriptor state in that cache. Thus, subpages in invalid de I descriptor state are not explicitly represented in the cache.

When the CEU references a subpage in an invalid descriptor, the local cache assigns one of the descriptors (in the correct set) to the SVA. De
After I descriptor allocation is complete, all the sub-page of the page has an invalid state.

If the local cache has a descriptor for a particular page, but the particular subpage does not exist in that cache, then that subpage is in the invalid state. The local cache attempts to obtain subpage data by communicating with other caches.

Read-Only State There is only one read-only state, that is, read-only state. Any number of caches may hold a particular subpage in a read-only state, provided that the subpage owner holds the subpage in a non-exclusive state. If the owner of the subpage has any other state (ie exclusive ownership state, ie one of exclusive, atomic, or transient atomic), the read-only copy cannot exist in any cell. The CEU cannot change a subpage that is in the read-only state.

Owner State There are two basic owner state types: non-exclusive and exclusive-ownership. When a particular cache holds a particular subpage in a non-exclusive state, some other cache may hold that subpage in a read-only state. Programmatically, the non-exclusive state is the same as the read-only state. CEUs cannot modify subpages that are in a non-exclusive state. The non-exclusive state is
It is basically a bookkeeping state used by the memory system, which defines subpage ownership.

Exclusive ownership states are exclusive, atomic and transient atomic. When a particular cache holds a particular subpage in exclusive ownership state, other caches may not hold a read-only or non-exclusive copy of the subpage. If the local cache holds the subpage in exclusive ownership state, the STT allows write access to the segment and the descriptor . CEU on the condition that no write flag is in the clear state can change the sub-page data.

Atomic State Atomic state is a stronger form of ownership than exclusive state. Only subpages can enter and leave the atomic state as a result of explicit requests by the program.

Basically, atomic state can be used for single-threaded access to any subpage of SVA space. The processor requests that the subpage enter the atomic state by gsp. NWT (subpage acquisition, without waiting) When executing the instruction is usually complete only if subpage is not in atomic state in already. Thus, the atomic state on a subpage can be used as a single lock. The lock is gsp. nwt
The instruction usually gets the subpage in exclusive state first,
By changing the state from exclusive to atomic,
Will be done upon completion. The lock is unlocked by executing the rsp (release subpage) instruction. The rsp instruction requires the subpage to reside in a cache that is in the atomic or transient atomic state. The local cache gets the subpage and then changes the subpage from the atomic or transient atomic state to the exclusive state. (If the subpage has transient atomic state, the behavior is more complicated, but the result is the same programmatically.)

It is important to note that atomic state is only associated with subpages. It is not associated with a particular running system process (typically a user program) or a particular cell. It is important that the process execute the gsp instruction to get the subpage in atomic state and then be switched by the system software to continue execution on other cells. The process continues execution on the second cell, eventually executing the rsp instruction to free the subpage. Between these two instructions, the entire memory system has only a single copy of its subpage, and it will be in atomic or transient atomic state. When various processors execute instructions that reference subpages, their single valid copy moves from cell to cell. It is also possible that one process gets the atomic state and another process releases the atomic state.

Atomic state is simply an additional flag associated with a subpage that allows protocols to be implemented that use the atomic state in addition to the subpage's data state. Atomic state protocols can be flawed, just as protocols implemented using only data can have errors. The hardware is STT and descriptor . No checks are placed on the use of atomic state beyond the access controls imposed by no atomics.

Transient atomic state gsp. The nwt instruction always completes within its limited execution time, but may or may not (depending on the current state of the subpage in the memory system). The second form of instruction is gsp. wt (get subpage, wait), but this does not complete until the subpage is acquired in the exclusive state and changes to the atomic state. gsp. The wt instruction relieves the programmer from the burden of determining whether the gsp instruction succeeded or not. If the processor is gsp. When executing the wt, if subpage atomic or transient atomic state to already, subpage is released, obtained by the local cache, the processor exclusively from state to change to the atomic or transient atomic state stops. g
sp. The use of the wt instruction can reduce the number of messages sent between caches as the cell waits for an opportunity to lock.

The transient atomic state is gs
p. Used automatically by the memory system to allow wt to function efficiently. Its use is completely transparent to the programmer. The subpage is in the atomic state and another cache has gsp. If executing wt, the subpage goes into transient atomic state in the holding cache. If the subpage is later released with the rsp instruction, the transient atomic state causes the subpage to be driven to an interconnect that is in a special released state. The freeing cache invalidates its own state for the subpage. Any cache running gsp will encounter the subpage and accept it. The accepting cache can then complete its gsp instruction and the subpage enters the transient atomic state in its cache. This action is done for each subsequent gs
It happens for p and rsp and continues until such time as the subpages being destroyed are no longer accepted by any other cache. The cache performing the release then changes the subpage from the invalid state (set when the subpage was released) to the exclusive state.

It is also possible for a packet with a release status to be accepted by the cache where the CEU is performing a load or store instruction. The original cache sees the subpage accepted and leaves the subpage in an invalid state. The accepting cache is C
The EU is allowed to execute a single instruction before withdrawing the subpage, set its own subpage state to invalid, and send the subpage in the released state. The cache that was doing the load or store is now the owner of the page for the purpose of this release. As before, if no other cache accepts the data, this cache will
Change subpage state to exclusive state and retain ownership.

Preferably, gsp. There should be no limit to how long a wt command can wait. The process that issued the order is
Wait until the subpage is freed. Instructions may be interrupted by various XIU signals. When such an event occurs, the CCU abandons its attempt to gain atomic access. If the subpage is released in the meantime and there are no other requesters, the subpage changes from the transient atomic state to the released state and finally to the exclusive state. In a typical system software operation, interrupts are processed and interrupted instructions are restarted,
The CCU will issue the request again. As before, it may go on or be put on hold.

State Transition This section presents some examples of subpage data and state flow. The basic mechanism for moving data from one's owning cache to another cache is instruction fetching and execution of load and store instructions by the processor local to those other caches. Different load and prefetch instructions require programs to have their local caches get read-only or exclusive-ownership state, and store instructions always ensure that subpages have exclusive-ownership state. I need. In some situations, a cache may get a read-only copy of the subpage it passes through on the interconnect. Subpage Post-Store (ps
tst) instruction makes a read-only copy of the subpage.
To communicate to the relevant cache of all hands. Finally, the owning cache may send an ownership state on the interconnect as part of recombining pages (discussed below).

Instruction fetch and load instructions can result in the local cache requesting a read-only copy of a subpage. This request is answered by the cache that owns the subpage. If the owning cache has subpages in the non-exclusive state, it supplies the requesting cache with a read-only copy, but does not change its own state. If the owning cache has a subpage in the exclusive state, it changes its own subpage state to the non-exclusive state and then supplies a read-only copy to the requestor (requestor). If the owning cache has a subpage in atomic or transient atomic state, it serves the subpage in that state and invalidates its own copy.

When the cache requires exclusive ownership,
The owning cache yields its copy of the subpage. If a subpage is owned non-exclusively, there may be read-only copies in other caches. Cache consuming all hands in response to the exclusive ownership request to invalidate their read-only copies.

If the cache acquires a subpage in exclusive ownership state to satisfy a store instruction,
Do not give ownership or read-only copy to other caches until the store instruction is completed. This rule is a strongly commanded property in memory systems because it guarantees that the leader of a memory location will encounter changes in the order in which they are made.

When a subpage is in the atomic state, this state can change to the transient atomic state, but it never changes to another state as a result of some load or store instruction. Whenever some other cache requests a subpage, that cache gets the subpage in atomic or transient atomic state. After the subpage is released to exclusive state,
The transition between exclusive and non-exclusive states may occur again and there may be a read-only copy if the subpage is non-exclusively owned.

[0109] in an invalid state particular subpage in a particular cache (i.e., a descriptor is assigned to already, there is no particular subpage), a copy of the subpage, from some other cache A cache with an invalid state obtains a read-only copy of its subpage if it is available at the information transfer level interconnect due to the request of. The effect of this mechanism is to accelerate parallel computing. This is because it can eliminate the latency associated with requesting a copy of a subpage from another cache.

It is important to note that the underlying mechanism provides a strongly ordered memory access model for programs that use simple load and store instructions. Programs can use the forms of load, store and prefetch that require specific states to improve their performance, and in many cases it is expected that the compiler will perform the necessary analysis. However, this analysis is optional.

In simple transactions, subpage atomic state is used purely as a lock. Subpage data is not relevant. Some of the more sophisticated forms of synchronization make use of subpage data held atomically. One technique uses atomic state on a subpage as a lock on the data within the subpage. Program is 1
Or put multiple subpages into atomic state, manipulate their contents, and release them.

The interaction between data copy creation strategy states is the time spent by load instructions waiting for a copy from another cache, the cache space and information transfer level bandwidth, and the time a store or gsp instruction is executed. It is a compromise between the time and bandwidth spent invalidating the copy in the other caches. If there are many read-only copies of subpages in the system, reads will be in the local cache.
Chance to find the data that exists in already is increased. However, if there are any read-only copies in the system, the owner must first invalidate them (by sending a single message) when modifying subpages. The heuristics described below attempt to dynamically distinguish dual read / write sharing from single read / writer access on a short term basis.

Duplex read / write sharing results in multiple read-only copies with high primary locality and write updates with low primary locality. Retaining a read-only copy is most efficient because multiple copies are read multiple times between updates. CE under change
U can get an exclusive-ownership state with a single information transfer level behavior. The CEU uses the pstsp instruction to announce changed data and distributes the data to the participating caches.

Single read / write access results in multiple read-only copies with low primary locality and write updates with much higher locality. Retaining a read-only copy is not very efficient because the copy is updated multiple times between reads. Single lead /
Write copy (exclusive-ownership state) does not require any information transfer level operation for writing.

The strategy for balancing these considerations is as follows. (1) When a copy of a subpage is sent across the information transfer level to satisfy a request, any cache that has a descriptor for that page but no copy for the subpage will be read-only copied from the message. To pick up. This mechanism
Accelerate application with high locality references. (2) When a cache that has an exclusive or read-only state needs to update a subpage, each other cache that has a copy of the subpage invalidates that copy and exclusive ownership is changing. Move to a cell. The cell being changed uses pstsp to announce the change. Attempts by other caches to read the subpage may result in copies not being distributed to the involved caches.

Cache Layout The preferred organization of the cache directory is shown in FIG. When a reference is made to the SVA, the cache must decide whether it has the information it needs. This is accomplished by selecting a set in the cache and testing all descriptors in that set. SVA [20:14] selects one set. In a typical architecture, each descriptor in the selected set is SVA [63:
21]. In the preferred embodiment having 240 megabytes, this means comparison with SVA [39:21]. If one of the elements of the set is a descriptor for the desired page, the corresponding comparator will indicate a match. The index in the set of matching descriptors concatenated with the set number identifies a page in the cache.

[0117] If one or more of the descriptor is a match, the cache to notify the multiple de I descriptor matching exception. If the descriptor is aligned, the cache allocates a de office <br/> descriptor and requests data from the interconnect. Either the allocation or the data request may be missed, but the cache indicates an error to the CEU.

Using SVA [20:14] to select a set is effectively a hash function on SVA addresses. The system software uses S so that this hash function will give good performance in the general case.
You must assign a VA address. Regarding distribution 2
There are two important cases. And when referring to a number of pages of a single segment, is the case of referring to the first page of many segments. This set selector provides good cache behavior for consecutive pages. Because there are 128 consecutive pages in 128 distinct sets. However, this selector does not have sufficient hashing behavior for many pages having the same value in SVA [20:40]. System software can avoid the latter situation by changing the local source of data within the segment.
For example, the user stack for each process can start with a different segment offset.

Descriptor Content When a cell responds to a subpage request, it supplies the subpage data and the value of some descriptor field to the local cache. When the response returns to the requester, the requester either copies these fields into the descriptor field (if it has no other valid subpages) or logically ORs these fields into the descriptor field. Some descriptor fields are neither supplied by the responder nor updated by the requester.

[0120] In a preferred embodiment, the de I descriptor fields are defined as follows: de I descriptor.
Tag (19 bits). Bits of SVA [39:21]. This field, SV identified by the corresponding de I descriptor
Identifies a particular page in the A space. For a given set in a given cell, 1 of this field is all hands
It must be unique among 6 of de I descriptor. The software "sets" this field when creating the SVA page. (This is also set during cache initialization.)

[0121] de I descriptor. atomic Change (1 bit) The cache sets this bit flag to 1 when any subpage of this page experiences a transition to or from the atomic state. Because g
This is because the sp or sp instruction was successfully executed. This also means that the subpage is transient from the atomic state.
It is also set when changing to the atomic state. This flag is not set if gsp is missing or rsp is missing. Because is in atomic state subpage in already in the former, because the subpage was not in atomic state in the latter. This flag is not set if gsp or rsp is missing. This is because de I descriptor. This is because no atomic is set. System software sets this flag to zero to indicate that it has acknowledged an atomic state change. This field is propagated from cache to cache.

[0122] de I descriptor. Change (1 bit) The cache sets this bit flag to 1 when any data is changed in the page. System software, to indicate that it has recognized the change of page, de I descriptor. Set changes to zero. This flag is not set when there is no attempt to change the data. This is because de I descriptor. This is because no writing is set. This field is propagated from cache to cache.

Descriptor . LRU position (4 bits) current location Most R de I descriptor in the set
The cache maintains this field when transitioning from ecently Used (0) to Least Resently Used (15).

Descriptor . Anchor (1 bit) The not obtained also recognized as valid data requests from other caches, for instructing that the de I descriptor can not be disabled, the software sets this field. Read or get requests from other caches return without responding to the requester and are treated as if there were no pages.
This field is set by the system software as part of creating or destroying an SVA page and as part of modifying the page descriptor.

Descriptor . Hold (1 bit) Software sets this field to indicate that the descriptor cannot be invalidated by the cache even if the subpage is not in the cache.

Descriptor . No Atomic (1 bit) Software sets this field to prevent the cache from changing the atomic state of the page for this page. There is no attempt to use gsp or rsp for active use and the processor is notified. The processor signals a no page atomic exception. Descriptor . No atomic can be modified even if some subpages have atomic state. This flag indicates that the descriptor . Easily prevent attempts to change atomic state in the same way that no writes simply prevent attempts to change data state. This field is propagated from cache to cache.

Descriptor No Write (1 bit) Software sets this field to prevent the page from being modified by the local processor. Rather than attempt to change the page, it is reported to the processor.
The processor signals a no page write exception. This flag does not affect the ability of the cache to get a subpage in exclusive or atomic / transient-atomic state. This field is propagated from cache to cache.

Descriptor . Summary (3 bits) Aggregates the subpage status field of a set of subpages. There is one 3-bit summary field for each set of sub-pages. Summaries are sometimes divided into individual subpages for subpages in the summary set. Ignore (override) the contents of the status field.

Descriptor . Sub page Status (4 bits) The subpage status consists of a 3-bit status field and a single-bit sub-cached status field. It is set by the cache to record the state of each subpage and to indicate if any part of the subpage is in the CEU subcache.

[0130] flag in the de I descriptors may be the case that out-of-date or incorrect. There are two reasons for this. That is, the latency between a CEU and its local CCU and the latency between CCUs of different cells. The former can occur when a page that was not previously modified is modified by the CPU. The local CCU does not know that a change has occurred until the changed subblock leaves the subcache. The latter may that to put <br/> when many cache containing a de I descriptor for each particular SVA page, state change affected by one cache is automatically notified to other caches of all hands Not done.

[0131] Since the valid bit is not associated with de I descriptor, de I descriptor has a tag of always SVA page. However, cache de I descriptor of all the hand with a tag to indicate a particular SVA page,
If not have a valid sub-page to their de-I in the descriptor, SVA page does not exist in the memory system. In the same way, if you do not have a tag cache de I descriptor to indicate a particular SVA page, that page does not exist in the memory system. Although it is possible to read the de I subscription <br/> descriptor fields for such a page, SVA page since been destroyed logically, field values are not valid.

[0132] For example, consider two caches with de I script <br/> data for a page. Cash A
Has a sub-page on the full and exclusive state, and de
I descriptor. Change is a clear state, other cache and does not have at all a de I descriptor for the page. The CEU in cell B executes a store instruction to change the subpage data. CEU B requests a subpage with exclusive ownership state from its local cache. Cache allocates de office <br/> descriptor for the page, and then requests the subpage using the ring. Owner (A) responds by giving cell B exclusive ownership. After subpage arrives, cell B
The response Karade I descriptor. To copy the changes (chestnut is in the A state). B's CEU then loads the subblock from the subpage into its data subcache and modifies that subblock. In this respect, CEU
The subcache indicates that the subblock has been modified, but the local cache still marks the subpage as unchanged. At some later time, CEU B
Send subblock data from that subcache to the local cache. This means that CEU B needs subcache blocks for other pages, or CPU
May be using an idle cycle to write back the modified subblock or some other cell has requested the subpage. Subsequently, the cell B cache de I descriptor. Set the change flag. Throughout this time, de I descriptor. Modifications are cleared state on cell A.

As a second example, assume that system software running on cell A now destroys an SVA page. Page by changing the de office <br/> descriptor of raising collect subpages of all hands in the exclusive state in cache A, then to have no SVA page no longer valid subpages A Destroyed. However, cache B is still has a de office <br/> descriptor for the page. De I descriptor. Fields like change don't make sense. SV followed by some other cache
Even re-creating the A page de I subscription <br/> descriptor cache B will remain unchanged until the first subpage arrives.

[0134] The system software is, when you try to use the de I descriptor information must ensure that the SVA page exists truly memory system.
One way to accomplish this is to always set the page anchor and get at least one valid subpage in the local cache. To ensure that changes and atomic change field is not set to absolute, the software must obtain each subpage in the first exclusive state. Anchor is prevented from obtaining any other cache other subpage obtained already asynchronously. When the operation is complete, system software de I script <br/> data in local cache. An anchor to clear. At that time, the other cache request for subpages of that page, would be owned again.

[0135] The system software is also, prior to creating the page, SVA page is it necessary to ensure that does not exist truly in the memory system. As described above, the mere presence of de I descriptor with the correct tag value does not indicate that there actually memory system SVA pages. The software may confirm the non-existence by setting the page anchor and then attempting to pull out the subpage that has the exclusive state. If the draw is successful, the software loses the competition and the page resides in the memory system. Otherwise, the software can use the mpdw instruction to create the page,
Set the SVA page address for all exclusively owned pages. Note that it is still necessary for a software interlock to exist for concurrent use of mpdw for this purpose.

Cache Descriptor Usage and Exchange The cache of the exemplary system 10 can be used by system software as part of a multi-level storage system. In this type of system, physical memory is multiplexed over large address spaces by demand paging. The cache has features that accelerate the implementation of a multi-level memory system in which software moves data between the cache and secondary memory in units of SVA pages.

[0137] of all hand cache, constitute the primary memory of the system together. However, for some purposes it is necessary to consider each individual cache as an independent primary store. This is because each cache has a limited number of S
This is because only VA pages can be held. That is, 2048 pages for each cache, 1 for any particular set
It is 6. There are several automatic and manual mechanisms to balance cache utilization.

[0138] Each cache maintains the LRU state for the presence page of all hands. LRU data is separately maintained for each of the 128 sets of the descriptor associative memory, in their set I follow the last reference time GENERAL prepare the 16 pages.

Each cache maintains the LRU-MRU ordering of descriptors within each set. The order setting is descriptor. LRU Maintained with priority. Each descriptor in one set is a descriptor. LRU
It has a value of 0 (MUR) to 15 (LRU) in priority.
Conceptually, when a page is referenced, it moves to the MRU, and all pages from the MRU that go to the old LRU priority of the referenced page then move one step towards the LRU. The descriptor is made the MRU in the next operation. That is, (1) including the increase of the sub-cache state strength, the local CE
On any load from U. (2) At any store from the local CEU. (3) By any local CEU writeback (back write) that does not store sub-cache. (4) Following the XCACHE operation (the cache requires the local CEU to change the sub-cache state). (5) After deallocation from the local CEU subcache.

A CEU instruction may reference an SVA page that is not in local cache (invalid-descriptor state). Cache must be assigned a de I script <br/> data as described below, then issues a request to the interconnect for the particular subpage which is required by the CEU. If the corresponding page exists somewhere in the system, the requesting cache copies certain descriptor fields and referenced subpage data from the responding cache. Finally, this process of assigning descriptors fills the elements of a particular cache set.

Each cache has the ability to automatically reuse a particular descriptor by changing the SVA page in that cache to the invalid-descriptor state. As a background activity, the cache also automatically moves owned subpages to other caches. These automatic operation, the total and is tuned or disabled by system software, and the descriptor. Holding and / or descriptor. When the anchor is set, it is inhibited. The following theory describes the activities of cache descriptor correction and background data movement.

Occasionally there may be one or more descriptors in the cache set that do not have valid subpages. This situation can occur as a result of requests issued by other caches or as a result of recombination activity (discussed below). For example, assume that a cache descriptor has only one valid subpage. In this case, the cache descriptor has an exclusive state. When some other cache requests ownership of a subpage, this cache no longer has a valid subpage for the page. What if the descriptor. Retention and descriptor. When the anchor is in <br/> cleared state in this cache, this descriptor, CEU invalid on the same set of this cache - when referring to some other page having a descriptor state can be reused.

On condition that the sub-cache is not stored in the sub-cache, the cache automatically drops a page having all pages in the read-only or invalid state (read-only page). This is possible because when the memory system destroys the read-only copy , no information is lost by the memory system. The cache has the option of either completely disabling dropouts of the copy or limiting it to a partial range of descriptors, according to the ordering of the LRUs.

The cache has all pages in an exclusive ownership state , and the descriptor. Modifications and descriptors. atomic Automatically drop the page with a change in the clear state. The fact that a cache has exclusive ownership of all subpages means that the other cache does not have a copy of the page with ownership or data state, and this cache has no data to communicate with it. Guaranteed to destroy state. descriptor. Modifications and descriptors. atomic The fact that both changes are cleared state indicates that the data or atomic state change (including transition from atomic state to transient atomic state as a result of gsp.wt instruction) has occurred. This occurs because the SVA page was created by system software or last polled. The memory system is assumed to have a copy of the atomic state and page data on secondary memory, and this can be used to recreate the SVA page and destroy it. Cache according LRU order number setting, either totally disable the shedding of pure pages, with a selection of some form to limit it to a partial range of descriptors. If any of the sub-pages are in the transient atomic state, the system software will use the descriptor. atomic
Never had that has been noted that should not be asynchronously clears the change.

Descriptor. Note that the hold settings do not guarantee that the individual subpages are in the local cache. In addition, the descriptor. Simply setting a hold does not prevent other caches from destroying the page. The system software must take explicit action to prevent the hardware from dropping pure pages automatically (eg descriptor. Set changes).

Rejoin is an operation in which owned subpages are expelled from the cache to the interconnect, giving other caches the opportunity to accept the ownership state. Cache in the acceptance is supposed have descriptors assigned to pages that include the already, the cache does not assign a descriptor to accept ownership of subpages is being recombined . Desired page recombination is to reduce the total number of de I descriptor assigned to a particular SVA.

The cache issues a rejoin message to rejoin the subpages. If the rejoin message does not find another cache that takes over the page, the rejoining cache retains the data. In fact, it turned out to be the goal of recombination in itself.
When some other cache accepts ownership (which can be non-exclusive, exclusive, atomic, or transient-atomic state), the issuing cache changes its subpage state to invalid.

The cache attempts to automatically rejoin subpages as background activity while a memory refresh is occurring. During each refresh interval, the cache tests a particular set and searches for an acceptable descriptor. Descriptors must not have subcached subpages, must have some subpages, and must not have all subpages in exclusive ownership state. When such a page is found, the cache issues a rejoin message for some owned subpage. The cache is
Depending on the LRU ordering, we have a structural choice to either disable background recombination entirely or limit it to a partial range descriptor. Background rejoins give the cache a greater chance of being able to allocate descriptors for newly referenced SVA pages, instead of causing an entire line error.

The CEU also provides instructions that allow system software to attempt to destroy owned subpages. Subpages must not be subcached, but no other constraints are imposed before the cache attempts to evict ownership.

Since the pages below the working set point are unlikely to be referenced, most of the recombination that actually moves the data moves it from the cells that are not referencing the data to the cells that are referring to the data.

[0151] When the new de I descriptor in the set is required, cash proceeds in the order through a number of only the following operations are required to find a useful de I descriptor. In other words, find a de I descriptor of the disabled (1). (2) Disable read-only copy. (3) Destroy a pure SVA page. (4) to be able to assign de I descriptor by the aforementioned means, for notifying the exception of line fill.

The individual steps are described below.

Find the invalid descriptor. If there is if invalid <br/> de I descriptor, it can be used immediately. This requires the following conditions. In other words, - sub-pages of all the hand is in the disabled state, and - de I descriptor. Retention and de I descriptor. Both the anchor is in the cleared state. -The sub-page of the page is not described by the PRT entry.

Read-only copy is invalidated. Hello ctl $ ccu lru If configu.cde is 1, the cache will attempt to identify the de office <br/> descriptor that contains only read-only copy of the sub-page. The cache supports LRU or MRU while waiting for a page with the following conditions :
Reach . All pages are read-only or invalid. -There are no subcached subpages. −ctl $ ccu lru LRU value greater than or equal to configu.cdl . - de I descriptor. Retention and de I descriptor. Cleared state anchors are both. -The sub-page of the page is not described by the PRT entry.

[0155] and is found if accepted can de I descriptor, sub-pages of all the hand is changed to invalid state, the <br/> de I descriptor is used.

Drop pure SVA. Hello ctl $ ccu lru
If config.pde is 1, the cache will try to identify SVA pages that may be corrupted (completely removed from the memory system). The cache searches for LRU or MRU while waiting for a page with the following conditions: -All pages are in (various) exclusive ownership states. -No subpages are stored in the subcache. - de I descriptor. Both changes and de I descriptor atomic changes in clear state. −ctl $ ccu lru LRU value greater than or equal to configu.cdl . - de I descriptor. Retention and de I descriptor. Cleared state anchors are both. -The sub-page of the page is not described by the PRT entry.

[0157] Once found if acceptable de I descriptor, subpages of all hand changed to invalid state (thereby destroying the SVA page), and its to Disk <br/> descriptor is used .

Line To reduce the chances of a full exception (all line exceptions) occurring, the cache attempts to reduce its ownership of subpages within recently unused pages. These operations are independent of descriptor allocation operations and occur during RAM refresh.

Hello ctl $ ccu lru When configu.bre is set, the CCU searches from the LRU while waiting for a page with the following conditions. -A subpage that is in the ownership state. -A subpage that is not in exclusive ownership state. -No subpages are stored in the subcache. -LR greater than or equal to ctl $ cculruconfig, ble
U value. -Descriptor. Retention and descriptor. Cleared state anchors are both. - sub-page of the page can not be more descriptive to the PRT entry.

When an acceptable descriptor is found, the cache issues a rejoin message to the interconnect.

The system software can monitor the LRU behavior of pages in individual caches and set a working set point for each cache line. Low de I descriptor with the LRU value is located in the Working Tsu in the grayed set, descriptor with a high LRU is not in the working cell Tsu in the door. Repeated tests of descriptor tags, LRU values and subpage state can be used to determine the working setpoint.

Change (or Atomic) When a (modified) page leaves the logical working set, system software may act to purify the page so that the descriptor may be reallocated in the near future.

If the descriptor holds a relatively small number of subpages in ownership, the software will
Try to rejoin these subpages to other caches using the csp (subpage rejoin) instruction. If not, you could get the entire SVA page and purify or destroy it.

When the working set changes, system software may change the morphological parameters that control the cache. The parameters for pure page loss, copy loss and background recombination can be modified to ensure that the hardware operation is consistent with the software working set policy.

A typical use of secondary memory is to extend the cache storage model provided by the hardware, with each SVA having a corresponding page assigned to it. A page initially exists only in secondary memory,
It is time referenced by the program, it is cached in memory. If the page is modified while in memory (data state or atomic state), a copy of the page on secondary memory can be reused for another SVA with the page of the containing memory system. Must be updated before. Such changes are made to the descriptor. Modifications and descriptors. atomic It can be detected by the change field. The act of updating secondary memory is referred to as "purifying" the SVA page. System software typically purifies SVAs that are below the working set point so that cache space is quickly available for other SVA pages.

Only some of the cells in the system are SV
It has a physical connection to a disk drive that provides secondary memory for A pages. A suitable I / O-capable cell must be able to obtain page data during I / O operations. System software may ensure this by allocating and holding descriptors on I / O cells before issuing I / O instructions. After the purification is complete, the system software will run the descriptor.
Modifications and descriptors. atomic The change flag click to <br/> Li A, or can destroy the SVA page. The actual protocol used may be more complicated depending on whether the system allows the program to access the page during the attempting purification.

In general, the system software will use the unless the memory system prevents pages from being corrupted.
Descriptor if any subpage has transient atomic state. atomic You should not click <br/> re-A changes. For example, if all descriptors have a retained flag set, or descriptor. If the change is set, the memory system will not corrupt the page.

When a particular set of a particular cache is full, software has no reason to assume that the entire memory system is correspondingly full. Thus, it is desirable for software to respond to a full set by moving a page from that set to a corresponding set in another cache.

In order to use memory efficiently, the software should use the appropriate pages to remove from the full set,
Some strategy must be used to identify the appropriate target cache (if any) for that page. The cache is equipped to facilitate certain strategies for this page exchange. To facilitate software selection of pages for exchange within the set,
Each cache is the most recently used page (M
RU) to the least recently used one (LRU). When a page is referenced, it is MR
Moved to U. Then when another page is referenced, it becomes stale towards the LRU. LRU information accelerates the strategy of replacing the most recently used pages.

Processor Data Access The processor makes data requests to its local cache to satisfy load and store instructions and co-execution unit operations. The cache requests its local processor to cause the processor to invalidate its copy of the subpage in the subcache.

Load and Store Instructions When the subblock containing the referenced address is not in the required subcache in the required state, the processor will pass load and store instructions to the local cache as a request. The different types of load and store instructions pass information to the local cache about the access patterns of the instructions below. For example, if the order of instructions is a load followed by a store and the subpage containing the data item is not already in the local cache, then take ownership of the load rather than get read-only for the load instruction, then It is more efficient to communicate through the information transfer level to take ownership of the store at.

The state of the sub-block in the sub-cache does not necessarily reflect the state of the corresponding sub-page in the cache. The instruction sub-cache always gets a read-only copy of the data. The data sub-cache may hold sub-blocks in read-only or exclusive state. Subcache has exclusive ownership state, and the descriptor. It may have an exclusive state only when no write is set. The sub cache is
No distinction is made between exclusive, atomic and transient atomic subpage states. If the sub-cache has the sub-block in the exclusive state, the CEU may simply store the new data in the sub-cache to execute the store instruction. st64. to the store instruction of all hands except nsc, when if the subblock is not described by the subcache, or invalid or if no read-only state, CEU is requesting an exclusive state before completing the store instruction Must. What if the descriptor.
If no write is set, or the subpage does not exist in the memory system, then a fault occurs.

When a subpage request arrives from another cache, the owning cache must respond. If the any portion of the subpage is in the data subcache, the local cache must ensure that obtain changes that may have Oh Rukamo be present only in the sub-cache. The cache also causes the CEU to change the sub-cache state for a sub-block to read-only or invalid, depending on the request. In some cases, the cache also ensures that the instruction subcache invalidates its read-only copy of the subpage.

It is also important to distinguish between sub-cache management units (blocks and sub-blocks) and cache management units (pages and sub-pages). Data travels between the CEU and its local cache in sub-blocks. Data also moves between caches in subpages. There are two subblocks per unit subpage.

The different types of load and store instructions are described below. Each description begins with a summary of the semantics of the instruction, followed by an overview of sub-cache and cache behavior.

Road (Read Only) [1d.ro] Road 64 (read only, sub cache memory)
[Ld64.ro.sc] The program continues the pattern of reading data. The least amount of work is done to get the data. If the containing subblock is subcached, it is used directly. If the local cache has no subpages, the local cache gets a copy. The local cache supplies sub-blocks to the sub-cache in an exclusive or read-only state, as appropriate.

Load (exclusive) [ld.ex] Load 64 (exclusive, sub-cached) [ld64.ex.s
c] The program writes the sub-block with the following command,
And the exclusive state is preferred to any other state. It is used by the program when the data is expected to have little sharing, or when a series of writes is occurring. This reduces the number of interconnect messages required before the CEU can modify the data. A specific example of the use of load (exclusive) is per program data such as a stack. Generally, there is no read-only copy of such data. Because the only copy is in use by the program. However, if the program moves from one processor to another, the local cache of the new processor is not been lifting the copy, local cache of old processor continues to hold the subpage in exclusive ownership state. If the program uses a (read only) load, the local cache gets the subpage in read-only state (unless the subpage is in the atomic or transient atomic state in which it was obtained). Subsequent stores require the cache to make another interconnect request (to obtain exclusive ownership state) before any CEU data changes can occur. As with ld.ro, a minimal amount of work is done to get the data. If the sub-blocks if present already in the subcache, it is used directly.
If the local cache does not have a subpage, the local cache requests the subpage in exclusive ownership state. If the local cache has sub-pages, the sub-block will be CEU in read-only or exclusive state as appropriate.
Is supplied to.

Store [st] Store 64 (sub-cached) [st64. if present subcache sc] If the sub-block in an exclusive state already, the subcache state is unchanged and data is written to the subcache. The sub-cache must have the sub-block in exclusive state. When needed, the sub-cache requests the exclusive state from the local cache, and the local cache requests the exclusive ownership state from the interconnect. If if de I subscription <br/> no-flops data document included flag is set and the error is CE
U is notified and CEU raises exception without page write. Otherwise, the subcache gets the subblock in an exclusive state and the data is written to the subcache.

Load 64 (read only, non-subcached) [ld64.ro.nsc] load 64 (exclusive, non-subcached) [ld64.ex.n
sc] The programmer uses exclusive and read-only instructions according to the expected reference pattern, as described for ld. However, the number of references to sub-blocks is expected to be low, and the sub-cache must not be disturbed during the fetching of this data. If the data is in the sub-cache , it will be used directly.
If the local cache has no subpages, the local cache gets a copy. The CEU gets a copy of the data and loads the instruction register.

Store 64 (non-subcache) [s
t64. The number of references to the nsc] sub-block is expected to be low (typically 1), and the sub-cache must not be disturbed while storing this data. If the subblock is stored in the subcache in an exclusive state, the subcache state is unchanged and the data is written to the subcache. If the subpage is read-only and stored in the subcache, the subpage is immediately invalidated. The CPU supplies the data to the local cache. If the cache does not have a subpage in exclusive ownership state, request it from the interconnect. When no de I descriptor write flag is set, an error is reported to the CEU, CEU generates an exception of no page write. Otherwise, the CEU data is written directly to the cache subpage.

Instruction Fetch Instruction fetch always pulls out a subpage that identifies the read-only state.

Subpage Atomic State Instructions Subpage atomic state instructions are the program interface to the get and release operations described above. These instructions exist in several forms to allow precise coordination of parallel programs.

Subpage acquisition [gsp.nwt] Subpage acquisition / waiting [gsp.wt] Subpage acquisition requests that the subpage should be set in the atomic state. For both forms of the get subpage instruction, if the subcache does not exist atomically in any cache, the local cache acquires it atomically. gsp. In the case of nwt, @ MEM status code indicates the success or failure of the attempt, the instruction, the trap option is present in the instruction, and a sub-page are trapped without changing the @MEM if atomic in already. gsp. The wt instruction causes the cache to stall the CEU until the subpage can be obtained atomically. This is the case if the atomic state must be obtained before the program can proceed.
Reduce the amount of interconnect traffic. If the subpage is already atomic in any cache (including local cache), the instruction waits until the subpage is freed. The local cache then gets the subpage atomically. The @MEM state is constantly changed to indicate success.

Subpage Release [rsp] Subpage release is used to remove a subpage from atomic state. If the subpage is not in the local cache, it is first requested via the interconnect. Once the local cache has exclusive ownership, rsp proceeds. If the subpage is not in atomic state, releasing the subpage does not change the subpage state. In this situation, the CRU will be trapped if a trap modifier is present for this instruction. If the subpage is in atomic state, it can be turned into exclusive state. If in sub-page transient atomic state, it is turned into an exclusive state and driven to the interconnect so that the waiting cell can get the atomic state.

Other Sub-Page Instructions Sub-Page Post Store [pstsp] The Sub-Page Post Store causes the program to drive a read-only copy of the sub-page to the interconnect. All the hands of the cache one di de I descriptor for the page, take a copy of the data. This instruction can be used to send data as part of completing an operation, but it makes a read request to the interconnect when some other cache needs to use the data. Reduce the likelihood of having to do it.

Subpage Prefetch [pcsp] Subpage prefetch requires that a copy of a subpage be retrieved in the local cache in a particular state. The instruction may require a read only or exclusive state. Subsequent references to the subpage will block until the subpage prefetch is complete.

"Manual" Control of the Memory System As mentioned above, the memory system is designed to direct the virtual memory system with automatic data sharing and LRU maintenance. However, software may take explicit control of the memory system for a particular application.

In normal operation, all processors are SV
A space is shared, data is command (and control operation)
Automatically move from cache to cache in response to. Software may dedicate some or all of the memory on the cache to its local non-shared use. This type of system partitions the SVA space between caches,
Explicit control operations must be used to move such data from cache to cache. System software, de I descriptor to each de I descriptor.
By setting a hold, you can prevent the cache from moving or destroying a page to make room for another page. The system software can then handle the exception or perform explicit destruction, as required to multiplex each cache memory.

In the automatic mode, the memory system can be configured as a shared memory multiprocessor.
When various automatic features are disabled, the memory system can be configured to emulate a more loosely coupled message-oriented architecture. Messages can be passed by reference to a special SVA range. Manual control of the memory system can be used to more closely implement a particular memory model.

Memory System Control Instructions Control operations allow the processor, eg 40A, to directly manipulate the memory system. There are two classes of control commands. That is, data movement and page state control. Data movement control instructions move pages and subpages of data in a system from cache to cache in a hierarchy. Page state control instructions operate on Pejide I descriptor.

The CEU instruction results in a cache instruction that is executed synchronously or asynchronously depending on the instruction. CE
U-cache instructions, while in progress, occupy one entry in the cache PRT (hardware table). PRT
Since has 4 entries, a maximum of 4 cache instructions can be executed in parallel. Most CEU instructions receive an allocation of one PRT entry that remains in use and provides a synchronous aspect until the request is satisfied. For example, because load / store instructions are executed synchronously, certain software controlled exceptions (such as lost pages and unwritable pages) can be resolved in a predictable manner.
pcsp (cache subpage prefetch) and p
The stsp (subpage post-store) instruction operates asynchronously, as described in the section below.

Synchronous errors usually result in the CEU executing a trap sequence.

Asynchronous errors can come from real hardware or can be caused by requests from some other cache. Errors of this kind are reported by memory system interrupts.

Prefetch Instructions Prefetch instructions require that a copy of a subpage be retrieved on a local cache in a particular state. pcsp prefetches subpages. Cache, when this instruction is detected, assign one PRT entry. If sub-pages long present in already,
PRT entry is released, PCSP is completed. If not, the cache makes a request and then directs the CEU to complete the instruction. This goes asynchronously. When the message returns as a request or response, the cache accepts the data (if any) and releases the PRT entry. There is no indication to the CEU that the data has arrived.

[0195] subpage Post - store instruction pstsp instruction is circulated on the copy interconnections subpages, read only copy of the cache sub-page with a de office <br/> descriptor for it by including page Request that you be able to get. pstsp refers to a subblock within a subpage. If a subblock is stored exclusively in the subcache and is modified in the subcache,
The CEU requests a post store operation from the local cache. Otherwise, the pstsp instruction has no effect. Cache allocates PRT entry and CEU
Request subpage data from. The cache then presents the post-store message to the interconnect,
To release the PRT entry, to indicate the completion of the instruction to the CEU. CEU proceeds asynchronously. The message is
When it returns to the issuing cache, it is discarded.

The subpage fetch instruction mfsva instruction causes the system software to fetch the subpage in a read-only or exclusive-ownership state and specify the SVA position of the subpage. This allows the software to use the DS as required by pcsp.
You can save the effort of setting TT conversion.

Sub-cached sub-page flush order
The mflsp command specifies that the specified subpage is the local CEU.
Causes the cache to guarantee that no sub-cache is stored in. If invalid subpage - to be in de I subscription <br/> descriptor state or invalid state, de I descriptor are not assigned, subpage is not requested via the interconnect.

The subpage rejoin instruction mrcsp instruction allows system software to reduce the number of active descriptors for a page by moving ownership to another cache. Unlike the cache background recombination activity, this instruction is not controlled by the cache configuration parameter.

Page state control instructions operate on individual pages of SVA space.

[0200] descriptor anchor instruction mpsa instruction provides a de I descriptor in which is anchored to the local cache for the SVA page. If if de I <br/> descriptor was present already before mpsa, its anchor flag is set. If not, the cache distributes de I descriptor, then sets the anchor flag. Page state control operations, de I descriptors anchored against SVA page requests that should be present on the local cache.

[0201] descriptor writing instruction mpdw instruction, creating an SVA page, and destruction, and is used to change the de I descriptor flags of existing SVA page. mpdw is the system software
Use psa, first requires to obtain <br/> de I descriptor moored for the page. The following discussion is tethered de I descriptor is assumed to be present on the local cache.

[0202] Following the creation mpsa of SVA page, but de I descriptor is present, all of the pages are in a disabled state. The system software executes mpdw which specifies that all subpage states should be set exclusively. This causes a message to be sent to the interconnect, thereby allowing the members of the ring concerned to be aware of the creation of the page.

Although there is an SVA page here, its data value is not limited. The software must use store instructions or I / O to initialize the page before letting the user reference the page. For this reason, software normally creates pages at SVA locations that are inaccessible to user programs,
The page data is initialized and then the address of the SVA page is changed as will be explained later. Page by executing the mpdw instruction to clear the anchor is released for general use.

After SVA page corruption mpsa, system software must get all subpages in an exclusive state. This is done using the mfsva instruction. The software then specifies that all subpages should be changed to the invalid state.
Execute the pdw instruction. This instruction sends a message to the interconnect so that members of the ring concerned are aware of page corruption. The SVA page is destroyed by this action. Software clear anchor
Release for re-use to Disk <br/> descriptor by performing a second mpdw which A.

Change Descriptor Field The mpdw instruction is used to change various fields of the local descriptor. This change, atomic changes, without writing, Atomic No and set the holding field or cleared to obtain, and may <br/> clear the anchor field. mpdw can also tag may thus vary the SVA space address associated with de I subscription <br/> descriptor.
(The descriptor's index forms part of the SVA, so the new tag is by definition in the same cache set.)

[0206] In order to ensure memory system consistency, system software, when you change the field or tag of de I descriptor, unexpected must comply with specific rules. mpdw is a descriptor . Anchor requests that should be set (although the instruction itself may sometimes result in a descriptor. Anchor clear). The various sequences require that all subpages be in exclusive cache with exclusive ownership.
This is, to set the de I descriptor anchor for each sub-page, mfsva. It is accomplished by cell Tsu door the ex. The various sequences require that all subpages are not subcached in the local cache. This is m for each sub-page that may be sub-cached in the local CPU.
This is accomplished by running flsp. (Mfs
va. Executing ex ensures that the subpage is not subcached by the CEU of any other cell. )

The following list gives valid constraints for each flag and tag.

[0208] (1) anchor is set, it is click <br/> Li A as part of the (usually) any de I descriptor tag or flag changes. This is can stay in any duration set state is soon clear as possible in a shared memory system.

[0209] (2) held, on a particular cell without restriction <br/> de I descriptor the possible changes in order to release the holding or holding.

[0210] (3) changing the no atomic changes and atomically, all the sub-pages that require it should be in exclusive ownership state in the local cache.

[0211] (4) clear the changes, to set the no write require that all subpages is not subcached in the local cache. This ensures that the local sub-cache does not have any sub-blocks in an exclusive state, nor does it have any modified sub-blocks. When changing the no-be-modify-write, the system software may determine whether to be recognized by all the hands of the cells to which the change refers to the page. Global changes require that all subpages be in exclusive ownership of the local cache. System software, usually, when clear the target change flag, form a wide-area changes. To make local changes, no subpage needs to be in exclusive ownership state. However, this results in a delay in recognizing the new state.

(5) In order to change the number of SVA pages by changing the tag, it is necessary that all subpages are in the exclusive ownership state and are not stored in the subcache in the executing cell.

Modifying a single bit field requires a single m
pdw. It is performed by the desc command. This instruction contains the new value of the changed flag, the old value for the other flags, and the tag. System software, unless have special reason to keep the page anchored state, to clear the anchor flag.

Changing the number of SVA pages in a descriptor is logically the same as destroying the old page and then creating a new page that may have the same data. The sequence is as follows:

[0215] (1) anchored to de I subscription <br/> descriptor against the old SVA page, obtaining the subpage in exclusive ownership state. If any subpage has an atomic or transient-atomic state, it will be retrieved in the executing cell. Once all sub-pages have been acquired, the old SV
Accessing A results in a missing page fault.

(2) Determination of atomic state of each subpage. This is for each subpage gsp. Most quickly accomplished by running nwt and testing the resulting @MEM indicator. The gsp. cells running wt to already leads to ultimately timer interruption, and gsp. When wt is restarted, it results in lost page faults.

(3) In order to change all subpages to the invalid state, mpdw. Use alli. This command is for the local CCU
The local CIU that the SVA page is being destroyed.
And RRC (if required). Even if all sub-pages are changed to invalid state, the data will not
Stay in.

(4) In order to change the tag and set flag to a desired state, mpdw. Use desc. The new anchor flag must be set.

(5) In order to change all subpages to an exclusive state, mpdw. Use allx. This command is a local CC
In U, local CI that SVA page is being created
Notify U and RRC (if required). The old data is now considered to be in an exclusive state.

(6) Restoring the saved atomic state of each subpage. For each subpage in the saved state, gsp. Issue nwt.

[0221] (7) for the anchor flag cleared m
pdw. Use desc.

Descriptor. Uses without writing include: That is, prevention of inadvertent modification of specific data, support of copy-on-write / copy-on-access protocol, and debugger monitoring point. In the first case, no write is set and remains set. In the second case, when the program tries to change the page, system software, make a copy of the pages available to other users, then can respond by clear the no writing. Software can make this change locally or globally. In the former case, no write failures will continue to occur on other cells referencing the page. Finally, the debugger watchpoint is intended to detect changes in specific areas of the context address space, for example to find where global variables are being destroyed. The system software does this by setting no writes for the page and trapping each modification attempt.
To changes in other than the monitoring range, software, anchored to the page, the no write clear and A, change the data, to release data from the sub-cache sets without writing, and proceeds. Watchpoint support can be implemented by making global changes to no writes.

Page Local Discovery-LRU Lookup
The instruction mfpl instruction searches a particular set in the LRU space of the cache for descriptors that meet the particular set of criteria. The search begins with the descriptor at LRU position 15 and progresses up until the criteria is met.

Instruction and data system address space
Split To ensure correct operation, the cache must know when subpages are also in the CEU. Thus, cache, when the interconnect (or even the local CEU) is from request to request subpage state change may require the sub-cache invalidation.

As part of this mechanism, the CEU communicates with the cache when changing the subcache storage state. However, the cache does not maintain per sub-cache information. As a result, the same subpage of SVA space should not appear in both subcaches at the same time. In general, this results in system software in which the same SVA area cannot be used as both instruction and data. Self-modifying programs, i.e. programs whose code and data are part of the same context address segment, are not supported.

System software must take special care in the following cases: (1) When changing an instruction, such as when inserting or removing a breakpoint. (2) Reading instructions as part of a trap analysis or program decomposition operation by a debugger. (3) Read the page from the I / O device that will be the instruction page.

In order to read the instruction as data, the system software must perform the following operations. (1) Configure a DSTT entry to write SVA. (2) Ensure that the subpage is not in the instruction subcache (using mflsp). (3) Read the command as data (1d64.nsc
use). (4) Invalidate the DSTT entry. To write the instruction subpage as data, the system software must do the following: (1) Configure a DSTT entry to write SVA. (2) Ensure that the subpage does not exist in any instruction subcache (mpsa to moor pages)
Using, Mfsva to invalidate subpages other caches and sub cache of all hands. use ex). (3) Extract the sub-block to be included (1d64.n
sc)). (4) Change the sub-block and write the instruction as data (st64.nsc). (5) Release page mooring. (6) Invalidate the DSTT entry.

The instruction page is normally pure and the system S
As part of VA space management
/ O device does not need to be written. Before writing the instruction page, the system software must: (1) mfsva.ex to also ensure that does not exist in the subpage which subcache (use the mpsa to anchor the page, invalidating the subpage other caches and subcaches all hands use).
If this cell never executes this page, this step is not needed. (2) Perform I / O. (3) Guarantee the page does not exist in the data cache (using mflsp). If this cell never executes this instruction page, then this step is not required. (4) Release page moorings.

SVA the instruction page from the I / O device
When reading into space, the system software must do the following: (1) Creating a page (using mpsa to allocate and moor pages, mp to complete page creation)
dw). (2) Perform I / O. (3) Ensure that the page is not in the data subcache (using mflsp). If this cell never executes this instruction page, this step is not needed. (4) a descriptor change flag clear and A, other descriptor attributes and descriptors, such as subpage atomic state. Set No write. (5) Release the mooring of the page.

RRC Organization and Data Structures The preferred RRCs constructed in accordance with the present invention are organized into two independent banks, each connected to a single sub-ring. Banks of 1-4 RRCs are interleaved based on the low order page address and the degree of interleaved configuration. Interleaved (in data Ribin <br/> grayed) is configured in the system configuration in Sutate I click. The RRC connected to Ring: 0 is referred to as RRC: 0 and the RRC connected to Ring: 1 is referred to as RRC: 1.

In the case of the single ring: 1 form, both RRC: 0 and RRC: 1 are interleaved based on SVA [14]. The Figure 19 A, depicts one such preferred embodiment.

In the case of a dual ring: 1 type (TwoRing: 1/1) having one RRC: 0, a single RRC: 0 is SVA [1
4] interleaved, 2 rings: 1RR
C is interleaved based on SVA [15:14]. FIG. 19B depicts one preferred such configuration.

Duplex ring with two RRC: 0:
In the case of form (Two Ring: 1/2), RRC: 0 is SVA [15:14]
Based on the two RRC: 1 SV
Interleaf based on A [15:14]. FIG. 19C depicts one such preferred form.

In case of 4-ring: 1 form, two RRCs:
0 is interleaved based on SVA [15:14] and four rings: 1 RRC are interleaved based on SVA [15:14]. FIG. 19D depicts such a preferred form.

Multiple RRC: 0 is TwoRing: 1/2 and FourR
For the ing: 1 form it is on a single ring: 0. Multiple RRC: 1 is Tworing: 1/1, TwoRing1 / 2 and fourRing: 1
In the case of a form, it is constructed on a separate ring: 1.

[0236] FIG. 6 shows a preferred ring routing cell pair to the present invention configured I slave the (RRC pair or RRCP). The pair comprises RRC: 0 and RRC: 1 connected by a single communication line, eg a bus, an optical transmission connection (hereinafter referred to as an inter-ring link). Downlink RR
C is the opposite RRC of the RRC pair. Same ring: 0
The RRC pair connected to is called the RRC array. When discussing cell or ring: 0, the local RRC
Or an RRC pair is an RRC or RRC pair that co-exists on the same ring: 0. Remote RRC or RRC pair are those of the whole hand RRC not localized.

Each RRC bank consists of three sections: a routing interconnect, a routing directory and an inter-ring buffer. The Routing Interconnect implements the RRC part of the Ring Interconnect and provides an interface to the Routing Directory,
The insert and extract buffers interface the ring to the RRC internal routing bus. The routing bus consists of two 32-bit paths. Packets from the extract buffer are transferred to the inter-ring drum buffer via the routing bus to the link. The packet is transferred from the IBU to the RI insertion buffer via the routing bus to the ring. The IBU also transfers the packet to the inter-ring drum buffer for error recording via the routing bus to the link.

[0238] Routing directory, ring RRC pair is connected: including an entry for all hands of allocated pages in the 0. The Routing Directory consists of a Routing Directory Unit (RDU) and the actual ram-based directory. The RDU controls the directory access sequence and contains routing rules. For each package, the routing interconnect is called pkt. addr, pkt. Command, and fif
Pass the o state to the routing directory. The routing directory returns instructions for updating routing commands and packet commands to the routing interconnect. The interleaved arrangement associated with the ring is RdMasterC.
Identified by the onfig position.

The inter-ring buffer buffers incoming packets from the ring interconnect extraction buffer and interfaces with the inter-ring link. The inter-ring buffer consists of an inter-ring buffer unit (IBU), an inter-ring drum buffer (IDB) and an inter-ring link interface. IDB, the packet will be transmitted to downlink RRC via the link between rings, and until confirmation is attributed to the buffering action into packets of all the hands from Routing interconnects. The IDB also contains the RRC error queue. The IBU buffers and formats outgoing inter-ring link packets and provides error checking and buffering for incoming inter-ring link packets.
Control the DB. The inter-ring link interface is
Converts 32-bit wide IBU outgoing and incoming packets to physical link format and vice versa.

RRC banks can be configured to connect to separate inter-ring links or to share a single inter-ring link. In a dual inter-ring link configuration, the inter-ring links are completely independent. In a single inter-ring link configuration, both RRC banks share a single inter-ring link. The inter-ring link is connected to the IBU of the RRC bank. RRC is arbitrated to share the inter-ring link. The IBU interleaf structure and packet address determine which bank an incoming packet is routed to.

RRC is consistent with the UCS system failure detection strategy. Bus of all hands, are parity and EEC protection. Parity or ECC is passed through the chip rather than regenerated when possible. Dram and large sram data structure of the entire hand, are EEC protected. Single bit soft or hard errors are corrected without causing a fatal system error (requiring a system reboot) and possibly in a software transparent manner. Data links that predict high error rates (inter-ring links) perform software transparent frame level operations. Frame level operation ensures error free transmission of frames, including retransmissions where errors are detected.

Routing Interconnection Configuration Each Routing Interconnection (RI) interfaces the RRC bank to the sub-ring, RRC internal routing bus and routing directory. The RI is implemented as two Routing Interconnect Units (RIUs). A preferred RIU constructed in accordance with the present invention is illustrated in FIG. Each RIU is interfaced to half a 16-byte wide ring and half an 8-byte wide routing bus. The two RIUs pass the address and command fields of the packet to the routing directory, which returns the packet routing command, new packet command and error status.

Each packet is a 1 of 10 while on the ring.
It consists of 6-byte double words (20 words). Each long packet (including data) is 9 in each RIU extraction buffer.
In words, 10 words are stored in the insert buffer. Each short packet (no data) is stored one word in each RIU extract buffer and two words in the insert buffer. RIU0 and RIU1 interface with ring data [63: 0] and ring data [127: 64], respectively. The RIU0 and RIU1 extract buffers are interfaced to the routed bus to the link [15: 0] and the routed bus to the link [31:16], respectively. R
The IU1 and RIU1 insertion buffers are routed to the ring [15: 0] and routed to the ring [31:16]
Interfaced to each.

The ring-in-ring-out part of the RIU consists of a 6-stage shift register. Packets can be copied or extracted from the shift register into the extraction buffer, and packets from the insertion buffer can be inserted into the shift register. The insert and extract buffers are managed by FIFO rules.

RI Operation Ring interconnects are routed from two sources to two destinations. The sources are the preceding ring interconnect and the insert buffer. Packets from the preceding ring interconnect are called ring packets,
Packets from the insert buffer are called insert packets. The destination is the next ring interconnect and extract buffer. The packet that exits the next ring interconnect is called a ring packet and the packet that exits the extraction buffer is called an extraction packet. The insertion packet is supplied to the insertion buffer, and the extraction packet is extracted from the extraction buffer by the inter-ring buffer unit (IBU).

The address and command fields of the packet are passed through the routing directory, which returns the packet routing command. RI
Buffers all packets until a packet routing command is obtained. The assertion of the ring empty field indicates that the ring packet is empty. Non-empty packets are always processed instead of insert packets. If the ring packet is empty and the insert packet is not empty, the next insert buffer packet is processed by the ring interconnect.

The commands from the route setting directory consist of packet route setting, packet field, extract buffer, and insert buffer commands.

The packet route setting commands for all packets are path and empty. The packet path command specifies that the ring interconnect should pass all packets unchanged to the next ring interconnect. The empty packet command specifies that the ring interconnect should empty the packet by setting the ring empty signal and the packet command CmdEmpty field. C
All packets whose mdRequestId field matches the ring interconnect CellNumber field are unconditionally emptied and no routing directory instruction is required. This allows the ring empty signal to be set faster than is possible when waiting for a command from the routing directory.

Extract buffer commands are no push and push. No push command extraction buffer identifies that does not store the current packet. The push command specifies that the current ring packet should be copied to the extraction buffer.

Insert buffer commands are no-pop and pop. The no-pop command specifies that the insert buffer top packet was not inserted. The pop command specifies that the insert buffer top packet has been processed and has been popped.

The packet field command specifies which outgoing packet command field is changed.
The packet command fields that can be modified are shown in the table below. pkt. All fields except Freeze are in the pkt instruction [31: 0].

[0252]

[Table 1]

The packet is extracted by the empty packet route setting command and the push extraction buffer command. The packet is copied by the path packet route setting instruction and the push extraction buffer instruction.

Extraction Buffer Management The extraction buffer holds a combination of short (2 words) and long (18 words) packets. This is equal to 512 words. It has a maximum of 18 for long and 256 for short.
, Or a combination of long / short packets. The extraction buffer management flow control algorithm is required because the peak sub-ring packet rate exceeds the rate at which IBM can empty the extraction algorithm. The control algorithm is based on the principle of maximizing forward progression in the context of reducing system load. Both of these goals are achieved by prioritizing the allocation of system resources towards completion requirements in the process. Because completing one request simultaneously reduces the load on the system and achieves the most efficient forward progression. The control algorithm is
Based on packet classification and the use of ring flow control mechanisms.

The empty space in the ring extraction buffer is 2
Two configurable pointers, ExtractAllThreshold and E
It is divided into four regions by xtractRspThreshold. The current region is selected by the number of empty words remaining in the extraction buffer. The area from the sky to the full state is ExtBuf: Al
l, ExtBuf.NonOpt, ExtBuf: PendReq and ExtBuf: Full. The extract buffer state is maintained on a per RRC bank basis. The boundaries of these regions are software configurable. Ring packets are grouped into three mutually exclusive categories for extraction buffer management.

(1) Optional packet The following remotely emitted packets are coherency-of
It is optional from the point of view. That is, -Rejoin with Pendle: None and SpState: {Inv | Ro}. -Local ring: Read-only copy of subpage in 0
-Duplicate.data regarding nothing. -Local ring: Read-only copy of subpage in 0
-No sva read ro.data. (2) Non-optional packet Based on SpState to complete a remotely issued request.
Must be extracted first. Optional packet
Not included. (3) Locally originated packet or RRC pende
Request (RRC: 0Pend {1,2,3}, RRC: 1Pend {1,2,3,4,5})
Satisfy the packet.

These packet categories are processed in the four extraction buffer areas as follows: That is,

[Table 2]

In the ExtBuf: All area, Enpty and Extra
Between ctAllThreshold, all packets normally copied to the extraction buffer are copied. Extbuf: In NonOpt region, between ExtractAllThreshold and ExtractRspThreshold, non optional packets and response packets of all would be copied manually is copied in the normal extraction buffer. All optional packets are passed. The extraction buffer and RRC discard it as more optional loads arrive. In the ExtBuf: PendReg area,
Between ExtractRspThreshold and Full, only packets for RRC pending request are accepted. Optional packets and non-optional packets of all the hand (including the new incoming requests) is responder busy chromatography (rsp b
usy) or in Bali Day Tha busy over (inv It is busy over by the assertion of the busy). The RRC gives up both the optional load and the new request to release the overload condition.

When the extraction buffer is full, the RIU
It will pass optional packets, assert responder busy over or Invaritar Day coater busy over to non optional request. If If the packet is a reply to a request issued by the RRC, packet remains on the ring, Pkt.Cmd.Timestamp field is cleared. R
Since only packets for RC pending requests will be accepted in the third region, the package will be copied on subsequent moves around the ring. The fact that a packet is in its second time of the rotation developing is transparent to the cells of the entire hand in the ring (apparent). If the packet is not a response to a request issued by this RRC, except for packets that satisfy the pending request (snarf), the pending request state is converted to Pend4: Reissue state.

Rooting Directory Rooting Directory Organization The rooting directory consists of a rooting directory unit (RDU) and the actual ram- based directory. The RUD controls the directory access sequence and contains routing rules. The preferred implementation of the rules hardcodes the SPA rules and allows the SVA rules to be loaded during RRC initialization.

FIG. 8 depicts the organization of the routing directory which is the preferred embodiment of the present invention. Routing Directory, ring RRC pair is connected: 0 all hands of assigned including <br/> No entries for pages in. For SVA pages allocated above a single local cache, a single routing directory entry is required. The root setting directory is 128
3 level btree for each of the subdirectories
Be organized as. The subdirectory is SVA [20:
As specified by the 14], including an entry for the page's assigned all hands in a single local cache set. A subdirectory may describe up to 2 ¹⁰ pages supporting up to 64 ring: 0 cells. The local sub-cache of each cell contains 128 sets, each containing 16 pages. The Ring: 0 subring is configured to interleave the transfer of subpages based on the cache set lsb, SVA [14]. Thus, half of the Routing Directory, each ring: 0 subring (Ring: 0 _A, Ring: 0 _B) and may be related.

[0262] L1 entry (Entry) is, Ru is indexed by SVA [39:25]. L1 entries are statically assigned to each 2 ⁴ pages in a set. Each valid L1 entry specifies a pointer to a dynamically allocated L2 entry. Up to 2 ¹⁰ L1 entries are available. Unassigned L2 entries are organized into a single linked free list. Index _i L2Free List Pointer
Points to the first L2 free list entry. Each L2Enry in the free list contains a pointer to the next L2 free list entry. The last L2 free list entry contains a zero pointer. Each L2 entry contains 16 sub-entries. Each valid L2 subentry contains a pointer to a dynamically allocated L3 entry. Up to 2 ¹⁰ L3 entries are available. Each L
Three entries describe a single page. Unallocated l3
The entries are organized into a single linked free list. Index _i L3Free List Pointer points to the first L3 free list entry. Each L3 entry in the free list contains a pointer against a next L3 free list entry. The end of the L3 free list entry, including a zero pointer. Each L3 entry consists of 16 L3 subentries, each of which describes a subpage state for 8 subpages.

Structure of Route Setting Directory FIG. 9 is a diagram showing the structure of the route setting directory which is the preferred embodiment of the present invention. The routing directory is L1, L2 and L3 for each sub-ring.
Physically organized into tables. L1 table is S
VA [20:14] (set number) and SVA [39:25] (index1)
Are addressed by concatenation of. SVA [39: 25:, 20: 14]
Is called index1. Each L2 entry consists of an L2 subentry and an L2V. L2 subentry is SV
A [20:14], L1Table [indexl] .12ptr and SVA [24:21] (i
Addressed by the concatenation of ndex2). L2V is SVA
Addressed by concatenation of [20:14] and L1Table [index1] .12ptr. The L2V field is interpreted as a level 2 subentry valid field if the L2 entry is valid, and as a level 2 next free list entry pointer if the L2 entry is invalid.
Each L3 entry consists of an L3 subentry and an L3 valid. L3 subentry is SVA [20:14], L2Table.12
Addressed by the concatenation of sub [index2] .ptr and SVA [13:10] (index3). The individual subpage status is SVA [2
0:14], L2Entry.12sub [index2] .ptr, SVA [13:10] and SVA [9: 7] (indexsp) are addressed. L
The 3V field is interpreted as a level 3 subentry valid field if the L3 entry is valid,
If the L2 entry is invalid, it is interpreted as a level next free list entry pointer.

A C-like description of the table structure of one bank is shown below.

[Table 3]

[Table 4]

FIG. 10 depicts the format of the routing directory entry in a preferred ring routing cell constructed in accordance with the present invention.

L1Entry L1Entry is an L2 entry pointer (12ptr), an L2 pointer valid bit (11 valid) and an L1 check bit (11
Check) is included. 12ptr is used to address the L2 entry as described below. L1Valid set indicates valid 12ptr, clear is invalid 12pt
Indicate r. The L1 check consists of a 5 bit single bit collect and a double bit detect ECC code.

L2Subentry L2Subentry is L3 entry pointer (13ptr) and L2
Includes check bit (11 checks). 13ptr is used to address the L3 entry as described above. The L2 check consists of a 5 bit single bit collect and a double bit detect EEC code.

L2VEntry L2VEntry has two formats, one of which is L
It is for use as a 2V entry (L2V.Valid) and is the second if it is part of the L2 free list (L2V.Invalid).
belongs to. L2V.valid is 16 valid bits and 5E
It consists of an array of CC check bits. L2V.valid.12va
lid [n] set indicates a valid L2 subentry [n], and cl
ear indicates an invalid L2 subentry [n].

L2V.invalid consists of a next free L2 entry, an unused field and an ECC check field. L2V.invalid.NextFree is used to address the next-strike L2 free list entry. Next
Freehighorderbitclear points to a zero pointer.

L3Subentry L3Subentry is an 8-subpage state field pointer (L3Subentry.SpState) and a check bit (L3Subentr).
y.check) is included. L3Subentry.check consists of 8-bit single-bit collect and double-bit detect EEC code.

L3VEntry L3VEntry has two formats, the first is L3VEntry.
For use as VEntry (L3V.Valid), L3freelis
If it is a part of t (L3V.Invalid), it is the second one. L3V.va
The lid consists of an array of 16-bit generated OwnerLimit, PagePend and ecc check fields. L3V.va
lid.13valid [n] corresponds to L3Subentry [n], and L3V. It is a function of generation.

[0272]

[Table 5]

[0273] L3V.valid.OwnerLimit is local ownership of subpages in the page ring: 0 to to limit, is set or cleared by software. When it is clear that any Ring: 0 also specifies that not own the subpages. When set, specifies that only cells in the local ring: 0 may own subpages, non-exclusive, atomic or transient atomic states. Remote Ring: 0 can receive read-only copy. Page faults, when OwnerLimit is set , R
The RC notifies the ownership request. The rejoin request is Ow
When nerLimit is set , local ring: No propagation beyond 0. During RRC descriptor allocation is clear.

When L3V.valid.PagePend is set,
The RRC page indicates that the deallocation sequence is in progress.

L3V.invalid is composed of a next free L3 entry, an unused field and an ECC check field.

L2invalid.Nextfree is used to address the next L3 free list entry. NextFree
highorderbitclear points to a zero pointer.

Routing Directory There are eight Routing Cell (RRC) directory operations that are operational. These operations occur within a single packet time. Single operation in packet type for all hands to be executed. Level 1 miss lookup operation Level 2 miss lookup operation Normal lookup operation Level 3 subentry deallocation Level 2 subentry deallocation SPA access playback or refresh

System Virtual Address (SVA) Looka
Tsu look-up operation of the flop operation all hand, the selection of sub-directory,
Level 1 table lookup, Level 2 table lookup, Level 3 table lookup and Level 3
The process proceeds through five steps of correcting the subpage state. Level 2 and Level 3 entries will need allocations to complete the lookup operation.

Normal Lookup Operation Normal lookup operation occurs when the level 1 table indicates that the level 2 entry is valid and the level 2 table indicates that the level 3 entry is valid.

L1EntryAddr, L2EntryAddr and L3EntryAdd
r is calculated as described in the Routing Directory Structure section. L1 Entry Addr → 11v being set
alid indicates that the level 2 pointer is valid.
The concatenation of SVA [20:14] and L1Entry [index1] .ptr identifies the L2Entry because the subdirectory is implied or implied by the 10-bit pointer. The level 2 subentry (L2EntryAdrr → 12sub [index2]) is indexed or indexed by SVA [24:21]. Is being set
L2Entry.12v [index2] is the level 3 pointer (L2EntryAdrr
→ Indicate that 12sub [index2] .ptr) is valid. S
The concatenation of VA [20:14] and L2Entry [index2] .ptr specifies L3Entry because the subdirectory is implied by the 10-bit pointer. Level 3 subentry (L3Ent
ryAdrr → 13sub [index3]) is indexed by SVA [13:10]. Each level 3 sub-entry contains sub-page state for 8 sub-pages and contains SVA
Indexed by [9: 7]. Index3 points to SVA [13:10], and Indexes points to SVA [9: 7]. L3En
try.valid [index3] and L3Entry.created specify whether the corresponding subentry is valid, and if invalid, the default subpage state.

Sub-page status is L3EntryAdrr → 13s
Presented to the RoutingDirectoryRules that generate new subpage states to be stored in ub [index3] .SpState [indexes].

Level 1 Miss Operation A Level 1 miss lookup operation occurs when a Level 1 table lookup indicates that the Level 2 entry pointer is invalid. The level 1 miss lookup operation allocates level 2 and level 3 entries to allow the routing directory to describe the page and subpage state of the requesting system virtual address. Level 1 miss is displayed by SVA [39:25] Address band page ring: 0 (Ring: 0)
Caused by the first reference in. If the packet command is ciu ex If summary, the request is part of a page generation operation.

The level 2 free list head is pointed to by the level 2 free list pointer. The free list head L2Entry.NextFree specifies the pointer to the next free level 2 entry. The level 2 free list head is assigned as a level 2 entry by writing LevelEntry.pointer with the level 2 free list pointer and setting L1Entry.valid. The level 2 free list pointer is updated by writing the level 2 free list pointer with the level 2 free list head, L2Entry.NextFree. L2Entry.12v.12valid [S
VA [24:21]] is set and the remaining valid bits are cleared.

[0284]

[Table 6]

[0285] If L2Free List Pointer is null (blank)
If so, a routing directory fault is signaled. Corresponding ring: 0 only if a route setting cell 2 allocates ¹⁰ or more unique page fails to correctly assign or unassign a level 2 entries within a single subdirectory, the error occurs. Two
Own pages that ¹⁰ corresponds to 64 processor cells (one per cell 2 ¹⁰ pages / 2 page ^4).

Level 2 Miss Operation A Level 2 miss lookup operation occurs when a Level 2 table lookup indicates that the Level 2 entry pointer is invalid. The level 2 miss lookup operation allocates a level 3 entry to allow the routing directory to describe the page and subpage state of the requesting system virtual address. A Level 2 miss is caused by the first reference in the Ring: 0 to the page in the address band indicated by SVA [39:21]. If the packet command is ciu ex summa
If ry, the request is part of a page generation operation.

The level 3 free list head is pointed to by the level 3 free list pointer. The free list head L3Entry.NextFree specifies the pointer to the next free level 2 entry. The level 3 freelist head is assigned by writing Level2Entry.12sub [index2] .13ptr with the level 3 freelist pointer. Level 3 Freelist Pointer Level 3 Freelist Point to Level Freelist Head L3Entry.Ne
Updated by writing with xtFree. L3Entry.13v.
13valid [SVA [13:10]] is set and the remaining valid bits are cleared. L3Entry.13sub ([SVA [13:10])]. Sp
State [SVA [9: 7]] is set and appropriate subpage state and the remaining 7 subs state is set to disabled. If the packet command is ciu ex If summary, L3Entry.13v.Va
lid.created is set.

[0288]

[Table 7]

[0289]System virtual address allocation release operation The system virtual address deallocation operation uses the packet command ci.
u inv sum alloc, ciu pure inv sum alloc and RrcDeall
Started by ocDesc.allAll allocation release operations are
Directory selection, level 1 table lookup
Level, level 2 table lookup, level 2 entry
To update and deallocate level 3 entries
It progresses through two stages. ifallLevel 2 Sube
If the entry is invalid, sequential level 2 entries are also
The patch is released.

[0290] If the level 2 sub-entry level 3 entry deallocation operation level 3 entry deallocation after all hands is invalid, the level 3 entry deallocation operation is performed. In the following operation, the packet command is ciu
inv If summary is executed.

[0291]

[Table 8]

In the following operation, the packet command is ciu wr des
cripto.data or ciu inv sum If allocate.data is executed.

[Table 9]

[0293] If an invalid level 2 Entry Level 2 subentry deallocation after all hand deallocation operation level 3 entry, level 2 entry deallocation operation is performed.

[0294]

[Table 10]

SPA operation SPA operation is read request and spa write request
Is started by two packet instructions. SpaDirec
There are two types of spa operations, t and SpaLookup.

The SpaDirect operation performs a directory-based spa read or write to the spa location specified in the packet address. Spa write
Request data is supplied to Pkt.DataWord0. Spa read
The request response data is supplied to Pkt.DataWord0.

The SpaLookup operation is used to emulate and deallocate an RDU lookup operation under controlled conditions from the spa address space. Pk
t.Address position offset and ctl $ RiuLookupCommand.Su
bAddr forms an emulated SVA and ctl
The remainder of $ RiuLookupCommand forms an emulated packet command. The RDU uses emulated packet instructions and performs emulation to perform normal lookup or deallocation operations. SpaLoo
The result of kup operation is L2Free List Pointer, L3Free List
Checked by accessing Pointer and SpaDirect locations.

Refresh Operation Refresh is accomplished in a distributed fashion by each RRC individually refresh controlling it. A single refresh requires one packet processing time. Thus, refresh can only be performed during empty or frozen packets. because,
These packets do not require any processing. RRC has to refresh
The use of empty and that frozen and hidden packet time of all hand and tries to refresh in the background.

The three dynamic RAM based data structures L1Table, L2Table and L3SubpageState require refreshing. During the refresh operation
One of the 512 columns is refreshed with its respective data structure.

ECC Correcting Operation When a routing cell detects a correctable ECC error,
Additional packet time is needed to modify the data structure. The freeze ring operation is used to allow for multiple packet times for modification and re-lookup.

When a correctable ECC error is detected, E
Since the CC error was unknown from the beginning in packet processing, RRC initiates the Full-1 Ring Freeze method. The first packet is treated as described in the Extract Buffer Full Section section, and the remaining packets are marked frozen for one revolution. The modified data structure that has been rewritten is saved in the RDU and is used to reprocess the packet that causes the ECC error after a sequential ring turn. Only the packets that originated the routing cell are processed, the non-occurring packets are extracted by another cell in the ring. The save correct packet operation ensures that the packet emitted by the routing cell is correctly processed for a single bit (correctable) hard error.

Initialization of Routing Directory The Routing Cell Hardware does not automatically initialize the Routing Directory. All hand Level1Entries (Level 1 entry), Level2Subentries (Level 2 entries) and Level3Subentries (Level 3 entries) must be initialized to invalid. Level 2 and Level 3 free list pointer and the free list, must be initialized to indicate that the entry and subentries all hands are free.

InterRing Buffer The InterRing Buffer buffers incoming packets from the Ring Interconnect Extraction Buffer and interfaces with the InterRing Link. The inter-ring buffer also controls the management of the routing cell bank buffer. The inter-ring buffer is an inter-ring buffer unit (InterRing Bu
fferUnit, IBU), inter-ring dynamic RAM buffer (InterRing Dram Buffer, IDB), and inter-ring link interface (InterRingLink Interface).

Inter-Ring Buffer Structure Inter-Ring Buffer Unit (IBU) The IBU buffers and formats inter-ring link outgoing packets, performs error checking and buffering on inter-ring link incoming packets, and inter-ring dynamic RAM buffers. Control. Out buffer is 7
It contains 2 words and holds packets up to 4 lengths, and the ingress buffer holds 72 words for packets up to 4 lengths. The sizing operation of the incoming buffer depends on the inter-ring link rate and the absolute delay (length and switching delay). 72 word buffer is 1gb
It is directly connected to the it / sec link part and sizing is performed for a maximum length of 10 km. A short packet consumes 2 words, and a long packet consumes 18 words in the out or in buffer. The outgoing and incoming buffers are managed by the fifo method.

Dynamic RAM buffer between rings (I
DB) Dynamic RAM buffer between rings (InterRing Dram)
Buffer, IDB) are ring packet queues (RPQ, Ring Pa)
cket Queue) and routing cell error queue (RRC Erro
rQueue) is included. The configuration for an inter-ring drum cue in a preferred routing cell (RRC) constructed in accordance with the present invention is shown in FIG. 11B. Ring packet queue until the packet transmitted and confirmed to respond to the downlink routing cells through the link between the rings (acknowledge) is returned to buffer the packets of all the hands from Routing Interconnect unit. When the Extract and Out buffers are full, Extract Buffer packets are injected into the IDB until the Out buffer can accept them. Ring packet queues are organized as circular queues. The Ring Packet Queue Push Pointer (RPQ PushPointer) points to the first freeword to queue for the next packet from the Extract Buffer. The Ring Packet Queue Pop Pointer (RPQ PopPointer) points to the first word of the next packet transmitted over the inter-ring link. The PRQ commit pointer points to the first word of the oldest packet not answered by the downlink routing cell. Packets between the ring packet queue push pointer and the ring packet queue pop pointer are queued or queued for transmission through the inter-ring link. RPQ
The packet between the pop pointer and the ring packet queue commit pointer is waiting for a response from the downlink routing cell. If the ring packet queue push pointer is equal to the ring packet queue pop pointer, then there are no queue packets in the ring packet queue for transmission on the inter-ring link. If the ring packet queue pop pointer equal to the PRQ commit pointer, transmitted packets are received all with response. If the ring packet queue overflows, a fatal error is indicated by IbuMasterConfig.RPQFull.

The inter-ring dynamic RAM buffer also contains the routing cell error queue. The routing cell puts an entry in the ErrorQueue for each error detected by the routing cell. Each error queue entry consists of a packet of error (if any) and a word of error information. The Error Queue Push Pointer (EQ Push Pointer) points to the first freeword for queuing the next error entry. The error cue pop pointer points to the first word of the least recent queued error. If the error queue pop pointer is equal to the error queue push pointer, the routing cell error queue is empty. The error queue pop pointer is not modified by the routing cell, only by SPA access. IbuMasterConfig.EQFull When Routing cell error queue has overflowed, is set.
The routing cell continues normal operation and no new error is recorded.

Inter-Ring Link Interface The Inter-Ring Link Interface converts the 32-bit wide Inter-Ring Buffer Unit output packet / input packet interface to or from the physical link format.

Routing Cell Buffer Management Each routing cell pair logically manages a pair of buffers in one of the respective directions between ring: 0 and ring: 1 by the fifo method (11th B).
Figure). The routing cell bank manages 5 buffers, 3 buffers between the ring and the outgoing inter-ring links, and 2 buffers between the incoming inter-ring link and the inter-ring links. Packets copied from the ring of the uplink routing cell bank are transmitted sequentially across the inter-ring link through the extraction buffer, and optionally the ring packet buffer, and the inter-ring link output buffer. Packets are sequentially passed through the inter-ring link input buffer and the downlink routing cell bank insertion buffer. Packets traveling in the opposite direction are copied from the ring of the downlink routing cell bank and pass through the extract buffer, ring packet buffer and inter-ring link out buffer. Packets sequentially pass through the inter-ring link input buffer and the uplink routing cell bank insertion buffer.

The extract and insert buffers adjust the speed or rate between the ring and the routing bus of the routing cell. The input and output buffers provide speed regulation between the inter-ring links and the routing buses of the routing cells. The ring packet queue handles possible overflow of these buffers by allowing the maximum number of outstanding or stuck packets to be stored, and prevents deadlock between routing cell pairs. The buffer is sized for occasional use to perform buffering between the extract and inter-ring link out buffers. Furthermore Ring packet queue, when a link error is required retransmission due are also used to record the packets of all the hands to be transmitted on the link between the rings.

The preferred routing cell bank implementation provides independent paths between the extraction buffer and the inter-ring link output buffer and between the inter-ring link output buffer and the insertion buffer. The inter-ring buffer unit forwards packets from the incoming buffer to the insert buffer when the inter-ring link incoming buffer is not empty and the insert buffer is not full.

The Inter-Ring Buffer Unit transmits packets from the Extract Buffer to the Out Buffer when the Extract Buffer is not empty, the Inter Ring Drum Queue is empty, and the Inter Ring Link Out buffer is not full. If the output buffer is full, the packet is transferred from the extraction buffer to the inter-ring drum queue. The packets contained in the inter-ring drum queue are always transmitted to the output buffer before any packets from the extraction buffer.

Inter-Ring Link Packet Protocol and Format The Inter-Ring Link Packet Protocol and Format are needed to be applicable over a range of link speeds for multiple generations of UCS computers.
Protocols and formats range from 25 meters in a single room to tariffed telecommunication networks1
It should cover interconnect distances in the range of up to 000 km. The format should be compatible with emerging telecommunications network standards such as SONET. That is SD
It is largely based on a simple concept introduced from the LC protocol. In 1978, "Communication Architecture" by RJ Cypser, published by Addison-Wesley Publishing Company.
for Distributed Systems, "or 1987, Addison W
"Telecommunic" by Mischa Schwartz, published by esley.
ations Networks: Protocols, Modeling and Analysis "
Please refer to. FIG. 12 illustrates a preferred inter-ring link frame format used in a ring routing cell constructed in accordance with the present invention.

Confirmation and Retransmission The connection between the routing cell pair is fully duplex and
It consists of two unidirectional inter-ring links. Due to the uncertain reliability of the inter-ring links, very certain and unambiguous confirmation is needed at the link level. The transmitting routing cell accepts a positive or positive confirmation over the incoming inter-ring link for each packet transmitted over the outgoing inter-ring link. Waiting for confirmation before transmitting the next packet results in very low efficiency. The inter-ring buffer unit attaches a sequential number to each transmitted packet, and up to 2 ¹⁶ packets are left pending before the first confirmation. Due to the duality capability, two independent sets of ordinal numbers are used, one for each direction of packet flow. This recovery between one ring link is enabled to proceed independently of the order number of links between other rings between Routing cell pair, and there avoid the resulting Ru timing ambiguity. Only data packets are examined sequentially.

Two ordinal numbers are maintained by each inter-ring buffer unit. The IbuXmitSeqNum operated modulo 2 ¹⁶ provides a count for each data transfer packet transmitted from the inter-ring buffer unit. Modulo 2 ¹⁶ IbuR
cvSeqNum is incremented by 1 for each valid in-order error relief data packet received by this inter-ring buffer unit.
One inter-ring buffer unit is the IbuRcvS
When the transmission to the downlink inter-ring buffer unit EgNum, it expected to a value indicated by the following directs the data packet and the packet reception order number (not including this value) all hand data received It is used to confirm packets. Each transmitted IbuRcvSeqNum subsequent confirms that the message preceding the whole hand is received again. Multiple data packets can be identified in a single packet.

Packet format The inter-ring link packet format is the frame start (FrameStart), preamble (Preamble), ring packet (RingPacket), CRC and frame end (Fram).
eEnd) consists of 5 fields. The start-frame and end-frame fields are link-specific framing fields and are added to or removed from the inter-ring packet by the inter-ring link interface.

The frame start and frame end fields depend on the type of physical link.

The preamble specifies a link control command, a sequence confirmation field, a transmission frame sequence number (TFSN) and a reception frame sequence number (RFSN). A control command clarifies the number of words and the contents of the ring packet subfield.

[0318]

[Table 11]

Instructions are divided into ordered and unordered instructions. The ordinal number for a link is incremented by the routing cell pair for ordered frames. The ordinal number for a link is not incremented for unordered frames.

The transmission frame sequence number (TFSN) represents the frame sequence number. Each consecutive ordered frame has its ordinal number incremented by one. Inter-ring link was used as the order number with the model di Interview Russia 2 ^16. When the sender reaches its maximum turn number, it is forced to stop sending and checking for pending packets until a frame in the opposite direction is accepted. The reception frame sequence number (RFSN) confirms the reception of RFSN-1,
And none of the frames preceding that number have been identified. RFSN is the number R in which the receivers are ordered.
Indicate that you are expecting an FSN. Transmission ring between the buffer unit buffers the entire hand frame not yet been clearly notified. Once securely notified, the frame can be wiped out or purge, and the order number is reused.

Inter-ring links, like HDLC, provide a "piggyback" nature in which the notification function is embedded in frames transmitted in the opposite direction. Multiple frames can be reported in a single frame. The three possible notifications are:

[0322]

[Table 12]

The idle command indicates that the ring packet field is null and the preamble and checksum or checksum residue is valid. The initialization request, initialization sequence and initialization confirmation command are used to initialize and synchronize the sequence protocol between IBUs at each end of the inter-ring link. The initialization sequence indicates that the inter-transmit ring buffer unit has reset its ordinal number. Receive routing cell resets sequence number, ring packet queue, IbuMas
It should clear terConfig.RPQFull and send a reset confirmation control command packet. Initialization request: downlink inter-ring buffer unit resets sequence number, ring packet queue, IbuMasterConfig.RPQFul
Clear l and send the initialization sequence packet. The short packet and long packet instructions indicate that the ring packet field contains a short or long packet, respectively.

Inter-ring links use two mechanisms for error recovery. 1- Refusal to recover. This would be needed whenever it is possible to increase the inter-ring link recovery rate. However, it can only be used once for a given frame. It cannot be called for a repeat of that frame. 2-Time out recovery. It must always be used in addition to reject recovery. Without time-out recovery, it would not be recovered if the last of the isolated, ordered frame or series of frames was mistransmitted. In addition, multiple rejections of a given frame have to be dealt with via the time out, since recovery rejection can only be used once per sequence frame.

For link packet error, Incorrect checksum Sequence transmission packet number out of range Clock recovery error Link frame error

The RNR confirmation gives flow control when the incoming buffer is full. In order to make optimal use of inter-ring link bandwidth, the smallest possible incoming buffer size is the number of frames received during a round trip inter-ring link delay.

The Cyclic Redundancy Field (CRC) is calculated from the preamble and ring packet fields.

IB operation Inter-ring buffer unit The buffer management control performs the following IB operation and transfers packets between the five routing cell buffers.

Packets from the Ring Interconnect Packet Extraction Buffer are forwarded through the Routing Bus to both the Stored Start and the Ring Packet Queue at the location specified by the Ring Packet Queue Push Pointer. The pointer is incremented by 18 words for both short and long packets. Ring packet is empty (RPQ push pointer is RPQ
(Equal to pop pointer) and the output buffer is not full, the packet is pushed further into the output buffer and the RPQ pop pointer is incremented. IbuXmit
SeqNum is incremented by 1 for each data packet transmitted on the inter-ring link.

Ring from dynamic RAM buffer
Interconnect Packets Packets are transferred from the Ring Packet Queue to the Egress Buffer when the Ring Packet Queue is not empty and the Out Buffer is not full. The ring packet queue pop pointer is incremented by 32 words.

Inter-Ring Link Packet The packet is transferred from the ingress buffer to the insertion buffer.
The count IbuRcvSeqNum is incremented by 1.

Reloading of dynamic RAM buffer between rings
A fresh refresh sequence is performed on the inter-ring drum when the IRB refresh count overflows. Refresh takes higher priority over all other routing operations.

Inter-Ring Link Retry Each packet received over the Inter-Ring link specifies the last received error clearing packet and whether an error was detected. If the confirmation does not indicate any error, the ring packet queue commit pointer and CommitPktNumbe
The r location is updated in the following manner. 1-RPQCommitPointer = RPQcommitpointer + (RcvPktSeqNum-CommitPktNum) * 32 2-CommitPktNum = RcbPktSeqNum

When a reject packet is received, the IBU
All packets must be retransmitted through the Ring Packet Queue Push Pointer with a sequence number after the Ring Packet Queue Commit Pointer. Ibis is performed by adjusting the ring packet queue pop pointer. 1-RPQCommitPointer = RPQcommitpointer + (RcvPktSeqNum-CommitPktNum) * 32 2-CommitPktNum = PktRcvSeqNum 3-RPQPopPointer = PktRcvSeqNum 4-Store received error packet in Error queue

For any error detected by the error record RI, the IB and RDU in the routing cell, the corresponding packet and the two words of the error state are recorded in the routing cell error queue. The packet should be further queued in the ring packet queue if it is being transmitted to the downlink routing cell.

Dynamics between SPA Read / Write Rings
A read or write is performed through the dynamic RAM location between the check RAM rings. If the output buffer is full or the routing cell detects an error, the dynamic RAM between rings
Is fixed.

Ring: 1 Operation The following description outlines the preferred operation of the routing cell in the context of the multi-level memory system hierarchy described above. The routing cell can be regarded as a packet switch that routes a packet based on a rule that is a function or function of a subpage state based on a packet type and a routing cell directory. First, the subpage state, the page state, the state based on the routing cell, and the undecided state will be described.
Next, a description of key rule concepts and sequences is provided. Then the full subpage rules are given. The page operation and the special operation will be sequentially described. The complete rules for accomplishing page behaviors and special behaviors as well as sub-page behaviors are described.

Outbound Rules FIG. 13 illustrates a standard request sequence,
It also defines the location terms used in this section. The originating or parent ring: 0 is the ring containing the requesting cell: 0 and issued the original request packet (step 1). If the requested subpage is not valid on the originating (or local) ring: 0, the routing cell: 0 extracts the packet and sends it to the local routing cell: 1 (step 2). Locally originated requests are sequentially routed by local routing cell: 1 ring: 1
(Step 3). The Routing Cell: 1 is connected to the remote Ring: 0 and holds a valid copy of the requested sub-page and in turn extracts requests originating from a distance (step 4). Requests originated from distant places are sequentially inserted into the remote ring: 0 by the routing cell: 0 (step 5). The response cell extracts the request and inserts the response (step 6). The request originated from a distance is extracted by the routing cell: 1 (step 7) and sent to the partner's routing cell: 1. This routing cell: 1 is calling the responding routing cell: 1 and inserts the response packet into ring: 1 (step 8). The packet travels around Ring: 1 to the Outgoing Routing Cell: 1 and the packet is extracted (step 9). The packet is sent to the routing cell: 0 of the originating ring: 0 and is inserted on the ring: 0 (step 10). The request sequence is sequentially completed by the originating (requesting) cell extracting the packet (step 11).

The packet is processed by the routing cell eight times during this standard sequence. Two Routing Cells: 0 and two Routing Cells: 1 interpret the packet in eight unique ways based on packet source and packet source, as illustrated in the table below. Routing cell operations and rules are configured based on these eight combinations of routing cell, packet source and packet source.

[0340] 殆 for which part, Routing cell: 1
The locally originated packet rule by is similar to the remotely originated packet rule by the routing cell: 1. From the perspective of the local ring: 0, the local routing cell:
0, for requests that are remotely originated, local Ring: 0 to appear logically as a single cell.

Pkt.RequestorId and RiuCellAddress are
A comparison is made to determine if the request originated locally.

Routing Cell Subpage State Description-Preferred
A preferred embodiment subpage and pending request state exists in any cache in ring: 0 and for which a request is stuck to or from nonlocal ring: 0 ( pending) All subpages are stored in the route setting cell pair. Routing cell: 0 stores the pending state for requests from remote ring: 0. Routing cell: 1 stores the pending state for local requests stuck on remote ring: 0. Both Routing Cell: 0 and Routing Cell: 1 store the sub-page state. Routing cell: 0 and routing cell: 1 subpage states are constant, except during periods when the subpage state is being updated. Rules and undetermined state processes the window frame of the whole hand (window Case) appropriately. Subpage states are described below for a preferred implementation of the invention.

[0343] RDU: 0 (RduState0) listed RDU: 0 state, Ru consists of two independent field and save field. If the subpage status field is
Local ring: 0 to describe the set state of the sub-pages of all the hands that are present. The Pending field describes whether any remotely originated requests for this Ring: 0 are pending. The individual states are described in the following subsections.

[0344]

[Table 13]

[0345] SubPageState: Inv ring: also to indicate the non-Ikoto valid copy of any of the sub-pages within the 0.

SpState: Ro SpState: Ro indicates that one or more caches in Ring: 0 may contain a read-only copy of the subpage, but none contain an owner state. Under certain circumstances, the routing cell pair records a read-only state (SpState: Ro) for the subpage, but the subpage is invalid at Ring: 0 (SpState: In).
v). This is because there is no read-only copy in any cell.

SpState: Nex SpState: Nex indicates the subpage owner, and zero or more readonly copies of the subpage are present on the local ring: 0. There may be zero or more read-only copies in the remote ring: 0. if,
A recombination or first invalidation operation indicating that there are no remote read-only copies, if no read-only copies are outside the local ring: 0, does not change the subpage state to exclusivity. Invite a transition.
Remote routing cells did not signal when they deallocate descriptors .

[0348] SpState: Ex SubpageState: Ex is a local ring: also to indicate the non-Ikoto 0 any of the valid copy of the Sabupeji on the outside of. Local Ring: 0 encompasses an exclusive owner with no read-only copy or a non-exclusive owner with zero or more read-only copies. Routing cell:
0, exclusive of exclusive ownership state of the local cell, (transient) atomic and transient recording the atomic as total and exclusive.

[0349] Pending: None PendingNone request state, the corresponding local ring related to the sub-page: to 0, pending activities that are remote caller to instruct any non-Ikoto.

Pend1: Ro Pendl: Ro is one remotely originated sva for Local Ring: 0. read ro Indicates that the request is pending. Additional remotely originated sva read The ro request is returned to the partner's Routing Cell: 1 with the RspBsy set until the request is closed.

Pend1: Owner Pend1: Owner indicates that one or more invalidation requests or remotely originated ownership requests for Local Ring: 0 are pending. In addition, one or more remotely originated sva read The ro request can also be pending. Remote originating ownership request all hand, read-only request and invalidate request, when in this state, the ring: is sent to 0. The ownership claim is sva read ex, sva
get and sva It's getw. Invalidation request is write inv and lookahead invalidation.

RDU: 1 (RduState1) Enumeration The RDU: 1 state consists of three independent fields and a storage field. Subpage state field, the local Ring: 0 to describe the state of aggregation of the sub-pages of all the hands are present. The Pending field describes whether any locally originated request for Ring: 1 is pending. Pending1 field needs to be reissued due to missing routing pages: 1 and missing pages for remotely originated ownership packets not extracted to satisfy pending request Describe if there is (snarf, snarf).

[0353]

[Table 14]

[0354] SubPageState: Inv ring: also to indicate the non-Ikoto valid copy of any of the sub-pages within the 0.

SpState: Ro SpState: Ro indicates that one or more caches in Ring: 0 may contain a read-only copy of the subpage, but none contain an owner state. Under certain circumstances, the routing cell pair recorded a read-only state (SpState: Ro) for the subpage, but the subpage is invalid within Ring: 0 (SpState: In).
v). This is because there is no read-only copy in any cell.

SubpageState: Owner SubpageState: Owner indicates that the subpage is owned exclusively (Ex) or nonexclusively (Nex) in the local ring: 0. Local Ring: 0 encompasses an exclusive owner with no read-only copy or a non-exclusive owner with zero or more read-only copies. Routing Cell: 1, non-exclusive exclusive ownership state of the local cell, exclusively, to record the atomic and transient atomic as owner whole.

[0357] Pending: None PendingNone request state, pending field and corresponding ring related to the sub-page: to 1, the local oscillator has been pending activity to indicate any non Ikoto.

Pend: Ro Pend: Ro is one locally originated s to local ring: 1
va read ro Indicates that the request is pending. Additional locally originated sva read The ro request is the routing cell of the partner with the RspBsy set until the request is closed:
Returned to 1.

Pend: Owner Pend: Owner indicates that one or more invalidation requests or locally originated ownership requests for Ring: 1 are pending. In addition, one or more locally originated sva read The ro request can also be pending.
Local originating ownership request all hand, read-only request and invalidate request, when in this state, the ring: is sent to 0. The ownership claim is sva read ex, sva get and sva ge
tw. Invalidation request is write inv and lookahead invalidation.

Pend1: ReIssue Pend1: ReIssue indicates that the routing cell: 1 may not lose a potential non-routing cell: 1 origination response packet for the routing cell: 1 origination request. Pend1:
The ReIssue state is a pseudo (spurious) mimic due to the distinction between remotely originated ownership packets that were not extracted to satisfy pending requests and missing page cases.
ssing Used by routing cell: 1 to prevent page errors (page faults) (snarf). If the request originated by the Routing Cell: 1 returns without any response and Pendl: Reissue, the revert request is used by the Routing Cell to cause the Request Processor Cell or Routing Cell to reissue the request. , Pseudo page errors are avoided. Pend1: Reissue is cleared when any locally generated request is pending or during the initial ownership (with data) response time.

Routing Cell Subpage State Description-Alternative
Embodiments Described below are the sub-page states used in the first alternative embodiment of the present invention.

[0362] RDU: 0 (RduState0) listed RDU: 0 state, Ru consists of four independent field and save field. If the subpage status field is
Local ring: 0 to describe the set state of the sub-pages of all the hands that are present. Pending 1 field is any remotely originated sva for this ring: 0 read ro, sva read e
x, or write Describe whether the invalidate request is pending. Undecided 2 fields sent remotely by sva get
Describe whether the request is pending. Undecided 3 fields sent remotely by sva getw (gate and wait, acquisition and wait)
Whether the request is pending, and an additional getw request
Describe whether this request was issued when it was pending. Multiple sva A getw request is recorded, signaling a transient atomic state to the requesting cache.

The three pending fields individually recorded the maximum number of one remotely originated pending request of each type per subpage. Multiple requests of the same type for the same subpage are returned to the partner's routing cell: 1 by the time the pending request is closed. The individual states are described in the following subsections.

[0364]

[Table 15]

[0365] SubPageState: Inv ring: a valid copy of the sub-pages to indicate any non-Ikoto in the 0.

SpState: Ro SpState: Ro indicates that one or more caches in Ring: 0 may contain a read-only copy of a subpage, but none contain an owner state. Under certain circumstances, the routing cell pair recorded a read-only state (SpState: Ro) for the subpage, but the subpage is invalid within Ring: 0 (SpState: In).
v). This is because there is no read-only copy in any cell. The first locally originated request, remotely originated invalid or remotely originated read only request adjusts the read only subpage state to the invalid subpage state when there is no read only copy. .

SpState: Nex SpState: Nex indicates the subpage owner, and zero or more readonly copies of the subpage are present on the local ring: 0. There may be zero or more read-only copies in the remote ring: 0. Subpage state transition towards exclusivity with recombination or first invalidation indicating that there are no remote read-only copies if no read-only copies are outside the local ring: 0. Invite. Remote routing cells do not signal when they deallocate descriptors .

[0368] SpState: Ex SubpageState: Ex is a local ring: also to indicate the non-Ikoto 0 any of the valid copy of the Sabupeji on the outside of. Local Ring: 0 includes a non-exclusive owner with any exclusive owner or zero or more read only copies read-only copies is also insignificant. Routing cell:
0, exclusive of exclusive ownership state of the local cell, records the atomic and transient atomic as total and exclusive.

[0369] Pending [1,2,3]: None PendingNone request state, local ring related to the pending field and the corresponding subpage: to 0, outstanding activity is any of the remote caller instructs the non Ikoto.

Pend1: Ro Pendl: Ro is a remotely originated sv for Local Ring: 0
a read ro Indicates that the request is pending. Additional remotely originated sva read The ro request is the routing cell of the partner with the RspBsy set until the request is closed:
Returned to 1.

Pend1: Ex Pend1: Ex is a remotely originated sv for Local Ring: 0.
a read ex Indicates that the request is pending. Preceding remotely originated sva read The ro request can be pending. Additional remotely originated sva read The ex request is returned to the partner's Routing Cell: 1 with the RspBsy set until the request is closed.

Pend1: OwnerInv Pend1: OwnerInv is a remote originated write to Local Ring: 0. invalidate request or lookahead invalidate (w
rite invalidate.lookahead, sva read ex.lookahead,
sva get.lookahead or sva getw.lookahead) Indicates that the request is pending. Preceding remotely originated sva re
ad The ro request can be pending. Subsequent lookahead invalidation from ring: 1 is emptied. Subsequent remote-transmitted sva read ex, write The inv request is used. Since each subpage has a single owner, and only the owner can issue a write invalidate request, additional write invalidations in this state indicate an error condition. Sva continues read The ro request is marked RspBsy and the duplicate data packet is emptied.

Pend2: Get Pend2: Get is a remotely originated s for Local Ring: 0.
va Indicates that the get request is pending. Subsequent remote-transmitted sva get request is pk until the request is closed
Returned to the originating cell that has the t.data set, indicating that the subpage is already atomic. The subpage may not yet be atomic, but will be atomic before subsequent requests are completed.

Pend3: Getw Pend3: Getw is one or more remotely originated sva for Local Ring: 0. Indicates that the getw request is pending. Unlike all other hand read request, of all hand ge
A tw request is sent during this state rather than the next request being busy. Ring: 0 getw response from is, when indicating that the page is atomic in already (getw.d
ata.RspBsy), Pend3: Atomic state is recorded.

Pend3: Atomic ownership packet is pending ge for atomic ownership.
tw indicates that it is required to satisfy the request.
The owning cache is guaranteed to be in a transient atomic state (tat). Subsequent remotely originated the getw or get request is returned, the instruction that Sabupeji is atomic on already. Subsequent ring: 0
Ownership packet (release.data or sva readex.data) is extracted and sent to Ring: 1 to send to other Ring: 0 that is atomic pending (RRC: 1Pend3: Atomic).

Pend3: Atomic is set under two conditions. First, it is time to getw response indicates that the page is atomic on already. Second, remotely activated re
It is when leas.data enters ring: 0, Pend3: Atomic is set, and the routing cell: 0 can extract subsequent release.data to send to another ring: 0 that is pending atomic. release.data will be locally originated if the ring: 0 cell gains atomic ownership with the remote originated release.

RDU: 1 (RduState1) Enumeration The RDU: 1 state consists of 6 independent fields.
Subpage state field, the local Ring: describes whether 0 the set state and the owner invalidate the subpage in all hand there is in progress. Pend1a field is locally originated sva read Describes whether the ro request is pending in Ring: 1. Pend1b field was locally originated
sva read Describes whether the ex request is pending in Ring: 1. Pend2 field is locally originated sva get
Describe whether the request is pending. Pend3 field is one locally originated sva Describes whether the getw request is pending, and whether additional getw requests were issued while this request was pending. Multiple sva ge
The tw request is recorded, signaling the transient atomic state to the request cache accordingly. The Pend5 field needs to be reissued due to missing routing pages: 1 and the missing cells originating remotely originated ownership packets that were not extracted to satisfy the pending request. Describe if there is (snarf, snar
f).

The five pending fields independently recorded the maximum number of locally originated pending requests of each type, one per subpage. Multiple requests of the same type are busy and returned to the partner's routing cell by the time the pending requests are completed.

[0379]

[Table 16]

[0380] SubPageState: Inv ring: also to indicate the non-Ikoto valid copy of any of the sub-pages within the 0.

SpState: Ro SpState: Ro indicates that one or more caches in Ring: 0 may contain a read-only copy of the subpage, but none contain an owner state. Under certain circumstances, the routing cell pair recorded a read-only state (SpState: Ro) for the subpage, but the subpage is invalid within Ring: 0 (SpState: In).
v). This is because there is no read-only copy in any cell. The first locally originated request, remotely originated invalid or remotely originated read only request adjusts the read only subpage state to the invalid subpage state when there is no read only copy. .

SubpageState: Owner SubpageState: Owner indicates that the subpage is exclusively (Ex) or nonexclusively (Nex) owned within the local ring: 0. Local Ring: 0 encompasses an exclusive owner with no read-only copy or a non-exclusive owner with zero or more read-only copies. Routing Cell: 1, non-exclusive are ownership status of the local cell, exclusively, to record the atomic and transient atomic as owner whole.

[0383]SubpageState: OwnerInvalidating Pend1: OwnerInv is a remote call to Local Ring: 1.
Write Indicates that the invalidate request is pending
It Each subpage has a single owner, and the owner
Only write Since you can issue invalidate request,
Additional write in state invalidates indicate an error condition
It sva read ex, sva read get or sva read by getw
The invalidations implied are Pend1b: Ex and Pend2: G, respectively.
Recorded as et or Pend3: Getw status. Continue to the station
Department sent sva read ro request marked RspBsy
And the duplicate data packet is emptied.

[0384] Pending [1a, 1b, 2,3] : None Pending: None request state, pending field and corresponding ring related to the sub-page: to 1, instructs the local oscillator has been pending activities any free Ikoto To do.

Pend1a: Ro Pendl: Ro is a locally originated sva for Ring: 1. re
ad ro Indicates that the request is pending. Additional locally originated sva read The ro request is the routing cell of the partner with the RspBsy set by the time the request is closed:
Returned to 1.

Pend1b: Ex Pend1: Ex is a locally originated sva for Ring: 1. re
ad ex Indicates that the request is pending. Additional remotely originated sva The read ex request is the partner's routing cell with the RspBsy set by the time the request is closed:
Returned to 0.

Pend2: Get Pend2: Get is a locally originated sva for Ring: 1. g
Indicates that the et request is pending. Subsequent remote-transmitted sva The get request is pkt.d by the time the request is closed.
It is returned to the local cell that has the ata set and the subpage
To indicate that it is atomic in already. The subpage may not yet be atomic, but will be atomic before subsequent requests are completed.

Pend3: Getw Pend3: Getw is one or more locally originated sva for Local Ring: 0. Indicates that the getw request is pending. Unlike all other hands read request, the whole hand ge
A tw request is sent during this state rather than the next request being busy. Ring: When getw response from the 1, page to indicate that it is atomic in already (getw.d
ata.RspBsy), Pend3: Atomic state is recorded.

Pend3: Atomic ownership packet is pending ge for atomic ownership.
tw indicates that it is required to satisfy the request.
The owning cache is guaranteed to be in a transient atomic state (tat). Subsequent remotely originated the getw or get request is returned, the instruction that Sabupeji is atomic on already. Subsequent ring: 0
Ownership packet (release.data or sva read ex.data)
Is extracted and sent to Ring: 1 to send to another Ring: 0 that is pending atomic (RRC: 1Pend3: Atomic).
Pend3: Atomic does not guarantee that one or more getw requests are still pending.

Pend3: Atomic is set under two conditions. First, it is time to getw response indicates that the page is atomic on already. Second, remotely activated re
When leas.data enters ring: 0, Pend3: Atomic is set, and the routing cell: 0 can extract subsequent release.data to send to another ring: 0 that is pending atomic (RRC. : 1Pend3: Atomic). release.data
Would be locally originated if the ring: 0 cell gained atomic ownership with the remote originated release.

Pend4: ReIssue Pend4: ReIssue indicates that the routing cell: 1 may not lose a potential non-routing cell: 1 origination response packet for a routing cell: 1 origination request. Pend4:
The ReIssue state is a pseudo (spurious) mimic due to the distinction between remotely originated ownership packets that were not extracted to satisfy pending requests and missing page cases.
ssing Used by the routing cell: 1 to avoid page errors (page faults) (snarf). If the request originated by the Routing Cell: 1 returns without any response and Pend4: Reissue, the return request is made busy by the Routing Cell and the requesting processor cell or the Routing Cell reissues the request. , Pseudo page errors are avoided. Pend4: Reissue is cleared when no locally generated request is pending or during the initial ownership (with data) response time.

Routing Cell Subpage State Description-Alternative
Example 2 The following is the subpage state used in the second alternative embodiment of the present invention.

Subpage State Describes the information associated with the routing cell about the subpage state.

[0394] Invalid: Inv invalid Sabupeji state ring: a valid copy of the sub-pages to indicate any non-Ikoto in the 0.

Read Only (Ro) The Read Only state indicates that one or more caches in Ring: 0 may contain a read only copy of the subpage. When the routing cell has a read-only state recorded, it is possible that any local cache contains a read-only copy of the subpage. Although the end of the cache that holds the specific Sabupeji to deallocated the descriptor, ring: If you have 0 different descriptors for similar SVA page cache is allocated for, the Routing cells immediately notice Not held, and temporarily holds the read-only state. In another case, reco
When mbine.nex.data departs from Ring: 0, consider that in rare circumstances (eg Pending Read Only Request, PlRo) the routing cell may have a Read Only copy in Ring: 0. That is something that must be done. Usually recombine.nex.inv Busy's claim indicates that a read-only copy of the subpage exists. The first invalidation request or the remote read-only request adjusts the read-only subpage state to the invalid subpage state when there is no read-only copy.

Non-Exclusive (Nex) The non-exclusive state indicates to the subpage owner that zero or more readonly copies of the subpage are present on the local ring: 0. There may be zero or more read-only copies in the remote ring: 0. If no read-only copy exists outside the local ring: 0, a recombination or first invalidation indicating that there is a read-only copy that is not far away will cause a subpage state transition to exclusivity. Occurs. Remote routing cells did not signal when they deallocate descriptors .

[0397] Exclusive (Ex) Exclusive state, local Ring: 0 no valid copy of Sabupeji the outside also instructs the non Ikoto. Local Ring: 0 contains an exclusive or non-exclusive owner and zero or more read-only copies. Routing cells, exclusively an exclusive ownership state of the local cell, records the atomic and transient atomic as total and exclusive.

Pending Requests Subpage Status Pending Requests status is the type of pending request, whether additional requests have been deleted, whether the request needs to be reissued or whether the request has been completed, and routing. Indicates whether the cell is waiting for the original request to be returned. A value of 0 or 1 for # indicates pending ring: 0 or ring: 1, respectively.

[0399] Pend: None (P # None) PendingNone request state is, what these pending activities related to the corresponding sub-page also indicates the non-Ikoto.

PendWait (P # Wait) PendingWait is a read request (sva read {ro, nex, ex},
The get, getw) but is still pending, indicating that the response or the erase response is issued to the already. When a return request packet is received, the packet is emptied and pending requests are closed. If the return request packet contains data and ownership, an error should be signaled.

PendWaitMergeRo (P # WaitMrgRo) PendWaitMergeRo indicates that the read request is still pending and one or more sva readro.nodatapac
ket (s) has been deleted while waiting for a return request.
And instruct. The second request cannot be issued until the first request is closed. When a return request packet is received, the packet is emptied, pending requests are closed, and sva read A ro request is normally issued. If the return request packet contains data and ownership, an error should be signaled.

PendWaitMergeEx (P # WaitMrgEx) PendWaitMergeEx indicates that a read request is still pending and one or more sva readex.nodatapac
Indicates that ket (s) has been erased while waiting for a return request. The pending state always admits the strongest request to be erased, so this state also has zero or more erased sva. read Includes ro packets. The second request is the first
Can not be issued until the request of is closed. When a return request packet is received, the packet is emptied, pending requests are closed, and sva read ex requests are normally issued. If the return request packet contains data and ownership, an error should be signaled.

PendWaitMergeAt (P # WaitMrgAt) PendWaitMergeAt indicates that a read request is still pending and one or more {get, getw} .nodatapac
Indicates that ket (s) has been erased while waiting for a return request. The pending state always admits the strongest request to be erased, so this state also has zero or more erased sva. Includes read {ro, ex} packets. The second request is
It cannot be issued until the first request is closed. When a return request packet is received, the packet is emptied, pending requests are closed, and getw requests are normally issued.
If the return request packet contains data and ownership, an error should be signaled.

PendReIssue (P # Rei) PendReIssue indicates that the read request is pending and that the request should be reissued when it returns.
This state is usually populated when an event detection while a request is pending indicates that the request is returned without a response or the response is returned with stale data. For example if sva read If the ro is pending and invalidation is detected, the returned data will be stale. Sometimes pendreissue
State is populated and local ring: 0 transactions are completed prior to transfer of ownership to remote ring: 0.

PendReissueMergeRo (P # ReiMrgRo) PendReissueMergeRo is a read request pending, and the request should be reissued when it returns, and one or more sva read ro.nodatapacket (s)
Indicates that it has been erased while waiting for a return request. The pending state always admits the strongest requirement to be erased, so this state also has zero or more erased svs.
a read Includes ro packets. The second request can not be issued until the first request is closed. When the return request packet is received, the packet is emptied and the stronger svad read ro or pending request is reissued.

PendReissueMergeEx (P # ReiMrgEx) PendReissueMergeEx indicates that a read request is pending and the request should be reissued when it returns, and one or more sva read ex.nodata packet (s)
Indicates that it has been erased while waiting for a return request. The pending state always admits the strongest requirement to be erased, so this state also has zero or more erased svs.
a read Includes ro packets. The second request cannot be issued until the first request is closed. When the return request packet is received, the packet is emptied and the stronger svad read ex or pending request is reissued.

PendReissueMergeAt (P # ReiMrgAt) PendReissueMergeAt indicates that a read request is pending and the request should be reissued when it returns, and one or more {get, getw} .nodata packet ( s)
Indicates that it has been erased while waiting for a return request. The pending state always admits the strongest requirement to be erased, so this state also has zero or more erased svs.
a read Includes {ro, ex} packets. The second request is the first
Can not be issued until the request of is closed. When a return request packet is received, the packet is emptied and the getw request is reissued.

PendRo (P # Ro) PendRo is sva read ro Indicates that the request is pending. Sva received in any pending ro state read
ex request causes conversion to ReissueMergeEx pending status.

PendRoMergeRo (P # RoMrgRo) PendRoMergeRo is sva read ro request is pending and one or more sva read ro.nodata packet (s)
Indicates that it has been erased while waiting for a return request.

When a return request is received, the packet is marked for erasure. Sva received in any pending ro state read ex request causes conversion to ReissueMergeEx pending status.

PendEx (P # Ex) PendEx is sva read ex Indicates that the request is pending.

PendExMerge (P # ExMrg) PendExMerge is sva read ex request is pending, and
sva read {ro, ex} Indicates that the request has been deleted. When the return request is received, the packet is marked as erased.

PendExMergeAt (P # ExMrgAt) PendExMerge is sva read ex request is pending, and
Indicates that the get or getw request has been cleared. When the return request is received, the packet is marked as erased.

PendAtomic (P # At) PendAtomic indicates that a get or getw request is pending or getw.
Indicates that a nodata response is being returned.

PendAtomic (P # AtMrg) PendAtomic indicates that the get or getw request is pending or g
The etw.nodata response is returned and get, getw or s
va read {ro, ex} Indicates that the request has been deleted. When the return request is received, the packet is marked as erased.

PendRecombine (P # Rec) PendRecombine indicates that recombine.data. {Tat, at, ex, nex} is in a pending or pending state from the local ring: 0. PendRecombine is only used with RDU: 1.

PendMrsp (P1Mrsp) PendMrsp indicates that the response including the erased request from this ring: 0 is currently circulating in ring: 0. The erased request from this ring: 0 will be indicated by P0RoMrg ^* , P0ExMrg ^* or P0AtMrg which is Rdu: 1 pending. During PendMrsp, additional sva to meet cell transaction completion criteria The read request is free.

PendMrspMrgRo (P # MrspMrgRo) PendMrspMrgRo is currently responding on Ring: 0, including responses to be erased from this Ring: 0, and during this period, the remote sva read ro Indicates that the request has been deleted.

PendMrspMrgEx (P # MrspMrgEx) PendMrspMrgEx is currently sending a response containing an erased request from this ring: 0 on ring: 0, and during this period the remote sva read ex request & sva read ro Indicates that the request has been deleted.

[0420]PendMrspMrgAt (P # MrspMrgAt) PendMrspMrgAt receives erasure requests from this ring: 0
Inclusive response is currently circulating on Ring: 0, and
Remote get or getw and sva during this period read ex or sva
read ro Indicates that the request has been deleted.

PendInvalidate (PInv) PendInvalidate is pending write invalidate, write inva
Indicates that lidate.lookahead is pending. sva re
Invalidation implied by ad {ro, ex}, get or getw is Pend
Recorded as Ro, PendEx, or PendAt state.

RDU: 0 state (RduState ₀ ) enumeration RduState ₀ is ring: 0 subpage state and ring: 1
Store pending request (from remote ring 0) state. Illegal combinations of ring: 0 subpage states and ring: 1 pending states are shown as dashed lines. Each RduState ₁ is given a name by appending the ring: 1 pending state name as a suffix after the ring: 0 subpage state.

RDU: 0 used in this alternative embodiment
A table showing pending states is shown in FIG.

RDU: 1 state (RduState 1) enumeration RduState ₁ is ring: 1 subpage state and ring: 0
Store pending request (from remote ring 0) state. Illegal combinations of ring: 0 subpage state and ring: 0 pending state are shown as dashed lines. Each RduState ₁ is given a name by appending the ring: 1 pending state name to the ring: 0 subpage state as a suffix. A table showing the RDU: 1 states used in this alternative embodiment is illustrated in FIG.

Page States The discussion in the following sections assumes the use of the preferred subpage states (for alternative embodiments).

Owner limit (owner limit, OwnerLim
it) OwnerLimit is set or cleared by software,
Restrict ownership of subpages within a page to local ring: 0. It is stored in both Routing Cell: 0 and Routing Cell: 1. When clearing, OwnerLimit
Specifies that any ring: 0 can own a subpage. Upon configuration, it specifies that only local ring: 0 cells can own subpages in non-exclusive, exclusive, atomic or transient-atomic state. Remote Ring: 0 can receive read-only copy. When the OwnerLimit is set, a page fault is signaled to the ownership request by the routing cell returning the request with InvBsy and RspBsy clear. OwnerL
When imit is set, the recombination request is a local ring:
It does not propagate beyond 0. Routing cell descriptor
The default state is cleared for other assignments.

[0427]

[Table 17]

DupLimit DupLimit specifies the conditions that cause duplicate.data to be sent from local ring: 0 to ring: 1. DupLimit
Is stored by the routing cell: 0.

[Table 18]

Dup: CondForward The Duplicate.data packet in the page is conditionally sent to ring 1 by the routing cell: 0 if the page is not exclusively owned by the local ring: 0.
Dup: CondForward is default state for routing the cell descriptor allocation <br/> hit.

Dup: AllForward Duplicate.data is always sent to Ring: 1 by Routing Cell: 0.

Pending Request Page State An pending routing cell operation determines how the routing cell processes an SVA request to which subpage in a page. The pending request page status field of the corresponding Level 3 VEntry identifies this pending information.

Page: None Any pending page-level route setting cell operation is also supported
Not in progress for descriptors . Page: None is def for the route setting cell descriptor assignment O
A Le door state.

[0433] Page: PendDeallocate Routing cell: 0 inside, the route setting cell pair is, di
Indicates that the process is in the process of determining whether the scripter should be deallocated or its routing cell: 1 is being signaled to deallocate the descriptor . Routing cell: Within 1, the routing cell pair indicates that it is in the process of deallocating the descriptor .

[0434] Routing cell descriptor, this de
Switch to Page: PendDeallocate in response to a single cell indicating that the descriptor has been deallocated. If the descriptor does not exist in any cell in ring: 0, then routing cell: 0 and routing cell: 1 deallocate the descriptor . Ciu
inv sum.no data, ciu inv sum allocate.no data and c
iuex Other page level packets such as sum.nodata are busy during this state. Remotely originated subpage requests are made busy by Routing Cell: 0 and Routing Cell: 1. Locally originated subpage requests are not fulfilled in Routing Cell: 0. This is because the requesting local cache has an allocation descriptor to avoid the routing cell pair deallocating that descriptor . Locally originated subpage requests are made busy by the routing cell: 1. This is because the routing cell pair is willing to deallocate the descriptor . disk
After the page has been deallocated, the descriptor will continue to be reallocated on the next request.

Other Routing Cell States The following state fields are stored based on the Routing Cell or Routing Cell Bank. These status fields apply extensively to all subpages and pages unless otherwise stated or mentioned.

Extract Buffer Status The free space in the ring extract buffer is represented by two configurable pointers, ExtractAllThreshold and ExtractRspThresho.
It is divided into four bands by ld. The current band is selected by the number of free words remaining in the extraction buffer. ExtBuf.All, ExtBuf: NonOp are the bands that go from empty to full
t, ExtBuf: PendReq and ExtBuf: Full. The extraction buffer state is maintained based on the routing cell bank. The boundaries of these bands are software configurable. Ring packets are grouped into three mutually exclusive categories for management of the extraction buffer.

1) Optional Packet The following remotely originated packet is optional from the coherency perspective: -RecombinewithSpState: (Inv | Ro) -Duplicate.dataSpState: Inv-sva read ro.datawithSpState: Inv 2) Non-optional packet Must be extracted based on SpState to complete the remotely originated request. Does not include optional packets. 3) Response packet Locally transmitted packet or route setting cell undecided request (RRC: 0Pend (1,2,3), RRC:
Packets that satisfy 1Pend (1,2,3,4,5))

These packet categories are processed in the four extraction buffer bands as follows.

[0439]

[Table 19]

ExtBuf: All The ExtBuf: All band is between the free space and ExtractAllThreshold. Packets of all the hand is received in ExBuf.All band.

ExtBuf: NonOpt ExtBuf: NonOpt Band is ExtractAllThreshold and ExtractRes
It is between ponseThreshold. Non-optional packets and response packets are accepted in this band. If the optional packet is not accepted, no signal is required in the packet.

ExtBuf: PendReq ExtBuf: PendReq Band is ExtractResponseThreshold and extr
It is between act buffer full. Only response packets are accepted in this band. Non-optional packets are rejected by RspBsy and InvBsy claims. If the optional packet is not accepted, no signal is required in the packet.

ExtBuf: Full ExtBuf: Full bandwidth is when the extraction buffer is full. No packets are accepted in this band. Non-optional packets are rejected by RspBsy and InvBsy claims. When the optional packet is not accepted, no signaling in the packet is needed. Locally originated packets are passed along with pkt.TimeStampclear. Packets are extracted by this routing cell at the next ring turn. Both RspBsy and InvBsy are set for remotely originated response packets that carry no data. A remotely originated response packet with read-only data is passed. A remotely originated ownership response packet is passed, and the routing cell: 1 is Pend1:
Switch to ReIssue.

The Pend1: ReIssue state is a pseudo (spurious) missing due to the distinction between a remotely originated ownership packet that was not extracted to satisfy a pending request and a missing page case. Used by Routing Cell: 1 to avoid page errors (page faults). Upon configuration, Pend1: Reissue indicates that Routing Cell: 1 may lose a potential non-routing cell origination response packet to the Routing Cell Origination Request. If the request originated by the Routing Cell: 1 returns without any response and Pend1: Reissue, the return request is made busy by the Routing Cell and the Request Processor Cell or Routing Cell reissues the request. , Pseudo page faults are avoided. Routing cell: 0 uses SpState field, missing page er
ror is avoided. Routing cell: 0 SpState field records whether there is read-only copy or ownership in ring: 0 to satisfy the request. If the request originated by the Routing Cell: 0 returns without a response and there is sufficient copy or ownership on Ring: 0, the request is reissued.

Ring Frozen State In addition to the references that follow, the routing cell must keep track of the first and last packets marked as frozen.

Fz: None Fz: None indicates that the routing cell is not in the ring freeze sequence.

Fz: SingleEccErr Indicates that a correctable Ecc error has been detected in the lookup sequence corresponding to this packet time. Ring: The beginning of the freeze sequence. Always switch to Fz: Freeze during the next packet time. The corresponding packet is processed as described in the Extract Buffer Full section.

Fz: Freeze Routing Cell indicates that it is in a ring freeze sequence. All hand of packets that were not frozen marked is frozen and marked earlier.

The Fz: Unfreeze Routing Cell indicates that the ring freeze sequence is complete. Previous packet of all hand frozen marked on is unfrozen marked and treated by the usual route setting cell. Any condition that triggers the ring freeze sequence will be a transition to the Fz: SingleEccErr or Fz: Freeze state.

Rule table basics

Ownership The strong ordering basis is based on the concept of rule ownership. Each sub-page of memory device has a single owner. For values of Sabupeji to be modified, the owner must be exclusive property which means that a copy of all hand Sabupeji is disabled. When a read-only copy of a subpage exists, the owner is non-exclusive and the subpage cannot be modified. Subpage ownership can be transferred dynamically between caches, where the owning cache is a dynamic home or save location for subpages. Memory devices and routing cells are based in particular on the nature of inclusion. Routing cells directory, Ring: All hand Sabupeji state and Ring 0: 0 to charging to the ring: to keep track of pending requests in the state of all <br/> Te exiting 0.

Other Concepts Owner and Invalidate Requests All owner requests and write invalidate requests go to the wholly-owning cell, regardless of which other requests are pending. The owner requests and invalidates all hand, when the packet is inserted into the ring, the route setting cell Pend: O
Try to go to wner. That is, the routing cell: 1
Pend: Owner when the packet is inserted on Ring: 1
Go to Pend: Owner, and the Routing Cell: 0 goes to Pend: Owner when it inserts the packet into Ring: 0. The routing cell switches from Pend: Owner to Pend: None if: 1-The packet carried by the owner is extracted by the routing cell. The routing cell with 2-SpState: (Inv | Ro) is pkt.! I.
Extract invalidation or remote owner request with nvBsy. The routing cell with 3-SpState: Owner is pkt.! InvB
Extracts the remotely sent invalidation with sy. Always 4 pages breakdown, together with allocated page, the local Routing cells: 0 and Routing cells: for all hands Sabupeji of 1 page, SpState:: 1 and the total hand remote Routing cells Inv And set Pend: None. All pending states are cleared so that page faults are signaled to other pending requests (requests would otherwise be caught busy or timed out).

[0453] read-only requests that are read-only request the local oscillator is, if RC: 1 is Pen
If it is not d: Ro, it is only inserted by the local routing cell: 1. This is an optimal way to avoid flooding the Ring: 1 and consequently too many similar requests to responders. Instead of inserting such a request, the routing cell: 1 extracts it, and pkt.
Set RspBsy. By the way, the routing cell: 0 at the responder does not bounce the read-only request in this way. Instead, read-only requests are inserted at ring: 0.

[0454] The Routing Cell may change from Pend: Ro to Pend: No when the packet-carrying owner or packet-carrying read-only data is extracted by the Routing Cell.
Switch to ne.

A packet carrying read-only data is "erased" as it passes through a routing cell in the Pend: Owner state. sva read ro packet is pkt.Rsp
It is "erased" by setting Bsy, and the duplicate is made empty.

Clear operation and packet busy bit solution
Interpretation RspBsy can always clear at the time of the response operation of with the data to the request packet.

InvBsy is always clear on departure from Originating Ring: 0. Because the routing cell: 0
, If it is pkt.InvBsy, it will not extract the packet. The routing cell passes the packet, and pk
Set t.InvBsy, RspBsy so that the request is reissued. InvBsy is always clear when a remote ownership request or revocation inserts into the remote ring: 0. Because the routing cell: 1 is pkt.InvBsy,
This is because the request is not extracted. Routing cell: 1 passes the packet and sets pkt.InvBsy.RspBsy to allow the request to be reissued. The lookahead actually continues, and begins invalidation with the remaining ring: 0 with SpState: Ro.

Packet Routing Use Cell Bits sva read For use by ro, reassociation and release is done to indicate that the packet should be reissued by the originating cell or the routing cell if the request is not completed. Pkt.rrc is used in two ways.

[0459] Pkt.rrc uses sva instead of prematurely extracting packets by the routing cell: 0. read ro.nodata
Or used to allow a request in recombine.data to traverse the whole ring: 0, so that the request is completed locally. Pkt.rrc is a locally originated sva read ro.n
It is set when odata or recombine.data passes through the routing cell: 0. If the request is not completed before returning to the originating cell (sva read ro.data or recombine.
no data), the source cell passes the packet. ROUTING CELL: 0 extracts the packet with the set rcc and handles the request normally.

Route setting cell: 0 or route setting cell:
In order for 1 to satisfy the pending request (called snarf),
When non-originated recombine.data (non-originating recombination data) or release.data (release data) cannot be extracted, the rrc bit is set so that the packet can be recirculated. If release.data returns to the originating cell, it must be reissued. If release.data is extracted by another cell, no action is needed.
This is because the transient atomic subpage state is saved and another release is issued (ta
t). recombine.data should not be extracted by any cell and is passed by the originating CIU to reissue.

Pkt.rrc is always initialized to clear and passed by CCU and CIU. It is cleared by the routing cell: 0 when the locally originated packet is extracted.

Invalidation Method Multiple types of packets cause invalidation of read-only copy. Write-invalidate causes explicit invalidation, and sva-read-ex, get and getw packets cause implicit invalidation in addition to read requests.
Invalidate of all hands to disable all of the read-only copy in two ways. In the first method, an invalidate packet invalidates all read-only copies it passes through. In the second method, sva-rea
When d-ro sends a request with an invalidate packet, the requester asserts rsp-busy to indicate that the data in the packet has been invalidated and prevents spurious page defects. When duplicate-data sends a request with outstanding invalidate, the packet is emptied. The first method invalidates all cells or RRCs and the second method prevents old read-only data from being propagated to already invalidated cells.

In order to maintain a strong ordered integrity of events and a strong order, all subpages must be invalidated before the validator updates and distributes subpage B. If the invalidate packet is processed serially by each ring: 0, then the invalidation time is all rings with subpage copies: 0
Will be the sum of the ring transition times. If the invalidate was copied exactly by each RRC: 1, the invalidate of subpage A will be before the original invalidator updates subpage B and distributes it to processors that have not yet invalidated subpage A. , Not guaranteed to be completed. Strong order is violated because all processors do not see changes to subpages A, B in the same order.

A technique called invalidate-lookahead invalidates all rings: 0 in parallel, but maintains a strong order of events. When an invalidate packet reaches the first RRC: 1 that pulls out this packet, this packet is copied and sent to Ring: 1, and the lookahead packet sets the lookahead mode to Ring: 1 packet. Produced on Ring: 1. The look-ahead packet has a SpState:
Copied by each RRC: 1 in Ro, invalidating read-only copy before the true request packet arrives there. Original request packet is followed by RR
When C: 1 is reached, the packet is forced to pass at a ring: 1 rate, because the invalidate was completed previously by the look ahead packet. Look-ahead packets are always considered short packets.

14A and 14B show the preferred ring:
1 shows the order of invalidation. The write-invalidate is extracted from the ring: 0 if RDU: 0 SpState is Nex (step 2), and sv
a- (read-ex, get, getw) is RD
U: 0 Extracted from Ring: 0 if SpState is not Nex. The packet is sent to RRC: 1 and inserted on ring: 1 (step 3). The first RRC: 1 with read-only copy of the subpage copies the packet, puts the lookahead mode into the non-lookahead mode and sends the packet to ring: 0 (step 5). The packet generation mode of the passed ring: 1 is unchanged. Subsequent RRC: 1 with read-only state copies the invalidate-lookahead packet (steps 6, 7, 8, 9). Invalidate-LookAhead is sent to Ring: 0, inserted, and PendState is a transition to OwnInv. Ring: After invalidating 0, SpState
Is a transition to inv and PendState is a transition to none, the packet returns to RRC: 1, and RDU:
1SpState is updated to inv and the look-ahead packet is empty (steps 13, 14, 15, 1).
6).

On the other hand, when the first RCC: 1 and the corresponding ring: 1 complete their invalidation or ownership response, the packet is inserted into ring: 1 in a non-lookahead state (step 12). When this non-lookahead packet encounters a subsequent RRC: 1 (and corresponding ring: 1) that had previously been invalidated by the lookahead packet, the packet is sent at the full ring: 1 rate. (Steps 17, 18, 19, 20). Actual Invalidate-If the actual invalidate arrives before the lookahead is complete, the actual invalidate will also pass the ring: 0. When the look-ahead packet reaches RRCi at the requesting RRC: 1, the packet is emptied (step 11). Upon reaching the 1, it rings requests are extracted:: Non-lookahead or generate lookahead packet RRC request 0 is sent (step 22), and is returned to the requester (step 24).

If there is a false RRCSpState: Ro , the RRC pair records the read-only state (SpState: Ro) for the subpage, but since there is no readonly copy in any cell, the subpage is ringed. : Invalid within 0 (SpSta
te: inv). The RRC pair deallocates the descriptor only when the last cache in ring: 0 deallocates the corresponding descriptor (deallocat).
e). When the cache deallocates a descriptor, the state of the subpages in the cache is. Invalidation or read-only. When the cache misses the allocation of disk descriptor, read-only copy is simply erased, R
The RC is not notified of the individual read-only subpages involved. Thus, if the cache deallocates a descriptor with another cache in ring: 0 having an allocated descriptor, and only the read-only copy of the subpage in ring: 0 is erased, RR
C still incorrectly stores the read-only state for the subpa. Other examples are non-exclusive packet recombination state Ring: 0 left, also, the ring: 0 in the RRC when the read-only copy exists: 0 and RRC:
This is the case where 1 must record SpState: Ro. The false read-only state is SpState:
It is corrected by RRC: 1 which takes a copy of the read-only data packet when (Inv | Ro).

Duplicate Distribution Read-only copies of subpages are distributed to multiple caches in two ways. Copy is performed when the descriptor is locally allocated and SpState: (I
nv | Ro), it is distributed internally by a cache that copies remotely initiated packets with any RRC: 1 or read-only data. Thus, the number of read-only requests is reduced because the copy is distributed to all Rings: 0 bound to RRC: 1 (packets carrying read-only data send it to Ring: 1).

The duplicate data packet is DupL.
Sent based on imit. Momo Dup: Cond
For Forward, the duplicate packet is sent by RRC: 0 to Ring: 1 provided the page is not exclusively owned by Local Ring: 0. If Dup: AllForward, duplicate packets are always ringed with RRC: 0:
Sent to 1. Ring: 1 the duplicate data packet, it local ring: Also when having 0 a descriptor allocated for the page corresponding to the inside of,
Copied by the remote RRC: 1 when the ExtractBuffer is in the ExtBuf: All region.

Pkt. pcopy is set by RRC: 1 to indicate that the packet was copied from ring: 1. Packets copied by Ring: 1 in the set of pcopy do not need to be returned to RRC: 1 or Ring: 1, after successfully passing RingR: 1 and then RR
It is emptied by C: 0.

[0471] Li Combine (recombination) Li combine data packet, if the ring: If any other cells in 0. not accept the re combine, starting ring: sent from zero. RRC: 1 determines that the packet should be sent from the Departing Ring Ring: 0 with a false RRCSpState: Ro in the same manner as for the Departing Ro request. In the first pass of the recombined data, RRC: 0 is pkt. Set RRC. If the combine combine no data rrc returns to the starting cell, the combine combine is extracted and aborted.
If the recombine data rrc returns to the departure cell,
The cell sends a packet. RRC: 0 then extracts the recombined data rrc and sends it to RRC: 1 for insertion into Ring: 1.

[0472] Ring: 1 the re Combine data packet, a remote RRC: 1 local Ring: 0 has a corresponding a descriptor allocated for the page in, and EtractBuffer is extbuf: All it is in region For example, extracted by remote RRC: 1. Recombined data can also be used to satisfy any outstanding request.

Release When a locally generated release is inserted into the ring,
The Pend: Owner state is recorded so that subsequent releases can be returned to the departure ring: 0 to satisfy possible outstanding getw requests with preceding, already atomic responses. When a remotely generated release is inserted in Ring: 0, the Pend: Owner state is recorded so that subsequent releases can be returned to Ring: 1.

[0474] Page Fault A page fault, which RRC also does not detect the discriminator flop ball Tsu <br/> Ji, Ring: 1 ring on request passing completely through the: detected in 1 of the system. Packet no data is RspBsy clear and InvBsy
Clear to indicate page fault. The packet is then extracted by the departure RRC: 1, returned to ring: 1 and sent to the requesting cell.

When a page fault occurs, the local RR
C: 1 is placed in the Pend: Owner or Pend: Ro state. This condition must be cleared by system software prior to page creation. Mpdw.in from any cell in the fault (not to mention that the page is fixed)
Execution of the vsum command results in a CiuInvSum packet. The following Tables 20 to 30 are a series of rule explanations in this order.

[0476]

[Table 20]

[0477]

[Table 21]

[0478]

[Table 22]

[0479]

[Table 23]

[0480]

[Table 24]

[0481]

[Table 25]

[0482]

[Table 26]

[0483]

[Table 27]

[0484]

[Table 28]

[0485]

[Table 29]

[0486]

[Table 30]

[0487] page operation page allocation disk descriptor is assigned by the local cell prior to any cell SVA request. The RRC pair allocates the descriptor when it receives the first locally-sourced request packet (always no data) from the local ring: 0. If a page fault is issued for the request, the descriptor remains assigned to the cell and RRC pair. One or more descriptors can be allocated for pages that do not exist in the memory system. The descriptor must be assigned to the page to be created. The following table specifically shows the default state of the descriptor assigned. Descriptor Default State DescriptorTag Pktpageaddress SubpageState Invalid OwnerLimit OL: All DupLimit Dup: CondForward PendingPageState Page: None

Deallocate the local cache and the descriptor of the local RRC pair are Ci
Only deallocated by uInvSumAlloc.nodata, CiuPureInvSumAlloc.nodata, or RrcDeallocDesc.nodata packets. These three packets are deallocated (deallo
cate) packet. If the deallocation packet indicates that no other cache in Ring: 0 has a page allocated, the local RRC pair deallocates the descriptor corresponding to that page. While the descriptor is deciding whether to deallocate to prevent a race condition between multiple de-allocation packets and packets destined for other pages, RRC: 0 is Page: Pe
Transition to ndDealloc. If the other cell does not have a descriptor assigned to the page, the descriptor will be deallocated for some time. R
Both RC pairs must be in the Page: PendDealloc state before either RRC deallocates the descriptor. The local RRC pair does not inform the remote RRC that the descriptor has been deallocated. First, the deallocation sequence is described, and then subpages and page behaviors are considered.

The deallocation (dialocate) sequence is initiated by the CCU sending CiuLoadDopReg to the CIU. Address of packet and CCU
The prt entry corresponds to the page to be deallocated. The CIU loads the address into its dropping register, extracts the packet and returns it to the CCU. When loaded at a legal address, the CIU's dropping register causes the CIU to assert RspBsy for requests for its own subpages, ignoring all other page or subpage addresses that match the dropping register. . Dropping register match takes precedence over cache group match.

CiuLoadDopR with CCU returned
When it receives an eg packet, it deallocates the old descriptor by writing a new descriptor in its place, deallocates the prt entry and issues a CiuInvSumAlloc packet if the de-allocated page is not pure. C if the deallocated page is pure
Issue the iuPureInvSumAlloc packet.
The deallocation packet is the address of the page to be deallocated (pkt.address [5: 5] contains the CCU descriptor group), and dataworks0 contains the page address to be allocated. Unprocessed CEU requests can be processed even with the newly assigned address. This is because prt is not needed for deallocate packets. The mrdd (RRC deallocate descriptor) instruction causes the CCU to issue a RrcDeallocDesc packet without allocating prt.

FIG. 15 illustrates the preferred deallocation sequence used in the system of the present invention. The CIU deallocates the old descriptor by writing the newly allocated descriptor to the location of the old descriptor, invalidates the dropping register, and inserts the packet to the CiuIn.
vSumAlloc or CiuPureInvSumA
Process the lloc packet (step 1). The CIU processes the RrcDeallocDesc packet by inserting the packet into the CIU's directory without any modification.

Since the deallocation (dialocate) packet traverses ring: 0, it is claimed by the CIU with the descriptor assigned RsyBsy (steps 2, 3, 6).

RRC: 0 is CiuInvSumAlloc or RrcD if RsyBsy is asserted.
Empty eallocDesc to indicate that the descriptor has been allocated to another cache in Ring: 0 (step 3). If RspBsy is not asserted, the RRC: 0 deallocation packet must traverse Ring: 0 completely to probe all local caches for this allocated page. Because this packet is deallocated cache and RRC:
This is because only part of the ring: 0 passes between 0. Ring 0 passes the packet, sets the RRC bit, clears RspBsy, and Page:
Transition to PendDeallocate. The RRC bit is used to indicate that the packet has passed RRC: 0. If RRC is updated (refres
h) or ecc. When the packet cannot be processed due to the error, the RRC sets IndBsy and then deallocates the de-allocated packet to the next revolution
Process n). Subsequent allocation (dialocate). !! RCC packet is PagePendDeall
Received while the ocate is emptied. This is because the cells that issued these deallocation packets do not claim RspBsy during the current rotation of the first deallocation packet. The reason why RRC: 0 rejects RspBsy during the first deallocation packet is that the cell issuing the subsequent deallocation packet asserted RspBsy during the first partial rotation of the first deallocation packet. For processing.

[0494] If the deallocation packet is CiuPur
If it is eInvSumAlloc, RRC: 0 extracts the packet, sets pkt.cmd-pend-dealloc, sends the packet to local RRC: 1 and jumps to step 8 (RR
C: 1 treatment). CiuPureInvSumAlloc
Is found by the cell holding all subpages within the page, and any other allocated descriptors guarantee that they have all page invalidations. Therefore, RRC: 0 is skipped and CiuPure is skipped.
InvSumAlloc packets are sent around the ring to determine if any cells have pages allocated. Pages allocated by the RRC, the ring: one cell in the 0 next requests a page inside the subpages.

All remote-origin packets received from Ring: 1 in the Page: PendDealloc state are returned to Ring: 1. Hello sva-read-ro package
When the state is returned (ring: 0 state is SPState:
Ro) Other rings: 0 will supply a copy of the subpage. The descriptor of the cell and RRC pair is about to be deallocated, so reconbin
e. data is returned. Since the page destroy sequence always sets all pages invalid prior to deallocation, SpState: {Nex, Ex} will not be present.

If the configuration does not include RRC, both InvBsy and RRC are denied. The original requester is the packet. re
The quest-id is cell-address ,! In
vBsy and! When the RRC is matched, the deallocation packet is emptied (step 5).

If the cache page is RspBsy
, Then RRC: 0 empties the deallocation RRC packet after a complete ring: 0 pass (step 7).
A denial of RspBsy indicates that no cache in Ring: 0 has any page allocated. RR
C: 0 extracts the deallocation packet and pkt.cmd-pend
-Set dealloc and set this packet to local RR
Send to C: 1. Both RRC: 0 and some 1 must be in Page: PenDealloc before the descriptor is deallocated to avoid contention.

RRC: 1 sends this packet to Page: P
Transition to endDealloc, pkt.cmd-dealloc-pa
set ge, extract packet, and RR this
Return to C: 0 (step 8). Page: PendDe
All remotely sourced packets received from Ring: 1 in the alloc state are returned to Ring: 1.

RRC: 0 processes the packet by deallocating the descriptor, and returns Page: Non.
transition to e, extract the packet and return it to RRC: 1 (step 9). RRC: 1 also deallocates the returning packet and transitions to Page: None,
Open a packet.

Packets for pages and subpages are P
age: specially processed in the PenDeallocate state. CiuInvSum.nodata, CiuInvzSumAlocate.nod
Other page-level packets like ata and CiuExSum.nodata are busy. Remote-sourced subpage requests are made busy by RRC: 0 and RRC: 1. Locally derived subpage requests are not affected by RRC: 0. What happens is that the requesting local cache has an allocated descriptor, R
This is because it prevents the RC pair from deallocating the descriptor. RR for locally-sourced subpage requests
C: 1 made busy. This is because the RRC pair deallocates the descriptor. Descriptors are subsequently reallocated on the next request after the page has been deallocated. RR
Locally derived packets sent by C: 0 during steps 7-9 are also made busy by RRC: 1. Locally derived requests processed by RRC: 0 after step 9 are guaranteed to be processed by RRC: 1 after step 10, and the descriptors are reallocated.

Page create memory systems assume that the system's software ensures that the existing page in memory is not reclaimed. The system software must also ensure that the page was not generated by more than one cache at the same time. The memory system hardware does not immediately detect these conditions.

The mpsa instruction provides an anchored descriptor in the local cache for SVA pages. If the descriptor already exists before mpsa, its anchor flag is set. Otherwise, the local cache allocates descriptors as described above in the page allocation / deallocation section.

The local cache is then mpdw. alle
Issue the ciu-ex-sum.nodata packet as a result of the x instruction and transition all owned subpages in the page to the owned state.
RRC: 0 extracts ciu-ex-sum.nodata packet, allocates descriptor if necessary, and R request
RC: Send to 1. Momo Page: PendDeall
If it is oc, RRC: 0 issues a request and pkt.
Set InvBsy. RRC: 1 extracts the packet, allocates descriptors if necessary, transitions all subpages to SpState: Owner, and returns the request to RRC: 0. RRC: 0 inserts a packet and spstas all subpages within the page.
te: Transition to Ex. The originating CIU extracts the packet and returns it to the CCU, which returns a completion status to the local processor. The mpdw instruction is then issued to clear the anchor bit.

Page Destruction Problem Pages can be destroyed in three ways. Whenever a page is destroyed, all subpages within that page must be exclusively owned by the local cache.
In the first method, when pure pages are deallocated, they are destroyed. CiuPu
The reInvSumAlloc packet indicates that the pure page has been deallocated. In the second method, if the descriptor address is changed,
The old page is destroyed and the new page has its contents. RrcDeallocDesc Packet Local RRC
Indicates that the page should be deallocated. In the third method, the page and its contents are explicitly destroyed.
CiuInvSum packet fingers view the possible local RRC should set to an invalid state for all subpages.

When the page is destroyed, CiuPu
reInvSumDearloc, RrcDeallo
cDesc and CiuInvSum packets ring:
1 through remote unprocessed or Rei for page
Clear the sue (reissue) status. Due to the different behavior of the local cache and RRC for the three cases, three different packet types are needed.

Page Destruction The mpsa instruction provides a local cache anchored descriptor for an SVA page. If a descriptor already exists before mpsa,
The anchor flag is set. Otherwise, the local cache allocates descriptors as described in the deallocation / reallocation section above.

The mfsva instruction is used by the local cache to gain exclusive ownership of all subpages.

[0508] Next, the local cache is ciu-inv-sum.nodata.
Mpdw.packet. occurs as a result of the allinv instruction,
All subpages in the page are transited to invalid (invalid) state. RRC: 0 extracts the ciu-inv-sum.nodata packet and sends the request to RRC: 1. If Page: PendDeallc, RRC: 0 passes the request and pkt. Set InvBsy. RRC: 1 extracts the packet, transitions all subpages to SpState: Inv, and sends the request to
Return to RC: 0. RRC: 0 inserts the packet,
And SpState:
Transition to Inv. Departure
The CIU extracts the packet and returns it to the CCU, C
The CU returns a completion status to the local processor.

Page destruction sequence and especially ciu-inv-su
m.nodata does not deallocate local cache or RRC to descriptor. The local cache and RRC-to-descriptor are not deallocated until the descriptor is reused.

Change Descriptor A change descriptor page address is logically to destroy the old page and then create a new page of the same data. The memory system assumes that the software of the system guarantees that no existing page in memory will be created. The system software must ensure that pages are not created by more than one cache at the same time. Note Rishi stearyl <br/> hardware of the beam is not detected immediately these states.

The change descriptor uses three types of instructions. That is, mpdw, mrdd, and mrwd
Is. mpdw instruction is a local cell (CCU and CIU)
Affects only, not inserted in ring: 0.

The RRC Deallocate Descriptor (mrdd) instruction from the CCU Reserved Memory System is opcoded. When the CCU codes this instruction from the CEU, the RCUDeallocDsk. Send out the nodata packet. !! m0 supplies the page address to be deallocated in the same manner as other memory system instructions.
The CIU inserts this packet on Ring: 0. RRC
Initiates the deallocation sequence by copying the packet. If the RRC cannot copy or process this packet, it sets the packet RspBsy (responder busy). CIU returns with RrcDa
llocDesc. Extract the nodata packet and send it to the CCU. If RspBsy is issued,
The request is reissued by the CCU. What if Rsp
If Bsy is clear, CCU indicates successful completion CEU
Return to. If RRC: 0 is configured (configure
e) If not, return packet RrcDeall
ocDesc. nodata. !! RspBsy is the same as the original request and the CCU reports the successful completion status to C
Send it back to the EU. This instruction does not affect the CCU or CIU data structure. The addition of this command is transparent to the CEU.

The RRC write (write) descriptor (mrwd) instruction from the CCU reserved memory system is operation coded. When the CCU codes this instruction from the CEU with RrcWrDesc. Send out the nodata packet. !! m0 supplies the page address to be allocated in the same manner as other memory system instructions. The CIU inserts this packet on Ring: 0. RRC: 0 extracts the packet. Local RRC: 1 allocates descriptors if necessary, sets descriptors to all exclusive, RRC resident descriptor bit (own
rlimit), extract the packet and send it back to ring: 0. If RRC cannot process the packet, packet. Set RspBsy. The CIU extracts the return RrcWrDesc, nodata packet and sends it to the CCU. If RspBsy is set, the CCU reissues the request. If RspBsy is clear, CCU is CE
Returns the successful completion status to U. If RRC: 0 is not configured, return packet RrcWrDesc,
nodata. RspBsy is identical to the original request and causes the CCU to return a successful completion to the CEU. This instruction does not affect the CCU and CIU data structures. The addition of this command is transparent to the CEU.

The change descriptor sequence is as follows. 1-Anchor (fix) the old page (mpsa). 2-Collect all sub-pages of the old page that are exclusively owned. 3-Deallocate RRC descriptor (mrd
d) (Old page). 4-Write a new RRC descriptor (mrw
d) (New page) 5-Write a new descriptor in the cell and clear the anchor bit (mpdw).

The mrdd and mrwd instructions are not needed if the tag portion of the descriptor is unchanged. Furthermore,
Collecting all sub-pages of an exclusively owned old page may not be necessary to change certain descriptor bits as described above. Atomic and transient atomic states are maintained within the cell. The RRC subpage state only records non-exclusive and exclusive ownership when atomic and atomic and atomic are in the latter category.
RRC retains the outstanding getw state that is lost in the change descriptor sequence. The getw requester times out.

Page operation rule Table 31 shows the allocation rule.

[Table 31]

[0517]

[Table 32]

[0518]

[Table 33]

[0519]

[Table 34]

[0520]

[Table 35]

Special operation RRC extraction buffer. Extract Buffer Full Since the peak packet rate is faster than the rate at which RRC can free the buffer, a flow control algorithm for the RRC extract buffer is needed. The control algorithm is based on the principle of maximizing forward while reducing system load. Prioritizing system resources to complete ongoing requests can accomplish both of these goals. This is because the completion of the request achieves the reduction of the system load and the advance of the maximum efficiency.

Packets that cannot be extracted or copied to the extraction buffer are processed based on whether the packet originated from this RRC. If the packet does not originate from this RRC, it is processed depending on whether the packet has no data, has read-only data, or has ownership. The packet busy bits RspBsy and InvBsy (conveniently positioned late in the packet), and the SpState and Pendl subpage state fields are used to make standard protocols do the right thing.

[0523] the total hand of packets emitted by the RRC, pkt. TimeStamp is cleared, thus allowing the packet to go around the ring again.

For all packets not derived from RRC, pkt. The TimeStamp is cleared, thus allowing the packet to go around the ring again if needed. All page operations are considered no data request packets. RspBsy and InvBsy are set for packets that have no data. R
spBsy and nodata are set for the sva-read-ro0.data response and duplicates are cleared. Ownership packet sent, RRC: 1 Pe
ndl: Transition to Reissue state. Pendl:
The Reissue state is a false missing-page-error (page fault) in RRC: 1, which distinguishes lost pages from cases of remotely-owned ownership packets that were not extracted to satisfy outstanding requests. ) Is used to prevent Pendl: R
eissue, when set, indicates that RRC: 1 may have lost a potential non-RRC-based response packet to an RRC-derived request. If RRC: 1 request comes back with no response and Pe
If it is ndl: Reissue, the return request is R
It is busy by the RC and causes the requesting processor cell or RRC to reissue the request, thereby preventing false page faults. RRC: 0 is missing-page-err using SpState field
prevent or. The SpState field of RRC: 0 is
Record whether the read-only copy or ownership is within Ring: 0 to satisfy the request.
If, RRC: 0 from request returns without response, also sufficient copy or ownership ring: if present 0, the request is reissued.

Next, the behavior of all cases is analyzed. Ring: 1 to RRC: 0, Ring: 0 to RRC: 1 RRC: 0 (1) to RRC: 1 (0) The packet received is usually on the ring: 1 (0), the packet is Rerouted back to RRC. This is done by immediately extracting the inserted packet. One example is unprocessed request (Pendl: sv in Ro state)
a-read-ex) is a local request of the same type as RR
This is the case when processed at C: 1. RspBsy is asserted and the request is returned to local ring: 0.

If the extract buffer cannot receive the rerouted packet, it is held in the insert buffer until the extract buffer is not full and empty packets are available. Rule notation insert. extract indicates a rerouted packet that follows this behavior. From Ring: 0 to RRC: 0, Local Ring: 0
The emitted packet request packet - returns to the originating location (perhaps be satisfied along the way), obtaining a reissue if necessary. Read Only Data Carrying Packets-Return to origin (maybe satisfied along path) and get reissues if necessary. Ownership Data Carrying Packets-Return to origin (maybe satisfied along path) and get reissues if necessary. But if there was a request from another ring: 0, this packet would not have been satisfied. When the packet returns to RRC: 1,
RRC: 1 will inform that ownership is still contained in Ring: 0 and will return the request to Ring: 0.

Ring: 0 to RRC: 0, remote ring:
Packets originating from 0 Request packet-ring: go around 0 (maybe satisfied along the way), R
Will be extracted on the next encounter with RC: 0. Read Only Data Carrying Packet-Ring: Go around 0 (maybe satisfied along the way), R
Will be extracted on the next encounter with RC: 0. Ownership Data Carrying Packet-Ring: Go around 0 (maybe not along the way),
Will be extracted on the next encounter with RRC: 0.

Ring: 1 to RRC: 1, local phosphorus
Packets originating from 0: Request packet-Ring: go around 0 (maybe satisfied along the way), R
Will be extracted on the next encounter with RC: 0. Read Only Data Carrying Packet-Ring: Go around 0 (maybe satisfied along the way), R
Will be extracted on the next encounter with RC: 0. Ownership Data Carrying Packet-Ring: Go around 0 (maybe not along the way),
Will be extracted on the next encounter with RRC: 0.

Ring: 1 to RRC: 1, remote ring:
Packets originated from 0 Request packet-return to origin (maybe satisfied along path) and get reissue if necessary. Read Only Data Carrying Packets-Return to origin (maybe satisfied along path) and get reissues if necessary. Ownership Data Carrying Packet-Will return to the origin and satisfy the request. However, if there were outstanding requests from ring: 0 bound to this RRC, this packet would not have been satisfied. When this packet returns to RRC: 1, R
RC: 1 notifies that ownership is still contained in Ring: 0 and returns the request to Ring: 0.

[0530] Ring freezing (freezing) Frozen (full rie's) bit is used by the cell to freeze all packets on the ring relative to the rotation of one. Ring freeze means that every packet on the ring is in a state where no member on the ring can change or copy the information in the packet. The cell freezes the ring by systematically asserting the freeze bit in all packets passing through it until all packets on the ring are frozen. During the freezing time, each cell is free to perform any operation without the need to process ring packets. Once each ring freezes a packet, the cell unfreezes each packet from the one that it first frozen, and unfreezes all frozen packets. The ring freezing sequence is started immediately if the condition to start the ring freezing sequence is detected during the defreeze period. R
The RC must remember the packet that was previously frozen and re-frozen. FIG. 16 shows a complete ring freeze (full ring freeze). Lightly shaded areas indicate packet slots that are not marked as frozen, while darkly shaded areas indicate slots that are marked as frozen.

[0531] To process a plurality of cells to freeze cents ring, behaves as if each cell has only one cell to be frozen 殆 etc. ring. However, the cell must only unfreeze the packets it has marked as frozen. Therefore, each cell must always store the packets it marks as frozen. The cell always stores the marked packet by knowing the first and last packet it marked as frozen. This simplification is possible because the freeze mechanism ensures that any cell freeze sequence freezes one adjacent string of packets. Therefore, it is possible to maintain the first and last frozen packets using the counter without following the shift register.

Ring freezing can be initiated in two ways: full ring freezing and full-1 ring freezing. cm
Since dFreeze is in the first part of the packet command, the situation leading to ring freeze must be known early in packet processing. Full ring freeze is when the freeze bit is used for the first packet to be frozen. If the conditions needed for ring freezing are not available early enough for packet processing, the Full-1 ring freezing method can be used.

The Full-1 Ring Freeze method does not freeze the first packet to be frozen. Instead, the packet busy bits, namely RspBsy and InvBsy (conveniently placed late in the packet), and SpState.
And the Pendl subpage status field is correct for standard protocols (Fz: SingleEccEr
r state). The remaining packets are labeled frozen and unfrozen, similar to full ring freezing. The first packet is the extraction buffer full (Extrac
In the section of tBufferFull), it is processed in the same way as described for packets that cannot be extracted or copied to the extraction buffer.

Update (Refresh) Update is performed by RiuMasterConfig. Rfresh
It is distributed by each independently updating RRC when enabled by Enable. 51 every 8 ms
It has to be updated twice. Update interval is 1
5.6 seconds, and RiuRefreshIntervc
Instructed in the ring clock by al.

One update requires one packet processing time. Therefore, the update can be done for blank periods or frozen packets. This is because these packets do not require any processing. RRC is blank on every update,
Attempts to update using all frozen or invisible packets. RRC uses RiuRef for each update
reshCount. The number of updates up to 31 is stored by increasing NumRef. N at the end of each renewal period
umRef is reduced by 1. If NumRef is 0, the update is immediately forced by the RRC using the full ring freeze method (NumRef is not incremented for the first frozen packet) All RRCs are formatted packets in the ring Can store a number of updates equal to the number of. The full ring freeze renewal method is only required if the ring has been fully utilized for a long period of time. Update interval is 0.94 μsec (15.6 * 31/512) to 1
It has been adjusted to 4.7 μs to give the effect of saving up to 31 updates.

ECC correctable error (correctable erro
r) When the RRC detects a correctable ECC error, additional packet time is needed to correct the data structure. Ring freeze is used to allow multiple packet times for correction and re-rulyup.

When a corrected EEC error is detected, RRC
Starts the full-1 ring freezing method. This is because ECC errors are not known early in packet processing.
The first packet is processed as described in the Extract Buffer Full section, and the remaining packets are marked frozen for one revolution. The corrected data structure is rewritten,
Saved in RDU, then E after one revolution of the ring
Reprocess packets that cause CC errors. Only packets originating from RRC are processed, other packets are extracted by other cells in the ring. Saving the correction packet ensures that RRC-derived packets are correctly processed for single-bit (correctable) hard errors.

RRC is pkt. RrcFault is set in the packet corresponding to the correctable ECC error.
The RRC packet also contains the pkt.txt when an uncorrectable ECC error is detected. Claim UnmarkedFault on RRC-derived packets.

The requesting processor cell interprets the RrcFault bit as follows. R
rcFault bit is R when an RRC fault is detected
Indicates to be stored by RC. If the remaining fault bits (MarkedFault, Local)
Fault, DescFault, and Unmarke
If dFault) is clear, then the packet request ends successfully. If the return packet is busy, the originating cell sends a fault in RrcAtt status and the request should not try again. Other CIUs and CCUs usually handle this packet. If the additional fault bits are set, the actions indicated by these bits ignore the actions of the RrcFault bit.

RDU: 0 Rule The RDU: 0 subpage and page rules defined previously are referenced in this rule.

[0541]

[Table 36]

[0542]

[Table 37]

RDU: 1 Rule The RDU: 0 subpage and page rules defined earlier are referenced in this rule.

[0544]

[Table 38]

[0545]

[Table 39]

[0546]

[Table 40]

Ring: 0 Rule Assumption Ring: N The following changes are needed for packet detail, ring detail, CCU rule, and CIU rule for operation.

Canceling Cell Descriptor Allocation Cell descriptor allocation has been changed. The write descriptor RCC requires two operations to perform the write descriptor operation. The RRC deallocates the descriptor of the old SVA and allocates the same descriptor to the new address. These two operations require two packet times for the deallocation address in the packet address field and the deallocation address in packet data word 0. The new sequence described in the Change Descriptor section is the write descriptor (mpd
w) describes how the two required RRC operations are split into two separate instructions and CCU requests.

CiuUpdateState CiuUpdateState has its only use c
iu. Assuming it is to change the pending state to disabled, there is no need to send it to the ring.

Packet operation Three RRC related operations, that is, RrcDeallocDesc, RrcWr
Added Desc and CiuPureInvSumAlloc. page
The ^ fault operation has been deleted because it is not needed for the Rrc operation. The following table shows the revised encoding.

[0551]

[Table 41]

Invalidator and Responder Busy The definitions of the invalidator busy and the responder busy, which are functions of each packet operation, are described in the following table. For certain operations, these bits are also set to indicate that cell or ring: 0 has a page allocated (descriptor match). This information is R
Used by RC: 0 to indicate whether the page is to be allocated in any additional cells in Ring: 0 during the cell deallocation sequence.

[0553]

[Table 42]

[0554]

[Table 43]

Packet Subpage State Fields The following table shows the valid (vali as a function of packet behavior.
d) Lists subpages.

[0556]

[Table 44]

Ring Freeze Ring Freeze is used to handle EEC single bit errors and foreground updates. Packet pcop
y pkt. as pcop bits for use of CIU, RIU, RRC. Assign cmd [33]. Always CI
Initialized by inserting U. Packets copied from Ring: 1 in the pcopy set are RRC: 1
Or it does not need to be returned to Ring: 1 and is cleared by RRC: 0 when successfully passing Ring: 0.

Packet freeze pk as freeze bit for use of CIU, RIU
t. cmd [32] is assigned. It is always initialized by inserting a CIU or RIU. If set, the packet contents are ignored by all cells.
It must be cleared by the same cell that sets it. If cleared, the packet is interpreted as normal. Valid for empty and non-empty packets.

Packet invalidate (invalid) look ahead packet . Change cmd [13] to hold. The following encoding resulted in packet. cmd-lookahea cmd [28]
Name it d [0]. packet. cmd [28] Description 0 NoLookahead 1 Lookahead Pkt. Lookahead always ignores the data and indicates that the packet is short.

Packet RRC bit packet . Assign cmd [8]. Always initialized and passed by CCU and CIU. Used by sva-read-ro to rejoin, release, and indicate that the packet should be reissued if the cell in ring: 0 does not fetch the subpage.

Packet RrcFault Bit The RrcFault bit has been modified by RRC. The RrcFault bit indicates that an RRC fault has been detected and stored by the RRC. If the remaining fault bits (MarkedFault, LocalFault, Des
If cFault and UnmarkedFault) are clear, the packet request was successfully completed. If the returning request is busy, the departure cell reports a fault in the RrcAtt situation and does not allow the request to enter. If additional fault bits are set, the actions indicated by these bits ignore the effect of the RrcFault bit.

Packet RrcCmd For the use of RRC and RIC pkt. cmd [23:
16] is changed to RmcCmd. RrcCmd uses RRC: 0 and RRC: 1 for the deallocation sequence.
Used to send status between.

[0563]

[Table 45]

CEU aborted request
Packets returned as busy corresponding to a CCU request where the est cell handling CEU has failed should not be reissued by the CCU. System error conditions are reported to the local cache, which does not hang up, but reports to the local processor, ensuring that failed requests are serviced by the local processor and the local cache.

The control location SPA space address is 64 bits and is divided into two parts, a cell address and a cell address offset. The cell address specifically identifies a particular cell in the hierarchy of the ring.
The cell address offset also identifies a particular location within the cell.

Cell Address Structure The cell address specifies the location of cells within the three levels of the ring interconnection hierarchy, Ring: 2, Ring: 1 and Ring: 0. Leaf cell address Haring: 2 addresses, ring: 1 address Ring: 0 address is hierarchically configured. The three ring: n address fields are used to construct absolute or relative addresses to leaf or non-leaf cells.

Referring to the following table, three types of rings are shown: n address, normal address, local address, and routing address. Normal address is an absolute reference against the one <br/> 128 cells within the level of ring hierarchy. The cccccc field specifies the absolute cell number. The local address is a relative reference inside level n-1 of the ring hierarchy. Routing-in addresses are used to address non-leaf cells in the hierarchy. Rrrr in routing address
The rr field specifies the number of rings or the route to ring: n + 1 to approach a non-leaf cell. Local address and routing coding are synonymous. Ring: n Address description 0cccccccc Normal address 1xxxxxxxxxx Local address 1xrrrrrrrr Routing ring: n + 1

Cell address modes are classified into leaf cell addresses and non-leaf (RRC) cell address modes. The cell address mode is determined by a combination of address types within the cell address.

[0569] The relative mode of Li Fuseru makes it possible to address their own without knowing the whole cell address cell is actual. Ring: 2 address, ring: 1 address and ring: 0 address specify local addresses. The leaf cells relative to the local ring: 0 mode allow the cell to address other cells within its local ring: 0 without knowledge of the ring: 1 and ring: 2 addresses. The Ring: 0 Address field identifies the leaf cell that was addressed. A leaf cell relative to a local ring: 1 allows the cell to address other cells within that local ring: 1 without knowledge of the actual ring: 2 address. Ring: 1 address specifies the number of rings: 0 in local ring: 2. The ring: 0 address specifies the number of cells in ring: 0. Ring: Allows a particular leaf cell within the system in bi-mode to be accessed with a full cell address. The Ring: 2 Address, Ring: 1 Address, and Ring: 0 Address fields hierarchically specify Ring: 1, Ring: 0, and the number of leaf cells accessed.

Local Ring: RRC: 1 (non-leaf) pair relative to 1 mode is non-local RRC: 1
Allows pairs to be addressed without knowledge of Ring 2 addresses. Ring: 2 address type is local, ring: 1 address type is normal and identifies non-leaf cells in ring: 1, and ring: 0 address type is routing and identifies routing. . RRC: 0 interleaved configuration (configur
and ring: 0 address fields are used to send packets to the specified ring: 1.
The RRC: 1 pair mode allows the RRC: 1 pair to be addressed by identifying the Ring: 2 address and the Ring: 1 address. RRC: 2 pair mode,
Allows RRC: 2 pairs to be addressed by specifying Ring: 2 address, Ring: 1 address routing and Ring: 0 address routing. In the table below, 1, 2, and 3 indicate rings, l indicates local,
n indicates normal and r indicates routing.

[0571]

[Table 46] SPA [39] controls whether the addressed RRC or its counterpart has been accessed.

Mutually coupled packet routing (SV
A, SPA) SVA request packet is RRC directory lookup, packet command, pkt.RequstorID, CellAddre
ss, RRC rules, and interleaved configuration (configurat
Ion). The SPA request packet is a packet command, pkt. Addle [6
3:40], cell address, pkt. Requ
StorID, are Routing based on CellAddres s及 beauty interleaved configuration. What if pkt.
If the RequesterID and the CellAddress match, the packet responds back and is returned to the requesting leaf cell. Otherwise, Requ
If the storeID and CellAddress match, the packet will be a request for a non-leaf or child leaf cell depending on the CellAddress mode. The code in the Packet RequesterID match and SPA address is a ring level function to which the RRC is bound.

[0573]

[Table 47]

[0574] Interleave configuration parameter is RduMaste
rConfig.RingInterleave, RduMasterConfig.RingValue and IbuMasterConfig.LinkInterleave.

SPA packet requests to leaf cells and non-leaf cells take the same route. The SPA packet request is a cell address interleaved matching ring: n address field (n is the root R
(Corresponding to the level of RC) based on the RRC bank. The RingInterleave configuration determines which bits from the Ring: n dress [3: 0] match. For non-leaf cell access, the Ring address [3: 0] specifies the correct route. For leaf cell access, the exact path of requesting ring: 0 and accessed ring: 0 is not required. Routing is based on the cell being addressed. Access to a non-leaf ring: 2 device requires that both ring: 0 and ring: 1 fields of the cell address be specified.

[0576]

[Table 48]

Both RRC Banks Have Single InterRin
In the gLink bound configuration, the incoming packet is routed to a bank based on IbuMasterConfig.LinkInterleave and a single bit address.

[0578]

[Table 49]

Offset Structure The offset portion of SPA selects a particular control register in the RRC pair, and SPA [39] is addressed to the RR.
Identify whether C or the other party has been accessed. SPA [39] Description 0 Addressed RRC 1 Addressed RRC counterpart When in each unit, control location is bounded by type and location offset. The range of location offsets for each type is specified by each description.

Cell Address [38:32] -Number of Units Units in banks A and B are associated with subrings A (number of odd rings) and B (number of even rings) respectively.
Unit cell address Ring: is accessible only when the unit as specified by n address field is accessed through the associated et been in <br/> are subrings.

[Table 50]

Cell Address [31:28] -Place Type The place types for unit numbers 0x0 and 0x2 that address the RDU are as follows.

[0582]

[Table 51]

Unit number 0x1 to address RIU
The types of places from 0 to 0x13 are as follows.

[0584]

[Table 52]

The location types for 0x1 and 0x3 addressing the RBU are as follows:

[0586]

[Table 53]

[Table 54]

Unit number 0x1 to address RIU
The location format for 0 to 0x13 is as follows:

[Table 55]

Unit number 0x1 to address IBU
And the location type for 0x3 is as follows.

[Table 56]

Routing Directory FIG. 9A depicts an array of dynamic random access memory (DRAM) configured in accordance with the present invention to implement a routing directory of the type shown in FIG. The array has three D's that serve as L1 table, L2 table and L3 table respectively.
It has a RAM. The DRAM has a conventional design, and has a row address input, a column address input, a row address strobe (RAS), and a column address strobe (CAS) data input and output, respectively.
Each DRAM is stored at its output in the entry corresponding to the specified row / column address in response to an address being applied on the row and column address inputs simultaneously with the assertion of RAS and CAS. A signal representing the data to be stored or the signal received at the data input is stored in that entry.

In the illustrated DRAM array, L1
The table DRAM contains level 1 entries referenced by the supply of the descriptor's set field to the row address input and the supply of the index 1 field to the column address input. The L2 table DRAM provides a concatenated set of descriptors for row address inputs and an index 2 field supply and L for column address inputs.
2-entry pointer (output by L1 table)
Referenced by the supply of. L3 table DRAM is
Concatenated set of descriptors for row address inputs, supply of index 3 and index sp fields and L3 entry pointer (L
2 output by the table).

Routing directory configured in the manner shown in [0591] Figure 9A, (including allocation and deallocation) fixed to access information relating a time period in the descriptor may update it. Thus, the time period varies depending on the number of tables (ie three) and is independent of both the degree of directory connectivity and the value of the descriptor being accessed.

This will be understood in connection with the exemplary lookup operation performed on the candidate descriptor received by the routing directory. For example, in time interval # 1, the descriptor set field is
It is supplied to the row address input of the L1 table DRAM along with the asserted RAS. In time interval # 2, the index part of that descriptor is asserted CA
It is supplied to the column address input of the L1 table DRAM together with S. At the same time interval (ie, at interval # 2), the concatenated descriptor set and index 2 fields, along with the asserted RAS, are in the L2 table
It is supplied to the row address input of the RAM.

At time interval # 3, the L1 table DR
The AM outputs the L2 entry pointer.
(2) The entry pointer is L with the asserted CAS
It is supplied to the column address input of the two-table DRAM. At the same time, the concatenated descriptor set, index 3 and index sp fields are asserted RA
It is supplied to the row address input of the L3 table DRAM together with S.

At time interval # 4, the L2 table DR
The AM outputs the L3 entry pointer which is supplied to the column address input of the L3 table DRAM along with the asserted CAS. Information relating to descriptors is then output by the L3 table DRAM at time interval # 5.

The example root directory will update information as needed in the DRAM during the lookup process.
By providing a data input for the same, a rapid fixed interval update operation on the contents of the directory can be performed as well.

Those skilled in the art will appreciate that the embodiments described above are merely exemplary and that other apparatus methods, including modifications, additions and omissions, are within the spirit of the invention. See. For example, it will be appreciated that the central processing units and sub-caches contained within the processing cells of the illustrated embodiment may be replaced with data allocated to those cells, or with other circuitry that may operate with the data. See.

[Brief description of drawings]

FIG. 1 is a diagram showing the structure of a preferred multiprocessing system constructed in accordance with the present invention.

2A is a diagram showing a preferred memory configuration providing data coherency over in the form multiprocessing system shown in Figure 1.

FIG. 2B is a diagrammatic representation of data movement in connection with ownership requests by processing cells not accessing the data in a preferred multiprocessing system constructed in accordance with the present invention.

FIG. 3 is a block diagram illustrating a preferred form of the exemplary level: 0 segment of FIG.

FIG. 4 is a block diagram showing a preferred configuration of a processing cell used to implement the present invention.

Figure 5 is a block diagram showing a preferred have structure of a cache directory in the configured processing cells in accordance with the present invention.

FIG. 6 is a diagram showing a preferred ring routing cell constructed in accordance with the present invention.

FIG. 7 is a diagram showing a preferred ring interconnect unit for ring routing cell used in an embodiment of the present invention.

FIG. 8 is a diagram showing the organization of a routing directory constructed in accordance with the present invention.

FIG. 9 is a diagram showing the organization of a routing directory constructed in accordance with the present invention.

FIG. 9A is a diagram illustrating an array of dynamic random access memory constructed in accordance with the present invention.

FIG. 10 is a diagram showing the format of a routing directory entry used in the practice of the invention.

FIG. 11A is a diagram showing a preferred configuration of an inter-ring buffer of a ring routing cell constructed in accordance with the present invention.

FIG. 11B is a diagram showing a preferred placement of inter-link dealers in a ring routing cell constructed in accordance with the present invention.

FIG. 12 is a diagram showing a preferred inter-ring link frame used in a system constructed in accordance with the present invention.

FIG. 13 is a diagram showing a preferred request routing sequence in a system constructed in accordance with the present invention.

FIG. 14A is a diagram showing a preferred invalidation sequence in a system constructed in accordance with the present invention.

FIG. 14B is a diagram showing a preferred invalidation sequence in a system constructed in accordance with the present invention.

FIG. 15 is a diagram illustrating a preferred deallocation sequence used in a system constructed in accordance with the present invention.

FIG. 16 is a diagram showing full ring freezing in a system constructed in accordance with the present invention.

FIG. 17 is a diagram showing a RUD: 0 state in another embodiment of the present invention.

FIG. 18 is a diagram showing a RUD: 1 state in another example of the present invention.

FIG. 19A is a diagram illustrating a preferred alternative ring interleave embodiment of the present invention.

FIG. 19B is a diagram illustrating a preferred alternative ring interleave embodiment of the present invention.

FIG. 19C is a diagram illustrating a preferred alternative ring interleave embodiment of the present invention.

FIG. 19D is a diagram illustrating a preferred alternative ring interleave embodiment of the present invention.

[Explanation of symbols]

10 System 12A-12F, 14A, 14B, 16 Segment 18A-18R Cell 20A-20F, 24A, 24B, 26 Bus 28A-28F, 30A, 30B Routing cell 40A, 40B, 40C Central processing unit (CPU) 48A, 48B , 48C bus 42A, 42B, 42C memory element 50A, 50B, 50C access request element 52A, 52B, 52C control element 54A, 54B, 54C store 56A, 56B, 56C directory element

Front Page Continuation (72) Inventor Paul A. Binda Mendelson Drive, Hollis, Massachusetts, USA 20 (56) References JP-A-1-281555 (JP, A) JP-A-4-230146 (JP, A) ) D. Commer, Kazue Ueda, Widely B-tree, bit extra volume, November 1980, Japan, Kyoritsu Shuppan Co., Ltd., November 15, 1980, p. 21-39 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06F 15/16-15/177 G06F 12/00-12/00 549 G06F 12/08-12/12 H04L 12/00-12 / 66 G06F 12/00 550-12/06 G06F 13/16-13/18

Claims (27)

(57) [Claims] 1. A routing directory for a multiprocessor computer system, wherein a plurality of descriptors each reference respective data, each said descriptor including a set identifier portion and a plurality of other portions. An input configured to receive a candidate descriptor; B. A directory comprising a series of tables each having a plurality of entries, each entry of the last table of the series of tables being configured to store information related to the one or more data, A directory, wherein each entry in a table other than the last table in the table is configured to store a pointer to at least one entry in the next table in the series of tables; A set of tables that make up a directory is used to determine whether the directory stores information related to the data referenced by the candidate descriptor, the set identifier portion and the plurality of A directory lookup element that accesses each table using both the corresponding ones of the other parts of D. And a utilization directory configured to utilize the decisions from the directory lookup element. 2. An update element is further referenced by said candidate directory in response to a determination that said update element does not store information related to said data referenced by said candidate descriptor in said directory. The routing directory of claim 1, wherein at least one entry in the table is updated to include information associated with the data. 3. The routing directory of claim 1, wherein the computer system includes an associative memory having a positive power of 2 associative degree. 4. The r-th table in the series of tables
(R is an integer of 1 or more and n or less), and the series of tables
The last table is the nth table and the jth table
(J is any integer of 1 or more and (n-1) or less)
Entries of the table are configured to store a pointer of q bits (q is an integer), the pointer pointing to at least one of the plurality of entries of the (j + 1) th table, and the nth table. The routing directory of claim 1, wherein each of the plurality of entries is configured to store information related to at least one of the data. 5. A. A first table (j = 1) has: i) an address input configured to receive an address including the set identifier portion of the candidate descriptor and a first portion of the plurality of other portions; ii) A memory element comprising a storage element configured to store a pointer, and iii) an output configured to output a pointer from the storage element identified by the address received by the address input, B. For any integer from j = 2 to j = (n-1), the memory of each jth table is combined with i) the output of the (j-1) th table to receive the pointer output. Column address input, ii) the set identifier portion of the candidate descriptor
The Chi caries said plurality of other portions of the candidate descriptor and
A row address input for receiving a row address representing the value of each j-th part, and iii) the pointer identified by the column address input and the row address identified by the row address input 5. The memory element of claim 4, including a memory element including a store for providing a value stored in the entry of the jth table, and iv) an output for outputting the value provided by the store. Root configuration directory. 6. A. The directory lookup element includes an access element for accessing the information in response to determining that the nth table stores information related to the data referenced by the candidate descriptor. , And further, B. 6. The routing directory according to claim 4 or 5 , comprising a accessed information generator element for generating said accessed information. If 7. At least one entry in the table of the n-th If the entry is service for storing sub-data indicating information relating to one or more data
A data store , and the one or more data is a descriptor in which the value of the set identifier and all the values from the first part to the n-th part of the other parts are common. Corresponding to the sub-data, i) the value of the set identifier, ii) stored in the entry of the (n-1) th table corresponding to the entry containing the sub-data of the nth table. pointer value, iii) the value of the portion of the first n among the plurality of other portions of a descriptor entry associated including the sub data of the n-th table, and iv), if any, said sub-data is associated among the plurality of other portions of the descriptor which is intended to be accessed as a function of the value of the portion of the (n + 1), Routing di according to claim 4 or 5 Directory. 8. The directory lookup element comprises:
The j-th table (j is at least one integer greater than or equal to 1 and less than or equal to n) is assigned to the plurality of candidate descriptors
The routing directory according to claim 4 or 5 , including an entry determining element for determining whether to include an entry associated with the j-th portion and the set identifier portion of the other portions . 9. The directory lookup element comprises:
The kth table (k is one integer that is greater than or equal to 1 and less than or equal to (n-1)) includes an entry associated with the kth portion of the set identifier portion of the candidate descriptor and the plurality of other portions . 9. The routing directory of claim 8 including a pointer generator for generating a pointer stored in that entry in response to the determination. 10. The directory lookup element is configured such that a k-th table (k is at least one integer greater than or equal to 1 and less than or equal to (n-1)) represents a set identifier portion of the candidate descriptor and the other portions. In response to the decision not to include an entry associated with the k-th part,
9. The routing directory of claim 8 including a miss indication generator for generating a level k miss indication indicating such a decision. 11. The directory includes an update element for updating at least one entry of the series of tables to include information related to the candidate descriptor in response to the level-k miss indication. Item 1
The root setting directory described in 0 . 12. A. With a free list including storage space, B. The update element is responsive to the level k miss indication to allocate storage space in the free list for one or more entries for the kth table (k is at least one integer between 2 and n). An allocation element for allocating, C. The update element further comprises a pointer generation and storage element for generating a pointer to the allocated storage space and storing the pointer in a corresponding entry of the (k-1) th table. 1 1
The listed root directory. 13. Responsive to the deallocation instruction selectively,
Wherein the k (where k is 2 or more and n or less at least one integer) of the table containing the deallocation elements to deallocate entries in claim 1 2 Routing Directory description. 14. is a n = 3, Routing directory according to any one of claims 4 or 5. 15. Each of the descriptors includes a set identifier part and a plurality of other parts, and the plurality of other parts are a first part, an INDEX1 part and a second part.
Minute, INDEX2 part, and third part, IN
The DEX3 part is included, and each part is information representing a value. A plurality of L1 entries for respectively storing pointers to at least one L2 entry, each L1 entry being a value of a set identifier part and an INDE.
The value of X1 moiety associated with one or more of the descriptors Ru common der related, and includes a plurality of L1 entries that are configured to be accessed as a function of their set identifier portion and INDEX1 part of the value L1 A table, and B. A plurality of L2 entries for respectively storing pointers to at least one L3 entry, each L2 entry being a value of a set identifier portion , INDEX1
Part of the value, and INDEX2 portion value Ru common der 1 also
It is associated with a plurality of the descriptor, and the following functions, namely, i) of the L1 entry corresponding to the L2 entry pointer, ii) L2 entry of the value of the set identifier portion and said INDEX2 portion associated L2 including a plurality of L2 entries arranged so as to be accessible as a function
A table; Each L3 entry is the value of the set identifier part , IND
Value of EX1 part, value of INDEX2 part, and INDE
Information relating to data values of X3 moieties are referred to by common der Ru 1 or more of said descriptors to memorize
A plurality of L3 entries is also of a, as the respective L3 entries for the next function, namely, i) the set identifier portion pointer L2 entry corresponding to the L3 entry, and ii) L3 entries associated and L3 including a plurality of L3 entries configured to be accessible as a function of the value of the INDEX3 part
Made from the table, according to claim 1 4 Routing Directory description. 16. At least one of the L3 entries comprises a set identifier value, an INDEX1 part value, IND
The value of EX2 part and the value of INDEX3 part are common, and
The fourth part of the plurality of other parts, IND
Storing subpage indicator representing information related to data referenced by one or more descriptors that have a EXSP portion, the sub-page indicator is stored as the next function, ie i) the subpage indicator L3 entry
The value of the pointer of the L2 entry corresponding to , ii) the value of the set identifier part and the INDEX3 part with which the L3 entry is associated, and iii) the IND with which the subpage indicator is associated
Configured to be accessed as a function of EXSP part of the value, according to claim 1 5 Routing Directory description. 17. The directory lookup element is such that the L1 table corresponds to the set identifier portion and the INDEX1 portion of the candidate descriptor,
And L for determining whether to include a valid L2 pointer
Including 1 table determinant, claim 1 5 Routing Directory description. 18. The method of claim 17, wherein the directory lookup element, wherein the L1 table corresponding to the set identifier portion and the INDEX1 part before Symbol candidate descriptor,
18. The routing directory of claim 17, further comprising an L2 pointer generator for generating a pointer in response to determining to include a valid L2 pointer. 19. The directory lookup element is an L1 table lookup and is an L2 entry pointer.
The routing directory of claim 17 , including a miss indication generator for generating an L 1 miss indication when indicating that is invalid . 20. A. A free list containing storage space, B. The directory includes an update element for updating the directory in response to the L1 miss indication, and the update element is i) free for an L2 entry corresponding to the set identifier part and the INDEX2 part of the candidate descriptor. An L2 entry allocation element for selectively allocating storage space to the list and storing a pointer to the L2 entry in the L1 entry corresponding to the set identifier part and the INDEX1 part of the candidate descriptor; and ii) of the candidate descriptor. Storage space is selectively allocated to the free list for L3 entries corresponding to the set identifier part and the INDEX3 part,
In addition, the L2 entry corresponding to the set identifier part and the INDEX2 part of the candidate descriptor has its L3
An L3 entry allocation element for storing a pointer to the entry, and iii) for storing information related to the data referenced by the candidate descriptor in the L3 entry corresponding to the set identifier part and the INDEX3 part of the candidate descriptor 20. The routing directory according to claim 19 , including the candidate descriptor information storage element of. 21. Responsive to a deallocation instruction selectively,
At least one of the L2 table and the L3 table
One of including deallocation elements to deallocate entries, claim 2 0 Routing Directory description. 22. The directory lookup element is an L2 whose L2 table corresponds to the set identifier portion and the INDEX2 portion of the candidate descriptor.
The routing directory of claim 18 including an L2 table determinant for determining whether to include an entry. 23. The directory lookup element is an L2 whose L2 table corresponds to the set identifier portion and the INDEX2 portion of the candidate descriptor.
Including L3 pointer generator for generating the pointer in response to determining that containing entries, claim 2 2
The listed root directory. 24. The directory lookup element is an L2 whose L2 table corresponds to the set identifier portion and the INDEX2 portion of the candidate descriptor.
In response to determining that no entries are included, including L2 miss generator for generating an L2 miss instruction, claim 2 2 Routing Directory description. 25. A. To assign the L3 entry corresponding to the set identifier part and the INDEX3 part of the candidate descriptor to the L3 table, and to store the pointer to the L3 entry in the L2 entry corresponding to the set identifier part and the INDEX2 part of the candidate descriptor L3 entry allocation element of B. Comprises the set identifier portion and a candidate descriptor information storage element for storing at least one default information relating to data referenced by the descriptor to the L3 entry corresponding to said INDEX3 portion of said candidate descriptor, claims 2 to 4, wherein Root configuration directory. 26. The directory lookup element stores in the L3 table an L3 entry corresponding to the set identifier portion and the INDEX3 portion of the candidate descriptor and generates the information stored in that entry. L3 table entries including identification element, according to claim 2 2 Routing directory description. 27. A directory consisting of a series of tables
And each table has a plurality of entries,
Each of the above entries in the last table of the series is 1
Storing information related to one or more data,
Other than the last table
Each entry is the next table in the series of tables.
Store pointer to at least one entry in
However, each of the above data is less than the number of descriptors.
Both are referenced by one of the descriptors,
The descriptor contains a set identifier part and several other parts.
Including, a multiprocessor computer system
Is a way to manipulate the rooting directory for
I, using the steps of: receiving a candidate descriptor, the series of tables constituting the directory
The candidate descriptor referenced by the candidate descriptor.
The system determines whether to store information related to the data.
And the step of utilizing the decision made by said step of making a decision.
And the step of determining comprises each table in the series of tables.
Each time the bull is accessed, before the candidate descriptor
The corresponding set identifier part and the plurality of other parts.
Characterized by the use of both
How to manipulate the directory settings directory.