JPH0922381A - Device for guaranteeing data consistency between processors - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent access from concentrating to a specified directory management node by holding directories for every main memory block and cache line and providing mode fields for indicating a form. SOLUTION: A main memory unit 103 is composed of plural memory blocks 106, and a main memory directory 107 is provided for every memory block 106. The mode fields 110 are provided inside the main memory directories 107. A cache memory 102 is composed of the cache lines 108 in the same size as the memory blocks 106 and a cache directory 109 is provided for every cache line 108. The mode fields are provided inside the cache directories 109. A cache control circuit 104 controls the cache memory 102 and the cache directories 109. A main memory control circuit 105 controls the read and write of the main memory unit 103 and the main memory directories 107.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an inter-processor data consistency guarantee device that guarantees consistency of data shared by a plurality of computers.

[0002]

2. Description of the Related Art A parallel computer in which a plurality of processors are connected to perform parallel processing is an important technique because it can increase the speed of calculation according to the number of processors. In a parallel computer, when each processor mutually accesses and caches all main memory on the parallel computer, guaranteeing cache coherency between the processors is an important issue.

Guaranteeing cache coherency means that when each processor independently holds the contents of the main memory in the cache memory of its own processor, the contents of the main memory look the same to all processors. Is to guarantee.

For example, processor A and processor B
Is caching block M in main memory,
When the processor C writes to the block M,
The contents of the cache memories of the processors A and B are different from the contents of the block M on the main storage device, which makes the consistency inconsistent. In this case, the corresponding cache lines of processors A and B need to be updated or invalidated to guarantee cache coherency.

In the case of a bus-coupled parallel computer, the snoop method is used as a method of maintaining cache coherency.
In the snoop method, the cache control device of each processor monitors the addresses of all write requests output on the bus. Then, when the other processor outputs a write request for a memory block (a block on the main memory is simply called a memory block) held in the cache of the own processor to the bus, the cache line is invalidated. As a result, when the memory block is accessed next time, the memory block has already been invalidated from the cache, so that the main memory is directly read and the latest data can be accessed. In the snoop method, when writing from another processor to a memory block held by the own processor, a method of updating the cache line of the own processor with the latest data instead of invalidating the cache line is also proposed.

In the bus-coupled parallel computer, memory access concentrates on the bus as the number of processors increases.
The scale of about a dozen processors is the limit of the number of processors.

When the number of processors is further increased, a method of guaranteeing cache coherency of each processor is used by adding a structure called a directory for each block to be cached on the main memory.

The directory is an area for holding the processor numbers of all processors that have cached the memory block (memory block corresponding to the directory). When a write request arrives for a memory block, cache coherency is guaranteed by reflecting or invalidating the write to the cache line of all processors indicated in the directory.

Conventionally, when the number of processors is relatively small, a full map directory is mainly adopted as a directory system. FIG. 14 shows an outline of the configuration of a parallel computer using the full map directory system and its operation.

In the parallel computer of FIG. 14, the main storage device 30
Is connected to the plurality of processors 10a through the network 20.
It is connected to 10d. Each processor 10a-10d
Have cache memories 12a to 12d, respectively. In addition, the main storage device 30 includes a plurality of memory blocks 3
1 and a directory 32 is added to each memory block 31.

The directory 32 is composed of bit strings corresponding to the number of processors (each bit corresponds to each processor). Each bit of the directory 32 becomes 1 when the cache added to each of the processors 10a to 10d caches the corresponding memory block 31,
It becomes 0 when not caching. In FIG. 14, the processors 10a and 10d cache the memory block M on the cache memories 12a and 12d. Therefore, in the directory 32 corresponding to the memory block M, the bit corresponding to the processors 10a and 10d becomes 1, and the processor 10a
The bits corresponding to b and 10c are 0.

In this state, for example, when the processor 10b writes to the memory block M, the write access is performed via the network 20 to the main storage device 30.
Sent to. The main storage device 30 writes the write data to the memory block M, and when it looks at the directory 32 and finds that the processors caching the memory block M are the processor 10a and the processor 10d, it invalidates the caches 12a and 12d. Issue a request to activate. The cache 12 that received the invalidation request
a and 12d delete the copy of the memory block M from the cache memory. As a result, the processor 10a or the processor 10b will be replaced by the memory block
When a read access is performed to the main memory device 30, a copy of the memory block M does not exist in the caches 12a and 12d, and the main memory device 30 is accessed, so that the latest data can be read.

As the number of processors increases, the ratio of the amount of memory required for the directory to the total amount of memory in the full map directory increases. If the number of processors is P, a P-bit directory is required for each memory block. A memory block is usually about 64 bytes to 1024 bytes in size, and if a full-mapped directory constitutes a parallel computer with several hundreds of processors or more, the directory is not practical because the number of bytes per memory block is about ten to 100 bytes.

Therefore, as a directory system of a parallel computer having several hundred processors or more, for example, "Computer Archi"
tecture News Vol.19, No.5, 1991, `` Two Economical D
irectory Schemes for Large-Scale Cache Coherent Mu
Ltiprocessors ”, Yeong-Chang Maa et al.” proposed a tree directory method and a hierarchical full-map directory method.

In the tree directory system, a directory for storing a plurality of processor numbers is provided for each memory block and cache line, and by connecting this to a tree structure, all the processors caching the memory block are recorded. Is. For example, assume that a directory for storing four processor numbers is provided for each memory block and cache line. In the directory of a certain memory block M, the processor numbers of the four processors caching the memory block M are stored. Further, in each of the four processors, the directory of the cache line that is caching the memory block M is
After all, the memory block M is cached 4
The processor number of one processor is stored. Similarly, all the processors that are caching the memory block M can be recorded in a tree structure.

When writing is performed on a memory block, all the processors caching the memory block M can be found by tracing the tree structure from the directory of the memory block. In this tree directory method, assuming that the total number of processors is P and the number of processor numbers that can be stored in one directory is N, a storage area of N log ₂ P bits is used per directory.

On the other hand, the hierarchical full-map directory system has a bitmap for each memory block and cache line, and by connecting this to a tree structure, all the processors caching the memory block are recorded. is there. Each bit of the bitmap provided for each memory block and cache line is associated with a lower processor (or a control node dedicated to directory management). The lowest bit map indicates the caching status of all processors in the corresponding processor group. The upper layer bitmap shows the caching status in processor group units.

When writing is performed to the memory block on the main memory, the control circuit of the main memory reads the bitmap from the directory of the memory block. Then, the bit in which 1 is set is searched for in the bitmap. When 1 is set, it means that the memory block is cached by any of the processors in the processor group corresponding to the bit. Therefore, the control circuit of the main memory issues an invalidation request to the directory management circuits of all the processor groups corresponding to the bits for which 1 is set in the read bitmap. In the directory management circuit of the processor group that receives this invalidation request, the cache control circuit reads the bitmap from the directory,
Look for the bit in the bitmap for which a 1 is set. 1
Means that the memory block is being cached by the processor corresponding to that bit. Therefore, the cache control circuit issues an invalidation request to all the processors corresponding to the bits for which 1 is set in the read bitmap. In this way, invalidation can be issued to all processors that are caching the written memory block.

[0019]

SUMMARY OF THE INVENTION Among the above prior arts,
In the tree directory system, since the height (depth) of the hierarchy becomes deep, it takes time to invalidate. For example, 1
If a directory that can point to 4 processors is used on a parallel computer consisting of 024 processors, 4
Since × 4 × 4 × 4 × 4 = 1024, the maximum number of layers is 6. Therefore, communication for invalidation involves communication in up to 5 steps, which takes time.

On the other hand, of the above-mentioned conventional techniques, in the hierarchical full-map directory system, since only the caching state in processor group units can be seen from the control circuit of the highest-level main memory, which processor in the processor group is concerned. It is not possible to select whether to issue the invalidation request. Therefore, in the hierarchical full-map directory system, is it necessary to provide each processor group with hardware that functions as a control node dedicated to directory management?
Alternatively, it is necessary for a specific processor in the processor group to manage the directory on behalf of the processor group. Therefore, there is a problem in that communication associated with cache control is concentrated on the directory management node.

An object of the present invention is to use only a directory having the same size as the tree directory system or the hierarchical full map directory system in a parallel computer consisting of a large number of processors, and the hierarchy is lower than that of the tree directory system. An object of the present invention is to provide an inter-processor data consistency assurance device using a directory management method in which access to a specific directory management node is not concentrated, such as the type full map directory method.

[0022]

The present invention maintains a directory for each main memory block and cache line in order to guarantee cache coherency in a parallel computer, and caches the main memory block in these directories. As a mode of recording the existing cluster 2
One mode is selected from the modes of more than one type for each main memory block or cache line, and the cluster that is caching the main memory block in the selected mode is recorded and the mode is shown. It is characterized by having a mode field. Then, the main memory control circuit and the cache control circuit control the directory and the mode field to issue the cache control communication to the plurality of clusters recorded in the directory via the network.

Particularly, it is advisable to use the directory of the main memory block as a root to manage the cache line directories of all clusters that have cached the main memory block by linking them in a tree structure. In addition, each main memory block and cache line directory has a pointer mode that operates as a pointer to a plurality of clusters and a bitmap mode that operates as a bitmap. In the operating mode, the state of caching of all clusters in the cluster group is indicated.

When writing to the main memory block, the main memory control circuit of the write target cluster issues an invalidation request to all the clusters pointed to by the main memory directory of the write target main memory block. In response to the invalidation request, the cache control circuit of the cluster invalidates the cache line corresponding to the main memory block and issues the invalidation request to all the clusters indicated by the corresponding cache directory. By doing so, the corresponding cache lines in all the clusters that have cached the same main memory block are invalidated. The write data may be transmitted to update the corresponding cache line in all the clusters caching the same main memory block.

When a request to invalidate or update the cache is issued when writing to the main memory block, the completion of writing to all clusters is guaranteed by waiting for the invalidation completion communication and the update response communication. You may do it.

A directory may be provided for each block, or another directory memory area is prepared, and the directory of the block is registered in the directory memory area only when there is a cluster to be pointed to from the block. May be. Further, the functions of the main memory control circuit and the cache control circuit may be replaced by a specific processor in the cluster.

[0027]

According to the present invention, one of two or more kinds of modes can be selected as a mode of recording a cluster in each directory. Therefore, it is possible to combine a method in which the height of the hierarchy is low, a method in which the size of the directory is not large, a method in which access to a specific node is not concentrated. In particular, each directory has a pointer mode that operates as a pointer to a plurality of clusters and a bitmap mode that operates as a bitmap of all clusters in a cluster group. As a result, the lower several layers of the tree structure configured by the tree directory can be replaced with one bitmap layer, and the number of layers is reduced as compared with the tree directory method. The upper layer has a tree structure with pointers, and can point to any cluster. For this reason, it is not necessary to provide a directory management dedicated circuit as in the hierarchical full map directory system, or to make a specific cluster manage the directory.
Communication is not concentrated on a specific circuit or cluster.

[0028]

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) First, a first embodiment of the present invention will be described. FIG. 1 shows a parallel computer according to the first embodiment. In this embodiment, a plurality of clusters 100 are connected by a network 200. In FIG. 1, reference numeral 101 denotes a processor, which may have the same configuration as a microprocessor in the related art. Cluster 100 is
Take a tightly coupled multiprocessor configuration consisting of one or more processors.

Each cluster 100 includes a plurality of processors 1.
01, one cache memory 102, main memory 1
03, cache control circuit 104, main memory control circuit 10
5 and the network interface circuit 112. Further, in the network 200, when a plurality of communications are performed from a certain cluster A to the same cluster B, the issuing order in the cluster A and the arrival order in the cluster B are assumed to be the same.

The main memory 103 is accessed by using the address line and the data line by using the conventional technique. The main storage device 103 includes a plurality of memory blocks 106, and a main storage directory 107 is provided for each memory block 106. A mode field 110 is provided in the main memory directory 107. Main storage device 103
Is accessed by the processor 101 in the cluster 100 and also by the processor 101 in another cluster 100. That is, each cluster 1
The main storage device 103 in 00 is a storage area shared by all clusters.

The cache memory 102 comprises a cache line 108 having the same size as the memory block 106, and the cache directory 10 is provided for each cache line.
9 are provided. A mode field 110 is provided in the cache directory 109. Caching is performed by storing a copy of the memory block 106 of the main storage device 103 in the own cluster 100 and another cluster in the cache line 108.

The cache control circuit 104 is a circuit for controlling the cache memory 102 and the cache directory 109.

The main memory control circuit 105 includes a main memory device 10.
3 read, write and main storage directory 10
7 is a circuit for performing control. The network interface circuit 112 is connected to a network interface circuit of another cluster through the network 200, and performs memory access from within the cluster 100 to the main storage device of its own cluster or another cluster, and from the other cluster to the main storage device 103 of the own cluster 100. Is a circuit that performs memory access to and from and to receive and send directory control communication between clusters.

In this embodiment, the main storage directory 107
A mode field 110 is provided in the cache directory 109 and a plurality of operation modes are provided in the directory.

FIG. 3 shows the structures of the main storage directory 107 and the cache directory 109. One directory consists of a mode field 110 and a cluster storage field 150. In this embodiment, the number of clusters of the parallel computer is 1024, the number of processors per cluster is 1, the size of the mode field 110 of the directory is 2 bits, and the size of the cluster storage field 150 is 64 bits, and all clusters are 64 clusters. However, the present invention can be used regardless of the number of clusters of the parallel computer, the number of processors per cluster, and the size of the directory.

Here, the cluster number will be described. Since the number of clusters is 1024 in the present embodiment, each cluster is assigned a cluster number of 0 to 1023 (expressed in binary number, 00 ... 0 to 11 ... 1 in 10 bits). A cluster group is formed so that clusters having the same upper 4 bits of the 10-bit cluster number belong to the same cluster group. For example, 64 clusters having a 10-bit cluster number of 0000 ******* are one cluster group, and 64 clusters having a 10-bit cluster number of 0001 ***** are one cluster. A group is such that 64 clusters having a 10-bit cluster number of 0010 ******* are one cluster group, and so on. *
Represents any 1-bit data of 0 or 1.

The structure of the directory will be described with reference to FIG. Directory is pointer mode 16
0, bitmap mode 170, and termination mode 18
Has 0. Cluster storage field 1 depending on mode
The usage of 50 is different.

The pointer mode 160 is a mode in which the mode field 110a is 00 (binary). In pointer mode 160, cluster storage field 15
0 has four 10-bit cluster numbers 161-0,
161-1, 161-2, 161-3 and cluster number existence flag 162-0, 1 corresponding to each cluster number
62-1, 162-2, 162-3 are stored. When the cluster number existence flag 162 is 0, it indicates that the cluster number corresponding to the cluster number existence flag is invalid. When the cluster number existence flag is 1, it indicates that the cluster number corresponding to the cluster number existence flag is valid and the same memory block 106 is cached in the cluster pointed to by the cluster number.

The bitmap mode 170 is a mode in which the mode field 110b is 01 (binary). In the bitmap mode 170, the cluster storage field 150 stores a 64-bit bitmap 171. Each bit of the bitmap 171 indicates the caching status in the cluster group including the cluster holding the directory, and the bitmap 1
If the bit of 71 is 1, it indicates that the cluster corresponding to the bit is caching the same memory block 106.

The correspondence between each bit of the bitmap 171 in the cluster group and the cluster is such that the lower 6 bits of the cluster number indicate the position of the corresponding bit in the bitmap 171. That is,
The kth bit in the bitmap 171 corresponds to the cluster whose lower 6 bits of the cluster number is k-1. For example, the first bit of the bitmap 171 corresponds to the cluster whose lower 6 bits of the cluster number is 0 (decimal), and the second bit of the bitmap 171 has the lower 6 bits of the cluster number is 1 (decimal). , The third bit of the bitmap 171 corresponds to the cluster in which the lower 6 bits of the cluster number is 2 (decimal), ..., The 64th bit of the bitmap 171 is the cluster number. It corresponds to the cluster in which the lower 6 bits of are 63 (decimal).

The termination mode 180 is set in the mode field 1
10c is a mode in which it is 10 (binary), and the cluster storage field 150 is not used. This mode indicates that the cache line of another cluster is not associated with the memory block or cache line.

Using the directory structure shown in FIG. 3,
For each memory block 106, a tree structure is created across all clusters with the memory block 106 as the apex, and all cache lines that cache the memory blocks are managed.

FIG. 2 is a diagram showing a method of managing cache lines in a tree structure formed by using a main memory directory and a cache directory. In FIG. 2, 301 is a main memory directory, and 321, 32
2,331,332,333,334,341,34
2, 343, 344 and 345 are cache directories. In this tree structure, the main memory directory 301 at the top is defined as hierarchy 1. The cache directories 321 and 322 which are directly pointed to by the main memory directory 301 of the tier 1 are defined as the tier 2. Cache directories 331, 332, 333, 334 directly pointed to by the cache directories 321 and 322 in the hierarchy 2
Is defined as Tier 3. After that, in the same way, layers 4, 5, ...
Is defined.

The tree structure of FIG. 2 is constructed according to the following rules.

(Rule 1) In the pointer mode 160, the number of clusters described in the cluster storage field 150 is Pp,
Cluster storage field 1 in bitmap mode 170
The number of clusters described in 50 is Pb, and the total number of clusters is P. At this time, xlog ₂ Pp + log ₂ Pb = lo
For x satisfying g ₂ P, the directory located in the layer x + 1 is in the bitmap mode or the termination mode, and the directory located in the layer x + 2 is in the termination mode.

(Rule 2) The cache line directly pointed to by the bitmap mode is only the termination mode.

(Rule 3) FIG. 4 shows a bit string in which the cluster number is represented by a binary number. The cluster number is log ₂ P
= Xlog ₂ Pp + log ₂ Pb, and x pointer fields 350 and one bitmap field 351. When the directory of the hierarchy j is in the pointer mode, the cluster numbers of all clusters in the subtree pointed to by the k-th subtree of the cluster number field of the directory are the clusters in which the directory exists from pointer field 1 to pointer field j-1. , And the pointer field j becomes k-1. When j = 1, when the directory of the layer 1 is in the pointer mode, the cluster numbers of all the clusters in the subtree indicated by the k-th cluster number field of the directory are pointer field 1 (the cluster number This means that the first pointer field) becomes k-1. The pointer field 35
Of 0, they are called pointer fields 1, 2, ... In order from the leading pointer field.

(Rule 4) The cluster numbers of all the clusters pointed to in the bit map mode are equal to the pointer fields of the cluster numbers in which the relevant directory exists.

Rules 1 to 4 will be described in detail. Rule 1 defines the height (depth) limit of the tree structure in FIG. When the total number of clusters is P, log ₂ P is the number of bits expressing the cluster number assigned to those clusters. Of the log ₂ P bits representing the cluster number, the log ₂ Pb bits represent the range specified in the bitmap mode, and the xlog ₂ Pp bits represent the range specified in the pointer mode (one layer is log ₂ Pp bits. Yes, that represents x layers).

Specifically, in this embodiment, the total number of clusters P =
Since it is 1024, log ₂ P = 10, and a 10-bit cluster number can represent all clusters. That is, the cluster number in FIG. 4 is represented by 10 bits. Since 64 clusters are described in the bitmap mode, Pb = 64 and log ₂ Pb = 6. In other words, 6 bits out of 10 bits of the cluster number (here, the lower 6 bits) are in a range that can be specified in the bitmap mode (that is, these 6 bits are IDs that specify one cluster in the cluster group). Has become. 4 excluding 6 bits of 10 bits of cluster number
The bit portion is the range specified in the pointer mode and corresponds to xlog ₂ Pp in the above formula. Cluster storage field 15 for one directory in pointer mode
Since 4 cluster numbers are described in 0, Pp = 4 and l
og ₂ Pp = 2. Therefore, x = 2. This means that if the tree structure in FIG. 2 has the maximum height, x = 2.
It is shown that the layers up to, that is, the layers 1 and 2 are in the pointer mode, the layer 3 therebelow is in the bitmap mode, and the layer 4 thereunder is in the termination mode. Although three pointer fields 350 are shown in FIG. 4, in the example of this embodiment, the entire cluster number is 10 bits and the 2-bit pointer field 35 is used.
It consists of two 0s and the remaining 6-bit bitmap field 351.

Rule 2 specifies that the hierarchy below the bitmap mode is only the termination mode.

Rule 3 defines the structure of cluster numbers. The latter part of Rule 3 represents the relationship between the cluster number set in the cluster number field of the pointer mode directory and the cluster number of the cluster connected to the tree structure. This relationship will be described with reference to FIG. In FIG. 2, the main memory directory 301 is in the pointer mode, and the cluster number a of the cluster Pa is stored in the second cluster number field and the cluster number b of the cluster Pb is stored in the third cluster number field. The first and fourth cluster number fields are invalid. This indicates that the memory block corresponding to this main memory directory is cached in the clusters Pa and Pb. In the four cluster number fields of the main memory directory 301, the first two bits (pointer field 1) of the cluster number are 00, 01, 10, 11 in order.
This is a field for storing clusters (0, 1, 2, 3 in decimal). Therefore, it can be seen that the first two bits of the cluster number a (10 bits) of the cluster Pa connected by the second cluster number field are 01. Similarly, it can be seen that the first 2 bits of the cluster number b (10 bits) of the cluster Pb connected by the third cluster number field are 10.

Moving to the hierarchy 2, 321 is a directory corresponding to the cache line which is caching the memory block corresponding to the main memory directory 301 in the cluster Pa, and 322 is corresponding to the main memory directory 301 in the cluster Pb. Is a directory corresponding to a cache line that is caching a memory block. The cache directory 321 is in the pointer mode, the cluster number c of the cluster Pc is stored in the first cluster number field, the cluster number d of the cluster Pd is stored in the third cluster number field, and the second cluster number field is stored. And the fourth cluster number field is disabled. This is because the cache line corresponding to this cache directory is the cluster P.
Being cached even in c and Pd (that is,
It means that the memory block corresponding to the main storage directory 301 of the tier 1 is cached). The four cluster number fields of the cache directory 321 store clusters in which the first 2 bits (01 in binary) and the next 2 bits (pointer field 2) of the cluster number are 00, 01, 10, and 11 in order. It is a field to do. Therefore, it can be seen that the first two bits of the cluster number c (10 bits) of the cluster Pc connected by the first cluster number field are 01 and the next two bits are 00. Similarly, the cluster number d (10 bits) of the cluster Pd connected by the third cluster number field
Indicates that the first 2 bits are 10 and the next 2 bits are 10.

The directory 322 of the layer 2 is also in the pointer mode, and the cluster numbers f and g of the clusters Pf and Pg are stored in the second and third cluster number fields thereof. Therefore, the memory block concerned (the memory block corresponding to the main memory directory 301 of the hierarchy 1)
Are cached by the clusters Pf and Pg, the first 4 bits of the cluster number f are 1001, and the first 4 bits of the cluster number g are 1010.

Moving to layer 3, 331, 332, 33
3, 334 are clusters Pc, Pd, Pf, P, respectively.
It is a directory corresponding to the cache line that is caching the memory block in g. The directories 331, 332, 333 are in the termination mode, and clusters are not connected to the hierarchy below this. The directory 334 is in the bitmap mode, and the clusters Ph, Pi, Pj, Pk, corresponding to the bits for which 1 is set are
It can be seen that the relevant memory block is cached in Pl. Further, the clusters Ph, Pi, Pj, P
It can be seen that k and Pl belong to the same cluster group as the cluster Pg, and the cluster numbers of these clusters have the first 4 bits (all pointer fields) of 1010 (the same as all pointer fields of the cluster number g).

According to rules 1 to 4, when investigating whether or not an arbitrary cluster is included in the tree structure as shown in FIG. 2, starting from the directory of the memory block at the top, any of the directories in the directory at all layers is checked. You just have to follow one pointer. For example, a cluster number Y
When investigating whether is included in the tree structure, you can search as follows.

(1) The directory of the hierarchy 1 of the tree structure to be investigated is read. The initial value of i is 1.

(2) Depending on the mode of the directory of hierarchy i, the following process is repeated until the directory of cluster number Y is read or the end condition is satisfied.

(2-1) When the directory of the hierarchy i is in the pointer mode: When the pointer field i in the cluster number Y is taken out and the value of the pointer field i is j, the (j + 1) th directory of the directory of the hierarchy i If the cluster number existence flag of the cluster number field is 1, the cluster number is read from the cluster number field, and the directory of the cluster of the cluster number is read as a new directory. This moves to the layer i + 1 (i + 1 is set as a new i). If the read cluster number matches the cluster number Y, it means that the cluster number Y was included in the tree structure. In addition,
If the cluster number present flag in the cluster number field to be read is 0, the process ends.

(2-2) When the directory of hierarchy i is in the bitmap mode: When the data is taken out from the bitmap field in the cluster number Y and the value of the bitmap field is j, the bits of the directory of hierarchy i If the (j + 1) th bit of the map is 1, the cluster directory corresponding to the bit is read. That cluster is the cluster with cluster number Y. If the bit is 0, the process ends.

(2-3) When the directory of hierarchy i is in the termination mode: The process is terminated.

If the cluster number of the directory finally reached by these processes is Y, it means that the cluster number Y was included in the tree structure.

Next, a method for realizing the write invalidation type cache coincidence control by the parallel computer having the above configuration will be described. The write invalidation type means that, when a write request for a certain memory block is received, the write is performed and invalidation is performed for all cache memories that are caching the memory block.

In the cluster 100, the read request communication and the write request communication from the network 200 to the main storage device 103 through the network interface circuit 112, the invalidation request communication, the addition request communication, the delete request communication, and the bitmap move to the cache memory 102 are performed. Request communication and pointer movement request communication can be received. Further, the read response communication to the processor 101 can be received.

Further, in the cluster 100, the cache control circuit 104 and the main memory control circuit 105 through the network interface circuit 112 to the cache memories of other clusters invalidation request communication, addition request communication, deletion request communication, bitmap move request communication. , And a pointer move request communication can be sent, and the main memory control circuit 105 can send the network interface circuit 112.
The read response communication can be sent to the other cluster through the, and the read request communication and the write request communication can be sent from the processor 101 to the other cluster through the network 200.

Details of these request communications will be described. First, the packet configuration of each request communication will be described. FIG. 5 shows a packet configuration of each request communication.

Each packet has a common packet header composed of packet transmission destination cluster numbers 400a to 400h and packet transmission source cluster numbers 401a to 401h. Packet transmission source cluster numbers 401a to 401h
Is the cluster number of the source cluster that transmitted the request communication packet. Packet destination cluster number 4
00a to 400h are cluster numbers of the destination clusters of the request communication packet.

The read request communication packet 420 includes a packet header and a destination intra-cluster address 403a.
Consists of

The write request communication packet 421 includes a packet header and a destination cluster address 403.
b, write data 404.

The invalidation request communication packet 422 includes a packet header, an invalidation target cluster number 402c, and an invalidation target cluster address 403c.

The add request communication packet 423 includes a packet header, an add target cluster number 402d, an add target cluster address 403d, and an add cluster number 40.
Consists of five.

The deletion request communication packet 424 comprises a packet header, a deletion target cluster number 402e, a deletion target cluster address 403e, and a replacement target cluster number 409.

Bitmap move request communication packet 425
Is the packet header, the cluster number 402f to be moved,
It consists of an intra-movement target cluster address 403f and a bitmap 406.

The pointer movement request communication packet 426 is
Packet header, migration target cluster number 402g, migration target cluster address 403f, and pointer 40
Consists of seven.

The read response communication packet 427 includes a packet header, a response target cluster number 402h, a response target cluster address 403h, and response data 408.
Consists of

The operation of each request communication is as follows.

(Read Request Communication) The read request communication to the cluster 100 is performed when the processor 101 in the cluster 100 or another cluster wants the data in the memory block 106 of the main storage device 103 in the cluster 100, and If a miss occurs in the cache memory 102 of the read request source cluster (that is, the data of the memory block is not cached in the read request source cluster), the cluster 100
Is issued to the network interface circuit 112. The operation of the cluster 100 when receiving the read request communication packet 420 is as follows.

(1) The network interface circuit 112 uses the destination intra-cluster address 403a of the read request communication packet 420 to determine the main storage control circuit 10.
5, a main memory read request is issued.

(2) The main memory control circuit 105 uses the destination intra-cluster address 403a to make the main memory 103
From the memory block 106 corresponding to the address, the main memory directory 107, and the mode field 11
Read 0.

(3) The main memory control circuit 105 returns the read contents of the memory block 106 to the network interface circuit 112. Specifically, the main memory control circuit 105 sends the read request communication packet 420 through the network 200 to the network interface circuit 112 of the cluster indicated by the packet transmission source cluster number 401a (which may be its own cluster or another cluster). A read response communication packet 427 is sent (directly in the case of the own cluster). Here, the response target cluster number 40 of the read response communication packet 427
2h is the cluster number of its own cluster 100 (that is, the cluster that is performing the process just described and holds the memory block 106 to be read), and the reply target intra-cluster address 403h is the received read request communication packet The transmission destination cluster address 403a and the response data 408 of 420 are the contents of the read memory block. Read response communication packet 427
The operation of the received cluster will be described later.

(4) Accessed memory block 106
Depending on the state of the main storage directory 107 of (4-1)
To (4-6). The content of the accessed memory block 106 is returned to the issuer of the read request as described in (3) above, but within the cluster (including its own cluster) receiving the request, the memory block 10 concerned is returned.
The contents of 6 will be cached (see read response communication). Therefore, it is necessary to newly register a cluster in which the contents of the memory block 106 are cached in the tree structure as shown in FIG. The processing is performed from (4-1) to (4
-6).

(4-1) Accessed memory block 1
06 main storage directory 107 in termination mode 180: mode field 1 of main storage directory 107
10 is changed to the pointer mode 160, and the cluster number of the read request issuing cluster is registered in the cluster storage field 150 according to the above-mentioned tree structure configuration rule. At this time, the cluster number existence flag of the cluster number field in which the cluster number is registered is set to 1.

(4-2) Accessed memory block 1
06 main memory directory 107 is pointer mode 16
0 and the entry (cluster number field) for registering the cluster number of the access issue source in the cluster storage field 150 of the main storage directory 107 is free (the cluster number existence flag of the cluster number field is 0): access The cluster number of the issuer is registered in the empty entry in the main memory directory 107.
At this time, the corresponding cluster number existence flag is set to 1.

(4-3) Accessed memory block 1
06 main memory directory 107 is pointer mode 16
0, the entry for which the cluster number of the access issuer in the cluster storage field 150 of the main storage directory 107 is to be registered is not empty (the cluster number existence flag of the cluster number field is 1), and all other entries are empty. (The cluster number existence flag is 0), and the pointer field of the cluster number of the access issue source, the pointer field of the cluster number of the cluster holding the memory block 106, and the entry which is not empty are stored. If the pointer fields of the existing cluster numbers are all the same: the mode field 110 of the main memory directory 107 is changed to the bitmap mode 170, and it corresponds to the cluster number registered in the entry of the cluster memory field 150 in the pointer mode 160. The bit corresponding to the Tsu door and access publisher of the cluster number to 1.

(4-4) Accessed memory block 1
06 main memory directory 107 is pointer mode 16
0, the entry to register the cluster number of the access issue source in the cluster storage field 150 of the main storage directory 107 is not empty (the cluster number existence flag of the cluster number field is 1), and another entry Is not empty, or
The pointer field of the cluster number of the access issuer, the pointer field of the cluster number of the cluster holding the memory block 106, and the pointer field of the cluster number registered in the entry to store the cluster number of the access issuer If at least two of them are not equal: the addition request communication is performed to the cluster having the cluster number registered in the entry that should store the cluster number of the access issuer. In particular,
The addition request communication packet 423 is transmitted. At this time, the addition target cluster number 4 of the addition request communication packet 423
02d is the cluster number of its own cluster 100 (that is, the cluster that is performing the process just described and holds the memory block 106 to be read), and the add-in-cluster address 403d is the read request communication packet 420. Destination cluster address 40
3a and the additional cluster number 405 are the packet transmission source cluster number 401a (cluster number to be added to the tree structure) of the read request communication packet 420. The operation of the cluster that receives the addition request communication packet 423 will be described later. In short, in the subtree structure connected to the cluster number registered in the entry that should store the cluster number of the access issuer, a new tree of the access issuer is newly added. The process for adding a cluster number is performed.

(4-5) Accessed memory block 1
The main storage directory 106 of 06 is the bitmap mode 170, and the pointer field of the cluster number of the access issuer is equal to the pointer field of the cluster numbers of all the clusters in the bitmap (in other words, the pointer of the cluster number of the access issuer). If the field is equal to the pointer field of the cluster number of the cluster that holds the memory block): The access issue is made by setting the bit corresponding to the access issue cluster number in the bitmap of the main memory directory 107 to 1. Incorporate the original cluster number into the tree structure.

(4-6) Accessed memory block 1
No. 06 main memory directory 107 is in the bitmap mode 170, and the pointer field of the cluster number of the access issuer is different from the cluster field pointer fields of all the clusters in the bitmap: From the bitmap of the main memory directory 107 One bit having 1 set is selected, and bitmap move request communication is performed with respect to the cluster corresponding to the bit. Specifically, the bitmap move request communication packet 425 is transmitted. At this time, in the bitmap move request communication packet 425,
The migration target cluster number 402f is the cluster number of the own cluster 100 (that is, the cluster that is performing the processing described now and holds the memory block 106 to be read), and the migration target cluster address 403f is a read request. The destination intra-cluster address 403a and the bitmap 406 of the communication packet 420 are the bitmaps stored in the cluster storage field 150 of the corresponding main storage directory 107. The operation of the cluster which receives the bitmap move request communication packet 425 will be described later, but in short, the process of moving the bitmap of the current directory in the bitmap mode to a lower hierarchy is performed.

After that, the mode field 110 of the main memory directory 107 is changed to the pointer mode 160,
The cluster number of the access issuer and the cluster number of the destination of the bitmap move request communication are registered in the cluster storage field 150 according to the above-described tree structure configuration rule, and the corresponding cluster number existence flag is set to 1. Here, if the cluster number of the access issuer and the cluster number of the destination of the bitmap move request communication match,
Perform (4-4).

(Write Request Communication) Write request communication to the cluster 100 is performed when the processor 101 in the cluster 100 or another cluster wants to perform memory write to the main storage device 103 in the cluster 100.
Network interface circuit 1 of cluster 100
Issued for 12. Write request communication packet 4
The operation of the cluster 100 when 21 is received is as follows.

(1) The network interface circuit 112 uses the destination intra-cluster address 403b of the write request communication packet 421 to determine the main storage control circuit 10.
5, a main memory write request is issued.

(2) The main memory control circuit 105 uses the destination intra-cluster address 403b and uses the main memory 103.
The write data 404 is written in the memory block 106 corresponding to the address in the memory block 106, and the main storage directory 107 and the mode field 110 corresponding to the memory block 106 are read out.

(3) Accessed memory block 106
Depending on the state of the main storage directory 107 of (3-1)
Alternatively, either (3-2) is performed.

(3-1) Accessed memory block 1
06 main memory directory 107 is pointer mode 16
0 or bitmap mode 170: cluster storage field 1 of the main storage directory 107
The invalidation request communication is transmitted to all the clusters registered in 50. Specifically, the invalidation request communication packet 422 is transmitted. At this time, the invalidation target cluster number 402c of the invalidation request communication packet 422 is the own cluster 100 (that is, the cluster that is performing the process just described, and the cluster that holds the memory block 106 that has written the data). ) Is the cluster number and the invalidation target intra-cluster address 403c is the destination cluster intra-address 403b of the write request communication packet 421
And The operation of the cluster that receives the invalidation request communication packet 422 will be described later. In short, since the content of the memory block 106 to which the writing has been performed is cached, the processing is performed to invalidate it. After that, the main memory control circuit 105 changes the main memory directory 107 to the termination mode 180.

(3-2) Accessed memory block 1
When the main storage directory 107 of 06 is in the termination mode 180: Since the relevant memory block is not cached anywhere, cache coherency control is not performed.

(Invalidation Request Communication) The invalidation request communication to the cluster 100 is performed when the cache control circuit 104 or the main memory control circuit 105 in the cluster 100 or another cluster issues the invalidation request communication (write request communication (3-1) and (3) of this invalidation request communication), is issued to the network interface circuit 112 of the cluster 100. The operation of the cluster 100 when receiving the invalidation request communication packet 422 is as follows.

(1) The network interface circuit 112 uses the address of the invalidation request communication packet 422 (the invalidation target cluster number 402c and the invalidation target cluster address 403c) to the cache control circuit 104. I do.

(2) The cache control circuit 104 reads the cache directory of the cache memory 102 using the address of the invalidation request communication packet 422. Specifically, since the memory block 106 indicated by the invalidation target cluster number 402c and the invalidation target cluster address 403c is supposed to be cached, the cache line 108 is searched for and the corresponding cache directory 109 is read. .

(3) When the read cache directory 109 is in the pointer mode 160 or the bitmap mode 170: The cache control circuit 104
The invalidation request communication packet 422 is transmitted through the network interface circuit 112 to all the clusters registered in the cluster storage field 150 of the cache directory 109. At this time,
The invalidation target cluster number 402c is the invalidation target cluster number 402c of the received invalidation request communication packet 422.
The same applies to the intra-cluster address 403c of the invalidation target communication packet 422 of the received invalidation request communication packet 422.

(4) Invalidate the cache line 108 that has received the invalidation request.

(Additional Request Communication) The additional request communication to the cluster 100 is performed by the cache control circuit 104 or the main memory control circuit 10 in the cluster 100 or another cluster.
5 issues an addition request communication (see (4-4) of read request communication and (2-4) of this addition request communication),
Network interface circuit 1 of cluster 100
Issued for 12. Addition request communication packet 423
The operation of the cluster 100 when receiving the is as follows.

(1) The cache directory 109 corresponding to the address of the addition request communication packet 423 is read. Specifically, since the memory block 106 indicated by the addition target cluster number 402d and the addition target cluster address 403d of the addition request communication packet 423 should be cached, the cache line 108 is searched for and the corresponding cache is searched. Directory 1
09 is read.

(2) One of (2-1) to (2-6) is performed depending on the read state of the cache directory 109.

(2-1) When the read cache directory 109 is in the termination mode 180: The mode field 110 of the cache directory 109 is changed to the pointer mode 160, and the additional cluster number 405 (tree The cluster number to be added to the structure) is registered in the cluster storage field 150. At this time, the cluster number existence flag of the cluster number field in which the cluster number is registered is set to 1.

(2-2) The read cache directory 109 is in the pointer mode 160, and the cluster storage field 15 of the cache directory 109 is read.
When the entry to register the additional cluster number 405 of 0 is empty (the cluster number existing flag of the cluster number field is 0): The additional cluster number 405 is registered in the cache directory 109. At this time, the corresponding cluster number existence flag is set to 1.

(2-3) The read cache directory 109 is in the pointer mode 160, and the cluster storage field 15 of the cache directory 109 is read.
The entry to register the additional cluster number 405 of 0 is not empty (the cluster number existing flag of the cluster number field is 1), all other entries are empty (the cluster number existing flag is 0), and the above empty If the cluster number of which the additional cluster number 405 and the pointer field are all the same is registered in the entry that is not: The mode field 110 of the cache directory 109 is changed to the bitmap mode 170, and the cluster storage field 150 is changed to the cluster storage field 150 in the pointer mode 160. After the bit corresponding to the registered cluster number is set to 1, the bit corresponding to the additional cluster number 405 to be added is set to 1.

(2-4) The read cache directory 109 is in the pointer mode 160, and the cluster storage field 15 of the cache directory 109 is read.
If the entry for which the additional cluster number 405 of 0 is to be registered is not empty (the cluster number existence flag of the cluster number field is 1) and there is another entry that is not empty, or the cache directory 109 of that cache directory 109 Pointer field of cluster number registered in entry to register additional cluster number 405 of cluster storage field 150 and additional cluster number 4
If the pointer field of 05 is different: The received addition request communication packet 423 is transferred to the cluster having the cluster number registered in the entry to register the additional cluster number 405.

(2-5) When the read cache directory 109 is in the bitmap mode 170 and the pointer field of the additional cluster number 405 is equal to the pointer fields of the cluster numbers of all clusters in the bitmap: bits of the cache directory 109 The additional cluster number 405 is incorporated into the tree structure by setting the bit at the position corresponding to the additional cluster number 405 in the map to 1.

(2-6) When the read cache directory 109 is in the bitmap mode 170 and the pointer field of the additional cluster number 405 is different from the pointer fields of the cluster numbers of all clusters in the bitmap: bits of the cache directory 109 One bit in which 1 is set is selected from the map, and bitmap move request communication is performed with respect to the cluster corresponding to the bit. Specifically, the bitmap move request communication packet 425 is transmitted. At this time, the movement target cluster number 40 of the bitmap movement request communication packet 425
2f is the addition target cluster number 402d of the received addition request communication packet 423, the movement target cluster address 4
03f is the intra-cluster address 403d to be added, and the bitmap 406 is the bitmap of the cluster storage field 150 of the cache directory 109.

After that, the cache directory 10
The mode field 110 of 9 is changed to the pointer mode 160, and the additional cluster number 405 and the cluster number of the destination of the bitmap move request communication are registered in the cluster storage field 150 according to the above-mentioned tree structure configuration rule, and the corresponding The cluster number existence flag is set to 1.

(Delete Request Communication) The delete request communication to the cluster 100 is performed by the cache control circuit 104 or the main memory control circuit 1 in the own cluster 100 or another cluster.
05 issues a delete request communication (see (2-1) Operation of cache replacement described later and (2-2) of this delete request communication), issued to the network interface circuit 112 of the cluster 100. It Cluster 1 when the delete request communication packet 424 is received
The operation of 00 is as follows.

(1) The main storage directory 107 or cache directory 109 corresponding to the address of the delete request communication packet 424 is read. Specifically, the deletion target cluster number 402 of the deletion request communication packet 424
e and the main storage directory 107 corresponding to the memory block 106 indicated by the deletion target intra-cluster address 403e, or the cache directory 109 corresponding to the cache line 108 in which the memory block 106 is cached.

(2) Either (2-1) or (2-2) is performed depending on the state of the read directory 107 or 109.

(2-1) Read directory 107
Or 109 is the pointer mode 160, and the cluster storage field 15 of the directories 107 and 109
If the packet source cluster number 401e (cluster number to be deleted) of the received delete request communication packet 424 is registered in 0: delete request communication packet 424
If the replacement cluster number 409 is valid, the cluster number in the cluster storage field 150 is changed to the replacement cluster number 409. If the replacement cluster number 409 of the deletion request communication packet 424 is invalid, the cluster number existence flag of the cluster number in the cluster storage field 150 is set to 0 and the cluster number is deleted. At this time, when the number of cluster numbers registered in the cluster storage field 150 becomes 0, the mode of the directory is changed to the termination mode.

(2-2) Read directory 107
Or 109 is the pointer mode 160, and the cluster storage field 15 of the directories 107 and 109
If the cluster number of the cluster different from the packet transmission source cluster number 401e (cluster number to be deleted) of the received deletion request communication packet 424 is registered in 0: The cache control circuit 104 deletes it through the network interface circuit 112. The deletion request communication is transferred to the cluster registered in the entry in which the target cluster number (the packet transmission source cluster number 401e of the received deletion request communication packet 424) is to be stored.

(2-3) Read directory 107
Or 109 in the bitmap mode 170: cluster to be deleted in bitmap (packet source cluster number 401e of received delete request communication packet 424)
The cluster number is deleted by setting the bit at the position corresponding to the cluster) to 0. At this time, if all the bitmaps held in the cluster storage field 150 become 0,
Change the mode of the directory to terminal mode.

(Bitmap move request communication) Cluster 1
Bitmap move request communication for 00 is in cluster 1
When the main memory control circuit 105 or the cache control circuit 104 in 00 or another cluster issues a bitmap move request communication (read request communication (4-6), addition request communication (2-6), cache replacement (2-3) of the operation at the time, and (3- of the pointer movement request communication)
2)), issued to the network interface circuit 112 of the cluster 100. Cluster 1 when receiving the bitmap move request communication packet 425
The operation of 00 is as follows.

(1) The network interface circuit 112 uses the address of the bitmap move request communication packet 425 to make a bitmap move request to the cache control circuit 104.

(2) The cache control circuit 104 reads the cache directory 109 and the mode field 110 of the cache memory 102 using the address of the bitmap move request. Specifically, the move target cluster number 402f and the move target cluster address 403f of the bitmap move request communication packet 425.
Since the memory block 106 indicated by must have been cached, the cache line 108 is searched for and the corresponding cache directory 109 and mode field 110 are read.

(3) The mode field 110 of the read cache directory 109 is changed to the bitmap mode 170, and the bitmap 406 of the received bitmap move request communication packet 425 is written in the cluster storage field 150 of the cache directory 109. . However, when all the bits of the bitmap are 0, the mode field 110 is set to the termination mode.

The bitmap move request communication packet 425 is always transmitted from the upper layer to the lower layer,
Since the lower layer of the bitmap mode directory is the termination mode, the destination directory of the bitmap move request communication packet 425 is always the termination mode.

(Pointer move request communication) Cluster 100
The pointer move request communication to the cache is performed when the main memory control circuit 105 or the cache control circuit 104 in the cluster 100 or another cluster issues the pointer move request communication (operation (2-2) at the time of cache replacement,
And (3-1) of this pointer movement request communication),
Network interface circuit 1 of cluster 100
Issued for 12. The operation of the cluster 100 when the pointer movement request communication packet 426 is received is as follows.

(1) The network interface circuit 112 uses the address of the pointer move request communication packet 426 to make a pointer move request to the cache control circuit 104.

(2) The cache control circuit 104 uses the address of the pointer move request to set the cache memory 1
02 cache directory 109 and mode field 110. Specifically, the movement target cluster number 402 of the pointer movement request communication packet 426
Since the memory block 106 indicated by g and the address 403g within the cluster to be moved is supposed to be cached, the cache line 108 is searched for, and the corresponding cache directory 109 and mode field 110 are read.

(3) Depending on the mode of the cache directory 109, any of (3-1) to (3-3) is performed.

(3-1) When the read cache directory 109 is in the pointer mode 160: select one cluster pointed to by the cache directory 109, remove the selected cluster number from the cache directory 109, and select it The pointer move request communication is transmitted to the cluster thus set, and the cache directory 109 is set in the pointer 407 and passed.

Thereafter, the number of the destination cluster to which the pointer movement request communication packet 426 is transmitted from the own cluster 100 is added to the pointer 407 (directory) sent by the pointer movement request communication packet 426 for the own cluster 100, Write to the cache directory 109 of the own cluster 100.

(3-2) When the read cache directory 109 is in the bitmap mode 170:
One cluster pointed to by the cache directory 109 is selected, the bit corresponding to the selected cluster is set to 0, the bitmap move request communication is transmitted to the selected cluster, and the cache directory 109 is set in the bitmap 406. And pass it on.

After this, the number of the destination cluster to which the bitmap move request communication is transmitted from the own cluster 100 is added to the pointer 407 (directory) sent in the pointer move request communication packet 426 for the own cluster 100, and Write to the cache directory 109 of the cluster 100. Also, the mode field 110
Changes to pointer mode.

(3-3) When the read cache directory 109 is in the termination mode 180: The directory (pointer 407) sent by the received pointer move request communication packet 426 is set to the cache directory 109 of the own cluster 100. , And the mode field 110 is changed to the pointer mode 160.
However, when all the pointer fields 350 are invalid, the mode field 110 is set to the termination mode 180.

(Read Reply Communication) The read reply communication to the cluster 100 is performed when the main memory control circuit 105 in the cluster 100 or another cluster issues a read reply communication (see (3) of read request communication). Network interface circuit 112
Issued to. Read response communication packet 427
The operation of the cluster 100 when receiving the is as follows.

(1) The network interface circuit 112 transfers the reply result to the cache control circuit 104. (2) The cache control circuit 104 records the response result in the corresponding cache line 108 of the cache memory 102, and registers the cache directory 109 as the termination mode 180. (3) The network interface circuit 112 sends the reply result of the read reply communication to the bus 1 in the cluster 100.
It outputs to 11 and responds to the processor 101.

(Operation at cache replacement) From the processor 101 in the cluster 100 to the local cluster 100
When it is desired to read data from the main storage device 103 in the internal cluster or another cluster, it is first checked whether the content of the memory block 106 to be read is cached in the cache memory 102 in the own cluster 100. Therefore, when a cache miss occurs, data is actually read from the memory block 106 desired to be read. At this time, if the content of another memory block has already been registered in the cache line 108 for storing the read result, it is necessary to replace the cache line. The operation when replacing the cache line is as follows.

(1) The cache control circuit 104 reads the cache directory 109 of the cache line 108 to be replaced.

(2-1) When the read cache directory 109 is in the termination mode 180: The deletion request communication of the own cluster number to the cluster holding the memory block 106 cached in the corresponding cache line 108 To send. Specifically, the delete request communication packet 424 is transmitted. At this time, the deletion target cluster number 402 of the deletion request communication packet 424
e is the cluster number of the cluster holding the memory block 106 cached in the cache line, the deletion target cluster address 403e is the cluster address of the memory block 106, and the replacement cluster number 409 is invalid.

(2-2) When the read cache directory 109 is the pointer mode 160: One cluster number registered in the directory 109 is selected and the cluster number is assigned to the directory 109.
Then, pointer movement request communication is performed for the cluster of that cluster number, and the cluster storage field 150 of the directory 109 is transferred. That is, the target cluster number 402g is the cache line 1
The cluster number of the cluster holding the memory block 106 cached in 08, the intra-cluster address 403g of the moving target cluster is the intra-cluster address of the memory block 106, the pointer 407 is the cluster storage field 150 to be moved, and the pointer move request communication packet 426. To send.

After this, the corresponding cache line 108
The delete request communication of the own cluster number is transmitted to the cluster having the memory block 106 that is cached by the. That is, the deletion target cluster number 402e
Is the cluster number of the cluster holding the memory block 106 cached in the cache line 108, the intra-cluster address 403e to be deleted is the intra-cluster address of the memory block 106, and the replacement cluster number 409 is the pointer move request communication packet 426.
A delete request communication packet 424 is transmitted as the cluster number of the destination cluster, and a request to delete the own cluster number is issued.

(2-3) When the read cache directory 109 is in the bitmap mode 170, one cluster number held in the directory 109 is selected and the bit corresponding to the cluster is set to 0.
Then, bitmap mobile communication is performed with respect to the cluster, and the cluster storage field 150 is transferred. After that, the replacement cluster number 409 is set as the destination cluster number of the bitmap mobile communication, and the self cluster number deletion request communication is transmitted to the cluster having the corresponding memory block 106.

(3) After deleting the cluster number of its own cluster from the tree structure as described above, the data of the new memory block 106 can be cached in the cache line 108.

(Operation Example) An operation example of the cache directory system of this embodiment will be shown below. In this operation example, the number of clusters is 1024, and the pointer field 1 of the cluster number
The pointer field 2 is represented by 2 bits and the bitmap field is represented by 6 bits. The number (2) added after the number indicates that the number is binary. A cluster having a cluster number of n is simply called a cluster n.

(1) FIG. 6 shows the initial state of the directory of the memory block M of the cluster 0100000011 (2). In the initial state, the memory block M is in the termination mode.

(2) Cluster 0100000001 in the state of (1)
(2) is memory block M for cluster 0100000011 (2)
Read request communication of the cluster 0100000011 (2)
Reads the contents of memory block M to cluster 0100000001 (2) and returns it in response communication. At this time, cluster 0100
The cache directory of 000001 (2) is in the termination mode, and the main storage directory of the cluster 0100000011 (2) is in the pointer mode according to the read request communication (4-1). This state is shown in FIG.

(3) In the state of (2), the cluster 0100000100
(2) is memory block M for cluster 0100000011 (2)
Read request communication of the cluster 0100000011 (2)
Reads the contents of memory block M to cluster 0100000100 (2) and returns it in response communication. At this time, cluster 0100
The cache directory of 000100 (2) is in the termination mode, and the main storage directory of the cluster 0100000011 (2) is in the bitmap mode according to the read request communication (4-3). This state is shown in FIG.

(4) In the state of (3), the cluster 10010000
01 (2) is a memory block for cluster 0100000011 (2)
When read request communication of M is performed, cluster 0100000011 (2)
Reads the contents of memory block M to cluster 1001000001 (2) and returns it in response communication. At this time, the cluster 1001
The cache directory of 000001 (2) becomes the termination mode, and the main memory directory of the cluster 0100000011 (2) becomes the pointer mode according to the read request communication (4-6). The cluster 0100000011 (2) performs bitmap mobile communication and moves the bitmap to the cluster 0100000001 (2). Accordingly, the cluster 0100000001 (2) is in the bitmap mode. This state is shown in FIG.

(5) In the state of (4), the cluster 10110000
10 (2) is a memory block for cluster 0100000011 (2)
When the read communication of M is performed, the cluster 0100000011 (2) returns the content of the memory block M to the cluster 1011000010 (2) by read response communication. At this time, cluster 10110000
The 10 (2) cache directory is in termination mode.
The cluster 0100000011 (2) uses the read request communication (4-
According to 4), additional request communication is performed to the cluster 1001000001 (2). The cluster 1001000001 (2) receives the add request communication and enters the pointer mode. This state is shown in FIG.

(6) In the state of (5), the cluster 10010000
When the memory block M is invalidated by the cache replacement at 01 (2), the cache replacement operation (2-
According to 2), cluster 1001000001 (2) becomes cluster 10110.
[0010] Pointer movement request communication is performed with respect to (2). Also,
The replacement cluster number is set to 1011000010 (2) for the cluster 0100000011 (2), and the deletion request communication is performed.

The cluster 1011000010 (2) receives the pointer movement request communication and maintains the state of the termination mode according to the pointer movement request communication (3-3). Cluster 0100000011
(2) receives the delete request communication, and delete request communication (2-1)
In accordance with this, the cluster 1011000010 (2) is registered in the pointer field. This state is shown in FIG.

(7) In the state of (6), the cluster 01000000
When the write request communication of the memory block M arrives at 11 (2), the cluster 01000000 11 (2) becomes the cluster 010000000.
The invalidation request communication is transmitted to 1 (2) and the cluster 1011000010 (2). Upon receiving the invalidation request communication, the cluster 0100000001 (2) transmits the invalidation request communication to the cluster 0100000100 (2). All the clusters that have received the invalidation request communication invalidate the cache line that is caching the memory block M.

According to the above embodiment, the main memory directory 107 and the cache directory 109 are provided with the pointer mode 160, the bitmap mode 170, and the termination mode 180, and the communication described above is provided, so that the configuration shown in FIG. It is possible to manage all clusters that cache memory blocks in a tree structure.

(Second Embodiment) A second embodiment of the present invention will be described. Since the present embodiment is a modification of the first embodiment, only differences will be described. In this embodiment, by changing the operations of the main memory control circuit 105 and the cache control circuit 104 in the cluster 100, all the cache lines that cache the memory block 106 to be written at the time of writing to the main memory device 103. The difference is that the completion of the invalidation of 108 is guaranteed.

In this embodiment, the operations of the write request communication and the invalidation request communication of the first embodiment are changed, and write reply communication and invalidation reply communication are provided for them.

(Changes in Write Request Communication) In the write request communication of this embodiment, the following is added after the processes (1) to (3) of the first embodiment. (4) Wait for receipt of invalidation reply communication from all clusters registered in the cluster storage field 150. (5) A write response communication is transmitted to the issuer of the write request communication.

(Changes of Invalidation Request Communication) In the invalidation request communication of this embodiment, (3) and (4) of the first embodiment are changed to the following processing.

(3) Either (3-1) or (3-2) is performed depending on the mode of the read cache directory 109. (3-1) The read cache directory 109 is the pointer mode 160 or the bitmap mode 170.
If so, the cache control circuit 104 transmits the invalidation request communication through the network interface circuit 112 to all the clusters registered in the cluster storage field 150. (3-2) When the read cache directory 109 is in the termination mode 180, the cache control circuit 1
09 invalidates the cache line 108 and sends the invalidation reply communication to the invalidation request communication issuer. (4) The following processing is not performed.

(4) Wait for receipt of invalidation reply communication from all clusters registered in the cluster storage field. (5) The cache line is invalidated, and the invalidation reply communication is transmitted to the invalidation request communication issuer.

According to this embodiment, writing can be reflected after the cache lines of all clusters are invalidated.

Example 3 Example 3 of the present invention will be described. Since this embodiment is a modification of the first embodiment, only different points will be described. The present embodiment is different in that the write / update cache coincidence control is realized by changing the operations of the main memory control circuit 105 and the cache control circuit 104 in the cluster 100. The write update type means that when a write is performed on a certain memory block, the write data is reflected on all the cache lines that are caching that memory block.

In this embodiment, the operation of the write request communication of the first embodiment is changed to provide the update request communication.

(Changes in Write Request Communication) The write request communication of this embodiment is the same as (3-1) and (3) of the first embodiment.
-2) is changed to the following processing.

(3-1) Accessed memory block 1
06 main memory directory 107 is pointer mode 16
If it is 0 or the bitmap mode, the update request communication is transmitted to all the clusters registered in the cluster storage field 150. (3-2) When the main storage directory 107 of the accessed memory block 106 is in the termination mode 180,
Since the memory block 106 is not cached anywhere, cache coherency control is not performed.

(Update Request Communication) Update request communication with the cluster 100 is performed by transmitting an update request communication packet consisting of data to be updated by writing and its address. The process when the update request communication packet is received is as follows.

(1) The network interface circuit 112 makes a cache write request to the cache control circuit 104 using the address of the update request communication packet. (2) The cache control circuit 104 uses the address of the cache write request to write to the cache memory 102.
Of the cache directory 10 while writing to the cache line 102 corresponding to the address.
9 and the mode field 110 are read.

(3-1) When the accessed cache directory 104 is in the pointer mode 160 or the bitmap mode 170, the update request communication is transmitted to all the clusters registered in the cluster storage field 150. (3-2) Accessed cache directory 109
Is the termination mode 180, the memory block 10
Since 6 is not cached in the layers below, cache coherency control is not performed.

According to the present embodiment, the write / update type cache coincidence control can be realized by the directory system of the present invention.

Also in this embodiment, the above-mentioned embodiment 2 is used.
Similarly, it is possible to guarantee the completion of the update of the caches of all clusters caching the memory block in response to the write request. Specifically, the cluster that issued the update request communication waits until it receives the update response communication, and when it receives the update response communication from all of the update request destination clusters, the cluster that issued the update request to its own cluster. Issue an update reply communication to.
Then, in the write target cluster, when the update response communication is received from all the clusters indicated by the main memory directory corresponding to the write target memory block, the update response communication is transmitted to the cluster that issued the write, and the write is sent. The cluster that issued the command waits until it receives the update response communication, thereby guaranteeing the completion of writing to all clusters.

(Fourth Embodiment) A fourth embodiment of the present invention will be described. FIG. 12 shows the configuration of the parallel computer according to this embodiment. Since this embodiment is a modification of the first embodiment, only different points will be described. In addition, in FIG. 12, a device common to FIG.
It is assumed that the circuits are represented by the same numbers with -1 added thereto, and the parts to which the description of the first embodiment is applied are replaced with the numbers with -1 added.

In the first embodiment, the cache directory 109, the main storage directory 107, and the mode field 110 are provided for each block of the cache memory 102 and the main storage device 103 in the cluster 100. On the other hand, in the fourth embodiment, the cache directory 10
By providing the 9-1 and the main memory directory 107-1 separately, the difference is that the amount of these memories is reduced.

In FIG. 12, the cache directory 109-1 holds the cache line numbers of cache lines other than the termination mode in the cache line 108-1, the cache directory 109-1, and the mode field 110-1. The main memory directory 107-1 holds the memory block numbers of the memory blocks other than the termination mode in the memory block 106-1, the main memory directory 107-1, and the mode field 110-1.

In this embodiment, the read request communication, the write request communication, the addition request communication, the deletion request communication of the first embodiment,
The following changes and additions are made to the bitmap move request communication and pointer move request communication.

(Changes in Read Request Communication) (2) The main memory control circuit 105-1 uses the destination intra-cluster address 403a to transfer from the main memory 103-1 to the memory block 106-1 corresponding to the address. Read out. Further, if there is a main memory directory 107-1 and a mode field 110-1 corresponding to the memory block 106-1, they are read.

(4-1) Accessed memory block 1
If there is no main memory directory 107-1 and mode field 110-1 corresponding to 06-1, it means that the memory block 106-1 is in the termination mode. In this case, the main memory control circuit 105-1 allocates the empty entry of the main memory directory 107-1 to the memory block 106-1. Main memory directory 1
If there is no empty entry in 07-1, the main memory control circuit 105-1 selects one entry from the main memory directory 107-1 and replaces it. After this,
This entry is used as the main storage directory 107-1 corresponding to the main storage memory block 106-1 read by the read request communication. The subsequent processing is the same as in the first embodiment.
It is the same as (4-1).

(Changes in Write Request Communication) (3-1) The same processing as (3-1) in the first embodiment is performed. However, “the main storage directory 107 is set to the end mode 1
Change to 80 "instead of the main memory control circuit 105-1
Changes the entry of the main memory directory 107-1 corresponding to the memory block 106-1 of the main memory which has received the write request communication, to an empty entry.

(Changes in Addition Request Communication) (1) The cache directory 109-1 is read in the same manner as in (1) of the first embodiment. However, in the fourth embodiment, there may be no cache directory.

(2-1) When there is no cache directory, it means that the cache line 108-1 is in the termination mode. In this case, the cache control circuit 104-1 allocates the empty entry of the cache directory 109-1 to the cache line 108-1. If there is no empty entry in the cache directory 109-1, the cache control circuit 104-
1 selects one entry from the cache directory 109-1 and replaces it. After this, this entry is changed to the cache line 1 corresponding to the addition request communication.
It is used as the cache directory 109-1 of 08-1. Subsequent processing is the same as (2-1) of the first embodiment.

(Changes in Delete Request Communication) (2-1) The same processing as (2-1) in the first embodiment is performed. However, when the number of cluster numbers registered in the cluster storage field 150 becomes 0, the directory is changed to an empty entry instead of changing the mode of the directory to the termination mode.

(2-3) The same processing as (2-3) of the first embodiment is performed. However, when all the bitmaps held in the cluster storage field 150 become 0, the directory is changed to an empty entry instead of changing the mode of the directory to the termination mode.

(Changes in Bitmap Move Request Communication) (2) Same processing as (2) in the first embodiment, except that the move target cluster number 402f and move target cluster address of the bitmap move request communication packet 425 are the same. 403
Since the cache line 108-1 in which the memory block 106-1 indicated by f is cached is in the termination mode, there is no cache directory 109-1 corresponding to the cache line 108-1.
Therefore, the cache control circuit 104-1 that has received the bitmap move request communication causes the cache directory 10
9-1 empty entry to the relevant cache line 108-
Assign to 1. If there is no empty entry in the cache directory 109-1, the cache control circuit 104-1 selects one entry from the cache directory 109-1 and replaces it. Thereafter, this entry is added to the cache directory 1 of the cache line 108-1 corresponding to the bitmap move request communication.
Used as 09-1.

(3) The same processing as (3) in the first embodiment is performed, but when all the bits of the bitmap are 0, instead of changing to the termination mode, the cache directory 109-1 is set to a free entry. Change to.

(Changes in Pointer Move Request Communication) (2) The same processing as (2) in the first embodiment, but there is a case where the cache directory 109-1 corresponding to the cache line 108-1 does not exist, That is the case in termination mode. If it is the termination mode, proceed to (3-3).

(3-3) The cache control circuit 104-1 having received the pointer movement request communication allocates a free entry in the cache directory 109-1 to the cache line 108-1. Cache directory 1
If there is no empty entry in 09-1, the cache control circuit 104-1 determines that the cache directory 10
One entry is selected from 9-1 and replaced.
After that, this entry is used as the cache directory 109-1 of the cache line 108-1 corresponding to the pointer movement request communication.

According to the present embodiment, the cache directory and the main memory directories, which are smaller in number than the cache lines and main memory blocks, are used in the first embodiment.
It is possible to realize an inter-processor data consistency assurance device similar to.

(Fifth Embodiment) A fifth embodiment of the present invention will be described. FIG. 13 shows the configuration of the parallel computer according to this embodiment. Since this embodiment is a modification of the fourth embodiment, only different points will be described. Note that, in FIG. 13, the devices and circuits common to those in FIG. 12 are represented by changing the number -1 to -2, and the numbers in the description of the fourth embodiment will be replaced.

The present embodiment is different in that the cache directory 109-2 and the main storage directory 107-2 in the cluster 100-2 are provided on the cache memory 102-2 and the main storage device 103-2.

According to this embodiment, the same inter-processor data consistency guarantee device as that of the first embodiment can be realized without preparing a memory for storing a directory in addition to the cache memory and the main storage device.

(Embodiment 6) Next, a sixth embodiment of the present invention will be described. Since this embodiment is a modification of the first embodiment, only different points will be described. The present embodiment is different in that the processors in the cluster share part of the cache coherency control processing. That is, the present embodiment is characterized in that the changing process when the operation mode of the main memory directory 107 or the cache directory 109 is changed from the pointer mode to the bitmap mode is executed by the specific processor 101 in the cluster 100. However, the present invention can be used regardless of how the processing is shared between the main memory control circuit and the cache control circuit and the processor.

In this embodiment, the read request communication and the addition request communication of the first embodiment are changed as follows.

(Changes in Read Request Communication) (4-3) Network Interface Circuit 112
Receives a processor processing request notification from the main memory control circuit 105 and interrupts the processor 101 that controls the directory. The processor 101 reads the main memory directory 107 of the memory block 106 to be accessed, changes the mode field 110 of the main memory directory 107 to the bitmap mode 170, and is registered in the entry of the cluster memory field 150 in the pointer mode 160. The bit corresponding to the cluster number is set to 1.

(Changes in Addition Request Communication) (2-3) Network Interface Circuit 112
Receives the processor processing request notification from the cache control circuit 104 and performs the directory control on the processor 10.
Interrupt 1 The processor 101 reads the cache directory 109 of the cache line 108 to be accessed, changes the mode field 110 of the cache directory 109 to the bitmap mode 170, and sets all the cluster numbers registered in the cluster storage field 150 in the pointer mode 160. After setting the bit corresponding to 1 to 1, the bit corresponding to the cluster number to be added is set to 1.

According to the present embodiment, the change processing required when shifting from the pointer mode to the directory mode can be performed by the software on the processor 101, and the hardware amounts of the cache control device and the main storage control device can be reduced and implemented. An inter-processor data consistency guarantee device similar to that of Example 1 can be realized.

[0190]

According to the present invention, a tree directory system is provided without providing a special control circuit dedicated to directory management for managing the caching states of all clusters in the lower hierarchy, or allowing a specific cluster to perform directory management. It is possible to realize a cache directory system with a lower hierarchy compared to In other words, in a parallel computer consisting of a large number of processors, only a directory of the same size as the tree directory method or the hierarchical full map directory method is used, and the hierarchy is lower than the tree directory method. An inter-processor data consistency assurance device using a directory management method that does not concentrate access to a specific directory management node is provided.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a parallel computer according to an embodiment of the present invention.

FIG. 2 is a diagram showing a method of managing cache lines using a tree-structured directory.

FIG. 3 is a configuration diagram of a directory in this embodiment.

FIG. 4 is a configuration diagram of cluster numbers in the present embodiment.

FIG. 5 is a configuration diagram of a packet in this embodiment.

FIG. 6 is a diagram showing directory connection according to an operation example (1) of the first embodiment of the present invention.

FIG. 7 is a diagram showing directory connection according to an operation example (2) of the first embodiment of the present invention.

FIG. 8 is a diagram showing directory connection according to an operation example (3) of the first exemplary embodiment of the present invention.

FIG. 9 is a diagram showing directory connection according to an operation example (4) of the first embodiment of the present invention.

FIG. 10 is a diagram showing directory connection according to an operation example (5) of the first exemplary embodiment of the present invention.

FIG. 11 is a diagram showing directory connection according to the operation example (6) of the first embodiment of the present invention.

FIG. 12 is a configuration diagram of a parallel computer according to a fourth embodiment of the present invention.

FIG. 13 is a configuration diagram of a parallel computer according to a fifth embodiment of the present invention.

FIG. 14 is a diagram showing an operation of a conventional full map directory system.

[Explanation of symbols]

10a to 10d ... Processor, 12a to 12d ... Cache, 20 ... Network, 30 ... Main storage device, 31 ...
Memory block, 32 ... Directory, 100, 100
-1 to 2 ... Cluster, 101, 101-1 to 2 ... Processor, 102, 102-1 to 2 ... Cache memory, 1
03, 103-1 to 2 ... Main storage device, 104, 104-
1-2: cache control circuit 105, 105-1-2
... Main memory control circuit, 106, 106-1 and 2 ... Memory block, 107, 107-1 and 2 ... Main memory directory, 108, 108-1 and 2 ... Cache line, 10
9, 109-1 and 2 ... Cache directory, 11
0, 110-1 to 2 ... Mode field, 111, 11
1-1 and 2 ... bus, 112, 112-1 and 2 ... network interface circuit, 200 and 200-1 and 2 ...
Network, 110a to 110c ... Mode field, 150 ... Cluster storage field, 161-0 to 1
61-3 ... Cluster number, 162-0 to 162-3 ... Cluster number existence flag, 160 ... Pointer mode directory, 170 ... Bitmap mode directory, 180 ... Termination mode directory, 171 ... Bitmap, 301 ... Main Storage directory, 321, 32
2, 331 to 334, 341 to 345 ... Cache directory, 350 ... Pointer field, 351 ... Bitmap field, 400a to 400h ... Packet transmission destination cluster number, 401a to 401h ... Packet transmission source cluster number, 402c to 402h ... Target cluster No., 403a to 403h ... In-cluster address, 404
... write data, 405 ... additional cluster number, 406
... Bitmap, 407 ... Pointer, 408 ... Reply data, 409 ... Replacement cluster number, 420 ... Read request communication packet, 421 ... Write request communication packet, 422 ... Invalidation request communication packet, 423 ... Addition request communication packet, 424 ... Delete request communication packet, 42
5 ... Bitmap move request communication packet, 426 ... Pointer move request communication packet, 427 ... Read response communication packet.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hiroaki Fujii 1-280, Higashi Koikekubo, Kokubunji, Tokyo Inside the Central Research Laboratory, Hitachi, Ltd.

Claims (11)

[Claims] 1. A plurality of clusters each including one or more processors, a main storage device, and a cache memory are provided, and a network connecting the plurality of clusters is provided. A data consistency assurance device between processors of a parallel computer capable of caching data in a main memory device in another cluster in a cache memory in its own cluster in block units, wherein the main memory device in the cluster is for each block. A main memory directory is provided for recording a plurality of clusters that are caching the main memory block, and the cache memory in the cluster stores the same main memory block that is cached in the cache line for each cache line. A cache for recording multiple cached clusters. The main memory directory and the cache directory, one of two or more types of modes for recording clusters caching the main memory block is provided for each main memory block or cache line. A cluster that is selectable and caches the main memory block in the selected mode is recorded, and a mode field indicating the mode is provided, and the cluster has access to the main memory device from its own cluster and other clusters. At times, a main memory directory for controlling the main memory directory in the local cluster and a mode field of the main memory directory and issuing a cache control communication via the network to a plurality of clusters recorded in the main memory directory. A memory control circuit, The raster controls the cache directory in the self-cluster and the mode field of the cache directory when receiving the cache control communication from the self-cluster and the other clusters, and sets the network to the plurality of clusters recorded in the cache directory. An inter-processor data consistency guarantee device for a parallel computer, which is provided with a cache control circuit for issuing a cache control communication via the cache. 2. The main memory directory records a plurality of clusters in which corresponding main memory blocks are cached, and in each cluster recorded in the main memory directory,
The cache directory corresponding to the cache line records a plurality of other clusters that cache the same main memory block, and the cache directory corresponding to the cache line in each cluster recorded in the cache directory has the same main memory. By recording multiple other clusters that cached blocks, a tree-structured directory chain with the main storage directory as the root node and the cache directory as an intermediate node or leaf node was created, and the same main storage block was cached. By representing all clusters in the tree structure,
2. The cache state of a main memory block is managed.
The data consistency guarantee device between processors of the parallel computer described in. 3. A pointer mode is provided as one of the modes for recording the cached cluster, and when the main memory directory or the cache directory is in the pointer mode, a plurality of cluster number parts are provided in the directory. , A cluster number existing flag for indicating the validity or invalidity of the cluster number portion is provided corresponding to each cluster number portion, and the mode field of the directory is set to the pointer mode,
3. The interprocessor data consistency assurance device of a parallel computer according to claim 1, wherein the cluster number part in which the cluster number existence flag is valid points to the cached cluster. 4. All the clusters of the parallel computer are divided into cluster groups consisting of one or more clusters in advance, and a bitmap mode is provided as one of the modes for recording the cached clusters, and the main memory directory or When the cache directory is in the bitmap mode, the directory is provided with a bitmap in which each bit indicates whether each cluster in the same cluster group is valid or invalid. 3. The inter-processor data consistency guarantee device for a parallel computer according to claim 1, wherein the cluster that is being cached is pointed to. 5. When writing to a main memory device on its own cluster or another cluster, the main memory control circuits of the write target clusters are all clusters designated by the main memory directory of the main memory block of the write target. The cache control circuit of the cluster that issued the invalidation request to
All clusters that have cached the same main memory block by invalidating the cache line corresponding to the main memory block and issuing invalidation requests to all clusters pointed to by the corresponding cache directory. The data consistency guarantee device between processors of a parallel computer according to claim 1 or 2, which invalidates a corresponding cache line in the. 6. A main memory control circuit of a write target cluster, when writing to a main memory device on its own cluster or another cluster, all clusters pointed to by a main memory directory of a main memory block to be written. To the main memory block, the cache control circuit of the cluster that receives the write data copies the write data to the cache line corresponding to the main memory block, and The corresponding cache line in all the clusters that have cached the same main memory block is updated by transmitting the write data to all the clusters designated by the directory.
Alternatively, the data consistency guarantee device between processors of the parallel computer according to claim 2. 7. When writing to a main storage device on its own cluster or another cluster, the main storage control circuit of the write target cluster is configured to store all clusters pointed to by the main storage directory of the main storage block to be written. The cache control circuit of the cluster that issued the invalidation request to
The cache line corresponding to the main memory block is invalidated, the invalidation request is issued to all the clusters indicated by the corresponding cache directory, and all the clusters that are the counterparts of the invalidation request is issued. When the invalidation completion communication is received, the invalidation completion communication is issued to the cluster that issued the invalidation request to the own cluster, and the main memory control circuit of the write target cluster is the main memory block corresponding to the write target main memory block. When the invalidation completion communication is received from all the clusters indicated by the storage directory, the invalidation completion communication is sent to the cluster that issued the write, and the cluster that issued the write waits until the invalidation completion communication is received. According to claim 5, it is possible to guarantee the completion of writing to all clusters. Interprocessor data consistency assurance device of the computer. 8. When writing to a main storage device on its own cluster or another cluster, the main storage control circuit of the write target cluster is all pointed to by the main storage directory corresponding to the write target main storage block. The write data written in the main memory block to the cluster, and the cache control circuit of the cluster receiving the write data copies the write data to the cache line corresponding to the main memory block and Send an update request containing the write data to all clusters pointed to by the cache directory,
When the update response communication is received from all the clusters that are the destinations of the update request, the update response communication is issued to the cluster that has issued the update request to the own cluster, and the main memory control circuit of the write target cluster writes When the update response communication is received from all the clusters pointed to by the main memory directory corresponding to the target main memory block, the update response communication is sent to the cluster that issued the write, and the cluster that issued the write sends the update response 7. The inter-processor data consistency guarantee device for a parallel computer according to claim 6, wherein the completion of writing to all clusters can be guaranteed by waiting until communication is received. 9. A directory memory area is prepared for each of the main memory block or the cache line instead of having a directory for each block, and the directory memory area is provided only when there is a cluster to be pointed to from the block. 9. The inter-processor data consistency guarantee device for a parallel computer according to claim 1, wherein the directory of the block is registered. 10. When accessing a main memory from its own cluster or another cluster, one of the processors in the cluster controls a main memory directory and a mode field of the main memory directory in place of the main memory control circuit. The parallel processing according to any one of claims 1 to 9, wherein a part or all of a process of issuing a cache control communication to the plurality of clusters recorded in the main memory directory via the network is performed. Data consistency assurance device between processors of computer. 11. When a cache control communication is received from a self cluster or another cluster, one of the processors in the cluster controls a cache directory and a mode field of the cache directory on behalf of the cache control circuit, and cache The processor of the parallel computer according to any one of claims 1 to 10, which performs a part or all of a process of issuing a cache control communication via the network to a plurality of clusters recorded in a directory. Inter-data consistency assurance device.