JP2005242500A - Read request arbitration control system and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an NUMA distributed shared memory parallel computer system for obtaining a read request arbitration control system for improving the effective performance of data transfer. <P>SOLUTION: Generally, it is not necessary to secure ordering between read requests in data transfer in an NUMA type distributed shared memory parallel computer. The access latency of the request is calculated from a request destination physical address, and ordering is changed so that the request whose access latency to the request destination physical address is much more longer can be given priority, and a read request arbitrating unit 10 for issuing the read request is installed on a data transfer path. Thus, the efficiency of the transfer of the read request can be attained, and the performance improvement of the data transfer efficiency can be realized as a result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a read request arbitration control system and method, and more particularly to a read request arbitration control system in a NUMA (Non Uniform Memory Access) type distributed shared memory parallel computer.

  As an example of a conventional shared memory system, there is a UMA (Uniform Memory Access) system as shown in FIG. In this system, it is assumed that the memory access latency is the same without depending on the physical address, and the reply of the read request to the main storage unit 806 of each processor 801 to 804 is returned by the control unit 805 in the order of issue of the read request. It has come to be.

  Therefore, arbitration between read requests is not necessary, and as shown in FIG. 18, the read request 110 is registered in the read request registration table 102 in the read request arbitration unit 11, and the read requests are issued in the registered order. By adopting the so-called FIFO (First In First Out) method, no problem was caused.

  In recent years, there has been a NUMA (Non-Uniform Memory Access) system as shown in FIG. 19 because of cost advantages. Referring to the figure, this system includes a plurality of processor nodes 20 and 21 and an interprocessor connection device 40 for connecting these nodes. The processor nodes 20 and 21 have the same configuration. For example, the processor node 20 includes a plurality of processors 200a and 200b, a processor node connection unit 202, and a main storage unit 201.

  In this system, the memory access latency differs depending on the physical address. Therefore, in the conventional FIFO method in which read requests are issued in the order of requests, the efficiency of read request transfer deteriorates and the data transfer between nodes is effective. Performance decreases.

  Here, referring to Patent Document 1, a multiprocessor system for the purpose of equalizing the memory access time of each processor is disclosed, and an arbitration unit that arbitrates between requests is provided on a data transfer path. In this arbitration unit, it has the longest transmission time based on the accumulated transmission time (added to the request) from the request source (that is, the longest waiting time since it was issued by the request source) The request is issued with priority. In other words, this technique is designed to improve the performance of requests from a plurality of request sources to one request destination.

JP-A-2-294867 JP 2003-21689A

  In the NUMA type distributed shared memory parallel computer shown in FIG. 4 described above, the memory access latency differs depending on the physical address. Therefore, in the conventional FIFO method in which read requests are issued in the order of requests, read request transfer is performed. In addition to the problem that the effective performance of inter-node data transfer deteriorates, the existing systems of other companies are combined and some of them are developed in-house, so it is difficult to change to other companies' products. However, if it is not a general-purpose method, there is a problem that it is difficult to improve the performance only by the in-house developed part.

  Further, in the above-described technology of Patent Document 1, the performance of requests from a plurality of request sources to one request destination is improved, and the performance in the case of a request from one request source to a plurality of request destinations. There is a problem that improvement cannot be achieved. In this technique, it is necessary to add special priority information to the request. Therefore, in actual application of this technique, there is a disadvantage that a system change is required. Therefore, the NUMA type distributed shared memory parallel is required. There is a problem that it cannot be applied universally to computers.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a read request arbitration control system and method that can be applied universally to a NUMA type distributed shared memory parallel computer and can improve data transfer effective performance by improving data transfer efficiency. That is.

  A read request arbitration control system according to the present invention is a read request arbitration control system in a distributed shared memory parallel computer comprising a plurality of nodes having one or more processors and a main storage device, and a request destination address of a read request Control means for performing arbitration control of the read request in accordance with the access latency for.

  A read request arbitration control method according to the present invention is a read request arbitration control method in a distributed shared memory parallel computer that forms a system with a plurality of nodes having one or more processors and a main storage device, and a request destination address of a read request And a control step of performing arbitration control of the read request in accordance with the access latency with respect to.

  A program according to the present invention is a program for causing a computer to execute a read request arbitration control method in a distributed shared memory parallel computer that forms a system with a plurality of nodes having one or more processors and a main storage device. Including a process for performing arbitration control of the read request in accordance with the access latency for the request destination address.

  The operation of the present invention will be described. In general, in data transfer in a NUMA type distributed shared memory parallel computer, since it is not necessary to guarantee ordering between read requests, the access latency of the request is calculated from the request destination physical address, and the request destination physical A configuration is made such that a read request is issued by changing the ordering so that a request with a longer address latency is given priority. Thereby, the transfer of the read request is made efficient, and as a result, the performance of the data transfer efficiency can be improved.

  The first effect of the present invention is to give priority to a request having a long access latency to the request destination physical address in accordance with the access latency depending on the request physical address, which is a feature of data transfer in the NUMA type distributed shared memory parallel computer. As described above, since the read request is issued by changing the ordering, the read request transfer is made efficient and the data transfer effective performance can be improved.

  The second effect of the present invention is that, in general, the ordering is changed between read requests that do not require ordering guarantee in the data transfer in the computer. Therefore, the NUMA type distributed shared memory parallelism is not particularly limited. It can be applied to computers, and the data transfer effective performance can be improved.

  The third effect of the present invention is that the read request arbitration unit can be provided at any location where arbitration between read requests can be performed on the data transfer path, so that the processor node of the NUMA type distributed shared memory parallel computer, I This means that the data transfer effective performance can be improved by providing it at any location such as in the / O node or the inter-node connection device.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the present invention. FIG. 1 shows a NUMA type distributed shared memory parallel computer as an embodiment of the present invention. As shown in the figure, the NUMA type distributed shared memory parallel computer includes a plurality of processor nodes 20 to 23, a plurality of I / O nodes 30 to 33, and an inter-node connection device 40 for connecting these nodes. In this example, the number of processor nodes and the number of I / O nodes are 4 respectively, but in practice, any number can be realized.

  Referring to FIG. 2, the detailed structure of each of the blocks of FIG. 1 is shown. The processor node 20 includes one or more processors 200 a to 200 b, a main storage unit 201, and a processor node control unit 202, and the processor 200 and the processor node control unit 202 are connected by a system bus 210. The I / O node 30 includes an I / O node control unit 300.

  The inter-node connection device 40 includes an inter-node connection control unit 400. Furthermore, in this embodiment, the read request arbitration unit 10 according to the present invention is connected on the data transfer path, that is, between the processor node 20 and the internode connection apparatus, and between the I / O node control 30 and the internode connection apparatus 40. Is done. Since the read request arbitration unit 10 is provided on the data transfer path, it may be provided in each node as in the present embodiment, or may be provided in the inter-node connection unit 400 as shown in FIG. It is also possible to provide for both.

  The distributed shared memory is distributed in the main storage unit 201 of each processor node 20-23. As shown in FIG. 4, the first example of the distributed arrangement is a system in which the main memory of each processor node is stacked to form the main memory space of the system, that is, the processor node is designated by the upper bits of the memory address. .

  In this embodiment, from the address 000 (hexadecimal) of the main storage space, the 3FF address is the processor node 20, the 400-7FF address is the processor node 21, the 800-BFF address is the processor node 22, and the last FFF address is C00. Are distributed to the processor nodes 23, and the processor nodes are designated by the upper 2 bits of the address.

  A second example of the distributed arrangement is disclosed in Patent Document 2, that is, as shown in FIG. 5, each of the processor nodes 20 to 23 includes an inter-node interleave control unit, and between the main storage units of each processor node. There is a method of performing interleaving, that is, a method of specifying a node by a bit in the middle of an address.

  In this example, main memory space address 000 (hexadecimal) to address 0FF, address 400 to 4FF, address 800 to 8FF, address C00 to CFF are the processor node 20, address 100 to 1FF and address 500 to 5FF. Addresses 900 to 9FF, D00 to DFF are assigned to the processor node 21, addresses 200 to 2FF, 600 to 6FF, A00 to AFF, E00 to EFF are assigned to the processor node 22, and addresses 300 to 3FF are assigned to 700. To 7FF, B00 to BFF, and F00 to the last FFF are interleaved and distributed in the processor node 23 in units of 256 bytes, and the processor node is specified by 2 bits above the lower 8 bits of the address.

  Referring to FIG. 6, there is shown a detailed configuration of the read request arbitration unit 10 according to the present invention shown in FIG. In the figure, the latency search table for each physical address 101 stores in advance the memory access latency for each physical address, that is, the time from when a request is issued until a reply is returned, and when the request physical address 111 is input. The access latency 112 is output.

  In this example, for example, as shown in FIG. 4, when the main storage space is distributed in each processor node, as shown in FIG. 7, the latency search table 101 by physical address holds the access latency. An entry 120 and a selection circuit 121 are included. At this time, one entry is required for each processor node, for a total of four entries. For example, the access latencies for each of the processor nodes 20 to 23 are held in advance as 200 nS, 400 nS, 600 nS, and 800 nS, respectively. The selection circuit 121 selects an entry with the upper 2 bits of the 11-bit request destination physical address 111, that is, the bits 11 and 10.

  Also, for example, as shown in FIG. 5, even when the main storage space is interleaved and distributed in each processor node, as shown in FIG. 8, the latency search table 101 by physical address shows the access latency. An entry 120 to be held and a selection circuit 122 are included. At this time as well, one entry is required for each processor node, for a total of four entries. For example, the access latencies of the processor nodes 20 to 23 are previously held as 200 nS, 400 nS, 600 nS, and 800 nS, respectively. In addition, the selection circuit 122 selects an entry with the upper 2 bits, that is, the bit 9 and the bit 8 from the lower 8 bits of the 11-bit request destination physical address 111.

  As shown in detail in FIG. 9, the read request registration table 102 includes a plurality of entries, and one entry includes read request information of a normal read request 110 such as a request destination physical address and a request data length. In addition, the access latency 112 obtained from the request destination physical address 111 by the latency search table 101 by physical address and the registration time 113 output from the timer 103 when registering the read request are stored.

  As shown in detail in FIG. 10, the arbitrating unit 100 waits for registration after subtracting the request registration time 130 from the current time 115 indicated by the timer 103 for each entry in the read request registration table 102. 131 is calculated, and the access latency 132 is further added to calculate the latency 133 from the registration time, and the read request 116 having the longest latency 133 from the registration time is selected and issued. The timer 103 holds the current time.

  The configuration of the embodiment has been described in detail above. The NUMA type distributed shared memory parallel computer of FIG. 1, the processor node 20, the I / O node 30, the inter-node connection device 40 of FIG. 2, and the configurations of FIGS. The distributed arrangement method of the distributed shared memory is well known to those skilled in the art and is not directly related to the present invention, and thus the detailed configuration is omitted.

  Next, the operation of the read request arbitration unit 10 shown in FIG. 6 will be described. The read request arbitration unit 10 performs a registration operation from the generation of the read request 110 to the entry of the read request registration table 102, and arbitration between the read requests registered in the read request registration table 102 by the arbitration unit 100. 116 is issued.

  First, the registration operation from the generation of the read request 110 to the entry of the read request registration table 102 will be described with reference to FIG. In the figure, when a read request 110 is given to the read request arbitration unit 10, read request information such as a request destination physical address 111 and a request data length of the read request 110 is registered in an empty entry of the read request registration table 102 ( FIG. 11 (a)).

  Further, the time 113 at this time is obtained from the timer 103 and registered in the same entry as the registration time (FIG. 11B). Furthermore, the request destination physical address 111 of the read request 110 is given to the latency search table 101 by physical address (FIG. 14C), and the access latency 112 to the physical address, that is, a reply is returned after issuing the read request. Since the time until it comes is output, it is registered in the same entry as the access latency (FIG. 11 (d)).

  When there is no empty entry in the read request registration table 102, flow control or the like is performed on the request source. This is well known to those skilled in the art and is not directly related to the present invention. Is omitted.

  Next, with reference to FIG. 10, description will be given of an operation in which the arbitration unit 100 performs arbitration between read requests registered in the read request registration table 102 and issues a request 116. In FIG. 10, when the read request 116 can be issued due to the result of flow control or the like, the request registration time 130 is subtracted from the current time 115 obtained from the timer 103 for the entry registered in the read request registration table 102. Then, the waiting time 131 from the registration of the request to the current time is calculated (FIG. 10A).

  Next, the access latency 132 is added to the waiting time 131 to obtain a latency 133 from the registration time until the read reply is received (FIG. 10B). The arbitration unit 100 calculates the latency 133 from these registration times for all entries other than the empty entry, selects the request with the longest latency, and issues it as a read request 116 (FIG. 10C). The issued read request 116 is deleted from the occupied entry.

  Further, the operation of the embodiment will be described using a specific example. In the I / O node 30 shown in FIG. 3, a read request of 4 Kbytes is generated in the I / O node control unit 300, and the read divided into 256 bytes from the I / O node control unit 300 to the arbitration unit 10 Assume that 16 requests are issued. It is assumed that a request is issued once every 25 nS from the arbitration unit 10 by flow control.

  As shown in FIG. 4, when the main storage space is distributed to each processor node, in the conventional FIFO method in which requests are issued in the order of requests, read requests are sent at intervals of 25 nS after 0 nS, as shown in FIG. The reply of the last request issued is returned after 1275 nS.

  On the other hand, when the arbitration unit 10 according to the present invention having the latency search table by physical address shown in FIG. 7 is provided, as shown in FIG. 13, ordering is performed so that a request with a long access latency to the request physical address is prioritized. Since the read request is issued, the read request is issued at intervals of 25 nS from 0 nS, and the reply of the last request is returned after 875 nS. That is, it can be seen that the reply is returned 400 nS earlier than before, the read request transfer is made more efficient, and the effective data transfer performance is improved.

  As shown in FIG. 5, even when the main storage space is distributed in an interleaved manner in each processor node, in the conventional FIFO method in which requests are issued in the order of requests, as shown in FIG. A read request is issued, and the reply of the last request is returned after 1275 nS.

  On the other hand, when the arbitration unit 10 according to the present invention having the latency search table by physical address shown in FIG. 8 is provided, as shown in FIG. 15, ordering is performed so that a request with a long access latency to the request physical address is prioritized. Since the read request is issued, the read request is issued at intervals of 25 nS from 0 nS, and the reply of the last request is returned after 875 nS. That is, it can be seen that the reply is returned 400 nS earlier than before, the read request transfer is made more efficient, and the effective data transfer performance is improved.

  As another embodiment of the present invention, the basic configuration is as described above. With regard to the latency search table 101 for each physical address, in the previous embodiment, the access latency depending on the physical address set in advance is used. Is further devised so that it is dynamically set based on the actual read request and its reply.

  The configuration is shown in FIG. In this figure, when the read request 116 is issued by the arbitration unit 100, the access destination physical address and the request issue time 140 obtained from the timer 103 are registered in the latency calculation unit 104 (FIG. 16A). When the read reply 117 for the read request 116 is returned, the reply arrival time 141 obtained from the timer 103 is registered (FIG. 16B). Further, the access latency 118 is calculated by subtracting the request issue time 140 from the reply arrival time 141 (FIG. 16C). Finally, the access latency corresponding to the physical address 119 in the latency search table 101 by physical address is updated with the access latency 118 (FIG. 16 (d)).

  As described above, in this example, the access latency depending on the physical address set in advance in the previous embodiment is dynamically updated based on the actual read request and its reply. There is no need for initialization, and the data transfer is dynamically optimized according to the dynamic configuration change of the NUMA-type distributed shared memory parallel computer and the load of the execution job, and the data transfer effective performance is improved. It is done.

  In this configuration, a plurality of latency calculation means 104 are provided, and the access latency can be dynamically set for a plurality of read requests. Further, the latency search table 101 for each physical address may be updated by calculating the average latency of not only a single request but also a plurality of requests.

  It is obvious that the operation of each of the above-described embodiments can be configured such that the operation procedure is recorded in advance on a recording medium such as a ROM and is read and executed by a computer.

It is a block diagram of a NUMA type distributed shared memory parallel computer system to which the present invention is applied. It is a figure which shows the specific example of each part of FIG. It is a figure which shows the modification of FIG. It is a figure which shows the example which stacked the main memory of each processor node, and formed the main memory space of the system. It is a figure which shows the example which formed the main memory space of the system by interleaving between the main memories of each processor node. It is a figure which shows the detail of an example of the read request arbitration unit 10 of FIG. It is a figure explaining an example of the latency search table 101 according to physical address. It is a figure explaining the other example of the latency search table 101 by physical address. 5 is a diagram for explaining an example of a read request registration table 102. FIG. 3 is a diagram for explaining an example of arbitration means 100. FIG. FIG. 6 is a diagram for explaining the operation of the read request arbitration unit 10. It is a figure which shows the time relationship between the read request by the FIFO method of the conventional distributed arrangement method, and a read reply. It is a figure which shows the time relationship between the read request and read reply by the distributed arrangement system of this invention. It is a figure which shows the time relationship between the read request and read reply by the FIFO method of the conventional interleave distributed arrangement method. It is a figure which shows the time relationship between the read request and read reply by the interleave distributed arrangement system of this invention. It is a figure which shows the detail of the other example of the read request arbitration unit 10 of FIG. It is a figure which shows the example of the conventional UMA system. It is a figure which shows the example of issue of the read request by the conventional FIFO system. It is a figure which shows the example of the conventional NUMA system.

Explanation of symbols

10 Node request arbitration unit
20-23 processor node
30-33 I / O node
40 Internode connection device
100 Mediation means
101 Physical address latency search table
102 Read request registration table
103 timer 200a, 200b processor
201 Main memory unit
202 processor node control unit
300 I / O node control unit

Claims (9)

  A read request arbitration control system in a distributed shared memory parallel computer comprising a plurality of nodes having one or more processors and a main storage device, the read request depending on an access latency for a request destination address of the read request A read request arbitration control system, comprising control means for performing arbitration control.   2. The read request arbitration control system according to claim 1, wherein the control means preferentially issues a read request having a longer access latency.   The control means stores a table in which the addresses of the read request destinations and latencies corresponding to the addresses are stored in advance, and refers to the table by the request destination in response to the occurrence of the read request. The read request arbitration control system according to claim 1, further comprising: means for calculating   4. The read request arbitration control system according to claim 3, further comprising means for updating the latency of the table based on an actual response to the read request.   A read request arbitration control method in a distributed shared memory parallel computer comprising a plurality of nodes having one or more processors and a main storage device, the read request depending on an access latency for a request destination address of the read request A read request arbitration control method, comprising a control step for performing arbitration control.   6. The read request arbitration control method according to claim 5, wherein the control step preferentially issues a read request having a longer access latency.   In response to the generation of the read request, the control step calculates the latency by referring to a table in which each address of the read request destination and a latency corresponding to each address are stored in advance. 7. The read request arbitration control method according to claim 5 or 6, further comprising steps.   8. The read request arbitration control method according to claim 7, further comprising the step of updating the latency of the table based on an actual response to the read request.   A program for causing a computer to execute a read request arbitration control method in a distributed shared memory parallel computer comprising a plurality of nodes having one or more processors and a main storage device, and accessing a request destination address of a read request A computer-readable program comprising processing for arbitrating the read request according to latency.

Images