JP3569341B2 - Parallel computer system - Google Patents

Description

[0001]
[Industrial applications]
According to the present invention, in a computer system in which a plurality of processor elements and a control processor are connected by a network, I / O processing from a processor element having no I / O device via a control processor having an I / O device is performed. The present invention relates to a parallel computer system which can be executed at high speed and efficiently.
[0002]
[Prior art]
FIG. 8 is an explanatory diagram of the prior art.
In the parallel computer system shown in FIGS. 8A and 8C, a processor element PE [PE (0), PE (1),...] Is a processor that specializes in operations. The control processor CP is a device that manages the processor element PE, and is connected to an I / O device such as a storage device 80. Further, the control processor CP and each processor element PE are connected by a crossbar network XB.
[0003]
When each processor element PE refers to data stored in the storage device 80, since the storage device 80 is not connected to the processor element PE, an I / O request is issued to a server existing in the control processor CP. The server of the control processor CP that has received the I / O request executes I / O to the storage device 80 and transfers the read data to the requesting processor element PE via the crossbar network XB.
[0004]
[Problems to be solved by the invention]
In FIG. 8, it is assumed that the data transfer speed between the storage device 80 and the control processor CP is 800 Mbyte / sec, and the data transfer speed between the control processor CP and each processor element PE is 400 Mbyte / sec.
[0005]
Conventionally, as shown in FIG. 8A, a processor element PE (0) makes an I / O (1) request and an I / O (2) request, and the processor element PE (1) issues an I / O (2) request. When the request of O (3) and the request of I / O (4) are made in this order, the control processor CP processes these I / O requests one by one. Therefore, if it takes, for example, 6 seconds to transfer data from the storage device 80 to the control processor CP by one I / O process, as shown in FIG. 8B, all data from the storage device 80 to the control processor CP are transferred. In transfer, it took 6 seconds × 4 = 24 seconds.
[0006]
Further, as shown in FIG. 8C, when one read request is issued from the client of the processor element PE and the data transfer from the storage device 80 to the control processor CP takes, for example, 6 seconds, the I / O After the completion of the O processing, the I / O processing to the processor element PE of the I / O request source starts, so that it takes 12 seconds to transfer data from the control processor CP to the processor element PE, and one I / O request It took a total of 18 seconds until the process was completed, as shown in FIG.
[0007]
As described above, the conventional technology has a problem that it takes time to process an I / O request from the processor element PE.
In the control processor CP, various processes such as a process for managing the processor elements PE, a process related to operation, and a process related to a job are performed, and the number of the processor elements PE is, for example, 100 to 200. In such a case, there is a problem that I / O requests are concentrated on one control processor CP, and the load on the control processor CP may be extremely high.
[0008]
SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-mentioned problems, and to speed up I / O processing and reduce the load on a control processor by dispersing I / O processing and parallelizing I / O division. And
[0009]
[Means for Solving the Problems]
FIG. 1 shows a configuration example of the present invention.
Processor elements (PE) 10-0, 10-1,... Are processors that specialize in operations. A plurality of control processors to which an I / O device such as a storage device 31 is connected are provided, one of which is set as a master control processor (hereinafter, referred to as a master CP) 12 and the other is a slave control processor (hereinafter, a master control processor). Hereinafter, this is set as a slave CP) 26.
[0010]
The master CP 12, the slave CP 26, and the processor element PE are connected by a crossbar network (XB) 11.
An operation input device 13 is connected to the master CP 12 for processing related to management and operation of the processor element PE, processing related to jobs, and the like.
[0011]
The I / O division instruction input means 14 is a means for inputting the number of divisions for dividing an I / O request into a plurality of I / O execution units by a command from the input device 13 or the like. The I / O distribution instruction input means 15 is a means for inputting a command or the like from the input device 13 as to whether or not the I / O request is distributed to a plurality of control processors, or what percentage to distribute to each control processor. is there.
[0012]
The I / O request receiving means 16 is a means for receiving an I / O request from each processor element PE. The distribution determination processing unit 17 distributes the execution of the I / O request to a plurality of control processors based on the load status of the master CP 12 and the slave CP 26 and the distribution instruction information from the I / O distribution instruction input unit 15, or This is a means for deciding whether to perform the processing with a specific control processor.
[0013]
The I / O request transfer unit 20 is a unit that transfers the I / O request to the slave CP 26 when the I / O request is executed by the slave CP 26 according to the determination result of the distribution determination processing unit 17. The I / O processing result receiving means 21 is a means for receiving an I / O processing result when the slave CP 26 executes an I / O request.
[0014]
The I / O division processing unit 18 is a unit that divides an I / O request from the processor element PE into a plurality of I / O execution units according to an instruction from the I / O division instruction input unit 14. For the divided I / O execution units, the I / O parallel execution means 19 performs data input / output between the storage device 31 and the master CP 12, and communication between the master CP 12 and the processor element PE of the I / O request source. And data transfer in parallel.
[0015]
The I / O processing completion notifying means 22 sends the I / O processing result executed by the master CP 12, the I / O processing result executed by the slave CP 26, or both I / O processing results to the processor element of the I / O request source. This is a means for notifying the PE.
[0016]
The abnormality detecting unit 23 is a unit that detects an abnormality of the slave CP 26 by a timeout of the requested I / O or a predetermined operation control. The dispersion suppressing unit 24 is a unit that, when the abnormality detecting unit 23 detects an abnormality in the slave CP 26, issues an instruction to the distribution determination processing unit 17 so as not to allow the slave CP 26 to share the execution of the I / O request.
[0017]
The recovery processing means 25 is means for re-executing the I / O request being processed by the slave CP 26 when detecting an abnormality of the slave CP 26 which has been executing the I / O request in a shared manner.
[0018]
The I / O request receiving unit 27 in the slave CP 26 is a unit that receives an I / O request sent from the master CP 12 for dispersing the I / O. The I / O division processing unit 28 is a unit that divides an I / O request from the master CP 12 into a plurality of I / O execution units according to an instruction from the I / O division instruction input unit 14. The I / O parallel execution unit 29 performs, for the divided I / O execution units, data input / output between the storage device 31 and the slave CP 26 and data transfer from the slave CP 26 to the processor element PE or the master CP 12. It is a means to execute in parallel. The I / O processing result transfer unit 30 is a unit that transfers the I / O processing result executed by the slave CP 26 to the master CP 12 or the processor element PE that has issued the I / O request.
[0019]
In FIG. 1, there is one slave CP 26, but two or more slave CPs may exist.
[0020]
[Action]
In the present invention, the number of control processors is increased, and a slave CP 26 is provided in addition to the master CP 12. Then, the I / O requests from the respective processor elements PE are processed by being shared by a plurality of control processors. Here, this is called I / O decentralization.
[0021]
Focusing on data transfer between the storage device 31 and the control processor and between the control processor and the processor element PE, one I / O request issued from the processor element PE is divided into a plurality of I / O requests. This speeds up the total I / O processing. Here, this is referred to as split parallelization of I / O.
[0022]
By distributing the I / O and dividing and parallelizing the I / O as described above, it is possible to distribute the load of the control processor and improve the I / O performance. As a result, the throughput performance of the job and the peak performance of the I / O are improved, and the speed of the entire system can be increased.
[0023]
Since the master CP 12 receives all I / O requests from the processor element PE and distributes the requests to one or a plurality of slave CPs 26, exclusive control of data in the master CP 12 can be easily realized.
[0024]
In addition, if there is an abnormality in the slave CP 26, the distribution of I / O is automatically suppressed, and if the slave CP 26 becomes abnormal while executing I / O, the I / O process is retried in the master CP 12. By doing so, reliability can be improved.
[0025]
【Example】
FIG. 2 is a diagram illustrating I / O decentralization according to an embodiment of the present invention.
As shown in FIG. 2A, the plurality of control processors operate in a master CP 12 and a slave CP 26 separately. Only the master CP 12 receives an I / O request from the processor element PE. The master CP 12 that has received the I / O request from the processor element PE determines whether to perform the I / O processing by itself or to leave the I / O processing to the slave CP 26 according to a predetermined load algorithm.
[0026]
Whether a certain control processor performs I / O decentralization can be designated by an operator by a decentralization command as shown in FIG. The command name is "balance", and -s is a designation of decentralization. The CP identification number specifies to which control processor this command is applied. on / off indicates the start / end of decentralization. In the case of dispersing, for example, the dispersing rate can be specified such that the master CP 12 is 30% and the slave CP 26 is 70%. If the distribution ratio is omitted, a distribution ratio in which the sharing between the master CP 12 and the slave CP 26 is 1: 2 is given as a default value. The load on the master CP 12 is reduced by the default value because the processing load such as the reception of a request from the processor element PE and notification of completion is larger than that of the slave CP 26.
[0027]
The load algorithm used by the distribution determination processing means 17 shown in FIG. 1 when the distribution command is input is as follows.
(1) If the master CP 12 has not performed any I / O processing, the master CP 12 executes I / O. This is because communication cost is required to request the slave CP 26 for processing.
[0028]
{Circle around (2)} While the master CP 12 is performing a certain amount of I / O processing and there is a slave CP 26 with a low load and not performing I / O processing, the slave CP 26 is requested to perform I / O processing. .
[0029]
{Circle around (3)} When both the master CP 12 and the slave CP 26 are performing I / O processing, the I / O amount is shared, and the master CP 12 and the slave CP 26 balance. In the case of one master CP 12 and one slave CP 26 and a dispersion ratio of 1: 2, the master CP 12 shares 1/3 of the I / O amount and the slave CP 26 shares 2/3 of the I / O amount. I do.
[0030]
In FIG. 2A, for example, the processor element 10-0 issues a request for I / O (1) and I / O (3), and the processor element 10-1 issues a request for I / O (2) and I / O (4). ). According to the load algorithm described above, the master CP 12 transfers the I / O (2) and I / O (4) requests to the slave CP 26, and executes I / O to the storage device 31 by two control processors.
[0031]
As a result, the data transfer from the storage device 31 to each control processor is performed as shown in FIG. 2B. If one I / O process takes 6 seconds, the total is 12 seconds. In the prior art, it took 24 seconds as shown in FIG. 8 (B), and it can be seen that the effect of decentralization has appeared.
[0032]
FIG. 3 is a diagram for explaining the parallel division of I / O according to the embodiment of the present invention.
As shown in FIG. 3A, when the master CP 12 processes an I / O request (read request) from the processor element 10-0, the I / O execution unit is divided into several units. For example, if it is divided into two, I / O (1) and I / O (2), first, I / O (1) is executed, and the data read by the I / O (1) is processed by the processor element. While the data is being transferred to 10-0, the remaining data is read by I / O (2). That is, data input from the storage device 31 to the master CP 12 and data transfer from the master CP 12 to the processor element 10-0 are performed in parallel.
[0033]
When the slave CP 26 executes I / O, it can be similarly divided and executed.
This division of the I / O request is performed by, for example, assuming that the number of divisions is 2 when the I / O request packet from the processor element 10-0 is P1 in FIG. Can be performed by creating a packet P2 in which the data size is reduced to 1/2 and a packet P3 in which the address is advanced by 1/2 of the data size and the data size is reduced to 1/2.
[0034]
For example, assuming that the data transfer speed between the storage device 31 and the master CP 12 is 800 Mbytes / sec, and that the data transfer speed between the master CP 12 and the processor element 10-0 is 400 Mbytes / sec, the I / O request is divided into two. Is as shown in FIG. 3 (C). Times other than data transfer are ignored because they are very small. The total time until the completion of the I / O is 15 seconds due to the split parallelization of the I / O. In the prior art, it took 18 seconds as shown in FIG. 8 (D), and it can be seen that the effect of the split parallelization has appeared.
[0035]
Whether or not to perform the split parallelization can be designated by, for example, a split parallelization command shown in FIG. The command name is "balance", and -p is a designation of division parallelization. The CP identification number specifies to which control processor this command is applied. on / off indicates whether or not to perform split parallelization. When performing parallel division, the number of divisions can be specified. If the number of divisions is omitted, the default value is 2. Theoretically, increasing the number of divisions seems to improve efficiency, but the overhead, such as the number of communications, also increases at the same time, so the cost cannot be ignored.
[0036]
FIG. 4 shows the flow of the entire I / O processing according to the embodiment of the present invention.
The storage device 31 is a semiconductor storage device, and has a relatively high transfer speed. There are three processor elements PE. Assuming that the processor elements 10-0, 10-1, and 10-2 have issued I / O requests (1), I / O requests (2), and I / O requests (3), respectively, the master CP 12 Requests are queued and processed in the order of requests. In the execution of an I / O request, first, exclusive control of data is performed, and distribution determination processing is performed in consideration of the load on each control processor.
[0037]
As a result, when the I / O processing is performed by the own processor, the execution unit of the I / O is divided according to the specification and the data in the storage device 31 is accessed. On the other hand, when performing I / O processing in the slave CP 26, the slave CP 26 requests processing. Thereafter, the master CP 12 waits for I / O completion of the slave CP 26. When receiving the I / O request from the master CP 12, the slave CP 26 divides the I / O execution unit according to the designation and accesses the data in the storage device 31. When the I / O processing is completed, the processing result is notified to the master CP 12.
[0038]
Upon receiving the report of the I / O processing by the own processor and the completion of the I / O processing from the slave CP 26, the master CP 12 notifies the I / O requesting processor element PE of the completion of the I / O processing.
[0039]
For example, when an I / O request for reading data from the storage device 31 is performed only by the slave CP 26, the data read directly from the slave CP 26 to the processor element PE that has issued the I / O request. To transfer. However, since the master CP 12 is waiting for the completion notification of the I / O processing from the slave CP 26 for the recovery processing when the slave CP 26 goes down, the slave CP 26 also terminates the I / O to the master CP 12. Notify that.
[0040]
When the I / O for reading data is processed by the master CP 12 and the slave CP 26 in a shared manner, the master CP 12 collects the data read by the slave CP 26, and collects the data read from the master CP 12 by the own processor. At the same time, the data is transferred to the processor element PE.
[0041]
In the above processing, if an abnormality occurs in the slave CP 26 after the master CP 12 requests the slave CP 26 for I / O, or there is no response for more than a predetermined time from the slave CP 26, the I / O request times out. In this case, the master CP 12 performs the I / O requested to the slave CP 26 by its own processor, and performs a recovery process.
[0042]
FIG. 5 is a flowchart of the I / O decentralization process in the master CP 12.
First, at step 51, the I / O request queue is checked, and if there is an I / O request, it is removed from the queue. In step 52, exclusive control of data is performed. Various methods for exclusive control of the data are conventionally known, and a detailed description thereof will be omitted.
[0043]
In step 53, it is determined whether the decentralization flag of the master CP 12 is on. This decentralized flag is a flag for system operation control that is turned on / off by a decentralized command or other environment setting means. When the decentralization flag of the master CP 12 is off, the process proceeds to step 55.
[0044]
When the decentralization flag of the master CP 12 is on, in step 54, it is determined whether or not the master CP 12 is performing I / O processing. When the I / O processing is not being performed, the I / O execution device is set as the master CP 12. When the I / O processing is being performed, the process proceeds to the next step 55.
[0045]
In step 55, it is determined whether the slave CP 26 can be used. If the slave CP 26 cannot be used due to a failure or the like, the I / O execution device is set as the master CP 12.
[0046]
If the slave CP 26 is available, in step 56, it is determined whether the decentralization flag of the slave CP 26 is on. If the decentralization flag of the slave CP 26 is off, the I / O execution device is set as the master CP 12. During normal operation, it is generally considered desirable that the decentralized flag of the slave CP 26 is always on.
[0047]
If the decentralization flag of the slave CP 26 is on, in a step 57, it is determined whether or not the slave CP 26 is performing I / O processing. When the I / O processing is not being performed, the I / O execution device is set as the slave CP 26. During the I / O processing, the master CP 12 and the slave CP 26 share and execute the I / O according to a predetermined distribution ratio. In this case, the I / O requests may be distributed for each number, or one I / O request may be divided so that the master CP 12 and the slave CP 26 share the processing according to the distribution ratio. .
[0048]
FIG. 6 is an I / O division processing flowchart in the I / O execution device according to the embodiment of the present invention.
When executing I / O in the master CP 12 or the slave CP 26, first, in step 61 of FIG. 6A, it is determined whether the designated number of I / O divisions is two or more. If the I / O execution unit is not divided, that is, if the number of divisions is 1, the process proceeds to step 63.
[0049]
If the number of divisions is two or more, at step 62, the execution unit of the I / O request is divided so as to have the designated number of divisions.
In step 63, it is checked whether or not I / O can be issued to the storage device 31 at present. If it is possible, the process proceeds to step S64. If it cannot be issued, it waits until it can be issued.
[0050]
In step 64, I / O of the divided I / O request is issued to the storage device 31. Steps 63 to 65 are repeated until all I / O issuance is completed by the determination in step 65.
[0051]
When there is an I / O interrupt for the above I / O issuance, or when the master CP 12 receives an I / O processing result from the slave CP 26, the transfer processing shown in FIG. 6B is performed.
[0052]
First, in step 71, it is determined whether the own control processor (CP) is the master CP 12 or the slave CP 26.
[0053]
If the own CP is the master CP 12, it is determined in step 72 whether or not the I / O processing result can be transferred to the processor element PE of the I / O request source. The I / O processing result is transferred to the processor element PE. If transfer to the processor element PE is not possible, the process temporarily returns to the interrupt source and waits for the next transfer trigger.
[0054]
If the own CP is the slave CP 26, it is determined in step 74 whether the I / O processing result can be transferred to the master CP 12. If transferable, the I / O processing result is sent to the master CP 12 in step 75. Forward. If transfer to the master CP 12 is not possible, the process temporarily returns to the interrupt source and waits for the next transfer trigger.
[0055]
FIG. 7 is a time chart when the I / O decentralization and the split parallelization are used in one embodiment of the present invention.
It is assumed that there is one master CP 12 and one slave CP 26, and the designated number of I / O divisions is two. It is also assumed that the distribution ratio is master: slave = 1: 2. The read data of the I / O request from the processor element PE is executed by the master CP 12 and the slave CP 26 as shown in FIG. That is, in the master CP 12, the reading of the data A and B is executed, and in the slave CP 26, the reading of the data C and D is executed.
[0056]
The temporal flow is as shown in FIG. First, data is transferred from the storage device 31 to the master CP 12 for the time A1 by executing I / O relating to data A from the master CP 12 to the storage device 31. When the transfer of A1 is completed, the master CP 12 executes I / O relating to the data B, and transfers the data A input at A1 to the processor element PE that has requested the I / O over a time A2. During this time, data transfer from the storage device 31 to the master CP 12 is executed in parallel during the time B1. After the time A2, data transfer for a time B2 from the master CP 12 to the processor element PE is performed.
[0057]
In the slave CP 26 as well, data transfer is performed for each of the data C and D as shown in FIG. First, data transfer from the storage device 31 to the slave CP 26 is performed at time C1, D1, data transfer from the slave CP 26 to the master CP 12 is performed at time C2, D2, and the master CP 12 is transferred to the processor element PE. The operation is performed as at times C3 and D3.
[0058]
In this embodiment, when there is an abnormality in the I / O processing of the slave CP 26, the master CP 12 performs the recovery processing taking over the I / O processing. If there is an abnormality in the master CP 12, one of the other slave CPs becomes the master CP and can similarly take over the processing.
[0059]
【The invention's effect】
As described above, according to the present invention, the load of the control processor is distributed by dispersing the I / O and dividing and parallelizing the I / O, and the execution of the I / O requested by the processor element is completed. Time is reduced. Therefore, the throughput performance of the job and the peak performance of the I / O are improved, and the speed of the entire system can be increased. Further, if there is any abnormality in the I / O processing of the slave CP, the master CP recovers, so that the reliability can be maintained.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration example of the present invention.
FIG. 2 is a diagram illustrating I / O decentralization according to an embodiment of the present invention.
FIG. 3 is a diagram for explaining I / O division and parallelization according to an embodiment of the present invention.
FIG. 4 is a diagram showing a flow of an entire I / O process according to an embodiment of the present invention.
FIG. 5 is a flowchart of an I / O decentralization process in a master CP.
FIG. 6 is a flowchart of an I / O division process according to an embodiment of the present invention.
FIG. 7 is a time chart when the I / O decentralization and the split parallelization are used together in one embodiment of the present invention.
FIG. 8 is an explanatory diagram of a conventional technique.
[Explanation of symbols]
10-0, 10-1,... Processor element 11 Crossbar network 12 Master CP
13 input device 14 I / O division instruction input means 15 I / O distribution instruction input means 16 I / O request receiving means 17 distribution determination processing means 18 I / O division processing means 19 I / O parallel execution means 20 I / O request Transfer means 21 I / O processing result receiving means 22 I / O processing completion notifying means 23 abnormality detecting means 24 distribution suppressing means 25 recovery processing means 26 slave CP
27 I / O request receiving means 28 I / O division processing means 29 I / O parallel execution means 30 I / O processing result transfer means 31 Storage device

Claims (5)

A plurality of processor elements for performing operations, a plurality of control processors for processing I / O requests from each of the processor elements, a network connecting the processor elements and the control processor, and an I / O connected to the control processor. In a parallel computer system having a storage device or an input / output device targeted for / O,
One of the control processors is a master control processor,
This master control processor
Means for receiving an I / O request from each of the processor elements;
Distribution determination processing means for determining whether to distribute the execution of the I / O request to a plurality of control processors or a specific control processor based on the load status of each control processor and predetermined distribution instruction information;
I / O request transfer means for transferring an I / O request to another control processor when executing the I / O request;
An I / O processing result receiving means for receiving an I / O processing result when another I / O request is executed by another control processor;
I / O for notifying the processor element of an I / O request source of an I / O processing result executed by its own control processor, an I / O processing result executed by another control processor, or both of them. A parallel computer system comprising a processing completion notifying means. The parallel computer system according to claim 1 ,
I / O distribution instruction input means for externally inputting whether to distribute the I / O request to a plurality of control processors or externally input a ratio of distributing the I / O request to each control processor. And a parallel computer system. In the parallel computer system according to claim 1 or 2 ,
The master control processor comprises:
Abnormality detection means for detecting an abnormality of another control processor;
A parallel computer system, comprising: a distribution inhibiting unit that inhibits another control processor in which an abnormality has been detected from sharing execution of an I / O request. In the parallel computer system according to claim 1, claim 2, or claim 3 ,
The master control processor comprises:
When an abnormality is detected in another control processor that has been executing the I / O request in a shared manner, recovery processing means for re-executing the I / O request being processed by the control processor is provided. Parallel computer system. A plurality of processor elements for performing operations, a plurality of control processors for processing I / O requests from each of the processor elements, a network connecting the processor elements and the control processor, and an I / O connected to the control processor. In a parallel computer system having a storage device or an input / output device targeted for / O,
One of the control processors is a master control processor,
This master control processor
I / O distribution instruction input means for externally inputting whether to distribute an I / O request to a plurality of control processors,
I / O division instruction input means for externally inputting a division number for dividing an I / O request into a plurality of I / O execution units;
Means for receiving an I / O request from each of the processor elements;
Based on the load status of each control processor and the distribution instruction information from the I / O distribution instruction input means, it is determined whether the execution of the I / O request is distributed to a plurality of control processors or performed by a specific control processor. Dispersion determination processing means
I / O request transfer means for transferring an I / O request to another control processor when executing the I / O request;
An I / O processing result receiving means for receiving an I / O processing result when another I / O request is executed by another control processor;
I / O for notifying the processor element of an I / O request source of an I / O processing result executed by its own control processor, an I / O processing result executed by another control processor, or both of them. Processing completion notification means,
Processing means for dividing an I / O request from the processor element into a plurality of I / O execution units in accordance with an instruction from the I / O division instruction input means;
For a plurality of divided I / O execution units, input / output of data between the storage device or the input / output device and the control processor, and communication between the control processor and the processor element of an I / O request source I / O parallel execution means for executing data transfer in parallel.

Images