JPH08272729A - Parallel computer system - Google Patents

Abstract

PURPOSE: To speed up I/O processing and lighten the load on a control processor(CP). CONSTITUTION: A master CP 12, once receiving an I/O request to a storage device 31 from a processor element PE, decentralizes I/O by a decentralization decision processing means 17, and the master CP 12 and a slave CP 26 execute the I/O. Further, an I/O division processing means 18 divides the execution unit of the I/O request according to specification and an I/O parallel execution means 19 performs data input from the storage device 31 and data transfer to the processor element PE in parallel.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a control processor having an I / O device from a processor element having no I / O device in a computer system in which a plurality of processor elements and a control processor are connected by a network. The present invention relates to a parallel computer system capable of efficiently executing I / O processing via a high speed.

[0002]

2. Description of the Related Art FIG. 8 is an explanatory diagram of a conventional technique. FIG.
In the parallel computer system shown in (A) and (C),
Processor element PE [PE (0), PE (1),
...] is a processor that specializes in arithmetic operations. The control processor CP is a device that manages the processor element PE, and includes an I / O such as the storage device 80.
The O device is connected. The control processor CP and each processor element PE are connected by a crossbar network XB.

Each processor element PE is a storage device 8.
When referring to the data stored in 0, since the storage device 80 is not connected to the processor element PE, an I / O request is issued to the server existing in the control processor CP. The server of the control processor CP that has received the I / O request executes I / O to the storage device 80 and transfers the read data to the requesting processor element PE via the crossbar network XB.

[0004]

In FIG. 8, the data transfer rate between the memory device 80 and the control processor CP is 800 Mbyte / sec, and the data transfer rate between the control processor CP and each processor element PE is 400 Mbyte /. Let's say seconds.

Conventionally, as shown in FIG. 8 (A), the processor element PE (0) requests the I / O (1) and
/ O (2) request and processor element PE
When (1) requests I / O (3) and I / O (4) in this order, the control processor CP
Processed these I / O requests one by one. Therefore, if it takes, for example, 6 seconds to transfer data from the storage device 80 to the control processor CP by one I / O process, as shown in FIG. 8B, all data from the storage device 80 to the control processor CP is transferred. It took 6 seconds x 4 = 24 seconds for transfer.

Further, as shown in FIG. 8C, when one read request is issued from the client of the processor element PE, it takes, for example, 6 seconds to transfer data from the storage device 80 to the control processor CP. Since the I / O processing to the I / O request source processor element PE is started after the completion of the I / O processing, it takes 12 seconds to transfer data from the control processor CP to the processor element PE, and one I / O processing is performed. Until the I / O request is completed, as shown in FIG.
It took seconds.

As described above, the conventional technique has a problem that it takes time to process an I / O request from the processor element PE. In addition, the control processor C
In P, various processes such as a process for managing the processor element PE, a process related to operation, a process related to a job, and the like are running. When the processor element PE has a scale of, for example, 100 to 200 units, I / O
There is a problem that requests are concentrated on one control processor CP, and the load on the control processor CP may become very high.

The present invention aims to solve the above-mentioned problems, and I / O
The purpose of the present invention is to speed up the I / O processing and reduce the load on the control processor by decentralizing the processing and parallelizing the I / O division.

[0009]

FIG. 1 shows a structural example of the present invention. Processor element (PE) 10-0, 10-
1, ... are processors that specialize in arithmetic operations. A plurality of control processors to which I / O devices such as the storage device 31 are connected are provided, one of which is a master control processor (hereinafter referred to as a master CP) 1
2 is set, and the others are set as slave control processors (hereinafter referred to as slave CPs) 26.

The master CP 12, slave CP 26 and processor element PE are connected by a crossbar network (XB) 11. The master CP12 has
Management of processor element PE, processing related to operation,
An input device 13 for operation is connected for processing related to jobs.

The I / O division instruction input means 14 is means for inputting the number of divisions into which the I / O request is divided into a plurality of I / O execution units by a command or the like from the input device 13. I
The / O distribution instruction input means 15 determines whether or not the I / O request is distributed to a plurality of control processors or what percentage is distributed to each control processor.
It is a means for inputting by a command or the like from.

The I / O request receiving means 16 is means for receiving an I / O request from each processor element PE. Based on the load status of the master CP 12 and the slave CP 26 and the distribution instruction information from the I / O distribution instruction input means 15, the distribution determination processing means 17 distributes execution of I / O requests to a plurality of control processors, or
Alternatively, it is a means for deciding whether or not to use a specific control processor.

The I / O request transfer means 20 sends the I / O request to the slave CP according to the judgment result of the distribution judgment processing means 17.
26, the slave CP 26 sends an I / O request.
Is a means of transferring to. I / O processing result receiving means 21
When an I / O request is executed by the slave CP 26,
It is a means for receiving the I / O processing result.

The I / O division processing means 18 follows the instruction of the I / O division instruction input means 14 and the processor element P.
It is a means for dividing the I / O request from E into a plurality of I / O execution units. The I / O parallel execution means 19 determines the storage device 31 and the master CP1 for each divided I / O execution unit.
Input / output of data to and from the master CP12 and I / O
This is a means for executing data transfer with the processor element PE of the request source in parallel.

The I / O processing completion notifying means 22 is a master C.
I / O processing result executed in P12 or slave CP
I / O processing result executed in step 26 or I of both of them
The I / O processing result is the processor element P of the I / O request source.
It is a means to notify E.

The abnormality detecting means 23 determines whether or not the slave C has timed out due to a requested I / O timeout or predetermined operation control.
It is a means for detecting an abnormality of P26. Dispersion suppressing means 24
Is a means for issuing an instruction to the distribution determination processing means 17 so that the slave CP 26 does not share the execution of the I / O request when the abnormality detection means 23 detects an abnormality in the slave CP 26.

The recovery processing means 25 is means for re-executing the I / O request being processed by the slave CP 26 when an abnormality is detected in the slave CP 26 which has shared the I / O request. .

The I / O request receiving means 27 in the slave CP 26 uses the master CP 12 for decentralizing the I / O.
It is a means for receiving the I / O request sent from the. I
The I / O division processing means 28 is the I / O division instruction input means 14
It is a means for dividing the I / O request from the master CP 12 into a plurality of I / O execution units according to the instruction from the. I /
The O parallel execution means 29 performs parallel input / output of data between the storage device 31 and the slave CP 26 and data transfer from the slave CP 26 to the processor element PE or the master CP 12 for the divided I / O execution units. It is a means to execute. The I / O processing result transfer means 30
The result of I / O processing executed by the slave CP 26 is used as the master CP 12 or the processor element P of the I / O request source.
It is a means to transfer to E.

Although one slave CP 26 is shown in FIG. 1, there may be two or more slave CPs.

[0020]

In the present invention, the number of control processors is increased and the slave CP 26 is provided in addition to the master CP 12. I / O from each processor element PE
Requests are shared among multiple control processors for processing. Here, this is referred to as I / O decentralization.

Further, noting that data is transferred between the storage device 31 and the control processor and between the control processor and the processor element PE, one I / O request issued from the processor element PE is divided into a plurality of pieces. To do. This will speed up the total I / O processing. Here, this is called division parallelization of I / O.

By distributing the I / O and dividing and parallelizing the I / O as described above, the load distribution of the control processor and the I / O performance can be improved. By this,
Job throughput performance and I / O peak performance are improved, and the speed of the entire system can be increased.

The master CP 12 receives all I / O requests from the processor elements PE and shares them with one or more slave CPs 26, so that exclusive control of data in the master CP 12 can be easily realized. it can.

When the slave CP 26 has an abnormality, the decentralization of I / O is automatically suppressed, and the slave CP 26
When 26 becomes abnormal during I / O execution, the master CP 12 retries the I / O processing, so that the reliability can be improved.

[0025]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 2 is a diagram for explaining I / O distribution according to an embodiment of the present invention. As shown in FIG. 2A, the plurality of control processors operate by being divided into a master CP 12 and a slave CP 26. Only the master CP 12 receives the I / O request from the processor element PE. Upon receiving the I / O request from the processor element PE, the master CP 12 determines whether to perform the I / O processing by itself or to let the slave CP 26 handle the I / O processing according to a predetermined load algorithm.

Whether or not the I / O is distributed by a certain control processor can be designated by the operator by a distribution command as shown in FIG. 2 (C). The command name is "balance", and -s is the designation of decentralization. The CP identification number designates to which control processor this command is applied. on / off represents the start / end of decentralization. When distributed, for example, the master CP 12 has 30
%, The slave CP 26 can specify the dispersion ratio such as 70%. When the distribution ratio is omitted, the distribution ratio is set as the default value so that the master CP 12 and the slave CP 26 share 1 to 2. The default value reduces the load on the master CP 12 because the processing load on the master CP 12 is greater than that on the slave CP 26, such as receiving requests from the processor elements PE and notifying completion.

With the decentralized command input,
The load algorithm used by the distribution determination processing means 17 shown in FIG. 1 is as follows. If the master CP 12 does not perform any I / O processing, the master CP 12 executes I / O. Slave C
This is because a communication cost is required to request P26 for processing.

If the master CP 12 is performing a certain amount of I / O processing and there is a slave CP 26 that has a low load and is not performing I / O processing, this slave CP 2
Request 6 I / O processing.

Master CP12 and slave CP2
If both 6 are in the process of I / O, the I / O amount is shared and the master CP 12 and the slave CP 26 perform balancing.
One master CP12 and one slave CP26,
In the case of the distribution ratio of 1 to 2, the master CP 12 shares the I / O amount of 1/3 and the slave CP 26 shares the I / O amount of 2/3.
Share the amount.

In FIG. 2A, for example, the processor element 10-0 issues a request for I / O (1) and I / O (3), and the processor element 10-1 outputs I / O.
(2), I / O (4) is requested. According to the above-mentioned load algorithm, the master CP 12 is I / O
(2) The I / O (4) request is transferred to the slave CP 26, and the two control processors execute I / O to the storage device 31.

As a result, data transfer from the storage device 31 to each control processor is performed as shown in FIG. 2B, and if one I / O process takes 6 seconds, the total time is 12 seconds. . In the prior art, FIG.
Since it took 24 seconds as shown in (B), it can be seen that the effect of dispersion is exhibited.

FIG. 3 is a diagram for explaining division parallelization of I / O according to the embodiment of the present invention. As shown in FIG.
When the master CP 12 processes the I / O request (read request) from the processor element 10-0, the I / O execution unit is divided into some. For example, I / O (1)
And I / O (2), I / O (2)
While executing O (1) and transferring the data read by the I / O (1) to the processor element 10-0, the remaining read by the I / O (2) is performed. That is, data input from the storage device 31 to the master CP 12 and data transfer from the master CP 12 to the processor element 10-
Data transfer to 0 is performed in parallel.

Similarly, when the slave CP 26 executes I / O, it can be divided and executed. This I /
The O request is divided, for example, by the processor element 10-0.
3B, if the number of divisions is 2, a packet P2 whose requested data size is halved and an address whose data size is This can be done by advancing by ½ and creating a packet P3 whose data size is ½.

For example, the data transfer rate between the storage device 31 and the master CP 12 is 800 Mbyte / sec,
Assuming that the data transfer rate between the processor 12 and the processor element 10-0 is 400 Mbyte / sec, the time chart when the I / O request is divided into two is as shown in FIG. Note that the time other than data transfer is neglected because it is very small. The total time required to complete I / O is 15 due to the parallelization of I / O.
Seconds. In the prior art, it took 18 seconds as shown in FIG. 8 (D), so it can be seen that the effect of division parallelization appears.

Whether or not this division parallelization is performed can be designated by, for example, a division parallelization command shown in FIG. The command name is "balance",
-P is a designation of division parallelization. The CP identification number designates to which control processor this command is applied. on / off represents whether or not division parallelization is performed. When performing division parallelization, the number of divisions can be specified. If the number of divisions is omitted, the default value is 2. Theoretically, increasing the number of divisions seems to improve efficiency, but the overhead such as the number of communications increases at the same time, and the cost cannot be ignored.

FIG. 4 shows the overall I / O according to the embodiment of the present invention.
The flow of processing is shown. The storage device 31 is a semiconductor storage device and has a relatively high transfer rate. The number of processor elements PE is three. Processor element 10-
0, 10-1, 10-2 are I / O requests (1),
When the I / O request (2) and the I / O request (3) are issued, the master CP 12 queues these requests and processes them in order of request. In executing an I / O request, first, exclusive control of data is performed, and the distribution determination process is performed in consideration of the load on each control processor.

As a result, when the I / O processing is performed by the own processor, the I / O execution unit is divided according to the designation and the data in the storage device 31 is accessed. On the other hand, when performing I / O processing with the slave CP 26, the slave C
Request processing to P26. After that, the master CP12
Wait for the I / O completion of the slave CP 26. When the slave CP 26 receives the I / O request from the master CP 12, it divides the I / O execution unit according to the designation,
Access the data in the storage device 31. When the I / O processing is completed, the master CP 12 is notified of the processing result.

When the master CP 12 receives the I / O processing report from its own processor and the I / O processing completion report from the slave CP 26, it notifies the I / O request source processor element PE of the I / O processing completion.

For example, the data is read from the storage device 31.
In response to an I / O request to be issued, that I / O is a slave CP.
When only 26 is performed, the read data is directly transferred from the slave CP 26 to the processor element PE which is the I / O request source. However, due to recovery processing such as when the slave CP 26 goes down, the master CP 12
Waits for the completion notification of the I / O processing from the slave CP 26, the slave CP 26 also notifies the master CP 12 that the I / O has been completed.

When the master CP 12 and the slave CP 26 share the I / O for reading the data, the master CP 12 collects the data read by the slave CP 26, and the master CP 12 sends the data to the own processor. It is transferred to the processor element PE together with the read data.

In the above processing, after the master CP 12 requests the slave CP 26 for I / O, the slave CP 26
If an error occurs in the slave CP26 or if there is no response from the slave CP26 for a predetermined time or longer and the I / O request times out, the master CP12 determines that the slave CP26
The self-processor executes the I / O requested to perform recovery processing.

FIG. 5 is an I / O decentralized processing flowchart in the master CP 12. First, in step 51, the queue of I / O requests is checked, and if there is an I / O request, it is removed from the queue. In step 52, exclusive control of data is performed. Various methods have been conventionally known for exclusive control of this data, and therefore detailed description thereof is omitted here.

In step 53, it is determined whether the decentralization flag of the master CP 12 is on. This decentralized flag is a flag for system operation control that is turned on / off by a decentralized command or other environment setting means. The decentralization flag of the master CP12 is o
If ff, go to step 55.

When the decentralization flag of the master CP 12 is on, it is determined in step 54 whether the master CP 12 is in the process of I / O. If I / O processing is not in progress, I
The / O execution device is the master CP12. If I / O processing is in progress, the process proceeds to the next step 55.

In step 55, it is determined whether the slave CP 26 can be used. Slave CP due to failure
If 26 cannot be used, set the I / O execution unit to the master CP
12

If the slave CP 26 is available,
In step 56, it is determined whether the decentralization flag of the slave CP 26 is on. Slave CP2
If the distribution flag of 6 is off, the master CP 12 is set as the I / O execution device. It is generally considered desirable that the decentralization flag of the slave CP 26 is always on during normal operation.

If the decentralized flag of the slave CP 26 is on, in step 57, the slave CP 26 will execute I / O.
It is determined whether or not it is being processed. When the I / O processing is not in progress, the slave CP 26 is set as the I / O execution device. I / O
During processing, the master CP 12 and the slave CP 26 share and execute I / O according to a predetermined distribution ratio. In this case, it may be distributed according to the number of I / O requests.
One I / O request is divided into master CPs according to the distribution ratio
12 and the slave CP 26 may share the processing.

FIG. 6 is an I / O division processing flowchart in the I / O execution device according to the embodiment of the present invention.
I / O in master CP 12 or slave CP 26
When executing, first, in step 61 of FIG.
It is determined whether the designated I / O division number is 2 or more.
When the I / O execution unit is not divided, that is, when the division number is 1, the process proceeds to step 63.

If the number of divisions is two or more, in step 62, the execution unit of the I / O request is divided so that the designated number of divisions is reached. In step 63, the storage device 3 is currently
It is checked whether or not I / O can be issued for 1, and if it can be issued, the process proceeds to step S64. If not, wait until it becomes possible to issue.

At step 64, the I / O of the divided I / O request is issued to the storage device 31. Step 65
According to the above determination, steps 63 to 65 are repeated until all I / O issuance is completed.

When there is an I / O interrupt for the above I / O issuance, or the master CP 12 is the slave CP 26.
When the I / O processing result is received from
The transfer process shown in is performed.

First, at step 71, the self control processor (CP) is the master CP 12 or the slave CP 2
It is judged whether it is 6, and if it is the slave CP 26, step 7
Go to 4.

If the own CP is the master CP 12, the processor element P of the I / O request source is processed in step 72.
It is determined whether or not the I / O processing result can be transferred to E, and if transferable, the I / O processing result is transferred to the processor element PE in step 73. If transfer is not possible to the processor element PE, the interrupt source is temporarily returned to wait for the next transfer trigger.

When the own CP is the slave CP 26, it is determined in step 74 whether the I / O processing result can be transferred to the master CP 12, and if it is transferable, the I / O is transferred to the master CP 12 in step 75. Transfer the processing result. If it cannot be transferred to the master CP12,
Return to the interrupt source and wait for the next transfer trigger.

FIG. 7 shows the I / O in one embodiment of the present invention.
The time chart at the time of using decentralization and division parallelization together is shown. It is assumed that there is one master CP12 and one slave CP26, and the designated I / O division number is two. Further, it is assumed that the dispersion ratio is master: slave = 1: 2.
Read of I / O request from processor element PE
The data is executed by the master CP 12 and the slave CP 26, as shown in FIG. That is,
The master CP 12 executes reading of data A and B, and the slave CP 26 executes reading of data C and D.

The temporal flow is as shown in FIG. 7 (B). First, the I / O execution of the data A from the master CP 12 to the storage device 31 transfers the data from the storage device 31 to the master CP 12 for the time A1. When the transfer of A1 is completed, the master CP 12 executes the I / O related to the data B, and transfers the data A input at A1 to the processor element PE which is the I / O request source at the time A2.
To transfer. During this time, the memory device 31 to the master C
The data transfer to P12 will be performed in parallel during time B1. After time A2, data is transferred from master CP12 to processor element PE for time B2.

Also in the slave CP 26, the data C,
For each of D, data transfer is performed as shown in FIG. First, from the storage device 31 to the slave CP
Data transfer to 26 takes place at times C1, D1,
Data transfer from the slave CP 26 to the master CP 12 is performed at times C2 and D2, and from the master CP 12 to the processor element PE at times C3 and D3.

In this embodiment, the slave CP 26 I
If there is an error in I / O processing, the master CP 12
Recovery processing is performed by taking over the / O processing. Master CP
When there is an abnormality in 12, one of the other slave CPs becomes the master CP and the processing can be similarly taken over.

[0059]

As described above, according to the present invention,
By distributing the I / O and dividing and parallelizing the I / O, the load on the control processor is distributed and the time required to complete the execution of the I / O requested by the processor element is shortened. Therefore, job throughput performance, I / O
The peak performance of is improved and the speed of the entire system can be increased. Further, when there is some abnormality in the I / O processing of the slave CP, the master CP recovers, so that the reliability can be maintained.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration example of the present invention.

FIG. 2 is a diagram illustrating I / O distribution according to an embodiment of the present invention.

FIG. 3 is a diagram for explaining division parallelization of I / O according to an embodiment of the present invention.

FIG. 4 is a diagram showing a flow of overall I / O processing according to an embodiment of the present invention.

FIG. 5 is an I / O decentralized processing flowchart in the master CP.

FIG. 6 is an I / O partitioning processing flowchart according to an embodiment of the present invention.

FIG. 7 is a time chart when I / O distribution and division parallelization are used together in an embodiment of the present invention.

FIG. 8 is an explanatory diagram of a conventional technique.

[Explanation of symbols]

 10-0, 10-1, ... Processor element 11 Crossbar network 12 Master CP 13 Input device 14 I / O division instruction input means 15 I / O distribution instruction input means 16 I / O request acceptance means 17 Distribution determination processing means 18 I / O division processing means 19 I / O parallel execution means 20 I / O request transfer means 21 I / O processing result reception means 22 I / O processing completion notification means 23 Abnormality detection means 24 Dispersion suppression means 25 Recovery processing means 26 Slave CP 27 I / O request receiving means 28 I / O division processing means 29 I / O parallel execution means 30 I / O processing result transfer means 31 Storage device

Claims (7)

[Claims] 1. A plurality of processor elements for performing arithmetic operations, a control processor for processing I / O requests from each of the processor elements, a network for connecting the processor elements and the control processor, and a connection for the control processor. In a parallel computer system including a storage device or an input / output device targeted for I / O control, the control processor divides an I / O request from the processor element into a plurality of I / O execution units. Input / output of data between the storage device or the input / output device and the control processor for the plurality of divided I / O execution units, and the control processor and the I / O execution unit.
A parallel computer system comprising: an I / O parallel execution means for executing parallel data transfer with the processor element of the / O request source. 2. The parallel computer system according to claim 1, further comprising an I / O division instruction input means for externally inputting a division number for dividing the I / O request into a plurality of I / O execution units. A parallel computer system characterized by the above. 3. A plurality of processor elements that perform arithmetic operations, a plurality of control processors that process I / O requests from each of the processor elements, a network that connects the processor elements and the control processor, and the control processor. In a parallel computer system equipped with a storage device or an input / output device that is an I / O target connected to a processor, one of the control processors is a master control processor, and each master control processor is a processor. A means for receiving an I / O request from an element, and the execution of the I / O request is distributed to a plurality of control processors based on the load status of each control processor and predetermined distribution instruction information, or a specific control Distributed decision processing means for deciding whether or not to execute by a processor, and I / O by another control processor
An I / O request transfer means for transferring the I / O request to the control processor when executing the request, and an I / O processing result for receiving the I / O processing result when the I / O request is executed by another control processor. / O processing result receiving means and I / O processing result executed by its own control processor or I / O processing executed by another control processor
I / O processing result or both I / O processing results
I / O for notifying the processor element of the O request source
A parallel computer system comprising a processing completion notifying means. 4. The parallel computer system according to claim 3, wherein whether or not the I / O request is distributed to a plurality of control processors, or a ratio of distributing the I / O request to each control processor A parallel computer system comprising I / O distribution instruction input means for input from 5. The parallel computer system according to claim 3 or 4, wherein the master control processor includes an abnormality detecting means for detecting an abnormality of another control processor and another control processor in which the abnormality is detected. A parallel computer system comprising: a distributed deterrence means for deterring execution of an I / O request so as not to be shared. 6. The parallel computer system according to claim 3, claim 4, or claim 5, wherein the master control processor detects an abnormality of another control processor that has shared the I / O requests and executed them. If you do,
A parallel computer system comprising recovery processing means for re-executing the I / O request processed by the control processor. 7. A plurality of processor elements for performing arithmetic operations, a plurality of control processors for processing I / O requests from each of the processor elements, a network connecting the processor elements and the control processor, and the control processor. In a parallel computer system including a storage device or an input / output device that is an I / O target connected to the I / O, one of the control processors is a master control processor, and the master control processor is
I / O distribution instruction input means for externally inputting whether or not to distribute I / O requests to a plurality of control processors;
The number of divisions that divides an I / O request into multiple I / O execution units
I / O division instruction input means input from the outside, means for accepting I / O requests from each processor element, load status of each control processor and distribution instruction information from the I / O distribution instruction input means. Based on the distribution determination processing means for deciding whether the execution of the I / O request is distributed to a plurality of control processors or to be executed by a specific control processor, and when executing the I / O request by another control processor, I / O request transfer means for transferring the I / O request to the control processor, and I / O request receiving means for receiving the I / O processing result when the I / O request is executed by another control processor.
O processing result receiving means and I / O processing result executed by its own control processor, I / O processing result executed by other control processor, or both of them
A plurality of I / O requests from the processor element are issued in accordance with the I / O processing completion notifying means for notifying the processor element of the I / O request source of the I / O processing result and the I / O division instruction input means. Processing unit for dividing into I / O execution units, and data input / output between the storage device or the input / output device and the control processor for the plurality of divided I / O execution units, and the control processor. A parallel computer system comprising: an I / O parallel execution means for executing parallel data transfer with the processor element of the I / O request source.