JP2020126487A - Parallel processor apparatus, data transfer destination determining method, and data transfer destination determining program - Google Patents

Abstract

To reduce time required for completion of data broadcast communication in a parallel processor apparatus.SOLUTION: The present invention is directed to a parallel processor apparatus having a plurality of nodes connected to one another through a network. Each of the plurality of nodes has a calculating unit for, based on configuration information of the network, positional information of each node in the network, and origin node information indicating an origin node of broadcasting communication, calculating a transfer destination node which is a transfer destination of data of the broadcasting communication so that transfer distance should gradually become smaller as a transfer number is increased, a storing unit for storing positional information of the transfer destination node at every transfer number calculated by the calculating unit, and a communication unit for, upon reception of the data from other node in the broadcasting communication, determining the transfer destination node based on the information stored in the storing unit and for transferring the received data to the determined transfer destination node.SELECTED DRAWING: Figure 1

Description

The present invention relates to a parallel processing device, a data transfer destination determination method, and a data transfer destination determination program.

When broadcasting a plurality of data in a network including a plurality of nodes, by performing a predetermined number of times of data transfer from one node to another node and then mutually executing data transfer between two nodes, The transfer time is reduced (for example, see Patent Document 1). In addition, when transmitting individual data from one node to another, dividing the network into multiple equal areas and making the number of data transmissions for each area equal allows efficient communication to be realized. (For example, refer to Patent Document 2).

JP, 11-345220, A International Publication No. 2008/114440

In a parallel processing device such as a distributed memory type HPC (High Performance Computing) system, a plurality of nodes need to transfer data all at once at the time of executing a job for performing parallel calculation. For example, there is broadcast communication as a simultaneous transfer of data.

In broadcast communication, it is desirable that as many nodes as possible operate as transmitting nodes as soon as possible and as long as possible, and further, it is desirable that link sharing that lowers the transfer rate does not occur. However, for example, in a broadcast communication in which data is sequentially transferred to an adjacent node, if the transmitted node that has transmitted the data continues to participate in the broadcast communication, the second and subsequent data transfers will be performed via another transmitted node. Will be sent. As a result, the possibility of sharing the link with other nodes during the second and subsequent data transfers increases, the data transfer efficiency during broadcast communication decreases, and compared to the case where link sharing does not occur, It takes time to complete the communication.

In one aspect, the present invention aims to reduce the time it takes to complete data broadcast in a parallel processing device.

According to one aspect, in a parallel processing device including a plurality of nodes connected to each other via a network, each of the plurality of nodes includes configuration information of the network and a position of each node on the network. Based on the information and the origin node information indicating the origin node at the time of the broadcast communication, the transfer destination node which is the transfer destination of the data in the broadcast communication such that the transfer distance becomes gradually smaller as the number of transfers increases is obtained. A calculation unit, a storage unit that stores the position information of the transfer destination node for each transfer count calculated by the calculation unit, and information stored in the storage unit when data is received from another node during broadcast communication. And a communication unit that transfers the received data to the determined transfer destination node.

In one aspect, the present invention can reduce the time it takes to complete data broadcast in a parallel computer.

It is a figure which shows an example of the parallel processing apparatus in one embodiment. FIG. 2 is a diagram showing an example of a sub-network including a node as a job execution target in the network of FIG. 1. It is a figure which shows an example of the network coordinate table of the subnetwork of FIG. It is a figure which shows an example of the receiving stage number table of FIG. It is a figure which shows an example of the transfer destination node table of FIG. 6 is a flowchart showing an example of the operation of each node in the first phase when the parallel processing device of FIG. 1 executes broadcast communication. It is a figure which shows an example of the broadcast communication in the parallel processing apparatus of FIG. 6 is a flowchart showing an example of the operation of each node in the second phase when the parallel processing device of FIG. 1 executes broadcast communication. 6 is a flowchart showing an example of processing for determining a transfer destination of data executed by each node in FIG. 1. It is a figure which shows an example (comparative example) of the broadcast communication in another parallel processing apparatus.

Embodiments will be described below with reference to the drawings.

FIG. 1 shows an example of a parallel processing device in one embodiment. The parallel processing device 100 illustrated in FIG. 1 includes a network NW having a plurality of nodes ND and a management node 50 that manages each node ND. Each node ND has a calculation unit 10, a storage unit 20, and a communication unit 30. Although FIG. 1 shows an example in which the network NW is a two-dimensional mesh network, the network NW may be another network and the dimension may be other than two-dimensional.

For example, the parallel processing device 100 operates as a distributed memory type large-scale HPC system. A job that uses a plurality of nodes ND to execute parallel computation requires communication that a plurality of nodes ND execute all at once in accordance with a specific communication pattern. Such communication is called collective communication. In the following, broadcast communication will be described as an example of collective communication.

The storage unit 20 has a storage area that holds a network coordinate table 22, a reception stage number table 24, and a transfer destination node table 26. The storage area holding the reception stage number table 24 is an example of the reception condition holding area, and the storage area holding the transfer destination node table 26 is an example of the transfer condition holding area.

The network coordinate table 22 includes configuration information (coordinate information indicating network coordinates, etc.) of a predetermined number of nodes ND that execute jobs in parallel among the nodes ND included in the network NW. In other words, the network coordinate table 22 includes the configuration information of the node ND that is the target of the broadcast communication.

The network coordinate table 22 may be pre-distributed from the management node 50 to each node ND when the configuration of the network NW is determined, or may be pre-distributed from the management node 50 to each node ND before executing the job. Good. By referring to the network coordinate table 22 in the own node ND, each node ND can coordinate information of the own node ND and all the nodes ND to be broadcasted without communicating with other nodes ND (that is, Location information) can be acquired. Note that each node ND is notified in advance of the coordinate information of its own node ND from the management node 50, and knows the position of its own node ND in the network NW.

For example, the node ND included in the sub-network within the range indicated by the coordinate information included in the network coordinate table 22 becomes a group of the nodes ND participating in the same job. An example of the network coordinate table 22 is shown in FIG. In the following, a partial network including a plurality of nodes ND that execute jobs in parallel is also referred to as a sub-network SNW (FIG. 2). Then, all the nodes ND included in the sub-network SNW are the nodes ND targeted for the broadcast communication.

The reception stage number table 24 shows information indicating which transfer number of times each of the plurality of nodes ND targeted for the broadcast communication receives the data in the broadcast communication executed by the plurality of transfers. Including. That is, the reception stage number table 24 of the storage unit 20 of each node ND indicates which transfer number of times the data is received by not only the own node ND but also all the nodes ND targeted for the broadcast communication. Contains the information to indicate. An example of the reception stage number table 24 is shown in FIG.

The transfer destination node table 26 includes information indicating the transfer destination node ND, which is the transfer destination of the data transferred by the predetermined node ND, for each transfer count of the broadcast communication executed by a plurality of transfers. That is, the transfer destination node table 26 of the storage unit 20 of each node ND shows the transfer destination node ND for each number of times of transfer of the broadcast communication not only for the own node ND but also for all the nodes ND targeted for the broadcast communication. Contains information. An example of the transfer destination node table 26 is shown in FIG. The number of times of transfer of the broadcast communication indicates not the transfer for each node ND but the transfer number in the entire sub-network SNW. The data transfer is performed by relaying the node ND, and each relay indicates the state of the broadcast communication. Therefore, in the following, the number of transfer of the broadcast communication is also referred to as the number of relay stages.

The reception stage number table 24 may include only the number of relay stages at which the own node ND receives data in the broadcast communication. Similarly, the transfer destination node table 26 may include only information indicating the transfer destination node ND of the data transferred from the own node ND in the broadcast communication. However, the information stored in the reception stage number table 24 and the transfer destination node table 26 is generated, for example, by the data transfer destination determination program executed by each node ND. Therefore, in the case of generating the reception stage number table 24 and the transfer destination node table 26 including the information of all the nodes ND of the broadcast communication, the common data transfer destination determining program is used for all the nodes ND of the broadcast communication. Can be used in. As a result, the management node 50 has only to distribute one data transfer destination determining program to each node ND and execute the program, which can simplify the management of the node ND by the management node 50.

Based on the network coordinate table 22 stored in the storage unit 20 of the own node ND and the origin node information indicating the origin node ND at the time of the broadcast communication, the calculation unit 10 is a transfer destination of the data in the broadcast communication. The node ND is calculated so that the transfer distance becomes gradually smaller. For example, as the transfer distance, a distance (Manhattan distance) on the route between the nodes ND to which the data is transferred may be used. The network coordinate table 22 includes the configuration of the network targeted for the broadcast communication and the position information of the node ND included in the network targeted for the broadcast communication.

Then, the calculation unit 10 creates the reception time stage number table 24 and the transfer destination node table 26 based on the calculated transfer destination node ND, and stores the created reception time stage number table 24 and the transfer destination node table 26 in the storage unit. It stores in 20. The reception stage number table 24 and the transfer destination node table 26 are an example of a message transfer order database indicating the order of transferring messages such as data. The origin node information may be notified from the management node 50 to each node ND in advance, or may be included in the network coordinate table 22.

As described above, the calculation unit 10 includes the reception stage number table 24 and the transfer destination node table including the information of the transfer pattern for “transfer the message (data) to the node ND as far as possible in the early stage of the broadcast communication”. 26 is created in advance before the broadcast communication is started. At this time, the calculation unit 10 is based on the node ND included in the sub-network SNW (FIG. 2), the origin node ND (starting position) for starting the transfer of the broadcast communication, and the link usable in the sub-network SNW. , The reception stage number table 24 and the transfer destination node table 26 are created.

The function of the calculation unit 10 may be realized by a data transfer destination determination program executed by a processor such as a CPU (Central Processing Unit) (not shown) included in each node ND. That is, the reception stage number table 24 and the transfer destination node table 26 may be generated by the processor executing the data transfer destination determination program. The calculator included in the calculation unit 10 and each node ND is an example of a computer.

In this case, the storage unit 20 may be provided so as to be accessible by the processor and may have a storage area for storing the data transfer destination determination program 28, as indicated by a broken line frame. Then, the processor of each node ND executes the data transfer destination determination program 28 to implement a data transfer destination determination method for creating the reception stage number table 24 and the transfer destination node table 26. The function of the calculation unit 10 may be realized by hardware such as FPGA (Field Programmable Gate Array).

In this embodiment, the calculating unit 10 provided in each node ND determines the transfer destination node ND of the data in the broadcast communication, and the information indicating the determined transfer destination node ND is used as the reception stage number table 24 of the own node ND. And the destination node table 26. As a result, it is not necessary to notify the determined transfer destination node ND to other nodes ND, etc., and thus it is possible to suppress an increase in the communication load of the network NW. On the other hand, for example, when the management node 50 determines the transfer destination node ND of the data in the broadcast communication, the determined transfer destination node ND is transferred to each node ND, so that the communication load of the network NW increases. ..

When data is received from another node ND in the broadcast communication, the communication unit 30 transfers the data based on the reception stage number table 24 and the transfer destination node table 26 stored in the storage unit 20. And transfer the received data to the determined transfer destination node ND. In communication other than broadcast communication, the communication unit 30 has a function of storing the received data in the storage unit 20 or the like when the destination of the received data is the own node ND. Further, when the destination of the received data is another node ND, the communication unit 30 has a relay function of transferring the data to the destination node ND.

The management node 50 is individually connected to each node ND via the management network MNW used for managing the node ND, and manages each node ND. In FIG. 1, the management node 50 is connected to only some of the nodes ND via the management network MNW, but is actually connected to all the nodes ND included in the network NW.

For example, the management node 50 may be a job scheduler node that manages the process activation of each node ND. The job scheduler node allocates the job submitted to the parallel processing device 100 to the node ND which is a calculation node, and requests the allocated node ND to start the program described in the job. Information such as the network coordinates of all the nodes ND participating in the same job (for example, the network coordinate table 22) is passed to each node ND from the job scheduler node.

FIG. 2 shows an example of the sub-network SNW including the node ND which is the execution target of the job in the network NW of FIG. In FIG. 2, the sub-network SNW is a sub-mesh network, but the topology of the sub-network SNW is not limited to the mesh network. For example, the sub-network SNW includes all the nodes ND that execute data processing and the like based on the activation of the program described in the job. Information indicating the network coordinates of the node ND and the node ND included in the sub-network SNW is included in the network coordinate table 22 transferred from the management node 50 to each node ND.

In the example illustrated in FIG. 2, the sub-network SNW includes 60 nodes ND including 12 nodes ND arranged in the X-axis direction and 5 nodes ND arranged in the Y-axis direction. (0, 0) and the like attached to the upper left of each node ND in the sub-network SNW indicate network coordinates. In the sub-network SNW, like the network NW, it is preferable to control the assignment of the node ND to the job so that the sub-network SNW becomes a mesh network or a torus network. This makes it possible to prevent the links used for inter-process communication of different jobs from overlapping.

In the network NW, the area (size) of the coordinate axis of each dimension that includes the node ND targeted for broadcast communication defined by the network coordinate table 22 is called a shape parameter. That is, the sub-network SNW is represented by the shape parameter. For example, the sub-network SNW is given in Cartesian coordinates, and the coordinate range of each coordinate axis X, Y is predetermined. The network NW may be used as the sub-network SNW.

FIG. 3 shows an example of the network coordinate table 22 of the sub-network SNW of FIG. The network coordinate table 22 has a plurality of entries that store the rank number RANK assigned to each node ND in the sub-network SNW and the network coordinates (X, Y). In the following, network coordinates are also simply referred to as coordinates.

The rank number RANK is a serial number assigned to each node ND in the sub-network SNW. In the example shown in FIG. 3, the rank number RANK has the coordinates (0,0), (0,1), (0,2), (0,3), (0,4), (0,5), ( 1,0), (1,1),. ． ． , (11, 3), (11, 4) are sequentially assigned to each of the assigned nodes ND. The allocation of the rank number RANK is not limited to the example shown in FIG. Each node ND can identify the network coordinates (X, Y) of the node ND in the sub-network SNW by referring to the network coordinate table 22 stored in the storage unit 20 of the own node ND.

When one process is assigned to each node ND, the rank number RANK is assigned to each node ND. When a plurality of processes are assigned to each node ND, the rank number RANK is assigned to each process of each node ND. However, when a plurality of rank numbers RANK are assigned to each node ND, by registering the representative rank number RANK in the network coordinate table 22, the network coordinate table 22 shown in FIG. 3 can be used as it is.

The network coordinate table 22 may store the network coordinates of all the nodes ND included in the network NW shown in FIG. In this case, since it is not necessary to update the network coordinate table 22 every time the sub-network SNW is generated, it is possible to reduce the amount of communication between the management node 50 and each node ND.

FIG. 4 shows an example of the reception stage number table 24 of FIG. The reception stage number table 24 stores a plurality of coordinates (X, Y) of the node ND targeted for the broadcast communication, a rank number RANK for identifying the node ND, and the number of relay stages for receiving the data in the broadcast communication. Has an entry of. The node ND having the number of relay stages=“0” indicates the origin node ND that starts the broadcast communication. In FIG. 4, the node ND to which the coordinates (0, 0) are assigned is the origin node ND. In the following description, the origin node ND of the broadcast communication is also referred to as "Root".

For example, the node ND to which the coordinates (0,4) and (11,4) are assigned indicates that it receives data with the number of relay stages=“1”. The node ND to which the coordinates (0,1), (0,3), (4,2), (7,2), (11,0), (11,2) are assigned has the number of relay stages=“2”. Indicates that data will be received. In the broadcast communication, the same data is transferred to all the nodes ND, so each node ND may receive the data once. Therefore, only one relay stage number is stored in the relay stage number column of each entry. Each node ND can detect the number of relay stages that have received the data by referring to the entry of the own node ND of the reception stage number table 24 based on the reception of the data. Thus, as will be described later, the transfer destination node ND to which the data is transferred can be detected by referring to the transfer destination node table 26 based on the detected number of relay stages.

FIG. 5 shows an example of the transfer destination node table 26 of FIG. The transfer destination node table 26 stores a rank number RANK that identifies the node ND that transfers the data, the number of relay stages that transfer the data in the broadcast communication, and the coordinates (X, Y) of the node ND that is the transfer destination of the data. Has a plurality of entries. In addition, in this embodiment, the predetermined node ND transfers data at a plurality of relay stages. Therefore, a plurality of entries corresponding to one node ND (for example, the node ND having the rank number RANK=“0”) is assigned to the transfer destination node table 26. Further, in this embodiment, each node ND transfers data to two nodes ND in each relay stage number that executes the transfer of the broadcast communication. Therefore, each node ND stores two coordinates in the transfer destination node ND column.

FIG. 6 is a flowchart showing an example of the operation of each node ND in the first phase when the parallel processing device 100 of FIG. 1 executes broadcast communication. The operation illustrated in FIG. 6 is started for each node ND based on, for example, a broadcast communication start instruction from the management node 50. The number of relay stages at the start of the flow of FIG. 6 is “0”, and although not shown in the flow of FIG. 6, the node “ND” of “Root” receives the data to be broadcast from the management node 50. The “Root” node ND may hold data to be broadcast before the flow of FIG. 6 is started.

The first phase is an operation executed at an early stage of the broadcast communication and is an operation for transferring data to the node ND as far as possible. By transferring the data to the node ND that is as far as possible, the nodes ND that receive the data can be distributed within the sub-network SNW. In addition, by distributing the nodes ND that receive data in the sub-network SNW, the data can be transferred to other nodes ND by using more relay stages without sharing the link with more nodes ND. can do.

For example, in the first phase, each node ND transfers data to k nodes ND in each number of relay stages, and the state in which the number of nodes ND holding the data after the transfer becomes k+1 times continues. When the number of nodes ND that have received the data increases and the data transfer destinations overlap, the number of nodes ND that retains the data after the transfer becomes k+1 times or less. In this embodiment, the transfer after the number of the nodes ND becomes k+1 times or less is executed by shifting to the second phase instead of the first phase.

First, in step S10, the node ND shifts the processing to step S14 when the own node ND is “Root”, and shifts the processing to step S12 when the own node ND is not “Root”. In step S12, the node ND other than "Root" waits until data is received, and when the data is received, the process proceeds to step S14.

In step S14, the node ND searches the reception stage number table 24, and acquires the number of relay stages from the entry including the rank number RANK assigned to the own node ND or the coordinates (X, Y) of the own node ND. For example, when the node ND is “Root” which is the origin node ND of the broadcast communication, the number of relay stages at the start of the flow of FIG. 6 is “0”, and the number of relay stages at the time of executing step S14 is “0”. is there.

When the node ND is other than “Root”, the number of relay stages at the time of executing step S14 is the number of relay stages that received the data in step S12. That is, the number of relay stages acquired from the reception stage number table 24 is the current number of relay stages. The node ND holds the value obtained by adding “1” to the acquired number of relay stages as the counter value i.

Next, in step S16, the node ND searches the transfer destination node table 26, and from the entry including the rank number RANK assigned to the own node ND and the number of relay stages indicated by the counter value i, the coordinates of the transfer destination node ND ( X, Y) is acquired. The reception stage number table 24 and the destination node table 26 can be associated with each other via the relay stage number. Therefore, the node ND can also search as one table via the number of relay stages even when the transfer destination node ND is obtained by searching the reception stage number table 24 and the transmission destination node table 26.

Next, in step S18, the node ND transfers the data to the transfer destination node ND acquired in step S16. Next, in step S20, the node ND adds "1" to the counter value i.

Next, in step S22, the node ND has an entry including the rank number RANK assigned to its own node ND and the number of relay stages indicated by the counter value i updated in step S20 in the transfer destination node table 26. Or not. If there is an entry that matches the condition, there is a transfer destination node ND that transfers the data, so the process proceeds to step S16, and the data transfer operation in the broadcast communication is continuously executed. If there is no entry that matches the conditions, there is no node ND that transfers the data in the first phase, so the operation in the first phase ends.

Each node ND independently executes the first phase with respect to the other nodes ND, but executes the transfer operation based on the reception stage number table 24 and the transfer destination node table 26 common to all the nodes ND. Therefore, it is possible to prevent the data transferred from the plurality of nodes ND from being transferred to one node ND in a duplicated manner.

Note that each node ND may acquire the information corresponding to its own node ND from the reception stage number table 24 and the transfer destination node table 26 before starting the operation of FIG. 6. Accordingly, for example, in step S16, the process of acquiring the transfer destination node ND from the transfer destination node table 26 each time can be omitted.

FIG. 7 shows an example of broadcast communication in the parallel processing device 100 of FIG. The broadcast communication shown in FIG. 7 is an example in which each node ND executes the flow shown in FIG. 6 with reference to the reception stage number table 24 shown in FIG. 4 and the transfer destination node table 26 shown in FIG. .. In FIG. 7, a node ND indicated by a black circle indicates that data to be broadcast is received, and a node ND indicated by a white circle indicates that data to be broadcast is not received.

First, when the number of relay stages=“0”, only the node ND at the coordinate (0,0) which is “Root” has already received the data of the broadcast communication. The number of transferred nodes is “1” when the number of relay stages=“0”.

Next, when the number of relay stages=“1”, the node “ND” of “Root” refers to the transfer destination node table 26 and transfers data to the node ND of coordinates (0, 4) and (11, 4). The node ND having the coordinates (0, 4) and (11, 4) having received the data refers to the reception stage number table 24 and detects that the current relay stage number is “1”. The number of transferred nodes when the number of relay stages=“1” is “3”.

When the number of relay stages=“2”, the node ND of “Root” refers to the transfer destination node table 26, and corresponds to the number of relay stages=“2”, the transfer destination node ND (coordinates (0, 1), (11, 0)) is present. Therefore, the node “ND” of “Root” transfers the data to the coordinates (0, 1) and (11, 0).

The node ND (RANK=4) at the coordinate (0,4) refers to the transfer destination node table 26, and the transfer destination node ND (coordinates (0,3), (4, 2)) is detected. Therefore, the node ND (RANK=4) transfers the data to the coordinates (0,3) and (4,2).

The node ND (RANK=59) of the coordinate (11, 4) refers to the transfer destination node table 26, and the transfer destination node ND (coordinates (7, 2), (11, 2)) is detected. Therefore, the node ND (RANK=4) transfers the data to the coordinates (7, 2) and (11, 2). The number of transferred nodes when the number of relay stages=“2” is “9”.

As shown in FIG. 7, based on the transfer destination node table 26, each node ND transfers data to the node ND having a relatively large transfer distance when the number of relay stages is small, and the transfer is performed as the number of relay stages increases. Data is transferred to the node ND having a relatively small distance. As a result, the nodes ND that receive data can be distributed in the sub-network SNW, and the number of relay stages in which the nodes ND that receive data continue to participate in the transfer of the subsequent broadcast communication can be increased. Further, by distributing the nodes ND that receive data, it is possible to reduce the possibility that link sharing will occur in the communication of a plurality of node ND pairs that transmit and receive data.

In the first phase, for example, when each node ND transfers data to two nodes ND in each relay stage number of the broadcast communication, in the m-th stage, the data is held in the m-th power of “3” nodes ND. Can be made. When the number of nodes ND to which each node ND can transfer data is “k” in each number of relay stages of the broadcast communication, the number of nodes ND that have already received data in the m stages is “(k+1) ^m ”. Shown.

FIG. 8 is a flowchart showing an example of the operation of each node ND in the second phase when the parallel processing device 100 of FIG. 1 executes broadcast communication. In the second phase, the node ND that transfers the data in step S22 of FIG. 6 disappears, and the node ND that has completed the first phase starts. That is, FIG. 8 is executed for each node ND.

First, in step S30, the node ND searches the transfer destination node ND stored in all the entries of the transfer destination node table 26 of the own node ND, and selects the transfer destination node ND among the nodes ND adjacent to the own node ND. The adjacent node ND which does not correspond is detected. Whether or not it is the adjacent node ND can be determined based on the network coordinate table 22. For example, the node ND in which the X axis or the Y axis is deviated by “1” with respect to the coordinates (X, Y) of the own node ND is the adjacent node ND. The adjacent node ND that is not included in the transfer destination node ND is a node ND to which data is not transferred in the first phase, and it is necessary to transfer the data in the second phase.

Next, in step S32, when the node ND detects an adjacent node ND that has not transferred data, the process proceeds to step S34. On the other hand, when the node ND does not detect the adjacent node ND which has not transferred the data, the data is transferred to all the adjacent nodes ND, and thus the process ends.

In step S34, the node ND determines the adjacent node ND to which the data is transferred, with k as the upper limit among the adjacent nodes ND to which the data has not been transferred. k is the number of nodes ND to which each node ND can transfer data in each number of relay stages of the broadcast communication, and is, for example, two. In other words, k is the number of simultaneous transfer operations that maximize the total data transfer bandwidth from a certain node ND. k is the message length, the number of communication links connected to each node ND, the number of DMA (Direct Memory Access) transfer engines of the network device, the bandwidth of the system bus to which the network device is connected, the command of the network device It is decided based on the parallelism of the operation of the queue. The network device is included in each node ND and has a function of controlling communication between the nodes ND.

Next, in step S36, the node ND transfers the data to the adjacent node ND determined in step S34. Next, in step S38, if there is an adjacent node ND that has not transferred data among the adjacent nodes ND that have decided to transfer data, the node ND returns the processing to step S34, and executes the data transfer processing. .. On the other hand, the node ND ends the process when the data has been transferred to all the adjacent nodes ND. By executing the above operation in each node ND, the data is transferred to all the nodes ND, and the broadcast communication ends.

The four nodes ND adjacent to the arbitrary node ND determine the arbitrary node ND as the adjacent node ND. Therefore, the adjacent node ND may receive data from a plurality of surrounding nodes ND in a certain number of relay stages in the second phase. In this case, the adjacent node ND may validate the data received earlier and discard the data received later.

FIG. 9 shows an example of a process executed by each node ND of FIG. 1 for determining a data transfer destination. In other words, FIG. 9 shows the process of creating the reception stage number table 24 and the transfer destination node table 26 used in the above-described first phase. The process shown in FIG. 9 is executed by the calculation unit 10 shown in FIG. Note that the processing illustrated in FIG. 9 may be executed by the calculation unit 10 realized by a data transfer destination determination program executed by a processor such as a CPU included in each node ND. That is, FIG. 9 shows an example of a data transfer destination determination method and an example of a data transfer destination determination program.

First, in step S40, the calculation unit 10 sets the relay stage number m to “1”. Next, in step S42, the calculation unit 10 obtains the total number of nodes ND to which the data is transferred at the m-th stage by using the transfer number k and the relay stage number m. The transfer number k is the number of nodes ND to which data is transferred from each node ND in each relay stage number m. For example, when the transfer number k is “2”, the data is transferred to 6 nodes ND in the second stage, and the data is transferred to 18 nodes ND in the third stage.

Next, in step S44, the calculation unit 10 selects, among the nodes ND that have not received the data, for each node ND that is the data transfer source so that the nodes ND that are the data transfer destinations are distributed in the sub-network SNW. Determines the transfer destination node ND which is the transfer destination of the data. For example, the node ND to which the data is transferred can be calculated by solving an optimization problem whose objective function is to maximize the distribution of network coordinates of the node ND that has already received the data. The calculation unit 10 determines the node ND that has not received the data, for example, by referring to the transfer destination node table 26 updated in step S50 described below.

Next, in step S46, the calculation unit 10 determines whether the transfer destination nodes ND of the data determined in step S44 overlap. When the transfer destination nodes ND overlap, it is determined that it is difficult to further distribute the transfer destination nodes ND, and the process ends. When the process is terminated, the information indicating the transfer destination node ND determined in step S44 immediately before is discarded.

Data is transferred to the node ND not assigned as the transfer destination node ND as the adjacent node ND by the operation of the second phase shown in FIG. When the transfer destination nodes ND overlap, in the second phase, by assigning the adjacent node ND as the transfer destination node ND, it is possible to reduce the possibility that the data is transferred to one transfer destination node ND in duplicate. .. As a result, it is possible to prevent the data transfer efficiency in the broadcast communication from decreasing.

On the other hand, if the transfer destination nodes ND do not overlap, the process proceeds to step S48 to determine the node ND to transfer the data to the transfer destination node ND. In step S48, the calculation unit 10 transfers the data from each transfer source node ND to which transfer destination node ND among the transfer destination nodes ND determined in step S44, with the node ND that has already received the data as the transfer source node ND. Decide what to do. That is, the calculation unit 10 determines the combination of the data transfer source node ND and the data transfer destination node ND. The node ND that has received the data is a virtual node ND determined by the data transfer destination determination program.

When each node ND transfers data to k nodes ND in each number of relay stages (transfer number=k), the calculation unit 10 uses a combination (allocation) of one transfer source node ND and k transfer destination nodes ND. ) Is determined. Here, the calculation unit 10 determines the combination so that the data transfer paths do not intersect. As a result, it is possible to reduce the possibility that the common link is used to transfer the data to the plurality of transfer destination nodes ND.

When a common link is used in the transfer of data in the determined combination (when the links to be used overlap), the calculation unit 10 changes the allocation of the transfer source node ND and the transfer destination node ND. By doing so, we try to set a transfer route that does not use a common link. This can reduce the possibility of using a common link. When the transfer path using the common link cannot be completely eliminated, the calculation unit 10 changes the allocation of the transfer destination node ND so that the ratio of the transfer paths using the common link becomes the lowest. Good.

Next, in step S50, the calculation unit 10 stores the coordinates (X, Y) of each transfer destination node ND, the rank number RANK, and the number of relay stages m determined in step S44 in the reception stage number table 24, thereby receiving. The time table 24 is updated. Next, in step S52, the calculation unit 10 associates the coordinates (X, Y) of each transfer destination node ND determined in step S44 with the rank number RANK indicating the node ND of the data transfer source and the relay stage number m. It is attached and stored in the transfer destination node table 26. As a result, the transfer destination node table 26 is updated.

Next, in step S54, the calculation unit 10 increases the relay stage number m by "1", returns the process to step S42, and executes the process of determining the node ND of the data transfer destination at the next relay stage number m. .. As described above, the process of determining the data transfer destination node ND is repeatedly executed until the transfer destination nodes ND overlap. Note that the order of steps S50, S52, and S54 may be interchanged.

FIG. 10 shows an example (comparative example) of broadcast communication in another parallel processing device. In the broadcast communication illustrated in FIG. 10, the node ND to which the coordinate (0, 0) that is “Root” is assigned transfers the data to the adjacent node ND adjacent to the own node ND in the relay stage number=“1”. .. The node ND having received the data transfers the data to the adjacent node ND adjacent to the own node ND when the number of relay stages=“2”. Even after this, in each relay stage number, the node ND receiving the data transfers the data to the adjacent node ND adjacent to the own node ND.

In the broadcast communication in which data is sequentially transferred to the adjacent node ND, the data transfer direction is limited to the direction away from “Root”. In the example shown in FIG. 10, the direction away from “Root” is the direction in which the X coordinate increases or the direction in which the Y coordinate increases. Therefore, the node ND that has transferred data with a certain number of relay stages may not be able to transfer data with the subsequent number of relay stages. For example, the node “ND” of “Root” cannot participate in the broadcast communication when the number of relay stages=“2”. Strictly speaking, the root ND node ND can participate in the broadcast communication by sharing a link with another node ND, but in this case, the data transfer band becomes small.

Therefore, in the broadcast communication shown in FIG. 10, the data transfer efficiency is lower than that in the broadcast communication shown in FIG. In other words, in the broadcast communication shown in FIG. 7, the node ND that has transferred the received data to the other node ND can transfer the data to the other node ND even in the subsequent number of relay stages. As a result, in the broadcast communication shown in FIG. 7, the time (number of relay stages) required for the broadcast communication can be reduced as compared with the broadcast communication shown in FIG. 10, and the efficiency of the broadcast communication can be improved. it can.

By the way, when the data size (message size) is large and the data cannot be transferred at one time, the data transfer efficiency is improved by dividing the data and performing the pipeline transfer. On the other hand, when the amount of data is small and data can be transferred at one time, the time required to complete the broadcast communication is proportional to the number of relay stages of transfer. In this case, when “k” is an integer of 2 or more, the number of nodes that have already received data by the number of relay stages of m stages (m is a positive integer) in the broadcast algorithm using the k-ary tree is expressed by It is shown in (1).

The "tree" is a term in the graph theory meaning "graph or subgraph having no closed circuit", and can be used to represent the connection relation of the whole or a part of the corresponding computer network.

In the sub-network SNW in which the number of nodes is “N”, the number of relay stages required when data is transferred to all the nodes ND by broadcast communication is represented by the equation (2).

For example, in the broadcast communication of the sub-network SNW shown in FIG. 2, when data is transferred to all N nodes ND included in the sub-network SNW in a k-ary tree, the transfer count (that is, the number of relay stages) is “log”. It becomes about _k+1 N″. Assuming that the transfer data amount is D, the bandwidth per relay is B, and the overhead per transfer + communication delay time is L, the approximate communication time for the entire broadcast communication is given by equation (3). An approximate communication delay time is given by equation (4).

On the other hand, the approximate communication time of the entire broadcast communication in another parallel processing device which has a large data size, divides the data D into three, and performs pipeline transfer with a binary tree is given by equation (5). ..

In Expression (5), the transfer data amount is D/3, the bandwidth per relay is B, and the sum of the transfer overhead and communication delay time per transfer is L.

For example, assuming that N=2000, k=3, B=12.5 GiB/sec, D=1.25 MiB, and L=1 μsec (10 ⁻⁶ sec), the approximate communication time in the parallel processing device 100 is calculated by the formula ( According to the equation (6) when L in 3) is negligible, it becomes about 179 μsec. On the other hand, the approximate communication time is approximately 333 μsec from the equation (7) when L in the equation (5) can be ignored. The symbol * in equations (6) and (7) indicates multiplication. The communication time represented by the equation (6) is about 40% of the communication time represented by the equation (7).

Expression (6) represents the communication time when the data transfer destination nodes ND are distributed in the broadcast communication as shown in FIG. 7. Expression (7) indicates the communication time when data is transferred to the adjacent node ND in the broadcast communication. Therefore, the communication time required for the broadcast communication shown in FIG. 7 can be shortened as compared with the communication time required for the broadcast communication shown in FIG.

On the other hand, when L is the main factor of the communication time, if the ternary tree is used, the communication time in the parallel processing device 100 is estimated to be about 5.5 μsec based on the equation (4), and the other parallel processing devices can use it. The communication time of is roughly calculated by multiplying equation (2) by L and becomes about 6.5 μsec. The parallel processing device 100 is also advantageous for the communication time when L is the main factor of the communication time.

In the following, an example of creating the reception stage number table 24 and the transfer destination node table 26 is shown. That is, a method of calculating the data transfer order in the above-described first phase in the sub-network SNW that executes broadcast communication will be described below.

In the first phase in which each node ND transfers data to k nodes ND in each number of relay stages, and the number of nodes ND holding the data after the transfer becomes k+1 times, the number of relay stages is It is executed until the upper limit n is reached. The upper limit n is the maximum integer that satisfies “(k+1) ⁿ ≦N”. For example, when the "maximum integer that does not exceed x" for the real number x is expressed as [x] by the Gauss symbol, the upper limit n is expressed by the equation (8).
n=[log _k+1 N] (8)
Hereinafter, a method of determining the node ND of the data transfer destination at each relay stage number using the quadratic programming method will be described. For simplicity of explanation, the network topology is a mesh network. In order to realize the condition that "the message (data) is transferred to the node ND as far away as possible in the early stage of the broadcast communication", "the variance of the network coordinates of each dimension of the mesh network" is an objective function that maximizes ..
<Example 1>
The result obtained by the data transfer destination determination program that executes the following procedure is registered in the reception stage number table 24 and the transfer destination node table 26. The data transfer destination determination program executed in the first embodiment includes the following steps A and B, and is different from the process of the data transfer destination determination program shown in FIG. Hereinafter, the data transfer destination determination program executed in the first embodiment will be simply referred to as a program.

In step A, the program determines the coordinates of the data transfer destination node ND (that is, the node ND that receives the data) at each relay stage number. Step A includes the following sub-steps A1, A2, and A3, and is executed, for example, for each number of relay stages. Note that the output of the "quadratic programming subroutine for integer variables" used in step A does not have to be an exact solution. For example, the integer part of the solution of the "relaxation problem" without the constraint "is an integer" was taken. Approximate solution is acceptable.

In sub-step A1, the program allocates a node ND that receives data at each relay stage and an array that stores the coordinates thereof.

Next, in sub-step A2, the program inputs the coordinates of the (k+1) ^m−1 nodes ND that have already received the data at the m−1 stage, and sets “the coordinates of the node ND that receives the data up to the m stage”. Accept an objective function that maximizes the variance of.

Next, in sub-step A3, the program gives the objective function of sub-step A2, calls a quadratic programming subroutine of integer variables, and stores the coordinates of the receiving node ND output by the subroutine in an array. In sub-step A3, the program derives k×(k+1) ^m−1 network coordinates (that is, the node ND that receives data) for each number of relay stages. k is the number of nodes ND to which each node ND can transfer data.

After step A is completed, in step B, the program associates the node ND that has received the data up to the m-1th stage with the node ND that receives the data at the mth stage. Step B includes the following sub-steps B1, B2, and B3, and is executed, for example, for each number of relay stages.

In sub-step B1, the program receives k×(k+1) ^m−1 data to be received at the m-th stage for every (k+1) ^m−1 nodes ND that have received the data up to the m−1 th stage. Are assigned from among the nodes ND. That is, the program determines the correspondence relationship between the node ND that transfers data and the node ND that receives data in the m-th stage.

Next, in sub-step B2, the program adds an entry corresponding to each of the k(k+1) ^m-1 nodes ND that receives data at the m-th stage to the reception stage number table 24.

In sub-step B3, the program adds, to the transfer destination node table 26, an entry having the allocated k nodes ND as the transfer destination node ND for each of the nodes ND that has received the data up to the m-1th stage. To do. Note that the sub-steps B2 and B3 may be executed in reverse order.

By executing steps A and B, it is determined to which node ND each node ND transfers the data at each relay stage number. By solving the quadratic programming, the condition that "the message (data) is transferred to the node ND as far as possible in the early stage of the broadcast communication" is achieved. For this reason, it is possible to perform a communication procedure in which packet (data) collision is unlikely to occur and transfer performance is unlikely to be impaired even in the latter half of the first phase described above.
<Example 2>
In the first embodiment, the transfer destination node ND is determined without considering the use of the common link. Therefore, when the data is transferred using the common link, the transfer efficiency decreases. In the second embodiment, the transfer destination node ND is determined so as to reduce the possibility of using the common link.

The second embodiment executes the same processing as the first embodiment except that the sub-step B1 is different from the first embodiment. In sub-step B1 of the second embodiment, the program allocates k transfer destination nodes ND for each transfer source node ND, and then, for example, a path using a common link in the data transfer path by the dimension order routing. Determine if there is. If there is a route that uses the common link, the program changes the assignment of the transfer destination node ND so that there is no route that uses the common link.

When the program cannot completely eliminate the route using the common link, the program changes the assignment of the transfer destination node ND so that the ratio of the route using the common link becomes the lowest in each relay stage number. Good. Further, the program may change the allocation of the transfer destination node ND so that the ratio of the routes using the common link is the lowest in the plurality of relay stages.

In the second embodiment, it is possible to reduce the probability of using a common link when transferring data, and improve the data transfer efficiency, so that the broadcast communication time can be shortened.

As described above, in the present embodiment, the node ND that has received the data can continue to participate in the data transfer of the broadcast communication, and each node ND can transfer the data as compared with the case of transferring the data to the adjacent node ND. It is possible to increase the number of relay stages that can participate. As a result, the number of data transfer nodes ND can be increased in each number of relay stages, and the time required to complete the broadcast communication can be reduced.

Further, by transferring the data to the node ND as far as possible, the nodes ND that receive the data can be dispersed in the sub-network SNW. By distributing the nodes ND receiving the data in the sub-network SNW, the data can be transferred to other nodes ND by using more relay stages without sharing a link with more nodes ND. You can

Since the calculation unit 10 of each node ND executes the common data transfer destination determination program, the same reception stage number table 24 and transfer destination node table 26 can be created in the node ND. Therefore, each node ND does not have to notify the other node ND of the transfer destination node ND determined by the calculation, so that it is possible to suppress an increase in the communication load of the network NW.

The reception stage number table 24 and the transfer destination node table 26 include reception information and transfer information of all the nodes ND that are targets of the broadcast communication. Therefore, each node ND can generate the reception stage number table 24 and the transfer destination node table 26 by executing the common data transfer destination determining program. As a result, the management node 50 has only to distribute one data transfer destination determination program to each node ND and execute the program, and the management node 50 can easily manage the node ND.

When the transfer destination nodes ND overlap when determining the transfer destination node ND in the first phase, data is stored in one transfer destination node ND by allocating the adjacent node ND as the transfer destination node ND in the second phase. The possibility of duplicate transfer can be reduced. As a result, it is possible to prevent the data transfer efficiency in the broadcast communication from decreasing.

The packet transfer can be executed with a low possibility of using the common link at the same time, and the packet transfer efficiency can be improved as compared with the case where the packet is transferred using the common link at the same time. ..

A common data transfer destination determination program is executed for each group of nodes ND included in the sub-network SNW, and each node ND creates a common reception stage number table 24 and transfer destination node table 26. That is, the reception stage number table 24 and the transfer destination node table 26 are generated according to the shape parameter of the sub-network SNW and the number of nodes ND included in the sub-network SNW. Therefore, it is possible to optimally set the data transfer efficiency by the broadcast communication according to the size of the sub-network SNW.

The features and advantages of the embodiments will be apparent from the above detailed description. This is intended to cover the features and advantages of the embodiments as described above without departing from the spirit and scope of the claims. Further, a person having ordinary skill in the art can easily think of all the improvements and changes. Therefore, the scope of the embodiments having the invention is not intended to be limited to the above, and it is possible to rely on appropriate improvements and equivalents included in the scope disclosed in the embodiments.

10 calculation unit 20 storage unit 22 network coordinate table 24 reception stage number table 26 transfer destination node table 28 data transfer destination determination program 30 communication unit 50 management node 100 parallel processing device MNW management network ND node NW network RANK rank number SNW subnetwork

Claims (10)

In a parallel processing device including a plurality of nodes connected to each other via a network,
Each of the plurality of nodes is
Based on the network configuration information, the position information of each node on the network, and the origin node information indicating the origin node at the time of broadcast communication, the transfer distance is gradually reduced as the number of transfers increases. And a calculation unit that obtains a transfer destination node that is a transfer destination of data in the broadcast communication,
A storage unit that stores the position information of the transfer destination node for each transfer count calculated by the calculation unit;
When data is received from another node during the broadcast communication, a transfer destination node is determined based on the information stored in the storage unit, and a communication unit that transfers the received data to the determined transfer destination node. A parallel processing device having. The calculation unit performs, for each of the plurality of nodes, calculation of a transfer destination node that gradually reduces the transfer distance to the transfer destination node as the number of transfers increases.
The storage unit stores the transfer destination node calculated by the calculation unit for each of the plurality of nodes in association with the number of transfers,
The communication unit, when receiving data from another node, transfers the data to a transfer destination node stored in the storage unit for each transfer count corresponding to its own node. Parallel processor. When the transfer destination nodes that receive data from the transfer source node that is the data transfer source overlap, the calculation unit stores the transfer destination nodes up to the transfer count that is one transfer count before the transfer count at which the transfer destination nodes overlap. Stored in the department,
When the communication unit completes the transfer of the data to the transfer destination node stored in the storage unit, the communication unit transfers the data to a node that has not received the data among the adjacent nodes adjacent to the own node. The parallel processing device according to claim 1 or 2. The storage unit holds a reception condition holding area for holding the number of times of transfer for receiving data for each of the plurality of nodes, and a transfer condition holding area for holding for each of the plurality of nodes a correspondence relationship between the number of times of transfer and a transfer destination node. Have
The communication unit of each of the plurality of nodes refers to the reception condition holding area based on the reception of data, detects the number of times of transfer of data, and based on the detected number of times of transfer, stores the transfer condition holding area. 4. The parallel processing device according to claim 1, wherein the transfer destination node to which the data is transferred is determined by referring to the parallel processing device. The calculation unit
For each transfer count, from a node that has not received data, a predetermined number of nodes are assigned to the transfer destination node, and the correspondence relationship between the assigned transfer destination node and the transfer count is stored in the reception condition holding area,
A transfer source node and a transfer destination node that are sources of data are associated with each other for each transfer count, and the association between the transfer source node and the transfer destination node is stored in the transfer condition holding area together with the transfer count. The parallel processing device according to claim 4. The calculation unit changes to a combination of a transfer source node and a transfer destination node that do not share a link when data is transferred to a plurality of transfer destination nodes sharing a link. Parallel processor. 7. The calculation unit allocates a predetermined number of distributed nodes to transfer destination nodes from among nodes that have not received data for each transfer count. The parallel processing device according to item 1. 8. The parallel processing device according to claim 1, wherein the network targeted for broadcast communication is a sub-network that is a part of the entire network. In a data transfer destination determining method for determining a data transfer destination during broadcast communication of a parallel processing device including a plurality of nodes connected to each other via a network,
Each of the plurality of nodes is
Based on the network configuration information, the position information of each node on the network, and the origin node information indicating the origin node at the time of broadcast communication, the transfer distance is gradually reduced as the number of transfers increases. For the transfer destination node that is the transfer destination of data in various broadcast communications,
A method of determining a data transfer destination, characterized in that the calculated position information of the transfer destination node for each transfer count is stored in a storage unit in the node. In a data transfer destination determination program for determining a data transfer destination during broadcast communication of a parallel processing device including a plurality of nodes connected to each other via a network,
A computer included in each of the plurality of nodes,
Based on the network configuration information, the position information of each node on the network, and the origin node information indicating the origin node at the time of broadcast communication, the transfer distance is gradually reduced as the number of transfers increases. For the transfer destination node that is the transfer destination of data in various broadcast communications,
A data transfer destination determination program, wherein the calculated position information of the transfer destination node for each transfer count is stored in a storage unit in the node.

Images