WO2014102917A1

WO2014102917A1 - Parallel processing method and parallel computer system

Info

Publication number: WO2014102917A1
Application number: PCT/JP2012/083546
Authority: WO
Inventors: 幸二福田; 由子長坂; 拓実仁藤
Original assignee: 株式会社日立製作所
Priority date: 2012-12-26
Filing date: 2012-12-26
Publication date: 2014-07-03
Also published as: JPWO2014102917A1; JP5965498B2

Abstract

In parallel computation of a graph problem or the like, the time required for sorting (ordering) processing which is performed at a computation node of a receiving side becomes problematic. The present invention is a parallel computer system which has a plurality of computation nodes, which resolves the above problem by partitioning the objects to be processed with a first grouping so as to place the objects at each computation node and perform processing, stores results of the processing in a storage device group on the basis of a second grouping, and transmits the stored results of the processing to each computation node in accordance with the first grouping.

Description

Parallel processing method and parallel computer system

The present invention relates to a parallel processing method and a parallel computer system, and more particularly to transmission of processing results.

Non-patent document 1 discloses a calculation model called MapReduce shown in FIG. 2 as a general-purpose method for efficiently performing calculations on a parallel computing device. As will be described later, MapReduce is taken up merely as an example of the technical description, and the invention itself in this application does not assume the MapReduce calculation model.

As shown in FIG. 2, the MapReduce calculation model is a calculation model composed of three phases, a Map phase, a Sort phase, and a Reduce phase. In the Map phase, input data is read after being divided into a large number of processing units and input to the Map process. The Map process performs some calculation or processing for each processing unit and outputs a set of <Key, Value>. In the subsequent Sort phase, the set of <Key, Value> output in the Map phase is classified (aligned) for each key, and a set of multiple values for each Key is output. In the Reduce phase, a set of a plurality of values for each key is input to the Reduce process, and some calculation or processing is performed on the set to which the Reduce process is input, and a final result is output.

Here, in the Map phase and the Reduce phase, each Map process and each Reduce process have no dependency with other Map processes and Reduce processes, and can be executed in parallel. Therefore, by using the MapReduce calculation model, it is possible to perform calculation (processing) in parallel by a parallel calculation device including a plurality of calculation nodes.

FIG. 3 is a schematic diagram showing a state in which the MapReduce calculation model is operated on a parallel computing device composed of a plurality of computation nodes. As described above, a plurality of Map processes in the Map phase and a plurality of Reduce processes in the Reduce phase can be executed in parallel. Therefore, these processes can be allocated to a plurality of calculation nodes and executed in parallel. Note that the Map phase and the Reduce phase are not necessarily performed by the same parallel computing device. By the way, in many cases, the total number of Map processes and Reduce processes is larger than the total number of computation nodes. In that case, each computation node is necessarily responsible for a plurality of Map processes and Reduce processes. Consider a case where the total number of Reduce processes (the total number of types of keys output in the Map phase) is larger than the total number of computation nodes. In this case, the Sort phase in FIG. 2 is subdivided into two phases, a Shuffle phase (inter-computation node communication phase) and a Local Sort phase (receiving side alignment phase). In the Shuffle phase, a set of <Key, Value> output in the Map phase is transmitted to a calculation node in charge of a Reduce process that is uniquely determined for each Key. When each computation node is in charge of a plurality of Reduce processes (a plurality of Map output keys), the <Key, Value> pairs sent in the Shuffle phase are classified (aligned) for each Key in the computation node on the receiving side. There is a need. This classification (alignment) is performed in the Local Sort phase (reception side alignment phase).

As described above, by using the MapReduce calculation model, it is possible to perform calculation (processing) in parallel with a parallel calculation device including a plurality of calculation nodes. Furthermore, when the MapReduce calculation model is used in a parallel computing device including a plurality of calculation nodes, only the Map process and the Reduce process perform actual calculation and processing, and the Shuffle phase and the Local Sort phase are applications. This is the same regardless of the processing content. Therefore, by creating the Shuffle phase and the Local Sort phase as a common framework in advance, it is possible to easily create a plurality of applications by changing only the processing contents of the Map process and the Reduce process.

As described above, by using the MapReduce calculation model, it is possible to perform calculation (processing) in parallel with a parallel calculation device including a plurality of calculation nodes. However, as the amount of data to be processed increases, especially as the total number of Reduce processes increases compared to the total number of computing nodes, the Local Sort phase (reception side alignment phase) becomes heavier, and the overall processing time is reduced. Will occupy the majority. This is because the Map phase, Reduce phase, and Shuffle phase take time proportional to the amount of data to be processed, whereas in the Local Sort phase, using a simple alignment algorithm, This is because it takes time proportional to N × Log (N).

Worse, the data rearrangement performed in the Local Sort phase generally requires random access to the storage device. Normally, a random access memory (RAM), which is a temporary storage device provided in each computation node, has a small capacity. Therefore, when the amount of data to be handled is large, temporary access that allows random access at high speed is performed in the Local Sort phase process. It is necessary to use not only a RAM as a storage device but also a storage device such as a flash memory or a hard disk drive which has a large capacity but a low random access speed. Therefore, when the amount of data to be handled becomes larger than the capacity of the RAM which is a temporary storage device, the time required for the Local Sort will increase more rapidly.

The invention of the present application has been made in view of the above, and one of its purposes is to perform classification (alignment) processing for each key or a part thereof in the Map phase and the Shuffle phase. In the Shuffle phase, a means is provided so that data that has been classified (aligned) by key is delivered to the computation node on the receiving side as much as possible, and a means for shortening the processing time of the Local Sort phase is provided by using this means. There is. The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

The problem to be solved by the invention of the present application is a problem that generally occurs when a plurality of processing units are processed in parallel by a plurality of computing nodes, and MapReduce is taken as an example only for explanation. This is because. Therefore, the invention of the present application can be applied not only to the MapReduce calculation model but also to many cases where a plurality of processing units are processed in parallel by a plurality of calculation nodes. FIG. 4 shows a schematic diagram when a graph problem is calculated by a parallel computing device as an example of parallel processing of a plurality of processing units by a plurality of calculation nodes to which the invention of the present application can be applied, which is different from MapReduce.

並列 Parallel calculation of graph problems is to perform various calculations on a given graph structure composed of multiple vertices and edges connecting vertices. FIG. 4 shows an example in which the calculation nodes 1 to 4 perform processing on the graph portion given to each of them. Usually, the graph problem is composed of a plurality of calculation steps, and at each calculation step, some calculation is performed with respect to all the vertices as input of the calculation result of the vertex having the edge toward the own vertex in the previous step. When a graph problem is calculated by a parallel computing device composed of a plurality of calculation nodes, it is natural to assign a vertex to each calculation node and perform parallel processing considering that each vertex can be calculated independently. At this time, each calculation step is completed, the calculation result is transmitted to the vertex to which the edge is connected, and the receiving calculation node needs to classify (align) the received data for each destination vertex. This is the same processing as in the Shuffle phase and the Local Sort phase in the above-described MapReduce calculation model. As the number of vertices increases, that is, as the graph problem becomes larger, the classification for each destination vertex in the reception-side computation node is increased. The problem is that the (alignment) processing takes time.

The present invention is a parallel computer system having a plurality of computing nodes, which divides a processing target in a first grouping, arranges the processing target in each computing node, and processes the storage device group based on a second grouping. The processing result is stored, and the stored processing result is transmitted to each computation node according to the first grouping, thereby solving the above-described problem.

According to the present invention, it is possible to reduce the time required for the classification (alignment) processing performed in the calculation node on the receiving side. As a result, parallel computation can be accelerated.

It is a block diagram which shows the structure of the information processing system which is an Example of this invention. It is a figure explaining the calculation model called MapReduce disclosed by the nonpatent literature 1. FIG. It is a conceptual diagram at the time of operating the MapReduce calculation model disclosed by the nonpatent literature 1 on a parallel computer. As an example of the problem to be solved by the present invention, it is a conceptual diagram when a graph problem is operated on a parallel computing device. It is a figure explaining the receiving side alignment (Local Sort) speed-up by the bucket sort used as the beginning which considers this invention. It is a figure explaining the receiving side alignment (Local Sort) speed-up by the bucket sort used as the beginning which considers this invention. It is a figure which illustrates notionally reducing the number of memory | storage devices required for a bucket sort by sharing a bucket between each calculation node. It is a figure explaining the operation | movement of the parallel processing method and parallel computer system by 1st Example of this invention. It is a figure explaining the operation | movement of the parallel processing method and parallel computer system by 2nd Example of this invention. It is a figure explaining the operation | movement of the parallel processing method and parallel computer system by 3rd Example of this invention.

In the following embodiment, when it is necessary for the sake of convenience, the description will be divided into a plurality of sections or embodiments. However, unless otherwise specified, they are not irrelevant, and one is the other. Some or all of the modifications, details, supplementary explanations, and the like are related. Further, in the following embodiments, when referring to the number of elements (including the number, numerical value, quantity, range, etc.), especially when clearly indicated and when clearly limited to a specific number in principle, etc. Except, it is not limited to the specific number, and may be more or less than the specific number.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

FIG. 1 is a block diagram showing a configuration of an information processing system 101 according to an embodiment of the present invention. The information processing system 101 is a parallel computer system including a plurality of calculation nodes CALC_NODE_x (x = 1, 2, 3,...) And a communication switch COM_SW that mediates communication therebetween.

Each computation node CALC_NODE_x includes a central processing unit (CPU), a temporary storage device MEM, a storage device STOR, a communication device COM_DEV, and a bus BUS connecting the CPU, the storage device STOR, and the communication device COM_DEV. . In each calculation node CALC_NODE_x, the CPU reads necessary input data from the storage device STOR, performs calculation, and transmits the input data or calculation result to another calculation node using the communication device COM_DEV if necessary. . During the series of processing, the CPU stores temporarily necessary data, the progress of calculation, and the like in the temporary storage device MEM as necessary. As will be described later, one of the computation nodes CALC_NODE_x serves as a retransmission management node.

Here, a random access memory (RAM) is used as the temporary storage device MEM. On the other hand, a flash memory, a phase change memory, and a hard disk drive are used for the storage device STOR. Therefore, the temporary storage device MEM can be accessed at a higher speed than the storage device STOR, but has a small capacity. In addition, the temporary storage device MEM can be randomly accessed at high speed, whereas the storage device STOR cannot perform random access only by sequential access, or the random access speed is very low compared to the sequential access speed. It will have the characteristics.

In the parallel computer system shown in FIG. 1, the CPU, the temporary storage device MEM, the storage device STOR, and the communication device COM_DEV included in each calculation node are not necessarily the same, and each calculation node has a large size. The sheath performance may be different. Further, the case where there is a computation node that does not include all of the CPU, the temporary storage device MEM, the storage device STOR, and the communication device COM_DEV is also considered for the invention disclosed in the present application.

5 (a) and 5 (b) are diagrams for explaining a bucket sort method that is the beginning of thinking about the invention of the present application in solving the above-described problems. For example, consider a case where the Key output in the Map phase is an integer number greater than or equal to zero. Here, the case where Key is an integer number greater than or equal to 0 is taken as an example of explanation, and even if Key is a non-integer such as a character string, each bucket is assigned in advance. If a method is determined, the same method can be applied.

Here, it is assumed that there are 1000 types of key numbers of 0 to 999 for the Reduce process in which the computation node CALC_NODE_1 responsible for a part of the parallel computation is in charge of processing. FIG. 5A shows a processing flow in the calculation node CALC_NODE_1 at this time. In the Shuffle phase, CALC_NODE_1 receives a set of <Key, Value> whose Key numbers are 0 to 999. Since there is no agreement on the order of the received <Key, Value> pairs, all the received <Key, Value> pairs must be temporarily stored in the storage device STOR in preparation for the subsequent classification (alignment) processing. There is. After the calculation node CALC_NODE_1 has received all the <Key, Value> pairs, it reads out all the <Key, Value> pairs stored in the storage device STOR and classifies (aligns) them for each Key. Reduce processing is performed.

However, in the method shown in FIG. 5A, the classification (alignment) processing for each key needs to be performed on all <Key, Value> pairs received by the self-calculation node. (Alignment) takes time. In view of this, as shown in FIG. 5B, a plurality of storage devices are prepared and bucket sorting (bucket sorting) is considered.

In FIG. 5B, CALC_NODE_1 includes 10 storage devices from STOR1 to STOR10, and these storage devices constitute a bucket group. The calculation node CALC_NODE_1 distributes and stores the received <Key, Value> set to 10 storage devices according to the Key number. This distribution is performed by means such as, for example, a <Key, Value> set with a Key number from 0 to 99 in STOR1, and a <Key, Value> set with a Key number from 100 to 199 in STOR2. After completion of the Shuffle phase and reception of all <Key, Value> pairs, first, the <Key, Value> pairs stored in the storage device STOR1 are read out and classified (aligned) for each Key. Reduce processing. After all the <Key, Value> pairs in the storage device STOR1 have been processed, the <Key, Value> pairs stored in the storage device STOR2 are read out, classified (aligned) by key, and reduced. To do. After that, similarly, STOR3 to STOR10 may be classified (aligned) in order and reduced.

In the method shown in FIG. 5B, the classification (alignment) processing for each key may be performed for each storage device STOR1 to STOR10. Therefore, the classification (alignment) processing time is longer than that in FIG. It can be shortened. Here, access from each storage device (bucket) STOR1 to STOR10 is only sequential access, and a storage device with low random access can be used. In addition, when a storage device that can access multiple ports simultaneously can be used, it is possible to configure a bucket group with a single storage device by making each access port correspond to a different key number range (bucket). It is. Here, the bucket groups (storage devices STOR1 to STOR10) are assigned in ascending order of the keys. However, it is sufficient that the correspondence relationship between the key numbers and the storage devices (buckets) is determined, and the order of the key numbers is not necessarily limited. There is no need to assign. Further, the order of processing the storage device (bucket) after completion of the Shuffle phase is arbitrary.

However, the method shown in FIG. 5B has a problem that it is necessary to prepare a large number of storage devices (buckets) in each computation node. In contrast, the inventors of the present application have found the following configuration.

FIG. 6 shows an example in which each of the ten calculation nodes CALC_NODE_1 to 10 of the information processing system 101 is in charge of Reduce processing of 1000 types of key numbers. If ten storage devices (buckets) are arranged in each computation node by the bucket sort method shown in FIG. 5B, 100 parallel storage devices (buckets) are required for the entire parallel computing device. By the way, as described above, in the bucket sort method shown in FIG. 5B, after the completion of the Shuffle phase, the bucket stored in the self-calculation node is stored as a set of <Key, Value> having a small key number. Classification (alignment) and Reduce processing are performed in order. If the distribution of the Keys output in the Map phase is not extremely varied, the time required for CALC_NODE_1 to process the buckets with the key numbers 0 to 99 and CALC_NODE_2 to process the buckets with the key numbers 1000 to 1099 The required time is considered to be approximately the same. Therefore, the time when CALC_NODE_1 starts the processing of the buckets with the key numbers 100 to 199 and the time when CALC_NODE_2 starts the processing of the buckets with the key numbers 1100 to 1199 are almost the same time. The inventors of the present application need this at almost the same time. That is, the buckets that start classification (alignment) / reduce processing at almost the same time share the calculation nodes across the computation nodes. Found that the number of buckets to be reduced.

FIG. 7 is a diagram for explaining an example of parallel processing operation in the information processing system 101. In the description of the embodiment, FIG. 7 shows an example in which parallel processing is executed by 10 calculation nodes CALC_NODE_1 to CALC_NODE_10, and each calculation node is responsible for Reduce processing of 1000 types of key numbers. . However, as described above, the key is an integer number, the reduction process or the bucket allocation method is arbitrary.

In the parallel processing operation illustrated in FIG. 7, ten storage devices (buckets) BSTOR1 to BSTOR10 that temporarily store a pair of <Key, Value> output in the Map phase, and ten units that perform Reduce processing. The calculation nodes CALC_NODE_1 to CALC_NODE_10 and the controller BUCKET_CONT that performs bucket retransmission management function.

The computation nodes CALC_NODE_1 to 10 realize the functions of performing the MAP processing, the classification (alignment) for each key, and the Reduce processing with the storage devices STOR1 to STOR10. As shown in FIG. 7, the storage device STOR of the calculation node CALC_NODE_1 is associated with STOR1, and the storage device STOR of the calculation node CALC_NODE_2 is associated with STOR2.

A group of buckets BSTOR1 to BSTOR10 that temporarily store a set of <Key, Value> are BSTORE1 as the storage device STOR of the calculation node CALC_NODE_1, BSTR2 as the storage device STOR of the calculation node CALC_NODE_2, and BSTR3 as the storage device STOR of the calculation node CALC_NODE_3. In this way, the calculation node is also used as a storage device STOR of a calculation node that executes calculation by parallel processing. By doing so, parallel computation can be executed with a small amount of resources. In addition, for example, the storage device STOR of the calculation node CALC_NODE_11 is a storage device STOR of a calculation node that does not perform calculation in parallel processing, such as BSTORE1, the storage device STOR of the calculation node CALC_NODE_12 is BSTR2, and the storage device STOR of the calculation node CALC_NODE_13 is BSTOR3. It may be realized with. Also, the number of bucket groups BSTOR1 to BSTOR10 and the number of calculation node groups CALC_NODE_1 to CALC_NODE_10 that perform Reduce processing may be different. In a configuration in which the number of bucket groups and the number of calculation node groups are different and buckets are arranged on the calculation nodes, an integer number of storage devices (buckets) of 0 or more are arranged per calculation node.

The controller BUCKET_CONT that performs bucket retransmission management is realized by a retransmission management node that is one of the calculation nodes CALC_NODE_x. Here, the retransmission management node may be realized as a calculation node that contributes to parallel processing, or may separately use a calculation node that is not related to the parallel processing calculation.

In the parallel processing method shown in FIG. 7, the above-described Shuffle phase (inter-computation node communication) is further subdivided into two phases: a bucket transmission phase and a bucket retransmission phase. In the bucket transmission phase, instead of directly transmitting the <Key, Value> pair output in the Map process to the computation node that performs the Reduce process corresponding to the Key, the bucket group BSTOR1 shared by the entire parallel process is transmitted to the BSTOR10. To do. At this time, if the remainder of dividing the Key number by 1000 is 0 to 99, the destination bucket is assigned to BSOTR1, and if the remainder of dividing the Key number by 1000 is 100 to 199, the destination bucket is assigned to BSOTR2. Determine. In this way, each computation node gives a destination to one of BSTORs 1 to 10 for a set of <Key, Value> that is a result of the Map processing in the computation nodes CALC_NODE_1 to 10.

When all pairs of <Key, Value> output in the Map phase are stored in the bucket groups BSTOR1 to BSTOR10, the process proceeds to the bucket retransmission phase. In the bucket retransmission phase, in accordance with an instruction from the controller BUCKET_CONT, the set of <Key, Value> stored in each bucket is retransmitted to the original destination calculation node (calculation node performing the Reduce process). More specifically, first, the set of <Key, Value> stored in the bucket BSTOR1 is retransmitted to the original destination calculation node (calculation node that performs the Reduce process). At this time, each of the calculation nodes CALC_NODE_1 to 10 stores the received <Key, Value> set in a storage device STOR included in the own calculation node (for example, STOR1 corresponds to the calculation node CALC_NODE_1). When the retransmission of the set of <Key, Value> stored in the bucket BSTOR1 is completed, the controller BUCKET_CONT notifies the calculation node CALC_NODE_1 to CALC_NODE_10. Then, each of the calculation nodes CALC_NODE_1 to CALC_NODE_10 reads a set of <Key, Value> stored in the STOR10 from the storage device STOR1, classifies (aligns) each key, and performs a Reduce process. When all the computation nodes finish the Reduce process and the storage devices STOR1 to STOR10 become empty, the controller BUCKET_CONT starts retransmitting the next bucket BSTOR2. Similarly, retransmission and Reduce processing are sequentially performed for each bucket until the last bucket BSTOR10.

As described above, in the parallel processing method shown in FIG. 7, each computing node performs MAP processing in the first grouping, whereas each storage device of BSTOR 1 to 10 in the second grouping is processed. And retransmitted from each storage device to each computation node according to the first grouping. As a result, by preparing 10 buckets BSTOR1 to BSTOR10 in the entire parallel processing, the time required for the classification (alignment) processing in each computation node is prepared when 10 buckets are prepared for each computation node (FIG. 5). It can be shortened to the same as (b). On the other hand, since communication between calculation nodes is performed twice in the bucket transmission phase and the bucket retransmission phase, the amount of communication increases. However, in general, the time required for communication between computation nodes is proportional to N with respect to the data amount N, whereas the time required for classification (alignment) is proportional to N × Log (N). As the amount increases, the reduction in the processing time for classification (alignment) becomes more effective in reducing the overall processing time.

In the above description, the number of Keys responsible for Reduce processing to each computation node is simply 1000 in ascending order of the Key number, and the Key allocation method for each bucket is the remainder obtained by dividing the Key number by 1000. However, this determination method is arbitrary, and other allocation methods can be applied. On the other hand, in order to further enhance the effect of the present invention, as shown in FIG. 6, the assignment method of the key responsible for the reduction process to each computation node and the assignment method to each bucket are as orthogonal as possible. It is desirable to have a relationship. That is, it is desirable that the processing targets included in any one group in the first grouping are distributed so as to be included in each group in the second grouping. Furthermore, a desirable allocation method depends on the distribution of the Key output in the Map phase, and differs depending on the application and input data. Therefore, it is conceivable that the application designer or user can set the assignment method of the key responsible for the reduction processing to each computation node and the assignment method to each bucket. It is also possible to perform state transition between phases by exchanging necessary information between each bucket and each computation node without having to use a controller BUCKET_CONT for performing bucket retransmission management.

Further, as described above, graph processing data can also be used as a processing target of the information processing system 101. In the case of graph structure data, the graph is divided into the number of computation nodes performing parallel processing and assigned to each computation node. In each computation node, processing of the assigned vertex group is performed.

FIG. 8 is a diagram for explaining a second example of the parallel processing operation in the information processing system 101. The system of this embodiment applies the idea of a radix sort (Radix Sort) method, which is a method of performing more precise sorting by performing bucket sorting a plurality of times, and reuses the same bucket group BSTOR1 to BSTOR10. By performing the bucket sort twice, it is possible to obtain substantially the same effect as when the number of buckets is squared. In the present embodiment, a group of buckets BSTOR1 to BSTOR10 that temporarily store a set of <Key, Value> are stored, storage device STOR of calculation node CALC_NODE_1 is BSTOR1, storage device STOR of calculation node CALC_NODE_2 is storage of BSTOR2, and storage of calculation node CALC_NODE_3 A description will be given by taking, as an example, a format in which the device STOR is realized by being used also as a storage device STOR of a calculation node that performs calculation by parallel processing, such as BSTR3.

In the processing method shown in FIG. 8, a set of <Key, Value> output in the Map phase is transmitted from the bucket BSTOR1 determined based on the remainder obtained by dividing the Key number by 100 to the BSTOR10. After all the communication is completed, each calculation node reads out the stored contents of the bucket BSTOR1 and transmits it again from another bucket BSTOR1 to the BSTOR10. At this time, the destination bucket is determined based on the remainder obtained by dividing the Key number by 1000. Similarly, after the retransmission of the bucket BSTOR1 is completed, the retransmission is sequentially performed from the bucket BSTOR2 to the bucket BSTOR10. After the retransmission of all buckets is completed, each calculation node again reads out the stored contents of the bucket BSTOR1 and sends it again to the original destination calculation node (calculation node that performs the Reduce process). At each calculation node, Local Sort and Reduce processing is performed. After the processing of the bucket BSTOR1 is completed, similarly, the packet BSTOR2 to the bucket BSTOR10 are sequentially sent again, and the Local Sort and Reduce processing is performed at each calculation node.

As described above, in the processing method shown in FIG. 8, it is possible to reduce the time required for the classification (alignment) processing in each computation node to the same level as when 100 buckets are prepared for each computation node. It is. It is possible to increase the number of bucket transmissions and retransmissions more than two. By increasing the number of retransmissions, the time required for classification (alignment) for each key at each computation node is reduced, while the amount of communication between computation nodes increases, so it is optimal for the application and input data. There is a number of bucket retransmissions. Therefore, it may be possible for an application designer or user to set the number of bucket retransmissions.

FIG. 9 is a diagram for explaining a second example of the parallel processing operation in the information processing system 101. In the system of the present embodiment, in the first embodiment of the present invention shown in FIG. 7, the bucket is retransmitted after the bucket transmission phase is completed, that is, output in the Map phase <Key, Value>. Is started after all the buckets BSTOR1 to BSTOR10 are stored. Therefore, the original destination calculation node (the calculation node performing the Reduce process) cannot receive any <Key, Value> pair until the bucket retransmission phase starts. However, if the original destination computation node (compute node that performs the Reduce process) can receive the <Key, Value> pair from an early stage, it will receive the <Key, Value> pair in parallel. The inventors of the present application have considered that the entire processing time may be shortened by performing the classification (alignment) processing for each key little by little (online).

In the processing method shown in FIG. 9, each bucket BSTOR1 to BSTOR10 has a remainder obtained by dividing the Key number by 1000 from 91 to 181, 182 to 272, and so on. Similarly, 90 or 91 Key numbers are assigned. assign. 90 or 91 corresponds to a number obtained by dividing the number of keys (1000) for which one computation node is in charge of Reduce processing by the number of bucket groups (10) +1. In addition, the time required for completing all Map processes is estimated in advance, and the contents stored in the buckets BSTOR1 to BSTOR10 are sequentially read every time 1/10 of the time for completing all Map processes has elapsed. Then, the data is retransmitted to the original destination calculation node (calculation node performing the Reduce process). By doing so, the original destination calculation node (the calculation node that performs the Reduce process) can receive the <Key, Value> set from an early stage, and receive the <Key, Value> set while receiving the parallel. Thus, the classification (alignment) processing for each key can be performed little by little (online).

Specifically, prior to the start of the Map phase, the time Tmap required for all Map processes is estimated by some means. Usually, since the number of Map processes and the processing contents are known in advance, it is possible to estimate the processing time of the entire Map process. Even if the estimated time has some errors, the following operations are not hindered.

After the start of the Map phase, the <Key, Value> set output in the Map phase is applied to the bucket corresponding to the Key number, and if there is no bucket corresponding to the Key number. The data is sent to the original destination calculation node (the calculation node that performs the Reduce process). Therefore, a set of <Key, Value> with a Key number from 0 to 90 is not directly sent to the bucket group, but is directly transmitted to the original destination calculation node (calculation node that performs Reduce processing). .

After that, at the timing when the time Tmap / 10 has elapsed from the start of the Map phase, the storage contents of the bucket BSTOR1 are read out and retransmitted to the original destination calculation node (calculation node that performs the Reduce process). From now on, the <Key, Value> pair output by the Map processing is in charge of the bucket BSTOR1 in addition to the <Key, Value> pair whose key number is 0 to 90 and no corresponding bucket exists. A pair of <Key, Value> with key numbers 91 to 181 is also transmitted directly to the original destination calculation node (calculation node that performs Reduce processing). Thereafter, when the time Tmap / 10 further elapses and the time (Tmap / 10) × 2 has elapsed since the start of the Map phase, the stored contents of the bucket BSTOR2 are read and the original destination calculation node (Reduce processing is performed) Re-send to (compute node). From then on, the <Key, Value> pair output by the Map process is the same as the original destination calculation node (<Key, Value>) even when the <Key, Value> pair that was handled by the bucket BSTOR2 is output by the Map process. It is directly transmitted to a computing node that performs Reduce processing. In the same manner, each time when Tmap / 10 further elapses, the bucket is retransmitted toward the original destination calculation node (the calculation node that performs the Reduce process), and thereafter, the <Key, Value output in the Map process is output. The pair of> is assigned to the bucket if the bucket corresponding to the Key exists and the bucket has not been retransmitted, and directly to the original destination calculation node (Reduce processing) otherwise. To the computing node that performs the transmission.

In the above description, for the sake of simplicity, all the buckets are assigned the same number of keys, and the buckets are retransmitted at a timing obtained by dividing the time Tmap required for all Map processing at equal intervals. However, in actuality, when the number of <Key, Value> pairs stored in each bucket at the bucket re-transmission timing differs depending on the bucket, the use efficiency of the storage device and the network efficiency are reduced. There is. This is because the bucket BSTOR1 stores a set of <Key, Value> only for a time of Tmap / 10, whereas the bucket BSTOR10 stores a set of <Key, Value> for a Tmap. Therefore, if the distribution of the Key output in the Map phase is uniform, the bucket BSTOR1 stores only 1/10 <Key, Value> pairs of the bucket BSTOR10 at the timing of bucket retransmission. Will not be. In order to keep the number of <Key, Value> pairs stored in each bucket at the time of bucket retransmission as much as possible, the number of keys handled by each bucket is increased in the bucket BSTOR1, and in the bucket BSTOR10. It is possible to reduce it. Or, instead of simply dividing the Tmap into equal parts, instead of dividing the Tmap evenly, the time between retransmitting the previous bucket and retransmitting the next bucket is gradually shortened. Good.

As described above, in the processing method according to the third embodiment of the present invention shown in FIG. 9, the original destination calculation node (reduction processing is performed by retransmitting the bucket in order in parallel with the Map processing. Node) can receive <Key, Value> pairs from an early stage, and receive the <Key, Value> pairs in parallel (online) little by little (online) classification (alignment) for each Key. Processing can be performed, and the overall processing time can be shortened.

101: Information processing system, CALC_NODE_x: calculation node, COM_SW: communication switch, MEM: temporary storage device, STOR: storage device, COM_DEV: communication device, STOR: storage device, BUS: bus, BSTOR1 to 10: storage device (bucket) , BUCKET_CONT: A controller that performs retransmission management of buckets.

Claims

A parallel processing method in a parallel computer system having a plurality of computation nodes,
Divide the processing target in the first grouping and place it on each computation node for processing,
A destination for each second group to which each of the processing results belongs,
Save the processing result in a different storage device for each destination,
A parallel processing method, comprising: transmitting a stored processing result to each computing node according to the first grouping.
The parallel processing method according to claim 1,
A parallel processing method, wherein each storage device in which the processing result is stored is arranged on each computation node.
The parallel processing method according to claim 1,
When sending the stored processing result to each computing node according to the first grouping,
The parallel processing method, wherein the stored processing result is transmitted with a time difference for each second grouping.
The parallel processing method according to claim 1,
The parallel processing method according to claim 1, wherein the processing targets included in any one group of the first grouping are distributed so as to be included in each group of the second grouping.
The parallel processing method according to claim 1,
The parallel processing method characterized in that the processing target is graph structure data.
A parallel computer system having a plurality of computation nodes,
Divide the processing target in the first grouping and place it on each computation node for processing,
A destination for each second group to which each of the processing results belongs,
Save the processing result in a different storage device for each destination,
A parallel computer system, wherein a stored processing result is transmitted to each computation node according to the first grouping.
The parallel computer system according to claim 6,
A parallel computer system, wherein each storage device in which the processing result is stored is arranged on each computation node.
The parallel computer system according to claim 6,
When sending the stored processing result to each computing node according to the first grouping,
A parallel computer system, wherein the stored processing result is transmitted with a time difference for each of the second groupings.
The parallel computer system according to claim 6,
The parallel computer system is characterized in that at least one processing target included in any one group of the first grouping is distributed so as to be included in each group of the second grouping.
The parallel computer system according to claim 6,
The parallel computer system, wherein the processing object is graph structure data.