JP2509929B2 - Parallel sort processing method - Google Patents

Description

DETAILED DESCRIPTION [Outline] A processing method for improving the processing efficiency of parallel sorting in a computer system including a plurality of processing devices.

To each of the m processing devices, a sharing range obtained by dividing the data value range into m is assigned, and each processing device distributes the elements of the data to be sorted according to the sharing range.
The sorting of all the data is completed by individually sorting the distributed data.

By this method, the exchange of data between the processing devices for sorting is reduced, and the processing can be executed efficiently.

[Industrial applications]

The present invention relates to a parallel sorting processing method in a computer system including a plurality of processing devices.

Sorting, in which data is arranged in the order of size, is widely used in each field of information processing as is well known.

[Conventional technology]

FIG. 2 is a block diagram showing a configuration example of a computer system.

The processing devices 1-1 to 1 to m are respectively storage devices 2-1.
It is a device that can read out the data held in 2 to 2-m and independently execute sort processing, and can exchange information with each other through a communication path formed by the network 3.

The control processing device 4 communicates with the processing devices 1-1 to 1-m and the like via the network 3 to perform required control relating to the entire system.

If sort of a large amount of data is processed in parallel by distributing the data to the respective processing devices 1-1 to 1-m of such a computer system, there is a possibility that the processing can be speeded up. There is a well-known bitonic merge method as a typical method.

According to this method, the sort target data distributed to a plurality of processing devices are first subjected to the following processing for each set of two processing devices.

That is, as shown in FIG. 3, the data distributed to each processing device by the set of two processing devices A and B (the figure shows an example in which each four elements are distributed) is individually processed in parallel. Then, for example, A is sorted in ascending order and B is sorted in descending order (in the figure), and the corresponding elements of the data of both processing devices are compared, and for example, the smaller element is moved to A and the larger element is moved to B. Yes (in the figure).

By sorting the data in each processing device in ascending order, for example, the processing devices A and B can be sorted (in the figure).

The above processing is executed in parallel for the data that each of the other processing devices has, and then the results of the processing devices A and B and the processing of the different data by the different processing devices C and D are performed. The same comparison and movement processing between the elements as described above is performed for each double length data (in the figure) with the result (in the figure), and the result is set as described above in each set of two processing devices. By sorting, the processing devices A to D can be sorted.

In this way, by sequentially increasing the data length of 4 processing devices × 2, then 8 processing devices × 2, and performing similar processing, the sorting over all data is completed.

[Problems to be solved by the invention]

Although the sorting method can be performed in parallel by the above method, as easily thought from the above description, if it is necessary depending on the movement of 1/2 of all data elements for comparing data between processing devices and the comparison result. It is necessary to repeatedly move the data for each sort processing of each part. Therefore, there is a problem that the amount of data that needs to be transferred between the processing devices is relatively large, and the processing speed cannot be increased.

[Means for solving problems]

 FIG. 1 is a flow chart of processing showing the configuration of the present invention.

The figure shows the processing flow of each processing device, and 30 to 32 are processing steps.

[Work]

Each processing device receives information for dividing the range of data values into m sharing ranges equal to the number of processing devices in processing step 30.

In the processing step 31, the sharing range to which each element of the data distributed to each processing device belongs is identified, and the element is sent to the processing device to which the sharing range is assigned, and only the element assigned to itself is identified. Leave. Also, the element sent to itself is received by the same processing performed in parallel by another processing device.

After the mutual distribution of the elements is completed, in processing step 32, each processing device sorts the data allocated in processing step 31, respectively, thereby completing the sorting of the entire data.

By this processing method, the data can be transferred only once for each element, and the sorting can be completed.

〔Example〕

In the computer system shown in FIG. 2, the data to be sorted is divided into substantially equal numbers of elements, and m storage devices 2
It is assumed that the items are previously distributed to 1 to 2-m.

m number of processing devices 1-1 to 1-m (hereinafter PE _{1 to} P
E _m ) is, for example, a storage device 2-1 to each of which is connected to each processing device according to an instruction sent from a control processing device (hereinafter referred to as CP) 4 to all processing devices by the network 3.
Input data from 2-m and start sort processing.

Thisゝa, for example CP4 is, as the information indicating the boundary of the division ranges, the upper limit of the m-1 pieces of each share ranging from the smallest value _{(a 1, a 2, ~a} m-1) the entire process Notify the devices PE ₁ to PE _m .

Each of PE _{1 to} PE _m is assigned a number of _{1 to m} , and is assigned to each processing device in the order of numbers starting from the allocation range with the smallest value. The sharing range is recognized as follows, where x is the value of the data element.

PE ₁ sharing range x ≤ a ₁ PE ₂ sharing range a ₁ <x ≤ a ₂ .... PE _m-1 sharing range a _m-2 <x ≤ a _m-1 PE _m sharing range Range a _m-1 <x It is desirable from the standpoint of efficiency of parallel processing that the sharing range to be determined in this way is determined so that approximately the same number of elements belong to each range.

Therefore, for example, when it is known that the values taken by each element are almost evenly distributed, first PE _{1 to} P
From E _m to CP4, notify the maximum value and the minimum value of the element of each data in hand, CP4 knows the range of the value of the data to be processed by taking the maximum value and the minimum value from among them,
The upper limits may be calculated so as to divide this range into approximately m parts, and those values may be notified to PE ₁ to PE _m .

However, in order to make sure that the elements belong to each sharing range almost evenly, including the case where the distribution of the element values is biased, for example, as described below, For each of the "/ m" th elements, set i from 1 to m-
It is desirable to change to 1 to obtain m-1 pieces and set them as the upper limit value of the sharing range. Here, "x" indicates the smallest integer larger than x (an integer obtained by rounding up the fractional part of x), and n is the total number of data elements.

That is, the PE _{1 to} PE _m exchange the information with the CP 4 as to the sort target data input from the storage devices 2-1 to 2-m as described below, and the CP 4 sequentially determines the respective upper limit values. To notify you.

FIG. 4 is a flow chart of the upper limit value determination processing, in which the left side shows the processing of CP4 and the right side shows the processing executed in parallel by PE ₁ to PE _m .

CP4 initializes variable j to 1 in process step 10,
The variable k is set to “n / m” and the processing of PE ₁ to PE _m is started.

Each processing device PE _i of PE _{1 to} PE _m initializes a variable j to 1 in processing step 20, sorts the data respectively input from the storage device 2-i, and sets it as Si of the current process.

In processing step 21, the intermediate value Mi is calculated and notified to CP4.

CP4 receives _m from PE _{1 to} PE _m in processing step 11.
The intermediate value M is calculated from each Mi and notified to PE ₁ to PE _m .

In processing step 22, each PEi classifies the elements of Si by M, the set of elements smaller than M as Si ₁ M the set of elements larger than Si ₂ M as Si _3, and Si ₁ , Notify CP 4 of each number of elements bi ₁ and bi ₂ of Si ₂ .

CP4 is notified from all processing devices in processing step 12 b
i ₁ and bi ₂ are aggregated to obtain total b ₁ and b ₂ .

In process step 13, b _1, b ₂ and k are compared, and for example, k ≦ b if _{1 1, b 1 <k ≦} b 1 + 2 if b _2, b ₁ +
If b ₂ <k, all processing devices are notified of 3. In the case of 2, the variable j is incremented by +1 if the next upper limit value is processed.
And, also, to set up a "ni / m" -k-b ₂ +1 by the value of the new j as the value of the new k. In the case of 3, the variable k
Subtract b ₁ + b ₂ from the value of.

In the processing step 23, each PEi sets Si _{1 for 1} and Si ₃ for 3 according to the notification of 1 to ₃ as a new Si according to the notifications of ₁ to 3, returns to processing step 21, and repeats the processing for obtaining the intermediate value by this Si.

When 2 is notified, the value of M at that time is stored as the jth upper limit value a _j , and if j = m−1, the upper limit value determination processing ends.

If j <m−1, the process proceeds to the processing step 24 for determining the next upper limit value, and only the elements larger than the value of M, which is the current j-th upper limit value in the sorting result of the original data, Set them as the first Si for processing, and set j to +
1 and returns to the processing step 21, the new j-th upper limit value,
That is, it is the “ni / m” th from the bottom for all data elements, and the new k in processing step 13 of CP4 for this processing target.
As set to the value, the process of searching for the value to be “nj / m” −k−b ₂ + 1th from the bottom is started.

When m-1 upper limit values a _{1 to} a _m-1 are determined in this way, each PEi recognizes each sharing range as described above,
The i-th sharing range is set as its own sharing range, only elements of its own sharing range in the data are left, and other elements are transferred by the network 3 to the other processing devices PE ₁ to PE _m of the respective sharing range. , Also receives elements that are forwarded to it from other processing units.

As a result, if the data collected in each PE i are individually sorted in parallel, the sorting of all data from PE ₁ to PE _m is completed.

FIG. 5 shows three processing units PE ₁ , PE ₂ and PE ₃
This is an example in which data of 15 elements having the contents of 1 to 15 are sorted in parallel by the processing method described above.

In the figure, the data status of each processing device is shown in the columns labeled PE ₁ , PE ₂ and PE _3, and the processing of CP 4 is shown in the left column.

FIG. 5 (a) is the upper limit value determination processing, and CP4 sets k to "15 × 1/3" = 5 in step. It is assumed that data is input to each PEi like D _{1 to} D ₃ shown in the step.

Since the step results in sorting results of each PEi as shown in S _{1 to} S ₃ and each of the intermediate values M _{1 to} M ₃ shown in the step is sent to CP 4, as in the step, those intermediate values M = 8.
Will be decided and each PEi will be notified.

As in step, each PEi is classified by the intermediate value M = 8, and the number of elements in each set shown in step is CP4.
Will be notified to.

In CP4, the number of elements is calculated as in step, and k
Since k <b ₁ is obtained as compared with = 5, 1 is notified to all PEi.

Therefore, as in step, the new Si ₁ In the PEi
Set to Si and notify CP4 of the intermediate value as in the step. Here, when the number of elements is an even number, a small value is set as an intermediate value by agreement.

In step, M = 3 is determined in CP4, treatment with an intermediate value, is carried out as before in step ~, b ₁ + b ₂ <since the k, the value of k is 5-b ₁ -b Updated to ₂ = 2 and all PEi are notified of 3.

Therefore, in each PEi, Si ₃
Then, as in the step, the intermediate value is notified to CP4.

In step, M = 5 is determined in CP4, treatment with an intermediate value, is carried out as before in step ~, since the _{b 1 + b 2 = k,} 2 is notified to all PEi,
The current M = 5 is determined to be the fifth-largest element, which is the first upper limit value.

Next, by setting k = “15 × 2/3” = 10 and performing the same processing as the above not shown, the second upper limit is set to 10
Will be decided.

FIG. 5 (b) shows a situation in which each PEi distributes data to each other and sorts the resulting data in parallel by the upper limit values 5 and 10 determined above.

Since the upper limit values are 5 and 10, 5 or less is the first processing device
PE ₁ , 6 to 10 are PE ₂ , and those larger than 10 are PE ₃ 's assigned range, and the elements of the assigned range of each other processing device are transferred from the handheld data shown in the step as in the step.

As a result, data is collected in each processing device like a step, and the data is sorted in parallel in each processing device to obtain a sorting result arranged in ascending order over the three processing devices as shown in the step. .

〔The invention's effect〕

As is clear from the above description, according to the present invention, in parallel sort processing of a computer system having a plurality of processing devices, it is possible to significantly reduce the number of times of data transfer between the processing devices, thereby significantly improving the processing efficiency. It has an industrial effect.

[Brief description of drawings]

FIG. 1 is a flow chart of a process showing the configuration of the present invention, FIG. 2 is a block diagram of a configuration example of a computer system, FIG. 3 is an explanatory diagram of the bytenick merge method, FIG. 4 is a flow diagram of an upper limit value determination process, and FIG. FIG. 5 is an explanatory diagram of an example of sorting processing according to the present invention. In the figure, 1-1 to 1-m are processing devices, 2-1 to 2-m are storage devices, 3 is a network, 4 is a control processing device, 10 to 13, 20 to 24, and 30 to 32 are processing steps. Show.

Claims (2)

(57) [Claims] 1. A computer system in which m processing devices are connected by a communication path, wherein each processing device executes sorting of data consisting of a plurality of elements distributed to each processing device in parallel. The processing device receives a predetermined notification indicating a sharing range obtained by dividing the range of the value of the data element into m (30), and the data element held by the processing device is assigned to the m sharing range. The respective elements belonging to the above are distributed to the respective processing devices allocated so as to be shared respectively (31), and the sorting processing is executed for the distributed elements (3).
2) A parallel sorting method characterized by the following. 2. The predetermined notification indicating the sharing range has a predetermined magnitude relationship with ni / m, where n is the number of elements of the data and i is an integer having a value from 1 to m-1. And ni / m
To the nearest integer as j _i, the value of the elements of each of the j _i th size from the smaller elements of the data, as a value indicating the predetermined boundary between the m of the shares range said value The parallel sort processing method according to claim 1, wherein the sharing range is identified.