JPS63126030A - Sort processing system - Google Patents

Abstract

PURPOSE:To effectively utilize the resource of a computer system and to shorten the execution time of sort processing by executing the sort processing by one main sort process and plural sub-sort processes. CONSTITUTION:A sub-sort process forming mans 11 in the main sort process 1 divides data in the leading block number of an input file 4 into block groups 41-43 equal to the number of primary intermediate files 61-63 and forms sub-sort processes 31-33 corresponding to the block groups 41-43. When a replacing information storing means 2 confirms the end of the sort processing of all the formed processes 31-33, a merging means 12 inputs data records from the field 61-63 storing the sorted results of the processes 31-33, executes merge processing and outputs the final sorted result for all the data records in an input file 4 to an output file 5.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a sort processing method, and particularly to a sort processing method when data is sorted using an auxiliary storage device in a computer system.

[Conventional technology]

Conventionally, in this type of sort processing method, when one sort process is executed, the sort process is always executed by one process, and the process includes data input processing, key comparison processing, intermediate file input/output processing, and data processing. A single sorting process sequentially performs a series of processes such as output processing.

[Problem that the invention seeks to solve]

In the conventional sort processing method described above, only one process can execute the sort process at any time, so even if the computer system has sufficient capacity and number of main storage and auxiliary storage devices, it is difficult to use them. They cannot be used in parallel, and have the disadvantage that the sorting process takes a long time when sorting a large amount of input data.

In view of the above points, the present invention has one main objective.

In addition to the sorting process, multiple secondary sorting processes are generated, the input data to be sorted is divided, and the multiple secondary sorting processes are used to sort the data independently and in parallel. It is an object of the present invention to provide a sort processing method that can effectively utilize the resources of a computer system such as a main storage device and an auxiliary storage space and shorten the execution time of a sort process.

[Means for solving problems]

The sort processing method of the present invention requires an input file on an auxiliary storage device that can be randomly accessed, which stores data records to be sorted and an effective number of blocks, which is the number of blocks to be sorted; A main sorting process that is started by a user and has a sub-sorting process generation means and a merging means, reads data on the number of effective blocks stored in the input file, and divides the blocks in the input file into multiple blocks based on this number of effective blocks. A sub-sorting process that divides into block groups of a number equal to the number of primary intermediate files, sets manual file classification information based on this division in an exchange information storage means, and performs a sorting process on each of the divided block groups. a sorting process targeting the input block group by inputting one block group in the input file based on the sub-sorting process generating means and the input file classification information set in the exchange information storage means; the sub-sorting process, which stores the data record of the sorting result in the primary intermediate file when the sorting process is completed, and sets the completion information of the sorting process in the exchange information storage means; the main sorting process and the sub-sorting process; said exchange information storage means for storing information exchanged between said primary intermediate file storing data records subjected to sorting processing for a group of blocks by said sub-sorting process, and said sub-sorting process generation; After confirming the completion of the sort processing of all the sub-sorting processes generated by the means using the end information of the sorting processing set in the exchange information storage means, all the sorting results of the sub-sorting process are stored. and the merging means for inputting data records from the primary intermediate file, performing a merging process, and outputting a final sorting result for all data records in the input file.

[Effect]

In the sort processing method of the present invention, the input file stores data records to be sorted and the effective number of blocks, which is the number of blocks to be sorted, on an auxiliary storage device that can be randomly accessed, and the main sort process is activated by a sort user and has a sub-sort process generation means and a merging means, and the sub-sort process generation means reads the data of the effective number of blocks stored in the input file and processes the data in the input file based on the number of effective blocks. divides the blocks into block groups of a number equal to the number of the plurality of primary intermediate files, sets input file classification information based on this division in the exchange information storage means, and performs a sorting process on each of the divided block groups. A sub-sort process is generated, and the sub-sort process inputs one block group in the input file based on the input file classification information set in the exchange information storage means, performs sorting processing on the input block group, and performs sorting. When the processing is completed, the data record of the sorting result is stored in the primary intermediate file, the completion information of the sorting processing is set in the exchange information storage means, and the exchange information storage means exchanges information between the main sorting process and the sub-sorting process. , the primary intermediate file stores data records that have been subjected to sorting processing for block groups by the sub-sorting process, and the merging means stores the sorting processing of all the sub-sorting processes generated by the sub-sorting process generation means. After confirming the completion of the sorting process using the sorting completion information set in the exchange information storage means, data records are input from all primary intermediate files in which the sorting results of the sub-sorting process are stored, and the merge process is performed and the data is stored in the input file. Outputs the final sorting results for all data records.

〔Example〕

Next, the present invention will be explained in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of the sorting method of the present invention.
A main sort process 1 activated by a sort user to perform a sort process on data records in an input file 4, and an exchange for storing information exchanged between the main sort process 1 and sub-sort processes 31, 32 and 33. Information storage means 2, an input file 4 on a randomly accessible auxiliary storage device (not shown) that stores data records to be sorted, and an output file 5 to which the final sorting results are output. and input file classification sort processing information 21, 22 and 23 (
(see Figure 3) and input the block groups 41, 42 and 43 in the input file 4 into the secondary intermediate file 71.
．． A sub-sorting process that performs sorting processing using 72 and 73, and when the processing is completed, stores the data records of the sorting results in primary intermediate files 61, 62 and 63, and sets the end information of the sorting processing in the exchange information storage means 2. 31.32 and 33 and sub-sort process 31.32
and primary intermediate files 61, 62 and 63 that store data records subjected to sorting processing for block groups 41, 42 and 43 by 33, and temporary files 61. It consists of secondary intermediate files 71, 72 and 73 used for storing data records.

The main sorting process 1 reads the data of 40 effective blocks stored at the beginning of the input file 4 and divides the blocks in the input file 4 into block groups 41, 42 whose number is equal to the number of primary intermediate files 61, 62 and 63. and 43, and input file classification sort processing information 21, 22 and 23 based on this classification are set in the exchange information storage means 2 to generate sub-sort processes 31, 32 and 33 corresponding to block groups 41, 42 and 43. After confirming by the exchange information storage means 2 that the sorting process of all the sub-sort processes 31.32 and 33 generated by the sub-sort process generation means 11 and 33 generated by the sub-sort process generation means 11 has been confirmed by the exchange information storage means 2, the sub-sort process 31.32 and 33
A primary intermediate file 61. in which the sort results of are stored.
The merging means 12 inputs data records from 62 and 63, performs merging processing, and outputs a final sorting result for all data records in the input file 4 to the output file 5. .

The input file 4 includes the number of effective blocks 40 and block groups 41, 42, and 43.

Referring to FIG. 2, the number of effective blocks 40 is located at the beginning of the input file 4 and stores the number N (positive integer) of blocks included in the input file 4 that are the targets of the sorting process.

The block groups 41, 42, and 43 are divided into the same number as the primary intermediate files 61, 62, and 63, and contain the number of blocks that the sub-sorting processes 31, 32, and 33 input as targets for sorting. That is, the quotient M (positive integer) obtained by dividing N indicated by the effective block number 40 by 3, which is the number of primary intermediate files 61, 62, and 63, is used as the number of blocks input by the sub-sorting processes 31 and 32. The remaining number of blocks included in block groups 41 and 42 (N-2X
M) is included in the block group 43 as the number of blocks input by the sub-sorting process 33.

Referring to FIG. 3, the information set in the exchange information storage means 2 includes the block size 20 indicating the size of one block in the input file 4, and the sub-sorting process 31, 32.
Block group 41. and 33 are the targets of the sorting process.
The input file classification sort processing information 21, 22, and 23 are set corresponding to 42 and 43, respectively.

Input file classification sort processing information 21, 22 and 23
is the input start block number 211 indicating the first block number of the block input from the input file 4 for the sub-sorting processes 3L 32 and 33 to perform the sorting process.
221 and 231 and sub-sort process 31.32
and 33 indicate the number of blocks manually input from the input file 4 to perform sorting processing 212, 2
22 and 232 (input start block number 21
1,221 and 231 and number of input blocks 212.2
22 and 232 can be collectively referred to as input file classification information). Sub-sort process processing end flags 213, 223, and 2 are set to on state when sorting processing ends.
It consists of 33. As will be described later, the input start block number 211 is "1", the input block number 212 is "M", the input start block number 221 is rM+IJ,
The input block number 222 stores the information "M," the input start block number 231 stores the information "r2XM+IJ," and the input block number 232 stores the information "rN-2×M."

-g is one on the auxiliary storage device that can be accessed randomly.
Since all blocks are the same size within one file, once the block number to be input is determined, the calculation formula (block number - 1) x block size is used to calculate the block size within the file that block occupies. The position of the block can be simply calculated and the input of the desired block can be made.

From the above, sub-sort processes 31, 32 and 3
3, the position of the block to be input in the input file 4 can be easily calculated from the input start block numbers 211, 221, and 231, and the desired block group 41.4 can be easily calculated.
2 and 430 inputs can be made.

Referring to FIG. 4, the processing of the sub-sort process generation means 11 includes a step 101 of reading the number of effective blocks and storing the block size, a step 102 of determining the number of sub-sort processes, and a step 103 of calculating the number of sub-sort process blocks. , counter initial value setting step 104 , sub-sort process counter determination step 105 , exchange information storage means white information setting step 106 , sub-sort process generation startup step 107 , counter increase step 108 , and exchange information storage means white information setting step 106 It consists of a final sub-sorting process correspondence information setting step 109 and a final-side sorting process generation activation step 110.

Next, the operation of the sort processing method of this embodiment configured as described above will be explained.

When sorting the data records in the input file 4, the sorting user first sorts the data records in the main sorting process 1.
is activated, and the sub-sorting process generating means 11 in the main sorting process 1 is executed.

The sub-sort process generation means 11 reads N from the number of effective blocks 40 in the input file 4, and
The size of one block in the input file 4 obtained by an instruction from the sort user or by reading from the input file 4 is stored in the block size 20 of the exchange information storage means 2 (step 101).

Next, the number of primary intermediate files 61, 62 and 63 specified by the sort user (3 in this case) is checked, and the number is determined by the sub-sort process 31, 32 to be generated.
and 330 (this number is also the number of block groups 41, 42 and 43 in input file 4) (step 1o2).

Furthermore, N, which is the number of effective blocks in the input file 4, 40, is divided by 3, which is the number of primary intermediate files 61, 62, and 63, to calculate the number of blocks, M, which is input to the sub-sorting processes 31 and 32 (step 103). .

Here, 1 is set as an initial value to a positive integer J as a sub-sort process counter that counts the number of sub-sort processes 31, 32 and 33, and input start block number 211
, 221 and 231, a positive integer K serving as an input start block number counter is set to 1 as an initial value (
Step 104).

Next, the secondary sorting process counter J counts the primary intermediate file 61. ．． It is determined whether the number is 3 or more of the numbers 62 and 63 (step 105).

If J is smaller than 3 in this judgment, the input start block number 211 or 221 of the area corresponding to the fifth sub-sorting process (sub-sorting process 31 or sub-sorting process 32) in the exchange information storage means 2 is selected. The value is stored in the input start block number counter and the block number M calculated in step 103 is stored in the input block number 212 or 222, and the sub-sort process processing end flag 213 or 223 is set to the sub-sort process 31 or 222. 32 is set to the off state to indicate unfinished (
Step 106).

Furthermore, the automatic job generation function present in the operating system (not shown) is used to generate and start the fifth sub-sort process 31 or 32 (step 107).

Next, 1 is added to the secondary sorting process counter J, M is added to the input start block number counter K (step 108), control returns to the process of step 105, and the secondary sorting process counter J is added to the primary intermediate file 61. .6
The processes of steps 106 to 108 described above are repeated until the number is equal to 2 and 63 (in this case, 3).

If J is equal to 3 as determined in step 105, the input start block number of the area corresponding to the fifth sub-sorting process (i.e., the final sub-sorting process 33) in the exchange information record 19 means 2. 231 stores the value of the input start block number counter ((J-1)XM+1), that is, (2XM+1), and the number of input blocks is 23.
2 to (N-(J-1)XM), i.e. (N-2xM
) is stored, and the sub-sort process processing end flag 233
is set to an off state (step 109), and the fifth (final) sub-sorting process 33 is generated using the job automatic generation function and activated (step 110).

The sub-sorting processes 31, 32 and 33 generated and started in this way refer to the input file classification sorting processing information 21, 22 and 23 of the exchange information storage means 2, and the block group 41 in the input file 4, Data records are input from 42 and 43 and sorted (the sorting by the sub-sorting processes 31, 32 and 33 is executed in parallel). If intermediate files are required in this sorting process, secondary intermediate files 71, 72 and 73 are used.

When the sorting processes of the secondary sorting processes 31.32 and 33 are completed, the data records of the sorting results are stored in the primary intermediate files 61.62 and 63, and the exchange information of the completion of the sorting process of the secondary sorting processes 31.32 and 33 is stored. Setting in the storage means 2 (setting the sub-sort process processing end flags 213, 223, and 233 to the on state) is performed.

The merging means 12 in the main sorting process 1 merges all the sub-sorting processes 31 .
After the sort processing of 32 and 33 is completed (the sub-sort process processing end flags 213, 223 and 233 are on), the primary intermediate file 61.6 is GitUed.
2 and 63, the sorted data records targeting block groups 41, 42 and 43 are input and merge processing is performed, and the final data records targeting all data records in input file 4 are input through the merge processing. The results of the sorting process are output to the output file 5.

Note that in this embodiment, the block group 41 in the input file 4
．． 42 and 43 (the number of sub-sort processes 31 and 32 and 33) is 3, but this number is determined by the sort user's -next intermediate file 61.62
It goes without saying that the number can be freely set by specifying the numbers of and 63.

〔Effect of the invention〕

As explained above, the present invention executes the sorting process by one main sorting process and a plurality of sub-sorting processes, divides the input data to be sorted into a plurality of block groups, and targets each block group. To effectively utilize the resources of a computer system and shorten the execution time of sorting processing (especially sorting processing of a large amount of input data) by performing sorting processing independently and in parallel using multiple sub-sorting processes. It has the effect of being able to

[Brief explanation of the drawing]

FIG. 1 is a blog diagram showing the configuration of an embodiment of the present invention, FIG. 2 is a diagram showing the structure of data in the input file in FIG. 1, and FIG. 3 is an exchange information storage means in FIG. 1. FIG. 4 is a flowchart showing the processing of the sub-sorting process generating means in FIG. 1. In the figure, 1... Main sorting process, 2... Exchange information storage means, 4... Input file, 5... Output file, 11... Sub sorting process generation means, 12... Merging means. , 20...Block size, 21-23-Input file classification sorting processing information, 31-
33. Sub-sort process, 40... Number of effective blocks, 41-43. Block group, 61-63. Primary intermediate file, 71-73. Secondary intermediate file, 211.221,231... Input start block number. , 212.222, 232: Number of input processes, 213.223.233: Sub-sort process end flag.

Claims (1)

[Scope of Claims] An input file on an auxiliary storage device that can be randomly accessed and stores data records to be subjected to sort processing and data for an effective number of blocks that is the number of blocks to be sorted; and a sort user. a main sorting process that is started by and has a sub-sorting process generation means and a merging means; reads data on the number of effective blocks stored in the input file and converts the blocks in the input file into a plurality of primary sort processes based on the number of effective blocks; Generates a sub-sorting process that divides into block groups of a number equal to the number of intermediate files, sets input file classification information based on this division in the exchange information storage means, and performs a sorting process on each of the divided block groups. One block group in the input file is input based on the input file classification information set in the sub-sort process generation means and the exchange information storage means, and a sorting process is performed on the input block group. between the main sorting process and the sub-sorting process, the sub-sorting process storing the data record of the sorting result in the primary intermediate file when the processing is completed, and setting the completion information of the sorting process in the exchange information storage means; said exchange information storage means for storing information to be exchanged; said primary intermediate file for storing data records subjected to sorting processing for block groups by said sub-sorting process; and generated by said sub-sorting process generation means. After confirming the completion of the sort processing of all the sub-sort processes that have been performed based on the end information of the sort processing set in the exchange information storage means, all the primary intermediates in which the sort results of the sub-sort processes are stored are A sorting processing method comprising: the merging means for inputting data records from a file, performing merging processing, and outputting a final sorting result for all data records in the input file.