JPH04178726A - Mass storage data classification processing system - Google Patents

Abstract

PURPOSE:To minimize a time required for the positioning of a magnetic head by linking mass storage data within a paragraph in cylinder unit when the mass storage data are classified by using a magnetic disk device, etc., as an external memory device. CONSTITUTION:In a pre-sort phase 1, a data record is read in from an input file 6, and a string is generated, and it is outputted to the magnetic disk device 8 that is the external memory device, and simultaneously, a string directory 7 is generated, and it is unified in one cylinder unit, and it is paragraphed, and is stored in an internal memory device. The string directory 7 consists of a disk address at which the string is stored, a first key that is the key of the first record of the string, and a final key that is the key of the final record. In a link phase 2, the string directory 7 generated in the pre-sort phase 1 is linked. Thereby, it is possible to minimize a time required for the positioning of the magnetic head.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to a classification processing method for large-capacity data in office data processing, etc. by a computer, and particularly relates to a classification processing method for large-capacity data in office data processing etc. by a computer, and in particular, a large-capacity data classification method using a magnetic disk device etc. as an external storage device. Concerning a method for classifying capacity data.

[Conventional technology]

Conventionally, there is a link classification method as a method for classifying large-capacity data using a magnetic disk device as an external storage device (Patent Application No. 1982-258068). This consists of the following four phases, as shown in FIG.

(1) Presort phase (Phase 1) (2) Link phase (Phase 2) (3) Merge phase (Phase 4) (4) Last pass phase (Phase 5) In Presort phase 1, data records are extracted from input file 6. are read, grouped in order, and output to the magnetic disk drive 8 as ordered sets, that is, strings, and at the same time, the position (address) on the magnetic disk drive 8 where each of these strings was output and its A string direct IJ7 consisting of the key of the first record of the string and the key of the last record is output to the internal storage device.

In link phase 2, this string directory is input and sorted in ascending order of the keys of the first record, and the keys of the last record of the first string directory and the keys of the first records of subsequent string directories after sorting are input. If the latter is larger, both string directories are linked.

This allows the strings to be merged logically.

In the next merge phase 4, more strings processed in the link phase are merged until the number of these strings is less than or equal to the merge order of the last pass.

In the last pass phase 5, the strings merged in the merge phase 4 are created into one string and written to the output file 9.

[Problem to be solved by the invention]

In the conventional large-capacity data classification processing method described above, emphasis is placed on making use of random access, which is a characteristic of external storage devices such as magnetic disk devices, so that 0. Since strings are linked logically, ignoring their physical characteristics, when physically exchanging and merging logically linked strings, the time required to position the magnetic head ( The disadvantage is that the seek time is extremely long.

In other words, in the linking phase, input the string directory, sort it in ascending order of the keys of the first record, and after sorting, the key of the last record of the first string directory and the key of the first record of the subsequent string directories. Because of the process of comparing and linking both string directories if the latter is larger, there is a possibility that strings that are physically very far apart will be linked, which will increase seek time. .

[Means to solve the problem]

The present invention reads and writes data from a rotating storage medium using a moving head, and requires the head to seek when reading and writing data from two or more cylinders of the storage medium from records read from an input file to an external storage device. In a large-capacity data classification processing method that outputs a plurality of created strings, and physically replaces and merges these strings, each of the strings is created and output to the external storage device, and at the same time, each of the strings is (1) outputting a string directory having a position on the external storage device to be output and a key of the first record and a key of the last record of the string to the internal storage device; The cylinder directories are sorted in the order of the keys of the first records based on a predetermined rule, and the key of the last record of each directory after sorting is determined by the key of the first record of the string directory in the following order. and a physical merging means for physically merging the strings linked by the logical merging means in units of cylinders. ing.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing a data flow when classifying a large amount of data using a large amount of data classification processing method according to an embodiment of the present invention; FIG. 2 is a diagram showing an example of the format of a string directory; FIG. 3 is a diagram showing an example of links in link phase 2.

The large-capacity data classification processing method of this embodiment consists of the following five phases as shown in FIG.

(1) Presort phase (Phase 1) (2) Link phase (Phase 2) (3) One cylinder merge phase (Phase 3) (4
) Multi-cylinder merge phase (phase 4) (5)
Last Pass Phase (Phase 5) In the presort phase 1, data φ records are read from the input file 6, a string is created, and the string is stored in the external storage device, similar to the conventional large-capacity data classification processing method shown in FIG. The data is output to the magnetic disk device 8, but at the same time, a string directory is created and stored in the internal storage device in units of cylinders with delimiters attached. As shown in Figure 2, this string directory contains the disk address (including cylinder address) where the string is stored, the first key which is the key of the first record of the string, and the final key which is the key of the last record. It consists of.

In the linking phase 2, the strings are logically merged by linking the string directory riffs created in the presorting phase 1. This link first sorts the plurality of string directories created in presort phase 2 in ascending order of the first key in units of cylinders. The strings are then logically merged by comparing the final key of the first string directory and the first key of the second string directory within one cylinder, and linking these two string directories when the latter is larger. merge. 3
Similar comparisons are made for the string directories after the string directory, and links are made.

By performing this work for all string directories on a cylinder-by-cylinder basis, strings are logically merged without physically merging, thereby reducing the number of strings.

For example, as shown in Figure 3, when linking four string directories with disk addresses "A" to "D" on 50 cylinders, these string directories are sorted in ascending order from the one with the largest first key. and uB" u [) 11° "C",'A" (displayed by disk address). Next, compare the final key of "uB" and the first key of "A", and since the latter is larger, link the two. This will update "B-A""
”.

This results in merging into three strings of “C”.

Next, compare the final key of "IJD" and the first key of "C", and since the latter is larger, link the two. This creates two strings, "B-A" and "D-C". Similarly, two strings, "H-F" and "E-G", will be created for cylinder 51 by the same operation.

In one cylinder merge phase 3, link phase 2
physically merge the strings until the number of logically linked strings is one string for each cylinder.

During this time, the magnetic head of the magnetic disk 8 does not need to move to access data on the same cylinder, and the time required for head positioning (seek time) can be reduced to zero at the minimum.

In the multi-cylinder phase 4, the two cylinders that were made into one cylinder and one string in the one-cylinder merge phase 3 are merged into one string. Repeat this until the number of strings is 2 or less. This method is characterized by a sliding buffer method designed to perform reading and writing on the same cylinder as much as possible (Patent Application No. 1983-291388). By doing this,
The seek time in the merge phase can be minimized as much as possible. Furthermore, in the last bus phase 5, the strings created by merging in the merge phase 4 are created into one string and written to the output file, thereby completing the process.

〔Effect of the invention〕

As explained above, when classifying large-capacity data using a magnetic disk device or the like as an external storage device, conventionally the present invention unconditionally stores all data on a string directory as the first key in the link phase. Instead of linking based on the size of the final key, by dividing it into cylinders and linking by the same operation within that range,
This has the effect of physically merging the strings while minimizing the seek time as much as possible.

[Brief explanation of drawings]

FIG. 1 is a diagram showing the data flow when classifying large volumes of data using the large volume data classification processing method according to an embodiment of the present invention, and FIG. FIG. 3 is a diagram showing a link example in link phase 2 of FIG. 1, and FIG. 4 is a diagram showing a data flow in a conventional large-capacity data classification processing method. 1... Presort phase, 2... Link phase, 3... One cylinder merge phase, 4... Multi cylinder merge phase, 5... Last pass phase, 6... Input file, 7... - String directory, 8... Magnetic disk device, 9... Output file.

Claims (1)

[Claims] Data is read from an input file to an external storage device that reads and writes data with a moving head on a rotating storage medium, and requires the head to seek when reading and writing data on two or more cylinders of the storage medium. In a large-capacity data classification processing method that outputs a plurality of strings created from records, and physically replaces and merges these strings, the strings are created and output to the external storage device, and at the same time, the strings are The location on the external storage device where each is output and the first string of that string.
a string directory creation means for outputting a string directory having a record key and a final record key to an internal storage device; sorting the cylinder directory in the order of the first record key based on a predetermined rule on a cylinder-by-cylinder basis; A logical merge that links both strings when the key of the last record in each directory after sorting is later than the key of the first record in the subsequent string directory based on the above rules. and physical merging means for physically merging strings linked by the logical merging means in units of cylinders.