JPH0397027A - Data sorter - Google Patents

Abstract

PURPOSE:To quickly sort a large quantity of records exceeding the capacity of an internal memory by successively reading out records from an external storage means and arranging designated keywords in the order of priority level related to designation and adding storage position information to convert records into sort records. CONSTITUTION:A designating means (b) arbitrarily designates plural keywords as the sort object and priority levels of respective sort processings of these keywords. A converting means (c) successively reads out records from an external storage means (a) to convert them into sort records where keywords designated by the designating means (b) are arranged in the order of priority level related to designation and storage position information in the external storage means (a) is added. Sort records converted by the converting means (c) are stored in an internal storage means (d). A sorting means (e) sorts records based on sort records in the internal storage means (d). Thus, the record form is changed to perform the sort processing, and records are quickly sorted.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a data classification device suitable for quickly sorting record data. [Prior Art] Conventionally, when sorting a large amount of record data stored in an external memory such as a magnetic disk on a record-by-record basis, a group of records to be sorted is divided into multiple blocks according to the memory capacity of the internal memory, and Sorting is performed for each block in memory, and re-sorting is performed between each block based on the sorting results for each block.

Furthermore, when sorting is performed using a plurality of words as keywords, the sorting is performed for each keyword in sequence according to the order in which the keywords are arranged on the record.

[Problems to be Solved by the Invention] As described above, when sorting a large number of records, re-sorting must be performed, and the sorting process takes a long time. Also, the sorting process for each keyword is performed sequentially according to the order in which the keywords are arranged, so for example,
Shorten the sort processing time even when the same word exists across multiple records, such as a word indicating a major classification, which can be sorted quickly by changing the execution order of sort processing by keyword. I couldn't do it. This is considered to be due to the fact that the sorting process is always performed with the original record format intact.

If you do this, it is clear that if you can change the record format and perform the sorting process, the sorting process can be done quickly. An object of this invention is to enable sorting processing by changing the record format.

[A thousand steps to solve the problem] The means of this invention are as follows. The external storage means a (see the functional block diagram of FIG. 1, the same applies hereinafter) stores a plurality of records each consisting of a plurality of words.

The specifying means b arbitrarily specifies a plurality of keywords to be sorted and a priority order of sorting processing for each keyword.

The converting means C sequentially reads records from the external storage means a, arranges the keywords specified by the specifying means b in priority order according to the specification, and adds storage position information in the external storage means a to the sorting record. Convert.

The internal storage means d stores the sorting records converted by the conversion means C.

The sorting means e sorts the sorting records in the internal storage means d based on the specified keyword and priority order. The inverse conversion means f inversely converts the sorting records sorted by the sorting means e into the original record format based on the storage location information. [Operation] The operation of the means of this invention is as follows. Now, the external storage means a stores 50 records, each record has a word, one of which is a word indicating a major classification, and this word is stored across multiple records. exists, and the total data amount of the record is stored in the internal storage means C.
It is assumed that the memory capacity is slightly less than three times the memory capacity of . In addition, the specifying means b specifies the words (A, B, C, ...) indicating the above-mentioned major categories and the words indicating the order of precepts expressed in numerical values as keywords to be sorted. As for the processing priority, it is assumed that the word indicating the major classification is specified as the first priority, and the word indicating the order of precepts is specified as the second priority. Then, the converting means C sequentially reads the records from the external storage means a, arranges the words indicating the major classification specified by the specifying means b and the words indicating the order of precepts in the priority order according to the specification, and stores the records in the external storage. Convert to a sorting record with position information added in method a. Then, the internal storage means d stores the sorting record converted by the converting means C, but if the storage location information corresponds to the data amount of 1 word, the sorting record has a data amount of 3 words, and the original 17 words from 9 words of
3, all 50 sorting records corresponding to the 50 records in the external storage unit a are stored at once in the internal storage unit d.

Next, the sorting means e sorts the sorting records in the internal storage means d by word, in the order of the word indicating the major classification and the word indicating the order of performance. In this case, words A, B, C, etc. indicating major classifications exist across multiple records. For example, after sorting each record in the order of A, B, C, When sorting the words shown, sort the words A. B. This can be done for each group of records containing C-..., which reduces the number of records to be sorted within each group and reduces the total number of comparisons of grade ranking data. Sorting time can be reduced compared to when sorting is performed on the indicated words. Furthermore, since all 50 sorting records corresponding to the 50 records in the external storage unit a are stored in the internal storage unit d at once, the 50 records in the external storage unit a are stored at once.
For book records, without dividing them into blocks,
Sort processing is performed at once on the internal storage means d. Therefore,
There is no need to resort between blocks. Next, the inverse conversion means f inversely converts each of the 50 sorting records sorted by the sorting means e into the original record format based on the storage position information.

Therefore, you can change the record format and perform sorting, and the sorting process can be completed in a short time. [Embodiment 1] An embodiment will be described below with reference to FIGS. 2 to 6.

FIG. 2 is a block diagram of the data classification device, which includes an external magnetic disk 1, an internal memory 2, a sort processing section 3, a conversion section 4, a sort format specification section 5, a delimiter detection section 6, a counter 7, and a designated word detection section. It has a section 8 and an inverse transform section 9.

The external magnetic disk 1 stores a large number of variable length records that exceed the memory capacity of the internal memory 2. The internal memory 2 is utilized as a work memory for sorting variable length records stored on the external magnetic disk 1.

The sort processing unit 3 sorts the records transferred from the conversion unit 4 to the internal memory 2 based on keywords.

The conversion unit 4 sequentially reads the variable length records in the external magnetic disk l, deletes words other than the keyword specified by the sort format specification unit 5, and compresses them into sorting records with predetermined reconversion information added. Convert and internal memory 2
Transfer to. At this time, the conversion unit 4 converts the sort format specification unit 5
Place the specified keywords on the sorting records in the order specified by . When performing such record compression processing, the conversion section 4 utilizes the delimiter detection section 6, the counter 7, and the specified word detection section 8.

The inverse conversion section 9 inversely converts the sorting records sorted on the internal memory 2 by the sorting processing section 3 into the original record format based on the inverse conversion information added to the keyword, and stores the sorted records on the external magnetic disk. Store in 1. Next, the sorting process will be explained with reference to FIGS. 3 to 6. Now, a file consisting of a group of variable length records as shown in FIG. 3(a) is stored on the external magnetic disk 1, and in order to create a form as shown in FIG. 4 based on this file, It is assumed that the sort format shown in FIG. 5 is specified by the sort format specification section 5. That is, each record in the file includes the word wa representing the person in charge, the word wb representing the number of sales, and the word wd representing the district.As shown in Figure 5, sales is used as a keyword representing the word to be sorted. A word wb representing a number and a word wd representing a district are designated, their priorities are designated as 2nd and 1st, respectively, and their data formats are character type and character type, respectively.
Assume that it is specified as a numeric type, and the sort order is specified as ascending order and descending order, respectively.

Among such specified data, the priority order is input to the conversion unit 4, the keywords and data format to be sorted are input to the specified word detection unit 8, and the sort format is input to the sort processing unit 3.
is input.

Next, record compression processing by the conversion unit 4 based on such specifications will be explained. This record compression processing is conceptually shown in FIG. 6, so please refer to FIG. 6 from time to time. The conversion unit 4 first resets the counter 7, outputs a read address to the external magnetic disk 1 in synchronization with a clock pulse, and reads data on the variable length record. At this time, the delimiter detection unit 6 monitors the read data and detects the read of the delimiter. Further, the specified word detection section 8 monitors the read data and detects the keywords to be sorted specified by the sort format specification section 5, that is, the word wb representing the number of sales and the word wd representing the district. Then, when the delimiter detection M5 detects a delimiter "('゜) indicating the start of a record, the converter 4 latches the read address at that time. Also, the converter 4 latches the read address at that time. When the word wb representing the number of sales related to the specification or the word wd representing the district is detected, that word is latched. Then, when the next record start delimiter ゛ (゜, counter 7 at that point
Latch the contents as the data number (length) of the previous record, and create a sorting record consisting of the word wb representing the number of sales, or the word wd representing the district, the record start address, and the length. and load it into internal memory 2. At this time, according to the priority order specified by the sorting format specifying unit 5, the words are arranged in the order of the word wd representing the district → the word wb representing the number of sales. Next, the counter 7 is reset and similar processing such as latching the read address at that point is repeated.

In this way, the conversion unit 4 compresses the variable length records from the external magnetic disk 1 according to the specifications, creates a sorting record in which the specified words are arranged in the priority order according to the specifications, and stores it in the internal memory 2. Forward.

The sorting processing unit 3 performs sorting on a plurality of sorting records in the internal memory 2 based on specified words on each record.
Sorting processing is performed in the sorting format specified by the sorting format specifying section 5. At this time, the word wd representing the designated district and the word wb representing the number of sales are arranged in the specified priority order on the sorting record, so the sorting processing unit 3 If the word-by-word sorting process is executed in order, the word-by-word sorting process will automatically be performed in the priority order specified by the sort format specifying unit 5. The inverse converter 9 sequentially reads out the sorting records sorted on the internal memory 2 from the beginning, and accesses the external magnetic disk 1 by using each address of the length starting from the record start address on the record as a read address. The original record corresponding to the sorting record is read, and the original record is sequentially stored in a predetermined area of the external magnetic disk. To further explain the series of sorting processes mentioned above with reference to FIG. 3, each record of the external magnetic disk 1 is read out in order from the beginning, and the converter 4 reads the records as shown in FIG. 3(b). Word wd representing the district → word wb representing the number of sales → address (St1, s
t2,...) → Length of the record (N1112
,...> is converted into each sorting record in which the data is arranged in the order. In this case, since both the address and length can be expressed in 1 to 2 bytes, the amount of data of the storage location information consisting of the address and length usually does not exceed the amount of data for one word. Therefore, the sorting record is compressed to at least 3/5 of the original record. The sorting records compressed in this way are expanded into the internal memory 2, and sorted by word by the sort processing unit 3 as shown in FIG. → Sorting by word is performed in the order of the word wb representing the number of sales, but when sorting based on the word wb representing the number of sales is performed after sorting based on the word wd representing the district, as shown in Figure 3 (C) As shown in , since the sorting by district has already been completed, the number of sales can be sorted by the same district, and the total number of data comparisons is reduced, so the sorting can be performed at high speed. The sorted records are sent to the inverse converter 9
As shown in FIG. 3(d), the records are converted back to the original record format and stored in a predetermined area of the external magnetic disk 1 in a sorted state. Note that when outputting a form like the one shown in Figure 4, since it has already been sorted as shown in Figure 3(d), it is only necessary to specify the output word and its output order. Since the sorting process is performed using sorting records compressed in this way, when compressed to 3/5 or less as described above, the records on the external magnetic disk l, which is approximately 5/3 times larger than the internal memory 2, Groups can be sorted at once without dividing them into multiple blocks. Therefore, there is no need for re-sorting between blocks, and a large number of records exceeding the memory capacity of the internal memory 2 can be quickly sorted. Furthermore, even if all compressed sorting records cannot be expanded to internal memory 2 at once, the number of divided blocks can be reduced, and the time required for re-sorting between blocks can be significantly shortened. ．． Furthermore, as described above, the sorting time when sorting the sorting records by word can also be shortened, and the speed of sorting can be further promoted.

Note that the present invention is not limited to the above-described embodiment; for example, the records to be sorted may be fixed-length records. In this case, it is not necessary to monitor the record length by the counter 6. [Advantageous Effects of the Invention] According to the present invention, the record format can be changed to perform sort processing, and a large number of records exceeding the memory capacity of the internal memory can be processed at high speed. It can be sorted into

[Brief explanation of drawings]

Fig. 2 is a functional block diagram of the present invention, Fig. 2 is a block diagram of the embodiment, and Fig. 3 is a system diagram showing an example of the contents of a file to be sorted and the flow of a series of sorting processes for this file. Figure 4 shows an example of outputting a form in a sorted form from the file contents in Figure 3, and Figure 5 shows the specified contents of the sort format for sorting in accordance with the form in Figure 4. FIG. 6, a diagram showing an example, is a diagram conceptually showing the record compression processing performed in accordance with the specifications shown in FIG. 5. 1... External magnetic disk, 2... Internal memory, 3-
. . . Sort processing section, 4. . . . Conversion section, 5. . . - Count format specification section, 9-. . . Inverse conversion section.

Claims (1)

[Scope of Claims] External storage means for storing a plurality of records each consisting of a plurality of words; a specification means for arbitrarily specifying a plurality of keywords to be sorted and a priority order of sorting processing for each keyword; a converting means for sequentially reading records from the means, arranging the keywords specified by the specifying means in the priority order according to the specification, and converting the records into sorting records to which storage position information in the external storage means is added; an internal storage means for storing the sorting records converted by the means; a sorting means for sorting the sorting records in the internal storage means based on the keywords and priorities specified above; and sorting by the sorting means. A data classification device comprising: inverse conversion means for inversely converting the sorted records into the original record format based on storage location information.