JP3516365B2 - Method and apparatus for storing records in memory - Google Patents

Description

Detailed Description of the Invention

[0001]

The present invention relates to the field of information processing,
The present invention relates to a method and apparatus for reading a plurality of input records into a memory.

[0002]

2. Description of the Related Art In today's information processing, many programs including a sort-merge program are used to read a plurality of records in an input file into a memory for processing. For example, sort merge processing by a sort merge program is performed in order to sort the records in the input file under a certain condition. When reading variable length records or indefinite length records into memory in conventional sort merge processing, etc., the user specifies the upper limit (maximum record length) of the record length in the input file, and the processing device is specified. A plurality of record storage areas (BIN) are allocated based on the value.

FIG. 13 shows sort merge processing by a conventional sort merge program. In FIG. 13, the user first specifies the maximum record length 1 of the input file 2 and notifies the sort merge program 3 of it.
If the clear maximum record length is not known, for example, a large value with a little margin is designated as the maximum record length 1. The sort merge program 3 allocates n BINs each having the specified maximum record length 1 in the memory 4 and secures them as a record area. Next, the input records of the input file 2 are sequentially read into the input buffer from the beginning, the input records are moved from the input buffer to BIN, and the sort merge processing is performed.

[0004]

However, the conventional sort merge processing as described above has the following problems.

In order to efficiently allocate a plurality of BINs in the memory 4, the user needs to know the maximum record length in the input file 2 before executing the sort merge program 3. If a value that is significantly larger than the actual maximum record length is specified, the number of allocable BINs in the memory 4 decreases and the processing efficiency decreases. On the contrary, when an input record larger than the maximum record length 1 specified by the user is input, the sort merge program 3 has to interrupt the processing.

It is an object of the present invention to provide a record storage method and apparatus for efficiently storing variable length records or indefinite length records in a memory without the user designating a maximum record length.

[0007]

FIG. 1 is a principle diagram of a record storing method of the present invention. When the process is started in FIG. 1, it is first checked whether or not there is an empty record storage area for storing one input record in the memory (step S1), and such an empty record storage area is found in the memory. If so, the input record is stored there (step S3). If there is no free record storage area in the memory, the current record storage area is divided into a plurality of areas that are equal to or longer than the maximum record length input so far, and a free record storage area is generated (step S
2). Then, the input record is stored in the generated empty record storage area (step S3). Next, of the plurality of input records including the above-mentioned input record, which has already been stored, the maximum record length is stored (step S4), and the process ends.

[0008]

The record storage area (BIN) for storing the input record of the input file is provided in the memory of the information processing device, and the maximum record length input so far is stored as the maximum input record length. Has been done. First, it is checked whether or not the record storage area currently set in the memory is free (step S1), and if there is a free record storage area, a new input record is stored (step S3).

If there is no empty record storage area in the memory, the current record storage area in which the input record is already stored is divided into a plurality of areas having a length equal to or longer than the maximum input record length stored. Yes (Step S
2). The new record storage area generated by the division is smaller than the original record storage area, but it is guaranteed that it is longer than the length of the record input so far. Therefore, all existing input records are still stored and new empty record storage area is created.
A new input record can be stored (step S3). Of the plurality of input records including the new input record thus stored in the empty record storage area, the maximum record length is stored again as the maximum input record length (step S4). The number and size of the record storage areas in the memory can be changed according to the input record, and the memory utilization efficiency is improved. Further, since the maximum input record length is automatically determined, the user does not need to specify the upper limit of the record length in the input file.

[0010]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 2 is a configuration diagram of the information processing apparatus in the embodiment. The information processing apparatus of FIG. 2 includes an input / output terminal 11, a CPU (central processing unit) 12, and an external storage device 1.
3, a memory 14, and a bus 15 coupling these. The input / output terminal 11 is a user terminal provided with an input device such as a keyboard and a mouse and an output device such as a display device, and is used for starting a program for processing a record and outputting a processing result. External storage device 13
Is a storage device such as a magnetic disk device or an optical disk device, and stores input files and output files. The memory 14 is used as a work area by the CPU 12, and a record area and the like are set therein.

FIG. 3 shows the internal structure of the memory 14 in the embodiment. FIG. 3A shows the memory 14 when performing sort processing, and FIG. 3B shows the memory 14 when performing merge processing.

An input buffer 16, a record area 17, a work buffer 18, and a record length storage area 19 are provided in the memory 14 of FIG. 3A. The input buffer 16 is a buffer for reading the records of the input file 21 into the memory 14, and the work buffer 18 is a buffer for outputting the processed records to the work file 22.

The CPU 12 reads the input record contained in the input file 21 of the external storage device 13 into the input buffer 16 by executing the program, and a plurality of B's are read.
Sorting processing is performed by moving to the record area 17 composed of IN or the like. Then, the intermediate result of the processing is stored in the work buffer
And the work file 22 is temporarily output. .

At this time, the record length storage area 19 stores the record length (maximum record length) of the largest record in the record area 17. The CPU 12 refers to the maximum record length stored in the record length storage area 19, divides the current BIN length (BIN length) into n equal parts by an appropriate integer n, and divides into n areas equal to or larger than the maximum record length. Split into. Then, each of the n areas generated by the division processing is used again as BIN.

In the memory 14 of FIG. 3B, a work buffer 18, a record area 17 and an output buffer 2 are provided.
4 are provided. The work buffer 18 is a work file 22.
Are read into the memory 14 and the output buffer 24 is used to output the merged records to the output file 23. When all the input records included in the input file 21 are read, the CPU 12 performs merge processing of some work files 22 in the memory 14 and outputs the work file 22 as an output file 23 to the external storage device 13.

As described above, in this embodiment, the program dynamically changes the BIN length based on the maximum record length in the record area 17, so that an appropriate BIN length can be set according to the progress of processing. It is set automatically. Therefore, the user does not need to input the maximum record length in advance.

Next, the sort merge processing using the record storing method of the present invention will be described with reference to FIGS. FIG. 4 is a flowchart of the sort merge processing, and FIGS. 5 to 9 show an example of the record area 17 in the memory 14 during the processing. The sort merge processing of FIG. 4 is started by the CPU 12 executing the sort merge program.

When the processing is started in FIG. 4, the CPU
First, the area 12 secures the record area 17 in the memory 14 (step S11) and initializes the BIN length (step S12). By this initial setting, the BIN length is set to the area length of the record area 17. At this time, the number of BINs (BIN number) in the record area 17 is 1,
The record area 17 is as shown in FIG.

Next, a record is read from the input file 21 into the input buffer 16 (step S13), and it is determined whether the record at the end (EOF) of the input file 21 and whether the record to be processed is the last record in the input buffer 16. The determination is made (steps S14 and S15). If it is EOF, the processing from step S26 is performed, and if it is the last input buffer 16, the processing from step S14 is repeated. If it is neither the EOF nor the end of the input buffer 16, the length of the record moved from the input buffer 16 to the record area 17 is checked (step S16).

If the length of the record to be moved is longer than the current BIN length, sort processing is performed (step S24),
If it is not more than the current BIN length, the record is moved from the input buffer 16 to the BIN in the record area 17 (step S18). Then, the record length of the largest record among the records stored in the record area 17 is stored in the record length storage area 19 (step S1).
9). If the length of the record moved immediately before is longer than the maximum input record length already stored in the record length storage area 19, the record length of the record is newly stored as the maximum input record length. On the contrary, if the length of the moved record is shorter, the record length storage area 19
The maximum input record length in is not updated.

When a record is moved from the input buffer 16 in the state of FIG. 5, the record is stored as the first record at the beginning of the record area 17 as shown in FIG. 6 (step S18), and the record length is changed. The maximum input record length is stored in the record length storage area 19 (step S19).

Next, an empty BI in the record area 17
It is checked whether or not N is present (step S20), and empty B
If IN is present, the processing from step S14 is repeated. If there is no empty BIN, the length n times the maximum input record length in the record length storage area 19 is compared with the current BIN length (step S21). If the current BIN length is smaller than n times the maximum input record length, step S2
The processing after 4 is performed. If the current BIN length is n times the maximum input record length or more, the current BIN length is divided into n equal parts (step S22), a free BIN is secured (step S23), and the processes after step S14. repeat. Here, n is an appropriate number of divisions and is described in the program in advance.

For example, in FIG. 6, since there is no empty BIN (step S20, NO), twice the maximum input record length (n = 2) is compared with the BIN length. Current BI
Since the N length is equal to the area length of the record area 17 and is larger than twice the maximum input record length (step S21, Y
ES) and BIN are equally divided into two (step S22). As a result, the record area 17 becomes as shown in FIG. 7, and an empty BIN is generated (step S23). B at this time
The number of INs is 2, and the BIN length is 1/2 of the area length of the record area 17.

Then, the next record following the first record is stored in the generated empty BIN as shown in FIG. 8 (step S18). At this time, the maximum input record length is the longer record length of the first record and the next record (step S19). After this, similarly, the two BINs are further divided into two, and two new empty BINs are generated as shown in FIG. At this time, the number of BIN is 4, and the BIN length is the record area 17
It becomes 1/4 of the area length.

Such processing is performed in step S17 in BI.
Until the N length is shorter than the record length, or
This is continued until the BIN length becomes smaller than twice the maximum input record length in step S21. Here, the processing is performed with the number of divisions of the BIN length set to n = 2, but generally any number of divisions can be used, and the value of n can be changed during the processing.

Next, the processing performed in step S24 will be described. In step S24, the plurality of records in the record area 17 are once sorted and moved to the work buffer 18. Here, as a sorting method,
Sorting method by tournament method (Japanese Patent Application No. 2-27)
9953, JP-A-4-153826) or QUICK
Use (quick) sort.

FIG. 10 is a flowchart of the sorting process by the tournament method. The tournament method is a method of selecting the strongest record by repeating winning battles in which two records form a set. When the process is started in FIG. 10, the CPU 12 first determines the record area 17
Initial tournaments for multiple records in
The winning record is determined (step S31). And
The winning record is moved to the work buffer 18 (step S32).

Next, it is checked whether or not there is a next input record in the input buffer 16 (step S33), and if there is a next input record, the record length is compared with the BIN length (step S34). If the record length of the next input record is equal to or less than the BIN length, it is determined whether or not it is weaker than the winning record transferred to the work buffer 18 (step S35). If the next input record is weaker than the winning record, it is stored in the BIN where the winning record was stored, the tournament is performed again, and the next winning record is selected (step S36). Then, the processing from step S32 is repeated.

FIG. 11 shows an example of tournament sort for four records in the record area 17. For example, it is assumed that the process of FIG. 10 is started after the records A, B, C, and D are stored in the four BINs shown in FIG. 9, respectively. At this time, from record B to record A
, Record C is stronger than record D, and record A is stronger than record C, record A wins the initial tournament (step S31).
Therefore, the record A is moved to the work buffer 18 (step S32), the next input record E is moved to the BIN in which the record A was stored, and the records E, B, C, D are stored.
A tournament is held between (step S36).

Such processing is repeated, and step S
When the next input record disappears in 33, or the record length exceeds the BIN length in step S34, or when the next input record is stronger than the winning record in step S35, the remaining records in the record area 17 are deleted. Sort and output to the work buffer 18 (step S37),
The process ends.

For example, in FIG. 11, when there is no next input record E (step S33, NO), the remaining records B, C and D are sorted and the work buffer 1
8 (step S37).

In the tournament method, one winning record is selected from the records in the memory 14, and the next input record is stored in the BIN where the winning record existed.
It is superior to other sorting techniques in that records more than the number of records stored at initial input can be output as one string.

FIG. 12 shows another sorting method Q.
It is a flowchart of the sorting process by the UICK method.
When the process is started in FIG. 12, the CPU 12 first sorts all the records in the record area 17 by QUICK sort (step S41). Next, all the sorted records are moved to the work buffer 18 (step S42), and the process ends.

In the QUICK method, since no input record is added during processing, the number of records that can be stored in one string is smaller than that in tournament sort, but the string structure is simple.

When the sort process of step S24 is completed in this way, the CPU 12 outputs the contents of the work buffer 18 to the work file 22 (step S25), initializes again (step S26), and repeats the processes of step S14 and thereafter. . When the EOF is reached in step S27, it is checked whether or not the work file 22 is used next (step S27).

If the processing of step S24 has never been performed and the work file 22 is not used, the records in the record area 17 are sorted (step S28) and output to the output file 23. (Step S30), the process ends. However, when the work file 22 is used, the records in the record area 17 and the records in the work file 22 are sorted and merged (step S29). Then, the merged result is output to the output file 23 (step S30), and the process ends.

For example, in the conventional record storage method, when the record area 17 is 1 MB (megabytes) and the maximum record length specified by the user is 1 KB (kilobytes), the number of records that can be stored in the record area 17 is It is always 1000 (1MB / 1KB).

However, in this embodiment, the number of records that can be input differs depending on the average value of the record lengths of the actual input records. In other words, if there are variations in the length of the input records, enter a relatively small number of records at once,
The number of inputs is reduced only for records that are significantly larger.
For example, if the average record length is 0.1 KB, 1000
2000 records can be stored for 0 or 0.5 KB, and 1000 records can be stored even for the worst 1 KB.
Records will be stored. As a result, the number of input records that can be read in the memory 14 increases and the amount of data that can be sorted at one time increases. Therefore, the processing time of the sort merge processing is shortened.

[0039]

According to the present invention, it is not necessary for the user to specify the maximum value of the input record in the process of reading the variable length record or the indefinite length record into the memory of the information processing apparatus. In addition, the memory utilization efficiency is improved, and the processing speed of the read record is improved.

[Brief description of drawings]

FIG. 1 is a principle diagram of the present invention.

FIG. 2 is a configuration diagram of an information processing apparatus according to an embodiment.

FIG. 3 is a diagram showing an internal configuration of a memory in the example.

FIG. 4 is a flowchart of sort merge processing according to the embodiment.

FIG. 5 is a diagram showing a record area with a BIN number of 1;

FIG. 6 is a diagram showing a record area after the first record is stored.

FIG. 7 is a diagram showing a record area with a BIN number of 2;

FIG. 8 is a diagram showing a record area after the next record is stored.

FIG. 9 is a diagram showing a record area with a BIN number of 4;

FIG. 10 is a flowchart of tournament processing according to the embodiment.

FIG. 11 is a diagram showing an example of tournament sort.

FIG. 12 is a flowchart of a quick process according to the embodiment.

FIG. 13 is a diagram showing a conventional sort merge process.

[Explanation of symbols]

1 maximum record length 2,21 Input file 3 sort merge program 4,14 memory 11 Input / output terminals 12 CPU 13 External storage device 16 input buffers 17 record area 18 working buffer 19 record length storage area 22 working files 23 Output file 24 output buffer

Claims (11)

(57) [Claims] 1. An information processing apparatus for reading a record into a memory for processing, checks whether or not a free record storage area for storing one input record exists in the memory, and determines the empty record storage area. If it is in the memory, the input record is stored in the empty record storage area, and if the empty record storage area is not in the memory, the current record storage area is stored as the largest record input so far. It is characterized in that the input record is divided into a plurality of areas having a length equal to or larger than the length, the input record is stored in a generated empty record storage area, and the maximum record length among the plurality of stored input records is stored. How to store records. 2. When there is one or more generated empty record storage areas, one or more input records including the input record are stored in one or more empty record storage areas, and the one or more When the input records are stored in all of the empty record storage areas, the record storage area is further divided into a plurality of areas to generate an empty record storage area. The record storage method described in 1. 3. A sorting method comprising: sorting a plurality of input records respectively stored in a plurality of record storage areas by the record storing method according to claim 1 and outputting the sorted records to a file. 4. A tournament in which the plurality of input records are sorted by tournament sort, the determined winning record is output to the file, and the next input record is stored in the record storage area where the winning record was stored. The sorting method according to claim 3, wherein the sorting is repeated. 5. When the length of the input record exceeds the length of the empty record storage area, a plurality of records input so far are sorted by tournament sort, output to a work file, and newly input. 4. The sorting method according to claim 3, wherein when there are no more input records, the contents of the record storage area and the contents of the work file are merged and output to the output file. 6. When the current record storage area cannot be divided into a plurality of areas that are longer than the maximum record length that has been input so far, the plurality of records that have been input so far are sorted by tournament sort. If you output to a work file and there are no new input records,
4. The contents of the record storage area and the contents of the work file are merged and output to an output file.
The sort method described. 7. The sorting method according to claim 3, wherein the plurality of input records are sorted by quick sort and output to the file. 8. When the length of the input record exceeds the length of the empty record storage area, a plurality of records input so far are sorted by a quick sort, output to a work file, and newly input. 4. The sorting method according to claim 3, wherein when there are no more input records, the contents of the record storage area and the contents of the work file are merged and output to the output file. 9. When the current record storage area cannot be divided into a plurality of areas that are longer than the maximum record length that has been input so far, the plurality of records that have been input so far are sorted by a quick sort. 4. The sorting method according to claim 3, wherein when a new input record is output to the work file, the contents of the record storage area and the work file are merged and output to the output file. 10. An information processing apparatus for reading a record into a memory for processing, a means for storing the maximum record length input so far as the maximum input record length, and one input record Means for checking whether or not there is a free record storage area in the memory, and if the free record storage area is not in the memory, the current record storage area is divided into a plurality of records having a length equal to or larger than the maximum input record length. And a means for generating an empty record storage area and a means for storing the input record in the empty record storage area. 11. The apparatus according to claim 1, further comprising means for sorting a plurality of input records respectively stored in the plurality of record storage areas and outputting the sorted records to a file.
0 record storage device.