JPH1021053A - Data processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To execute the sort and merge of massive data at a high speed without considerably increasing hardware by providing 2nd input means at the sort processors of respective steps in addition to 1st input means, merging plural sorted record streams inputted from these 2nd input means, and outputting a single sorted record stream. SOLUTION: Memories 111-114 are connected to respective sort processors 121-124 and these one-dimensionally connected sort processors 121-124 are provided with 1st input means 161a-164a composed of receivers and registers and 2nd input means 161b-164b similarly composed of receivers and registers. Then, the output of the sort processor 121 is connected to the 1st input means 162a of the sort processor 122, circuit configuration for merge as a 2nd mode is provided in addition to the circuit configuration of pipeline margin sorter as a 1st mode, and both the modes are used while being switched.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for implementing sorting and merging in database processing by hardware, and more particularly to an improvement in a pipeline merge sorter.

[0002]

2. Description of the Related Art In general, a record which is an element of a database is composed of a plurality of fields. The process of sorting records in ascending or descending order using a certain field as a key is called sorting. When there are a plurality of sorted record strings, a process of reorganizing them into one sorted record string is called merging.

When sorting or merging is performed by software, a large amount of data is exchanged between a processor and an auxiliary storage device such as a memory or a disk, and a long time is required for processing. On the other hand, there is a pipeline merge sorter or the like as hardware for performing high-speed sorting.

FIG. 7 shows, for example, “VLSI sort processor” (information processing, Vol. 31, No. 4, 1990).
2) is a configuration diagram of a pipeline merge sorter described in (1). In the figure, 101, 102, 103, 10
4 is a one-dimensionally connected sort processor, 111 to 11
Reference numeral 4 denotes a memory connected to each sort processor.

In a pipeline merge sorter, an n-th sort processor inputs two sets of sorted record strings each composed of 2 ^n-1 records from an ( ^{n-1) -th} sort processor, and merges them into 2 ⁿ records. Output a set of sorted record strings consisting of The memory connected to each sort processor stores the first record string of the two sets of input record strings. FIG. 8 shows a process of sorting using a pipeline merge sorter.
In the figure, in the output stage from 101, the result of sorting in units of two numbers in the number string is recorded. Also, in the output from the next stage 102, the result of sorting in units of four numbers in the number string is recorded. In the output from 103, the result of sorting in units of eight numbers is recorded, and finally, in the output from 104, the entire sorting is completed.

With such hardware, the amount of data that can be sorted at one time is limited. Normally, when sorting data that exceeds the limit, first sort the entire data by 1
A method is adopted in which a record row is divided into record rows that can be sorted at a time, each record row is sorted by hardware, and a plurality of sorted record rows generated as a result are merged by software or hardware.

[0007] However, as described above, when merging is performed by software, it takes a long time to process. Japanese Patent Application Laid-Open No. 62-15 / 1987 discloses an example of means for performing merging with hardware.
FIG. 9 shows a configuration diagram of Japanese Patent No. 4140. In the figure, 2
01 and 202 are sorting devices, and 203 is a merging device. Each of the sorting devices 201 and 202 inputs and sorts a record sequence, and inputs the sorted record sequence to the merge device 203. The merging device 203 merges and outputs the input two sets of sorted record strings. With such a configuration, an amount of data that cannot be sorted at one time can be sorted at a high speed in a pipeline manner.

[0008]

In the conventional method, hardware dedicated to merging is required, and two sort devices are required to double the amount of data that can be sorted at one time. There was a problem that the wear scale became large. Further, in the conventional method, the amount of data that can be sorted at one time is increased at most twice, and in order to sort the data exceeding that amount, it is necessary to merge with software or add additional hardware. There is a problem that the processing time increases or the hardware scale increases.

The present invention has been made to solve such a problem, and provides a data processing apparatus capable of executing a large amount of data sorting and merging at high speed without a large increase in hardware. It is intended to be.

[0010]

According to a data processor according to the present invention, an n-th sort processor having a memory has two sets of 2 ^n-1 records each having 2 ^n-1 records from an ( ^{n-1) -th} sort processor. In a pipeline merge sorter for inputting and merging record sequences and outputting a set of sorted record sequences of 2 ⁿ records, apart from the first input means for inputting the sorted record sequences, each of the above-mentioned stages is The sort processor is provided with a second input means, and a plurality of sorted record strings input from the second input means are merged to output a single sorted record string.

[0011] Further, the n-th sort processor having a memory is two times from the (n-1) -th sort processor.
a pipeline merge sorter for inputting and merging two sets of sorted record strings consisting of ^n-1 records from a first input means and outputting a set of sorted record rows consisting of 2 ⁿ records; A storage device for storing a plurality of sets of sorted record sequences obtained by repeatedly performing the entire sort process in the sorter, and reading a predetermined fixed length record from each set of record sequences stored in the storage device; A data transfer control device for transferring the records to the sort processors of the respective stages; and a second input means for inputting the records to the sort processors of the respective stages. It is a record sequence.

Further, the transfer control device monitors the second input means of the sort processor at each stage, and when the record length of the second input means is equal to or less than a predetermined fixed length, the transfer control apparatus transmits the subsequent record to the second input means. The data is read from the storage device and transferred to the sort processor.

Each of the sort processors in each stage monitors the respective second input means, and when the record length of the second input means is equal to or less than a predetermined length, reads a subsequent record from the storage device. However, the transfer control device requests the transfer control device to transfer the data to the sort processor.

Further, the transfer control device includes a buffer for reading and storing a record of a predetermined fixed length from each set of record strings stored in the storage device.

The transfer control device monitors the buffer, and when the record length is equal to or less than a predetermined length, reads out a record subsequent to the record sequence from the storage device and stores the record in the buffer. is there.

Further, the data inputted to the first input means is stored in a memory of the sort processor.

Further, the data inputted to the second input means is stored in a memory of the sort processor.

Further, an n-th sort processor having a memory inputs two sets of sorted record strings each consisting of 2 ^n-1 records from the ( ^{n-1) th} sort processor from the first input means and merges them. And outputs a set of sorted record strings consisting of 2 ⁿ records, and stores a plurality of sorted record strings obtained by repeatedly performing the entire sort processing in the pipeline merge sorter. A data transfer control device that reads a record of a predetermined length that is set in advance from each set of record strings stored in the storage device, and transfers the read record to a memory connected to the sort processor at each stage. A second input means for inputting the stored records to the sort processors of the respective stages; To merge the over door already record column is intended to be a single sorted record column.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. FIG. 1 shows a data processing apparatus according to one embodiment of the present invention. In the figure, 111, 11
2, 113 and 114 are memories connected to each sort processor. Reference numerals 121, 122, 123, and 124 denote one-dimensionally connected sort processors, each including a first input unit 161a including a receiver and a register, and a second input unit 161b including a receiver and a register. The output of the sort processor 121 is connected to first input means of the sort processor 122, the output of the sort processor 122 is connected to first input means of the sort processor 123, and the output of the sort processor 123 is connected to the first input of the sort processor 124. Connected to the input means. The sort processor 121 has a circuit configuration for merging as shown in FIG. 1 as a second mode, separately from a circuit configuration as a pipeline merge sorter which is a first mode. Is configured to be used. In the figure, the sort processor 121
Is a comparison circuit 141 for comparing two inputs and outputting the smaller or larger one, and a second data input means 161b or memory 111 connected to one input of 141.
Is selected by a selection circuit 151 for selecting the data. The same applies to the sort processors 122, 123, and 124.

Next, the operation will be described. The first mode, sorting, is the same as the conventional example shown in FIG. At this time, the second input means of each sort processor is not used.

FIG. 2 illustrates the process of the second mode, the merge, according to the present invention. The figure shows an example in which four sets of sorted record strings are merged in ascending order, and each record string is composed of two records. Each sort processor first inputs the first record of the corresponding record sequence from the second input means, and then compares the record input from the first input means with the record input from the second input means. Then, the smaller record is output from the output means. Now, focusing on the k-th sort processor, the first input means inputs the smallest record in the k-1 set of record strings corresponding to the k-1th sort processor from the first-stage sort processor. Then, the smallest record in the record sequence corresponding to the k-th sort processor is input from the second input means, and as a result, the output means of the k-th sort processor outputs k rows from the first sort processor to the k-th sort processor. The smallest record among the k sets of record strings corresponding to the eye sort processor is output. Second
If there are no more records in the input means, subsequent sorted records are sequentially input, and the above-described process is repeated. As a result, four sets of sorted record strings are merged.

As described above, according to the first embodiment, merging can be performed using a pipeline merge sorter without adding large hardware.

Embodiment 2 FIG. 3, reference numeral 301 denotes a data processing device, 302 denotes a data transfer control device, 303 denotes a storage device, and both 301 and 303 are connected to 302. Reference numerals 311, 312, 313, and 314 are address registers which are located in the data transfer control device 302 and hold addresses in the storage device 303.

In the second mode in which merging described in the first embodiment is performed, the data transfer control device 302 monitors the second input means of the sort processors 121 to 124, and switches to the sort processor having no record in the register. The next record is sent. At this time, the address registers 311 to 314 in the data transfer control device 302 indicate the head addresses of the record strings corresponding to the sort processors 121 to 124, respectively, and the data transfer control device 302 uses the addresses indicated by these address registers. The record is read from the storage device 303 and transferred to the sort processor. Also, the address register 31
1 to 314 update the contents by the transferred record size, and always indicate the start address of the record string.

According to the present embodiment, since the data transfer control device 302 has the address registers 311 to 314, each sort processor and the data transfer control device 3
02 can be merged by the pipeline merge sorter without greatly increasing the number of signal lines between the two.

Embodiment 3 FIG. 4, reference numeral 301 denotes a data processing device, 302 denotes a data transfer control device, 303 denotes a storage device, and both 301 and 303 are connected to 302. 321, 322, 323, and 324 are sort processors 121, 122, 123, and 12, respectively.
4 is an address register that holds an address in the storage device group 303.

In the second mode for performing the merging described in the first embodiment, each sort processor holds the head address of the corresponding record string in the storage device 303 in the address registers 321, 322, 323, and 324, respectively. When the record no longer exists in the second input means of the own sort processor, the record is read directly from the storage device 303 or indirectly via the data transfer control device 302.

According to the present embodiment, when performing merging with the pipeline merge sorter, each sort processor manages the head address of the corresponding record sequence by itself, so that the data transfer control device 302 is provided with an address register. Although the number of signal lines increases as compared with the case,
It is possible to arbitrarily set the number of stages of the sort processor to be dimensionally connected.

Embodiment 4 5, reference numeral 301 denotes a data processing device, 302 denotes a data transfer control device, 303 denotes a storage device, and both 301 and 303 are connected to 302. Reference numerals 311, 312, 313, and 314 are address registers which are located in the data transfer control device 302 and hold addresses in the storage device 303. 3
Reference numeral 31 denotes a buffer located in the data transfer control device 302.

In the second mode in which merging described in the first embodiment is performed, the data transfer control device 302 reads a record from the storage device 303 based on the addresses indicated by the address registers 311 to 314 and reads the record from the
1 and the data transfer control device 302 monitors the second input means of the sort processors 121 to 124 and transfers the records from the buffer 331 to the sort processors whose records do not exist in the registers.

According to the present embodiment, when performing merging with the pipeline merge sorter, it is possible to reduce the overhead time for reading a record string.

Embodiment 5 Embodiments 1 and 2,
In the third and fourth modes, in the second mode, the record input from the first input unit is compared with the record input from the second input unit.
All or a part of .about.114 are used as a buffer memory, and records input from the first input means are stored in the buffer memory. Each sort processor can perform merging by comparing the first record in the buffer memory with the record input from the second input means and outputting the smaller record.

According to the present embodiment, when merging with the pipeline merge sorter, the flow of data between the sort processors is smoothened without adding a special buffer, and as a result, the time required for merging is reduced. be able to.

Embodiment 6 FIG. In the fifth embodiment, in the second mode, all or a part of the memories 111 to 114 are used as a buffer memory, and records input from the first input unit are stored in the buffer memory. In the present embodiment, the record input from the second input means is stored in the buffer memory. Each sort processor can perform merging by comparing the record input from the first input means with the first record in the buffer memory and outputting the smaller record.

According to the present embodiment, when performing merging with the pipeline merge sorter, it is possible to reduce the overhead time for reading a record string without adding a special buffer.

Embodiment 7 In the fifth and sixth embodiments, in the second mode, all or a part of the memories 111 to 114 are used as buffer memories, and records input from the first input unit or input from the second input unit are used. One of the records is stored in the buffer memory.
All or a part of the data 11 to 114 is used as a buffer memory, and both the record input from the first input means and the record input from the second input means are stored in the buffer memory. , And merging can be performed by outputting the smaller record.

According to the present embodiment, when performing merging with the pipeline merge sorter, the flow of data between the sort processors is smoothened without adding a special buffer, and the overhead time for reading a record sequence is reduced. Thus, the time required for merging can be reduced.

Embodiment 8 FIG. 6, reference numeral 301 denotes a data processing device, 302 denotes a data transfer control device, 303 denotes a storage device, and both 301 and 303 are connected to 302. Reference numerals 311, 312, 313, and 314 are address registers which are located in the data transfer control device 302 and hold addresses in the storage device 303. In the sixth and seventh embodiments, in the second mode, the memory 111
To 114 are used as a buffer memory, and the record input from the second input means is stored in the buffer memory. In the present embodiment, the data transfer control device 302 The columns are stored directly in the memories 111 to 114 and the sort processor 10
Records 1 to 104 read the records from the memories 111 to 114, respectively, and perform the merge.

According to the present embodiment, when performing merging with the pipeline merge sorter, it is possible to reduce the overhead time for reading a record string without adding a special buffer. Further, by connecting the memory to the second data input means, the number of signal lines of the sort processor can be reduced.

In the above description, a pipeline merge sorter in which four sort processors are connected one-dimensionally is taken as an example, but the present invention does not limit the number of sort processors.

[0041]

Since the present invention is configured as described above, it has the following effects.

By adding input means to a sort processor of a pipeline merge sorter conventionally used only for sorting, a plurality of sorted record strings can be merged.

[0043] Further, an input means is added to the sort processor of the pipeline merge sorter used only for sorting, and a storage device for storing sorted record strings and a data transfer control device for transferring records to the processor are provided. Accordingly, a large amount of data can be sorted at a high speed without adding hardware for merging.

Further, the transfer control device monitors, and when the record length of the second input means is equal to or less than a predetermined length, a subsequent record is read from the storage device and transferred to the sort processor. Therefore, merge processing can be performed continuously.

When the record length of the second input means is equal to or less than a predetermined length monitored by the sort processor, the transfer control device reads out the subsequent record from the storage device and transfers it to the sort processor. Since it is configured to request, the merging process can be performed continuously.

Furthermore, since a buffer for reading and storing a record of a predetermined length set from each set of record strings stored in the storage device is provided, an overhead time for reading the record string when performing a merge operation. Can be shortened.

Further, the transfer control device monitors the buffer, and when the record length is equal to or less than a predetermined length, a subsequent record of the record sequence is read from the storage device and stored in the buffer. , The merge process can be performed continuously.

Further, since the data input to the first input means is stored in the memory of the sort processor, the flow of data between the sort processors is smoothed, and as a result, the time required for merging is reduced. be able to.

Further, since the data input to the second input means is stored in the memory of the sort processor, the overhead time for reading the record sequence can be reduced.

Further, since the data transfer control device is configured to directly write the record string to be merged into the memory, the overhead time for reading the record string can be reduced when performing the merge, and the sorting can be further performed. The number of signal lines of the processor can be reduced.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an operation process according to the first embodiment of the present invention.

FIG. 3 is a configuration diagram showing a second embodiment of the present invention.

FIG. 4 is a configuration diagram showing a third embodiment of the present invention.

FIG. 5 is a configuration diagram showing a fourth embodiment of the present invention.

FIG. 6 is a configuration diagram showing an eighth embodiment of the present invention.

FIG. 7 is a configuration diagram of a conventional pipeline merge sorter.

FIG. 8 is an explanatory diagram showing an operation process of a conventional pipeline merge sorter.

FIG. 9 is a configuration diagram of a conventional merge device.

[Explanation of symbols]

111-114 memory, 121-124 sort processor, 302 data transfer control device, 303 storage device, 331 buffer, 161a-164a first input means, 161b-164b second input means

Claims (9)

[Claims] 1. An n-th sort processor having a memory is 2 ^n-1 from an (n-1) -th sort processor.
In a pipeline merge sorter for inputting and merging two sets of sorted record strings composed of records and outputting a set of sorted record strings composed of 2 ⁿ records,
Separately from the first input means for inputting the sorted record strings, the sort processors at each stage are provided with second input means, and a plurality of sorted record strings inputted from the second input means are merged. A data processing device for outputting a single sorted record sequence. 2. An n-th sort processor having a memory is 2 ⁿ⁻¹ from a ( ^{n−1) -th} sort processor.
Two sets of sorted record strings consisting of records are input from the first input means and merged, and 1 consisting of 2 ⁿ records
A pipeline merge sorter for outputting a set of sorted record strings, a storage device for storing a plurality of sets of sorted record strings obtained by repeatedly performing the entire sort processing in the pipeline merge sorter;
A data transfer control device that reads a record of a predetermined length that is set in advance from each set of record rows stored in the storage device and transfers the record to the sort processor of each stage, and inputs the record to the sort processor of each stage. A data processing apparatus comprising a second input unit, wherein a plurality of sets of sorted record strings are merged into a single sorted record string. 3. The transfer control device monitors the second input means of the sort processor at each stage, and when the record length of the second input means is equal to or less than a predetermined length, the transfer control apparatus transmits the subsequent record to the second processor. 3. The data processing device according to claim 2, wherein the data is read from a storage device and transferred to the sort processor. 4. The sort processor of each stage monitors each of the second input means, and when the record length of the second input means is equal to or less than a predetermined length, reads a subsequent record from the storage device. 3. The data processing device according to claim 2, wherein the transfer control device is requested to transfer the data to the sort processor. 5. The transfer control device according to claim 2, further comprising a buffer for reading out and storing a record of a predetermined fixed length from each set of record strings stored in the storage device. Data processing equipment. 6. The transfer control device monitors the buffer and, if the record length is equal to or less than a predetermined length, reads a record subsequent to the record sequence from the storage device and stores the read record in the buffer. Claim 5
The data processing device according to claim 1. 7. The data processing apparatus according to claim 1, wherein the data input to said first input means is stored in a memory of said sort processor. 8. The data processing device according to claim 1, wherein the data input to said second input means is stored in a memory of said sort processor. 9. The n-th sort processor having a memory is 2 ⁿ⁻¹ from the ( ^{n−1) -th} sort processor.
Two sets of sorted record strings consisting of records are input from the first input means and merged, and 1 consisting of 2 ⁿ records
A pipeline merge sorter for outputting a set of sorted record strings, a storage device for storing a plurality of sets of sorted record strings obtained by repeatedly performing the entire sort processing in the pipeline merge sorter;
A data transfer control device for reading a record of a predetermined length set from each set of record strings stored in the storage device and transferring the record to a memory connected to the sort processor in each of the above stages, and a record stored in the memory A second input means for inputting the data to the sort processors at the respective stages, and merging the plurality of sets of sorted record strings into a single sorted record string.