JPS62154139A - Data selecting device - Google Patents

Abstract

PURPOSE:To shorten the data selecting time by selecting two data that have possibility for coincidence between the attribute contents designated as keys in terms for two data groups having the variable length data consisting of variable length attributes. CONSTITUTION:One of independent buffers 11-14 of a fixed capacity stores the length (l) of one or plural attributes designated as keys in a single tuple, a pair of values (v) and the address of the corresponding tuple. A key extracting device 17 extracts the contents of the attribute of the variable length designated as a key out of the tuple of the variable length. The variable length tuple is given from a data input terminal. A hashing device 18 contains a pair of a bit array 7 and a hashing circuit 5 which has a hashing action with an appropriate hash function. A key transmitting device 19 outputs the keys to a data output terminal 25 after selection.

Description

[Detailed Description of the Invention] [Industrial Field of Application] The present invention relates to a data sorting device, and more specifically, the present invention relates to a data sorting device, and more specifically, the present invention relates to a data sorting device, and more specifically, the present invention relates to a data sorting device that matches two variable-length data groups in the content of a variable-length attribute designated as a key. The present invention relates to a data sorting device that sorts out data that may be corrupted.

[Conventional technology]

In the field of data management using information processing systems, 2
When extracting and combining data with matching attribute contents specified as keys from two data groups (the data is variable length), the processing time required for the combination process is long, so it is necessary to combine each data group in advance. Data screening operations are often performed to remove irrelevant data and reduce join processing time. This type of data sorting device generally uses a combination of a hashing circuit and a focusing array. Note that a conventional device of this type is, for example, LEBCH (D, R, McGregor.

R, H, Thomson, W, N, Dawson:
High Performance Hardware
re for Database Systems.

in Systems for Large Data
Ba5es, pp, 103-116, Nor
th-Holland, 1976).

Now, according to Figure 2, the basic operation of a data sorting device using a conventional hashing circuit and bit array combination;
Explain the steps. In Figure 2, 1 and 2 are data groups, 5 is a hashing circuit with an appropriately defined hash function, and 7
is a bit array, 10 is a selected data group, and 301 is a key content of data group 1 which is input to the hashing circuit 5 to perform hashing, and the bit position whose address is the hashing value of bit array 7 is set to '1'. In the operation (which will be called set processing), 302 inputs the key contents of data group 2 to the hashing circuit 5 and performs hashing, and the bit position corresponding to the hashing value of the bit array 7 is set to II I.
This shows an operation (referred to as refer processing) of selecting only data that is II (this is expressed as HIT).

First, all bits of the focus array 7 are initialized to "OII".Next, when selecting data group 2, for example, the bit array 7 is set to "1" in operation 301, and then the data group is set in operation 302. By hashing the key contents of 2, the data whose bit position is 1101+ whose address is the hashing value of bit array 7 is removed as unnecessary data, and the data whose address is "1" is selected as necessary data (this (referred to as sorting process).

[Problem that the invention seeks to solve]

In the conventional data sorting device as shown in Fig. 2,
We are dealing with fixed length attributes.

There was nothing that handled variable length attributes.

Therefore, it was necessary to use software to extract a variable-length attribute designated as a key from the variable-length data and convert it to a fixed length, that is, a key extraction process. Therefore, there is a problem that the key extraction processing time is long and the data sorting processing time is long as a whole.

The purpose of the present invention is to create two sets of variable-length data consisting of variable-length attributes, which have not been achieved in the past;
- To shorten the processing time for one sorting process by making it possible to sort out data whose attribute contents designated as keys are likely to match each other for evening groups.

[Means and effects for solving the problem] The present invention provides four independent buffers each having a constant capacity, a key extraction device for extracting the contents of a variable length attribute designated as a key from variable length data, and attribute contents. a hashing device consisting of a set of a hashing circuit and a bit array for hashing by an appropriate hash function, a key sending device for outputting the attribute contents after selection, and a buffer.
Equipped with a buffer clear device and a control device that perform 01'' clear processing, and according to the instructions of the control device, the four processes of key extraction, hashing, key sending, and buffer clear are performed sequentially in a pipeline manner. This method selects data that may have the same variable-length attribute content designated as a key between two data groups having variable-length data.

〔Example〕

FIG. 1 shows an embodiment of a data sorting device according to the present invention. Figure 3 shows the format of the input data (hereinafter referred to as a tuple) used in this embodiment.
D (TID), a pair of length (Q, ) and value (■1) of attribute l (referred to as Qv pair), relative address in the tuple of attribute 1 (oI: offset)
The configuration is as follows.

The configuration of FIG. 1 is as follows. 11,12゜13.
14 are independent buffers each having a constant capacity (for example, about 128 bytes). One buffer contains one or more 71-rebut length (Q) and value (V) pairs (Qv) specified as keys in one tuple.
pair) and the address of the tuple (TAD: if the memory is connected to the end of the data input terminal, the storage address in the memory) is stored. Reference numeral 17 denotes a key extraction device that extracts the contents of a variable-length attribute designated as a key from a variable-length tuple, and the variable-length tuple is supplied from a data input terminal. 18 is a hashing circuit 5 that performs hashing with the bit array 7 using an appropriate hash function.
This is a hashing device consisting of a set of 19 is a key sending device that outputs data to the data output terminal 25 of the selected key. 20 is a buffer clearing device, which initially clears all buffers 11, 12, 13, and 14 to 11011 when the power is turned on, but normally it clears all bytes in the first side of the connected buffer ``0''.Buffer 11.12,
13.14 and key extraction device 17, hashing device [118
, key sending device 19, buffer access [1 with 20]
The connections of m are mutually switched by a connection switching circuit 15.

21 is a control device that controls the entire device. The key extraction device 17, hashing device 18, and key sending device 19 each have control registers 22, 23, and 24 for performing set processing or clear processing, and values are set by the control device 21 prior to the set processing or clear processing. Ru.

Connection lines between the control device 21 and the key extraction device 17 include a control signal line 101, an address bus line 102. data bus line 10
3. It consists of a read/write control line 104. The control signal line 101 has a processing start signal (START) 1 for 1 tuple.
01-01, normal processing completion signal for l tuples (END)
101-02゜ Data group all tuple end signal (ENPT
Y) 101-03 etc. Address bus line 102 has a sufficient address bit width to indicate control registers and the like. Data bus line 103 has a data bit width sufficient to indicate control registers and the like. Read/write control 1104 is necessary to read/write data to a control register or the like. In addition, 5TART
Paired signals such as the signal and the END signal are handled by handshaking.

Connection lines between the control device 21 and the hashing device 118 include a control signal line 111, an address bus line 112, a data bus line 113, and a read/write control line 114. The control signal line 111-12 has a 5TART signal 111-o1.
: Performs END signal 111-02 and hashing,
There is a HIT signal 111-03 indicating that the bit position corresponding to the hashing value of the bit array 7 is '1'.

Connection lines between the control device 21 and the key sending device 19 include a control signal line 121, an address bus line 122, and a data bus line 1.
23 and a read/write control line 124. The control signal line 121 has a 5TART signal 121-01 and an END signal 121-02.

The connection line 131 between the control device 21 and the buffer clearing device 20 is for the 5TART signal and the END signal, and the connection line 135 between the control device 1i21 and the connection switching circuit 15 is for shifting the connection relationship of the buffer by one. It is an instruction signal.

Connection lines between the key extraction device 17, hashing device 18, key sending device 19, buffer clearing device W20 and connection switching device 17 are address bus lines 142, 152, 162.
, 172, data bus lines 143.153, 163, 17
3. Read/write control lines 144, 154. ．． 164
．． 174 mag.

The connection lines between the connection switching line 17 and each buffer 11, 12, 13°14 are also the same, and the address bus 8182.1
92,202,212. Data bus 41183.193
, 203, 213, read/write control lines 184, 19
4,204,214, etc.

First, the overall control procedure will be explained using the case of sorting data group 2. 1 by a control device 21 that controls the entire device.
Perform the following steps.

(1) Bit array 7 and buffers 11, 12゜13
．． 14 initial settings (2) Setting of control registers for set processing of data group 1 (3) Setting of control registers for set processing of data group 2 (4) Setting of control registers for refer processing of data group 2 (5) Refer processing These processes will be explained in detail later.

(1) Bit play and buffer clear control device 2
1, the pitch array 7 and buffers 11, 12, 13, and 14 in the hashing device 18 are transferred to 1101.
Clear 1.

(2) Control register settings for set processing Information on data group 1 includes: ■ Start address of data group 1 (MEMADR) ■ Effective data length of data group 1 (MEMLNG) ■ Number of attributes specified as keys (ATTNO ) ■ Offset total length (OFSTOTAL) ■ Offset value from the beginning of the offset column for the key (OFSLOC
i) (Attributes such as character/binary for keys required for C-hashing ■ TAD (ArTSPECi) (7) Write address, write length (TADWADR, TADWLN)
G) etc. in each control register 22 according to instructions from the control device 21.
．． Set to 23.24. The setting method is the same as the general memory access method, and the address bus line 102,
112, 122, data bus line 103.113, 123
, the read/write control lines 104, 114, and 124 may be used to write to the control registers 22, 23, and 24.

Note that since the key sending device 19 is not activated in the setting process, it is not necessary to set it in the control register 24.

(3) Set processing FIG. 4 shows an example of a time chart in the set process. At time to, for the first tuple (tuple 1) of data group 1 to be input first, key extraction bag! i! 17 (assert START signal 10f-01). Next, when the processing of the key extraction device 117 is completed (
END signal 101-02 is asserted), connection switching circuit 15 is activated and 5HIFT signal 135 is asserted), and connection switching is performed.

At time tl, hashing device 1 is used for tuple 1.
8, and starts the key extraction device 17 for the next tuple (tuple 2) (START signals 101-01 and 1
11-01).

Next, when the processing of the key extraction device i17 and the hashing device 18 is completed (END signals 101-02 and 11
1-02), activates the connection switching circuit 15 and asserts the 5HIFT signal 135), and switches the connection.

At the time of L2, p1 is a set process for tuple 1, so the key sending device 19 is not activated (the key sending device 19 is not activated for all tuples in the set process), and for tuple 2. starts the hashing device 18 and the key extraction device 17 for tuple 3 (START
assert signals 101-01 and 111-01)
. Next, the key extraction bag Fi17 and the hashing device 18
When the processing is completed (END signals 101-02 and 1
11-01), activates the connection switching circuit 15, and asserts the 'SI■■FT signal 135) to switch the connection.

At time t3, the buffer clearing device 20 is activated for tuple 1 and the 5TART signal 131-01 is asserted), the key sending device 19 is not activated for tuple 2, and hashing is performed for tuple 3. The device 18 (ST
ART signal 111-01), and key extractor 17 for tuple 4 (START signal 101-01).
01) respectively. Next, when the processing of the key extracting device 17, hashing device 18, and buffer clearing device 20 is completed (END signals 101-02, 1
11-02 and 131-02), activates the connection switching circuit 15 and asserts the 5HIFT signal 135), and switches the connection.

In order to repeat the above process, the connection is switched so that key extraction processing, hashing processing, and buffer clearing processing (however, key sending processing is not performed in set processing) are performed sequentially on one side of the buffer. When the set processing for all tuples of data group 1 is completed, the ENPTY signal 101-03 is asserted.

(4) Setting the control registers for refer processing Similarly to the FJ combination in (2), the information of data group 2 is stored in each control register.
Set to 2, 23, 24.

(5) Refer processing Refer processing is performed in the same way as set processing. The difference between refer processing and set processing is that the hashing device 18
When T signal 111-03 is asserted (HI
If the HIT signal 111-03 is negated, the key sending device 17 is activated and the tuple is output from the data output terminal 25.
If there is no key sending group "i19", the key sending group "i19" is not activated.

When the refer processing is completed for all tuples of data group 2, the ENPTY signal 101-03 is asserted, and the selection processing of data group 2 is completed. FIG. 5 shows an example of a time chart in refer processing.

The control procedure described above makes it possible to select data (tuples) that have a possibility of matching the contents of the variable-length attribute specified as the key.

In this section, we have explained the sorting process for only data group 2, but if you want to sort data group 1 and data group 2 at the same time, you can install two devices, one of which can handle the set processing of data group 1. This becomes possible by controlling the other device to perform set processing for data group 2 and then refer processing for data group 1 to the other device.

Next, the key extraction device 17. Each operation of the hashing device 18, the key sending device 19, and the connection switching circuit 15 will be explained.

FIG. 6 is a diagram illustrating the processing of the key extraction device 17.

In this example, the input data (tuple) has four attributes (A+, A2, A3, As), of which offsets are attached to A and A2. Also, as keys, attributes A + , A 2 are A 1 . This is a case where they are specified in the order of A x. In the key extraction device 17, it is assumed that the maximum flame of each attribute is given in advance, and the attributes have a fixed length. Note that A1 and A2 may be stored consecutively in the buffer without making them a fixed length.

As a result of the processing of the key extraction device 17, A+.

Write the Qv pair of A2 and the calculated TAD to the buffer.

A detailed algorithm for key extraction processing is shown in FIG. If it is a memory that is connected to the end of the data input terminal 16, then the start address of the data group in the memory MEMADR
Assume that a tuple is stored from , and the final address is M
It is EMADR + MEMLNG. When there are no tuples at an intermediate address, the end determination (determination of the end of the data group) is performed using QQ=O. address and final address), ■It is sufficient to specify the end of the data group.

Next, the processing of the hashing device 18 will be explained.

The hashing device 18 performs hashing processing using a hashing function that contains the value of an attribute designated as a key, one byte at a time.

In the set processing, the data is hashed and the bit position corresponding to the hashed address is set to "1".

On the other hand, in the refer processing, if the bit position whose address is the hashing value after hashing is II I IT,
The HIT signal 111-03 is asserted, but if it is 0'', the HIT signal 111-03 remains negated.

Next, the processing of the key sending device 19 will be explained. This is HI
The Qv pair and TAD of attributes A1 and A2 designated as the key of the tuple for which the T signal 111-03 has been asserted are output to the data output terminal 25.

FIG. 8 is a diagram showing how connection switching is performed by the connection switching circuit 15. The connection switching circuit 15 has four buffers 11,
12, 13, and 14, key extraction device 17, hashing device 18, key sending device 19, and buffer clearing device 20.
Each buffer is connected to one of the four devices. These connections are switched by asserting the 5HIFT signal 135, and the key extraction device 17, hashing device 18, key sending device 19, and buffer clearing device 20 are sequentially activated as shown in FIG. 4 or 5.

〔Effect of the invention〕

As described above, the present invention provides a key extraction device including four independent buffers each having a constant capacity, a control device, a key extraction device, a hashing device, a key sending device, a buffer clearing device, and a connection switching circuit.

While operating the hashing device, key sending device, and buffer clearing device sequentially in a pipeline manner, the contents of the variable-length attributes specified as keys match between two data groups each having variable-length data composed of variable-length attributes. Since this system selects data that may be used for data processing, it is able to handle variable-length attributes compared to conventional devices, and at the same time, it is 1. This has the advantage that the time required for sorting processing can be shortened.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an embodiment of a data sorting device according to the present invention, FIG. 2 is an explanatory diagram of the operating procedure of a conventional data sorting device, and FIG. 3 is a diagram showing an input data format used in this embodiment.
Fig. 4 is a time chart of the set processing in this embodiment, Fig. 5 is a time chart of refer processing in this embodiment, Fig. 6 is an explanatory diagram of the processing of the key extraction device in this embodiment, and Fig. 7 is the main FIG. 8 is a diagram showing a detailed algorithm of the key extraction process in this embodiment, and is a diagram showing the state of connection switching by the connection switching circuit in this embodiment. 5... Hashing circuit, 7... Bit array,
11.12, 13.14... Buffer, 15... Connection switching circuit, 16... Data input terminal. 17... Key extraction device, 18... Hashing device, 19... Key sending device, 20... Buffer clearing device, 21... Control device, 22, 23.24
...Control register, 25...Data output terminal. Representative Patent Attorney Seiichi Suzuki 1st Figure 2 [

Claims (1)

[Claims] In a device that selects data that has a possibility of matching the contents of variable-length attributes specified as mutual keys between two data groups having variable-length data composed of variable-length attributes, each has an independent fixed capacity. A hashing device consisting of a four-sided buffer, a key extraction device that extracts the contents of a variable-length attribute specified as a key from variable-length data, a hashing circuit and a bit array that hash the attribute contents using an appropriate hash function. , a key sending device that outputs the attribute contents after sorting, a buffer clearing device that clears the buffer, and a link between the four buffers, the key extracting device, the hashing device, the key sending device, and the buffer clearing device. It is equipped with a connection switching circuit that switches connections, and a control device that controls each of the devices and the connection switching circuit, and performs key extraction, hashing, key sending,
Data characterized in that four processes of buffer clearing are sequentially operated in a pipeline manner, and data in which variable length attribute contents designated as keys are likely to match each other in the two data groups is selected. Sorting equipment.