JPH0731686B2 - Data sorter - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a data selection device, and more specifically, to matching of variable-length attribute contents designated as a key between two variable-length data groups. The present invention relates to a data selection device that selects data that may be present.

[Conventional technology]

2 in the field of data management by information processing systems
It is possible to combine each data group in advance because the processing time required for the combination processing is long when the data with matching attribute contents designated as keys from each other is extracted from one data group (the data is variable length). A data selection operation is often performed to eliminate irrelevant data and reduce the join processing time.It is common to use a pair of hashing circuit and bit array for this kind of data selection device. Conventional devices of this type are, for example, LEECH (DRMcGregor, RHThomson, WNDawson: High
Performance Hardware for Database Systems, in Syste
ms for Large Data Bases, pp.103−116, North-Holland,
1976) and other systems.

Here, a basic operation procedure of the conventional data selection device using the set of the hashing circuit and the bit array will be described with reference to FIG. In FIG. 2, 1 and 2 are a data group, 5 is a hashing circuit having an appropriately defined hash function, 7 is a bit array, 10 is a sorted data group, and 301 is a hashing circuit for the key contents of the data group 1. 5, the hashing operation is performed by inputting the data to the 5th row, and the bit position having the hashing value of the bit array 7 as an address is set to "1" (set processing is called).
Hashing is performed by inputting to the bit array 7, and the bit position whose address is the hashing value of the bit array 7 is “1” (this is HI
An operation of selecting only data (expressed as T) (referred to as referrer processing) is shown.

First, all bits of the bit array 7 are initialized to "0". Next, for example, when selecting the data group 2, operation 30
After setting “1” to the bit array 7 by 1, the operation 302
The key contents of the data group 2 are hashed, and the bit position with the hashing value of the bit array 7 as the address is "0".
The data that is is removed as unnecessary data,
The data that is "1" is selected as necessary data (this is referred to as selection processing).

[Problems to be solved by the invention]

In the conventional data selection device as shown in FIG. 2,
It deals with fixed-length attributes and none with variable-length attributes. Therefore, it is necessary for the software to perform a process of extracting a variable-length attribute designated as a key from the variable-length data from the data to fix it, that is, a key extracting process. Therefore, there is a problem that the key extraction processing time is long and the data selection processing time is long as a whole.

An object of the present invention is to select, from two data groups having variable-length data composed of variable-length attributes, which has not been conventionally handled, data whose attribute contents designated as a key may match each other. The purpose is to shorten the sorting processing time.

[Means and Actions for Solving Problems]

The present invention has four independent buffers each having a fixed capacity, a key extraction device for extracting the contents of a variable length attribute designated as a key from variable length data, a hashing circuit and a bit for hashing the attribute contents by an appropriate hash function. A hashing device consisting of a set with an array,
A key sending device that outputs the attribute contents after selection,
A buffer clear device that clears the buffer "0",
And a control device, and in accordance with an instruction from the control device, four processes of key extraction, hashing, key transmission, and buffer clear are sequentially operated in a pipeline manner, and two data having variable length data composed of variable length attributes are provided. This is to select data in which the variable-length attribute contents designated as a key may match each other in the data group.

〔Example〕

FIG. 1 shows an embodiment of a data selection device according to the present invention. FIG. 3 shows a format of input data (hereinafter, this data is referred to as a tuple) used in this embodiment. In order to support a variable length type, a double length (ll) and a tuple ID are used.
(TID), a pair of a length (l _i ) and a value (v _i ) of the attribute i (called a l _v pair), and a relative address (o _i : offset) in the tuple of the attribute i.

The configuration of FIG. 1 is as follows. Reference numerals 11, 12, 13, and 14 are independent buffers having a fixed capacity (for example, about 128 bytes). 1 length of one or more attributes in the buffer specified as a key in one tuple (l) and value (v) of the pair (l _v pair) and the address of the tuple (TAD: the previous data input terminal If the connected memory is the memory, the storage address in the memory)
To store. Reference numeral 17 is a key extraction device for extracting the contents of a variable length attribute designated as a key from the variable length tuple, and the variable length tuple is given from the data input terminal 16. A hashing device 18 is composed of a bit array 7 and a hashing circuit 5 that performs hashing by an appropriate hash function. Reference numeral 19 is a key sending device for outputting the selected keys to the data output terminal 25. Reference numeral 20 is a buffer clear device, which initially clears all of the buffers 11, 12, 13, and 14 to "0" when the power is turned on, but normally clears all bytes of the connected buffer 1 side to "O". Buffer 11,12,13,1
4 and key extraction device 17, hashing device 18, key sending device 1
9. The connection with the buffer clear device 20 is switched by the connection switching circuit 15.

Reference numeral 21 is a control device for controlling the entire device. Key extractor
17, the hashing device 18, and the key sending device 19 are control registers for performing set processing or clear processing, respectively.
There are 22, 23 and 24, and the value is set by the control device 21 prior to the set processing or the clear processing.

The connection line between the control device 21 and the key extraction device 17 is a control signal line.
101, address bus line 102, data bus line 103, read /
It consists of a write control line 104. On the control signal line 101, a processing start signal (START) 101-01 for one tuple, a processing normal end signal (END) 101-02 for one tuple, an all tuple end signal (ENPTY) 101-03 for the data group, etc. is there. The address bus line 102 has an address bit width sufficient to indicate a control register or the like. The data bus line 103 has a data bit width sufficient to indicate a control register or the like. The read / write control line 104 is used to read / write data to / from a control register or the like.
It is necessary to write. In addition, START signal and END
Handshaking is done for paired signals such as signals.

Connection lines between the control device 21 and the hashing device 18 are composed of a control signal line 111, an address bus line 112, a data bus line 113, and a read / write control line 104. The control signal line 111 has a STAR
There are a T signal 111-01, an END signal 111-02, and a HIT signal 111-03 which indicates that the bit position where the hashing value of the bit array 7 is "1" is "1".

Connection lines between the control device 21 and the key sending device 19 are composed of a control signal line 121, an address bus line 122, a data bus line 123, and a read / write control line 124. Start on control signal line 121
There are signals 121-01 and END signals 121-02.

Connection line 131 between the control device 21 and the buffer clear device 20
Is a START signal and an END signal, and the connection line 135 between the control device 21 and the connection switching circuit 15 is an instruction signal for shifting the connection relationship of the buffer by one.

A key extracting device 17, a hashing device 18, a key sending device 19,
The connection lines between the buffer clear device 20 and the connection switching circuit 15 are the address bus lines 142, 152, 162, 172 and the data bus line 14
3,153,163,173, read / write control lines 144,154,164,174
Etc. The connection lines between the connection switching circuit 15 and the respective buffers 11, 12, 13, 14 are also the same, and the address bus lines 182, 192, 20
2, 212, data bus lines 183, 193, 203, 213, read / write control lines 184, 194, 204, 214 and the like.

First, the entire control procedure will be described in the case of selecting the data group 2. The following procedure is performed by the control device 21 that controls the entire device.

(1) Initial setting of bit array 7 and buffers 11, 12, 13, 14 (2) Setting of control register for set processing of data group 1 (3) Set processing (4) For reference processing of data group 2 (5) Referrer Processing Next, these processings will be described in detail in order.

(1) Clearing the bit array and buffer The bit array 7 and the buffers 11, 12, 13, and 14 in the hashing device 18 are cleared to "0" according to an instruction from the control device 21.

(2) As information of the setting data group 1 of the control register for set processing, the start address of the data group 1 (MEMADR) The effective data length of the data group 1 (MEMLNG) The number of aributes specified as the key (ATTNO) Total offset Long (OFSTOTAL) Offset value for the offset key from the beginning of the column (OFSLOCi) Control the write address and write length (TADWADR, TADWLNG) of the attribute TAD (ATTSPECi) such as character / binary for the key required for hashing It is set in each control register 22, 23, 24 according to an instruction from the device 21. The setting method is the same as the general memory access method, and the address bus lines 102, 112, 122, the data bus lines 103, 113, 123, and the read / write control lines 104, 114, 124 may be used to write to the control registers 22, 23, 24. Since the key sending device 19 is not activated in the setting process,
It does not have to be set in the control register 24.

(3) Set Process FIG. 4 shows an example of a time chart in the set process. At time t ₀ , only the key extracting device 17 is activated (asserts the START signal 101-01) for the first tuple (tuple 1) of the data group 1 that is input first. Next, when the processing of the key extracting device 17 is completed (END signal 101-02 is asserted), the connection switching circuit 15 is activated (SHIFT signal 135 is asserted), and connection switching is performed.

At time t _1, the hashing device 18 for tuple 1
For the next tuple (tuple 2)
(Start signals 101-01 and 111-01 are asserted). Next, the key extraction device 17 and the hashing device
When the processing of 18 is completed (END signals 101-02 and 111-02
Is asserted), the connection switching circuit 15 is activated (the SHIFT signal 135 is asserted), and connection switching is performed.

In time t _2, since for the tuple 1 is set processing, without starting the key delivery device 19 (a set processing, the key delivery device 19 does not start for all tuples), for the tuple 2 The hashing device 18 and the key extraction device 17 for the tuple 3 are activated (START signal 101-01 and
111-01 is asserted). Next, when the processing of the key extraction device 17 and the hashing device 18 is completed (END signal 101-
02 and 111-02 are asserted), the connection switching circuit 15 is activated (the SHIFT signal 135 is asserted), and connection switching is performed.

At time t ₃ , buffer clear device for tuple 1
20 is activated (the START signal 131-01 is asserted), the key sending device 19 is not activated for the tuple 2, and the hashing device 18 is asserted for the tuple 3 (the START signal 111-01 is asserted). , The key extraction device 17 for the tuple 4 (ST
Assert ART signal 101-01) Activate each. next,
When the processes of the key extracting device 17, the hashing device 18, and the buffer clearing device 20 are completed (END signals 101-02, 111-0
2 and 131-02 are asserted), the connection switching circuit 15 is activated (SHIFT formation 135 is asserted), and connection switching is performed.

In order to repeat the above processing, key extraction processing, hashing processing, and buffer clear processing are performed on one side with a buffer (however, key sending processing is not performed in the set processing).
The connection is switched so that the steps are sequentially performed. When the setting process is completed for all tuples in data group 1, ENPTY signal 101
Assert −03.

(4) Setting of control register for referrer processing As in the case of (2), the information of the data group 2 is set in each control register 22, 23, 24.

(5) Referrer processing The referrer processing is performed in the same manner as the set processing. The referrer process is different from the set process when the HIT signal 111-03 is asserted by the hashing device 18 (when HIT is performed).
Activates the key sending device 17 and outputs the tuple from the data output terminal 25, but does not activate the key sending device 19 when the HIT signal 111-03 is negated (when not HIT). Is. FIG. 5 shows an example of a time chart in the referrer process. At time t ₀ , only the key extraction device 17 is activated (asserts the START signal 101-01) for the first tuple (tuple 1) of the data group 2 that is input first. Next, when the processing of the key extracting device 17 is completed (END signal 101-02 is asserted), the connection switching circuit 15 is activated (SHIFT signal 135 is asserted), and connection switching is performed.

At time t _1, the hashing device 18 for tuple 1
For the next tuple (tuple 2)
(Start signals 101-01 and 111-01 are asserted). Next, when the processing of the key extraction device 17 and the hashing device 18 is completed (END signals 101-02 and 111-02 are asserted), the connection switching circuit 15 is activated (SHIFT signal 135
, And switch the connection.

Here, the processing of the hashing device 18 performs hashing, and if the bit position whose address is the hashing value of the bit array 7 is “1”, it means that a hit occurs.
If signal 111-03 is asserted, and if it is "0", H
To negate the IT signal 111-03.

In the example of Fig. 5, ・ Associate the HIT signal for tuple 1 ・ Assert the HIT signal for tuple 2 ・ Negate the HIT signal for tuple 3 ・ Negate the HIT signal for tuple 4, and so on The same process is performed for the tuples of.

Since the HIT signal 111-03 is asserted for the tuple 1 at time t ₂ , the key sending device 19 is activated to send the corresponding key to the data output terminal 25. The hashing device 18 is activated for the tuple (tuple 2), and the key extraction device 17 is activated for the tuple 3 (START signal 101-01 and
111-01 is asserted). Next, when the processes of the key extraction device 17, the hashing device 18, and the key transmission device 19 are completed (END signals 101-02 and 111-02 are asserted), the connection switching circuit 15 is activated (the SHIFT signal 135 is asserted). Yes), switch connection.

At time t ₃ , the buffer clearing device 20 that clears the corresponding buffer is activated for tuple 1 (START signal 130
Assert -01). HIT signal 1 for tuple 2
Since 11-03 is asserted (in the processing of the hashing device 18, the hashing is performed and the bit position whose address is the hashing value of the bit array 7 is "1"), the key sending device 19 is activated. Then, the data output terminal 25
Then, the corresponding key is sent to. For tuple 3, the hashing device 18 (asserts START signal 111-01) and for tuple 4 the key extraction device 17 (START signal 101-01).
Assert each). Next, the key extraction device 1
7. When the processes of the hashing device 18, the key sending device 19, and the buffer clearing device 20 are completed (END signals 101-02, 11
1-02 and 131-02 are asserted), connection switching circuit 15
Is activated (asserts the SHIFT signal 135) to switch the connection.

The HIT signals 111-03 are negated with respect to the tuples 3 and 4 (hashing is performed by the hashing device 18, and the bit position whose address is the hashing value of the bit array 7 is "0"). from, in each time t _4, t _5, that it does not hit, the key delivery device 19 is not activated.

When the referrer processing is completed for all tuples of the data group 2, the ENPTY signal 101-03 is asserted, and the selection processing of the data group 2 is completed.

With the control procedure described above, it is possible to select data (tuples) that may have matching variable-length attribute contents designated as a key.

This section describes the sorting process for only the data group 2. However, if you want to sort the data groups 1 and 2 at the same time, by equipping one of these devices, you can set the data group 1 in one device. It is possible to perform the processing, the contact processing for the reference of the data group 2, the other apparatus for the setting processing of the data group 2, and the subsequent processing for the data group 1.

Next, operations of the key extracting device 17, the hashing device 18, the key sending device 19, and the connection switching circuit 15 will be described.

FIG. 6 is a diagram for explaining the processing of the key extracting device 17. In this example, the input data (tuple) has four attributes (A ₁ , A ₂ , A ₃ , A ₄ ), and the offset is one of them.
Attached to A ₁ and A ₂ . This is also the case when the attributes A ₁ and A ₂ are designated as keys in the order of A ₁ and A ₂ . In the key extracting device 17, the maximum length of each attribute is given in advance, and the attributes are fixed length. In addition, A ₁ and A ₂ are not fixed length, A ₁
And A ₂ may be consecutively stored in the buffer.

As a result of the processing of a key extractor 17, writes the TAD and the calculated l _v pair A _1, A ₂ in the buffer.

The detailed algorithm of the key extraction process is shown in FIG. If a memory is connected to the end of the data input terminal 16, tuples are stored from the first address MEMADR of the data group in the memory, and the last address is MEMADR + ME.
It is MLNG. Ll = is the end judgment (judgment of the end of the data group) when tuples are lost in the middle addresses
However, the present invention is not limited to this, and the storage area of the data group (for example, the start address and the final address) and the end of the data group may be designated.

Next, the processing of the hashing device 18 will be described. The hashing device 18 performs a hashing process by using a hashing function that stores one byte of the value of the attribute designated as the key. In the setting process, hashing is performed and the bit position having the address as the hashing is set to "1". On the other hand, in the referr processing, if hashing is performed and the bit position whose address is the hashing value is “1”, HI
The T signal 111-03 is asserted, but if it is "0", the HIT signal 111-03 remains negated.

Next, the processing of the key sending device 19 will be described. This is to output a l _v pairs and TAD of the specified attribute A _1, A ₂ as the key of the tuple asserts the HIT signal 111-03 to the data output terminal 25.

FIG. 8 is a diagram showing a state of connection switching by the connection switching circuit 15. The connection switching circuit 15 is a four-sided buffer 11, 12, 13, 1.
4 and key extraction device 17, hashing device 18, key sending device 1
9. The connection between the buffer clear device 20 and four devices is switched, and each of the buffers is connected to any one of the four devices. These connection switching is SH
This is performed by asserting the IFT signal 135, and then, as shown in FIG.
As shown in the figure, sequential key extraction device 17, hashing device
18. The key sending device 19 and the buffer clearing device 20 are activated.

〔The invention's effect〕

As described above, the present invention includes four independent buffers each having a fixed capacity, a control device, a key extracting device, a hashing device, a key sending device, a buffer clearing device, and a connection switching circuit. , The key sending device and the buffer clearing device are sequentially operated in a pipeline manner, and the contents of the variable length attribute specified as a key can be matched with each other in two data groups having variable length data composed of variable length attributes. Since it is intended to select data with a certain nature, compared to conventional devices,
There is an advantage that a variable length attribute can be handled and at the same time, a time required for a selection process can be shortened as compared with a conventional device in which a variable length key must be extracted by software.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of a data selection device according to the present invention, FIG. 2 is an explanatory diagram of an operation procedure of a conventional data selection device, and FIG. 3 is a diagram showing an input data format used in this embodiment. FIG. 4 is a time chart of the set process in this embodiment, FIG. 5 is a time chart of the referrer process in this embodiment, FIG. 6 is an explanatory diagram of the process of the key extraction device in this embodiment, and FIG. FIG. 8 is a diagram showing a detailed algorithm of the key extracting device in the embodiment, and FIG. 8 is a diagram showing a state of connection switching by the connection switching circuit in this embodiment. 5 ... Hashing circuit, 7 ... Bit array, 11, 12, 13, 14
… Buffer, 15… Connection switching circuit, 16… Data input terminal,
17 ... Key extracting device, 18 ... Hashing device, 19 ... Key sending device, 20 ... Buffer clearing device, 21 ... Control device, 22,23,
24 ... Control register, 25 ... Data output terminal.

Claims (1)

[Claims] 1. An apparatus for selecting data in which contents of variable-length attributes designated as keys of two data groups having variable-length data composed of variable-length attributes may match each other. An independent fixed-capacity four-sided buffer, a key extraction device that extracts the contents of a variable-length attribute designated as a key from variable-length data, a hashing circuit that hashes the attribute contents with an appropriate hash function, and a bit array. A hashing device consisting of a set, a key sending device for outputting the attribute contents after selection, a buffer clearing device for clearing the buffer, a buffer and a key extracting device for the four sides, a hashing device, a key sending device, and a buffer clearing device. A connection switching circuit for switching a connection with a device, the devices and And a controller for controlling the continuous switching circuit, and in accordance with an instruction from the controller, the four processes of key extraction, hashing, key transmission, and buffer clear are sequentially operated in a pipeline manner, and the two data groups are mutually operated. A data selection device for selecting data having a possibility of matching variable-length attribute contents designated as a key.