JPH0419570B2 - - Google Patents

Description

[Detailed Description of the Invention] 1. Description of the field to which the invention pertains The present invention is a data sorting method that selects data whose data contents may be the same for all data groups from among a plurality of data groups. It is related to the method.

2. Description of Prior Art In the field of data management, in particular, when extracting and combining data whose data contents match each other from two data groups, the processing time required for the combining process is large, so it is necessary to prepare each data in advance. Data culling operations are often performed to remove uncombinable data from groups and reduce join processing time. For this data sorting, a hashing device and a bit array are generally used.

Traditionally, this type of equipment is called LEECH (DRMc
Gregor, RHThomson, WNDawson: High
Performance Hardware for Database
Systems,in Systems for Large Data Bases,
PP.103-116, North-Holland, 1976.) and
CAFS((1) E.Babb:Implementing a
Relational Database by Means of
Specialized Hardware, ACM Trans.Database
Syst.,Vol.4,No.1,PP.1-29,March 1979.(2)
E.Babb:Patent Application No.27093/74,
Used in systems such as June 1974 (Hashed Bit Arrays Store).

The prior art will be explained with reference to FIGS. 1, 2, and 3.

FIG. 1 shows the basic operating procedure of a conventional method using a bit array. 1 and 2 are data groups, 5
7 is a bit array, 10 is a selected data group, and 101 inputs the data contents of data group 1 to the hashing device 5 to perform hashing, and the hashing value of bit array 7 is obtained. The operation 201 is to input the data contents of the data group 2 to the hashing device 5 and perform hashing, so that the bit position whose address is the hashing value of the bit array 7 is ▼1▼. Indicates an operation to select only certain data.

First, all bits in bit array 7 are ▼0▼
Initialize to . Next, after setting ▼1▼ in the bit array 7 by operation 101, the data contents of data group 2 are hashed by operation 201,
Data whose bit position is ▼0▼ whose address is the hashing value of the bit array 7 is removed as unnecessary data, and data whose bit position is ▼1▼ is selected as necessary data (this is called selection).
The method shown in FIG. 1 has a drawback in that only one of the two data groups is selected.

Next, a case will be described in which data selection is performed on a plurality of data groups.

The method shown in Fig. 2 selects data whose data contents may match each other from two data groups, 1 and 2 are data groups, 4,
5 is a hashing device with the same hashing function;
7 and 8 are bit arrays, and 10 and 11 are selected data groups.

First, all bits in the bit array are initialized to ▼0▼.

Next, data group 1 is input to the hashing device 5 for hashing, the bit position corresponding to the hashing value of bit array 8 is set to ▼1▼ (operation 101), and then data group 2 is input to the hashing device 5. 4 and performs hashing, and sets the bit position corresponding to the hashing value of bit array 7 to ▼1▼ (operation 102).
Select only data whose address is the hashing value of bit array 8 and whose bit position is ▼1▼ (operation 201).Furthermore, operation 102 and operation 2
01 is completed, data group 1 is input to the hashing device 4 and hashed, and only the data whose address is the hashing value of the bit array 7 and whose bit position is ▼1▼ are selected (operation 202). .

FIG. 3 shows a time chart for the method shown in FIG. In FIG. 3, the numbers of data in data groups 1 and 2 are respectively m ₁ and m ₂ (m ₁ < m ₂ ), and S ₁ : Processing time required for operation 101 S ₂ : Processing time required for operation 102 R ₁ : Processing time required for operation 202 R ₂ : Processing time required for operation 201. In this case, S ₁ <S ₂ ,
R ₁ < R ₂ and max(R ₂ , S ₂ ) is R ₂ and S ₂
If it represents the larger value of
The processing time (T ₁ ) is T ₁ =S ₁ +max(R ₂ , S ₂ )+R ₁ .

The method shown in Figure 2 has an advantage over the method shown in Figure 1 in that data selection processing is performed for both data groups, but as shown in Figure 3, in general, , since the data group 10 selected in operation 202 is output after the data group 11 selected in operation 201 is output, there is a drawback that the processing time is longer than in the method shown in FIG. Furthermore, when two bit arrays are used as preprocessing when performing data group combination processing, only two data groups can be handled in principle. Therefore, when performing a combination process on three or more data groups, the method is to first perform a selection process and a combination process on two data groups, create a new data group, and then
There is no choice but to adopt a method of repeating the selection process and the combination process again between this new data group (intermediate data group) and the remaining data group. However, this method requires (n-2) operations to generate an intermediate data group, and (n-1)
The disadvantage is that the processing time becomes extremely long due to the need to perform multiple data sorting operations. Furthermore, in the conventional method shown in FIG. 2, as is clear from FIG. Now, in operation 102 and the next operation 202),
It's different. In other words, from operation 101 to operation 20
Immediately before shifting to 1, it was necessary to switch the data group to be input.

Regarding the switching control of this data group, it is necessary to prepare multiple data paths for inputting to the hashing device, and a switching control circuit is also required to connect the data group and the hashing device via the data path. However, the switching control is complicated and the configuration is complicated.

3. Purpose of the Invention In order to eliminate the drawbacks of the conventional method described above, the present invention provides an AND device that performs an AND operation between n sets of hashing devices and bit arrays and all bit arrays for a plurality of (n) data groups. By having one device, data that may have the same data content for all data groups can be selected at once, and its purpose is to shorten the selection processing time.

Hereinafter, the present invention will be explained in detail with reference to examples.

4 Description of Invention Structure and Effects FIG. 4 is a diagram for explaining the operating procedure in the embodiment system of the present invention. Here, to simplify the explanation, a case will be explained in which the number of data groups is three. In the figure, 1, 2, and 3 are data groups, 4, 5, and 6 are hashing devices with the same hashing function, 7, 8, and 9 are bit arrays, 13 is an AND device, and 10, 11, and 12 are selected data. Indicates a group.

First, all bits in the bit array are initialized to '0'. Next, one data group is assigned a pair of hashing device and one bit array, data group 1 is input to hashing device 4 to perform hashing, and the hashing value of bit array 7 is At the same time as setting ▼1▼ in the bit position to be the address (operation 301),
is input into the hashing device 5 to perform hashing, and set ▼1▼ in the bit device whose address is the hashing value of the bit array 8 (operation 302).
Further, the data group 3 is input to the hashing device 6, hashed, and ▼1▼ is set in the bit position corresponding to the hashing value of the bit array 9 (operation 303).The setting operations for all these data groups are completed. At this point, the bit arrays 7, 8, and 9 are input to the AND device 13, AND is performed, and the AND result is transferred to the bit arrays 7, 8, and 9.

The situation will be explained with reference to FIG. Bit array 7 at the time when all data group set operations are completed,
Let the contents of 8 and 9 be 20, 21, and 22, respectively.
The AND device 13 selects the bit array contents 20,2
Input 1 and 22 and perform AND for each bit position. For example, in the example shown in FIG.
The result of taking is 0.Similarly, since the contents of bit position 1+1 are 0, 1, 0, the result of taking the AND is 0.Similarly, the contents of bit position 1+2 are 1, 1, 1 Therefore, the result of the AND is 1, and so on, the result 23 of the AND for every bit position is updated for each bit position of bit arrays 7, 8, and 9. As a result, the updated contents of bit arrays 7, 8, and 9 are all the same, as shown at 24, 25, and 26 in FIG.

Here, the data group 1 is input to the hashing device 4 and hashed, and only the data whose address is the hashing value of the bit array 7 and whose bit position is ▼1▼ are selected (operation 401).
The data group 2 is input to the hashing device 5 and hashed, and only the data whose bit position is ▼1▼ with the hashing value of the bit array 8 as the address are selected (operation 402), and then the data group 3 is input to the hashing device 6. is input to perform hashing, and select only the data whose address is the hashing value of bit array 9 and whose bit position is ▼1▼ (operation 4).
03).

As described above, first, by performing the setting operations for data groups 1, 2, and 3 simultaneously, each bit array 7, 8, and 9 is set, and then, AND is performed for each bit position of bit arrays 7, 8, and 9. Based on the results, bit arrays 7, 8,
9, and then hash the data contents of each data group 1, 2, and 3, and refer to the updated bit arrays 7, 8, and 9 to find the bit position whose address is the hashed value. The contents of ▼1
By simultaneously performing the operation of selecting only the data that is ▼, that is, the REFER operation, it is possible to simultaneously select data that may have the same data content for all data groups.

FIG. 6 shows a time chart in the embodiment shown in FIG. Figure 6 shows the numbers of data in data groups 1, 2, and 3 as m ₁ , m ₂ , m ₃
(m ₁ <m ₂ <m ₃ ), and Si: processing time required for operation 30i (i=1, 2,
3) Rj: Processing time required for operation 40j (j=1, 2
3) Q: This shows the processing time required to perform AND between all bit arrays and transfer the result to all bit arrays, and the processing time (T ₃ ) is T ₃ = Q+max{S ₁ , S ₂ , S ₃ }+max {R ₁ , R ₂ , R ₃ }. Here, the processing time Q for performing the AND operation between all bit arrays and transferring the results to all bit arrays by the AND device becomes an overhead. However, since this AND device is a group of simple AND circuits, it can be expected to increase the speed and reduce this overhead by implementing LSI. Therefore, compared to a conventional method that uses two hashing devices and two bit arrays to select all n data groups, the processing time is reduced based on the fact that there is no need to create intermediate data groups. can be expected to be shortened.

FIG. 7 shows an example of a circuit that implements the embodiment shown in FIG. In Figure 7, 4, 5, 6
is a hashing device, 7, 8, 9 are bit arrays,
13 is an AND device, 60, 70, 80 are input terminals,
61, 71, 81 are data retrieval circuits, 62, 7
2, 82 are hashing circuits, 63, 73, 83 are buffers, 67, 77, 87 are transfer circuits, 64,
74, 84 are output terminals, 65, 75, 85 are SET signals that instruct the operation (SET operation) to set ▼|▼ at the bit position whose address is the hashing value of the bit array, 66, 76, 86 are the hashing signals of the bit array. The bit position where the value is the address is ▼｜
PEFER signal instructing operation to select only data that is ▼ (REFER operation), 67, 77, 87
90 is a transfer circuit, and 90 is an activation signal to an AND device that performs AND between all bit arrays and transfers the AND result to all bit arrays (note that the SET signal, REFER signal, and activation signal to the AND device) (The control circuit that generates the signal is omitted from the figure.) First, all bits in the bit arrays 7, 8, and 9 are initialized to ▼0▼. The data of data group 1 is input to input terminal 60, the data of data group 2 is input to input terminal 70, and the data of data group 3 is input to input terminal 80. Data retrieval circuit 61, 71, 81
is activated in synchronization with the generation of each SET signal 65, 75, 85 or each REFER signal 66, 76, 86, and transfers data to buffer 63.

Next, each hashing device 4, 5, 6,
It consists of each hashing circuit 62, 72, 82 and each buffer 63, 73, 83. For example, when the hashing circuit 62 detects the SET signal 65, the content of the data stored in the buffer 63 is hashed. Then, ▼1▼ is set at the bit position corresponding to the hashing value of the bit array 7 (SET operation), and the next data is transferred to the buffer 63 in order to repeat the SET operation of the data.
A signal requesting transfer to the data retrieval circuit 61 is issued to the data retrieval circuit 61. Further, when the hashing circuit 62 detects the REFER signal 66, the buffer 63
If the contents of the data stored in the bit array 7 are hashed and the bit position corresponding to the hashed value is ▼1▼, a start signal is issued to the transfer circuit 67, and the transfer circuit 67 starts buffering by this start signal. The data retrieval circuit 6 outputs the data contents of 63 to the output terminal 64 and sends a signal requesting to transfer the next data to the buffer 63.
Emit at 1. If the bit position is not ▼1▼, a signal is issued to the data retrieval circuit 61 requesting that the next data be transferred to the buffer 63 (REFER operation).

The hashing devices 5, 6 and the data retrieval circuits 71, 81 also operate in the same manner as described above, and send out the selected data to output terminals 74, 84, respectively.

When the SET operation is completed for all data groups, an activation signal for the AND device 13 is issued.
The AND device 13 inputs the bit arrays 7, 8, and 9, performs an AND operation, transfers the result of the AND operation to the bit arrays 7, 8, and 9, and updates the contents thereof.

As described above, by first performing the SET operations on data groups 1, 2, and 3 simultaneously, each bit array 7, 8, and 9 is set, and then the result of ANDing bit arrays 7, 8, and 9 is set. Using bit arrays 7, 8, and 9, each data group 1,
By performing two and three REFER operations at the same time, it is possible to select all data groups at once for data that may have the same data content.

5. Explanation of Effects As explained above, the present invention can simultaneously process data for all data groups by providing n sets of hashing devices and bit arrays and one AND device for a plurality of (n) data groups. Since the purpose is to select data that may have matching contents, two conventional hashing devices and two bit arrays are used.
This method has the advantage that the time required for the sorting process can be reduced compared to a method in which all n data groups are sorted using a separate device. As is clear from a comparison between FIG. 2 of the prior art and FIG. 4 of the present invention, there is only one data path for inputting to each hashing device, and there is no need to control data group switching. Furthermore, the AND device itself has a simple configuration, which has the advantage of simplifying the hardware configuration, making it easy to implement into an LSI, and increasing the speed by implementing an LSI.

[Brief explanation of the drawing]

1 and 2 are explanatory diagrams of the operating procedure of the conventional data sorting method, FIG. 3 is a time chart for the operation shown in FIG. 2, and FIG. 4 is an explanatory diagram of the operating procedure of the data sorting method of the present invention. , Figure 5 shows AND
6 is a time chart for the operation shown in FIG. 4, and FIG. 7 is a circuit diagram of an embodiment of the present invention. In the figure, 1, 2, 3 are data groups, 4, 5, 6 are hashing devices, 7, 8, 9 are bit arrays, 1
0, 11, 12 are selected data groups, 301,
302 and 303 are operations for setting the bit position of the bit array whose address is the hashing value of the data content of the data group to ▼1▼; 401, 402, 40;
3 is an operation to select only the data whose address is the hashed value of the data content of the data group and the bit position of the bit array is ▼1▼. 13 is an operation to input all the bit arrays and perform an AND operation, and based on the result of the AND operation. Represents an AND device that updates all bit arrays.

Claims (1)

[Claims] 1. In the case where there are a plurality of n data groups (n≧3), in a method for selecting data whose data contents may be mutually identical among all data groups, a hashing device for hashing data and a bit array are used. The number of sets is n equal to the number of data groups, one hashing device and one bit array are assigned to one data group, the n data groups are simultaneously hashed to set the bit array, and then The AND device performs an AND operation between all bit arrays, transfers the result to all bit arrays at the same time for updating, and then transfers the n data groups to the same hashing device used for the hashing. A data sorting method characterized by simultaneously inputting and hashing data, referring to each bit array, and selecting data that may match each other among all data groups.