JPS63271525A - Data processor - Google Patents

Abstract

PURPOSE:To decrease the number of times of checking on a main storage and to perform join processing at relatively low cost by performing matching operation by using plural hash functions and the applying plural hash functions. CONSTITUTION:One hash function (BMH) is applied to a keyword string to generate HBM 170 in synchronism with the data transfer of the keyword string from a secondary storage device, etc., and at the same time, the other hash function (HPH) is applied to this keyword string to generate HPM 180. Then the BMH is applied to a search key string in synchronism with the data transfer of the search key string to a result, which is compared with the HBM; and the HPH is applied to the search key string to obtain TPM, which is compared to perform search processing. Consequently, processing for checking whether or not the keyword string contains respective data of the search key string is realized by said processing. Consequently, a great decrease in channel load is made.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to a data processing device using a hash function that is suitable for dedicated hardware such as database processing or file processing, and in particular, it relates to a data processing device that uses a hash function and is suitable for search processing or join processing. The present invention relates to a data processing device.

[Conventional technology]

Search processing and join processing are basic operations in database processing or file processing. That is, search processing has a very high processing frequency and has a relatively high processing load, and join processing has a very high load, so speeding up these processes is important.

By the way, search processing by software.

If join processing is implemented, the processing efficiency will deteriorate in data comparison, data structure manipulation, etc., compared to the case where the processing is supported by dedicated hardware, and a general-purpose computer will inevitably have low efficiency.

Now, with the development of LSI technology and the ability to develop LSI chips at relatively low cost, various methods have been proposed to speed up search processing or join processing using hardware.

For example, regarding search processing and join processing, (1) Hash-based: According to Babb, E.

r I mples*enting a Re1a
tional Database by Means
of 5 pacializad HardvareJ
, (ACMTOD S g vol-41 & 1 @
March 1979 g p p -1-29), but regarding only search processing, (1) those based on dichotomy: Tanaka, Y.
According to etc. rPipeline Searching
and SortingModules as Co
a+ponents of a Data flowC
computerJ, (I F I P CC
ongress' 80゜pp, 427-43
2. Oct., 1980) (2) Based on associative memory: Fostar, C.C., r Cont.
entA addressableParalle
lProcessorsJ* (Van No5tra
nd Ra1nhold.

etc. have been proposed.

[Problem that the invention attempts to solve]

In database processing, for example, relational
In database processing, data to be processed exists on a secondary storage device, and each database operation requires reading and transferring a large amount of data.

Here, the database is assumed to consist of relations that can be seen by the user in the form of a two-dimensional table, and the tuples correspond to the rows of the table. Further, a tuple consists of one or more attributes (this is called a "column").

In such a database system, when a large amount of data is transferred, the data transfer time becomes a performance bottleneck of the database system.

Therefore, a method can be considered to effectively utilize the data transferred from the secondary storage device. This method completely overlaps the time required for database migration and the time required for database processing for the database, and is well known as a prior art.

Narrowing down that is processed on data transferred from the secondary storage device that stores the database.

Database operations such as extraction and matching are essential as long as the concept of storage hierarchy exists.

Below, the discussion will be divided into search processing and join processing.

Regarding the search process, when based on the Hasshu method, 0(
It is well known that 1) is possible. however,
When a hash function is set, there is a problem in that a conflict may occur even in search processing that uses a hashed bitmap, which has been well known in the past.

To deal with this, it is conceivable to perform matching using a hash function and then apply multiple hash functions in order to reduce the number of checks on the main memory.
There was no suggestion of such a checking method that would minimize conflicts.

A first object of the present invention is to synchronize data transfer from a secondary storage device by utilizing the characteristic that search processing can be realized in 0(1) when based on the hash method, and
The object of the present invention is to realize a search processing device at relatively low hardware cost.

Furthermore, regarding simi-in processing, that is, matching processing, conventionally the sort-merge method and the host loop method have been implemented.

If the number of tuples mentioned above is n, the former requires O(
nlogn), the latter requires 0(n) for the matching process, and the latter requires O(n'') for the matching process.

However, in order to execute the join process in synchronization with data transfer, a method of calculating with 0(n) is required. In this respect, the host loop method is basically unfeasible, and
Regarding the sort-merge method.

If O(logn) comparators are used for heat processing, it can be implemented in synchronization with data transfer.

However, the need for O(logn) comparators means that the hardware cost increases significantly as the amount of data increases.

Therefore, with these methods, it is difficult to synchronize the shimmy-in process with data transfer and to realize it at a relatively low hardware cost.

By the way, as mentioned above, it is well known that when based on the hash method, matching processing can be realized with 0(1). However, if you set the hash function,
For example, even in search processing that uses hashed bitmaps, which are well known in the past, there is a problem that conflicts may occur.
As mentioned above, it is possible to apply a plurality of hash functions in order to reduce the number of checks in the main memory.

A third object of the present invention is to synchronize data transfer from a secondary storage device and compare The object of the present invention is to realize a join processing device with a low hardware cost.

[Means for solving problems]

The first object of the present invention is to provide means for sequentially inputting a keyword string and a search key string to check whether or not a search key string is included in the keyword string; A means for outputting an evaluation value obtained by applying a prepared hash function, a means for storing a hashed bitmap that is a target of hashed bitmap calculation processing, and an evaluation value obtained by applying a prepared hashed function to the keyword string. In a data processing device, a hashed bitmap is created by preparing a plurality of hash functions, applying one of the hash functions prepared in advance to the keyword string, and simultaneously storing the logical product of the keyword string. means for creating a hash product memory by applying the other hash function prepared in advance to the keyword string; and a hashed bitmap for applying the one hash function prepared in advance to the keyword string; means for checking whether the contents of an address on the hashed bitmap, which is assigned an address by the evaluation value obtained by applying the hash function to the search key string, have already been set; Negating the addressed hash product memory using Harasha product memory applying the other hash function, and applying the other force hash function prepared in advance to each data in the search key string. This is achieved by a data processing device characterized in that it is provided with means for performing a logical OR with the result.

The second object of the present invention is to provide a means for storing a plurality of hashed bitmaps to be subjected to bitmap calculation processing, and a means for sequentially inputting join keys to which a plurality of hash functions prepared in advance are applied. , a data processing device comprising means for outputting an evaluation value obtained by applying a plurality of hash functions prepared in advance to the join key.

A means for determining a hash function that minimizes conflict as a result of applying a plurality of hash functions to create the hashed bitmap, and a hashed function determined by the means and a hashed bitmap based on the hashed function. , it is checked whether the contents of the address on the hashed bitmap that is numbered by the hashed function evaluation value has already been sected, and from the check result, it is determined that there is a possibility that it matches the join key. In this case, the present invention is achieved by a data processing device characterized in that it is provided with means for outputting a hash function evaluation value of a join key.

[Effect]

The search processing device according to the present invention includes a hashed bitmap (hereinafter also referred to as rHB MJ), a hashed product memory (hereinafter also referred to as rHPMJ), and a plurality of hash functions.

1. In the search processing device according to the present invention, 2.
1.Synchronizing with the data transfer of one data string from the next storage device, etc. One hash function is applied to the data string to create the HBM, and at the same time, the other hash function is applied to the data string to create the HPM.

Next, the results obtained by applying one hash function to the data string in synchronization with the data transfer of the other data string are compared with the aforementioned HBM, and the other hash function is applied to the data string in question. Search processing is performed by comparing the results obtained with the above-mentioned HPM.

Finally, a check is performed using one of the hash functions on the main memory. Specifically, using HBM that applies one hash function, it is checked whether each data of one data string ↓ and the other data string is included.

In addition, in case a conflict occurs, HPM that applies the other hash function is used to evaluate the other hash function for each data in one data string, and the hash function evaluation values of each data are logically multiplied. If the result of the negation and the result of applying the hash function to each data in the other data string is true, then one data string contains each data in the other data string. It turns out that there is a possibility that

Furthermore, in the join processing device according to the present invention, database processing overlaps with data transfer time, and a selection/constraint processor that exclusively performs filtering and join processing, a projection processor, and a join processor that exclusively performs the join processing are implemented. An input/output control device is placed between the main storage device and the secondary storage device.

The selection/control processor extracts a tuple from the transfer data, selects one, and determines whether the search condition is satisfied based on search condition information including column information to be subjected to constraint processing.

Furthermore, the projection processor extracts tuples that satisfy the search conditions based on the single room information and creates an output format. In both surface processing, if the number of tuples is n, then 0(n
) processing time is required.

Furthermore, since the join processor is expected to have a very high load, a hash-based algorithm is adopted for matching processing. According to the hash-based matching process, the processing time is 0(1).

By using this join processing device, it is possible to select a hash function that minimizes conflicts in synchronization with data transfer by overlapping filtering processing, and 0(
By using the result of the matching process using the hash table in 1), it is possible to efficiently obtain the join process result on the main storage device.

In other words, by narrowing down the tuples to be joined with high precision using the hash table, it is possible to reduce the CPU load through matching processing on the main memory, and to significantly reduce the channel load by filtering and narrowing down the tuples to be joined. realizable.

〔Example〕

Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a configuration diagram of a search processing device (search engine: 5E) 10 which is an embodiment of the present invention.

In this embodiment, the search process refers to checking whether one data string (keyword string) includes the other data string (search key string).

Specifically, first, data transfer of a keyword string from a secondary storage device etc. is synchronized, one hash function (hereinafter also referred to as rBMHJ) is applied to the keyword string to create the HBM, and at the same time The HPM is created by applying the other hash function (hereinafter also referred to as rHPHJ) to the keyword string. Next, in synchronization with the data transfer of the search key string, the results obtained by applying the BMH to the search key string are compared with the HBM, and the HPH is applied to the search key string. Search processing is performed by performing a comparison processing with the above-mentioned HPM obtained.

The above process realizes the process of checking whether each data of the search key string is included in the keyword string.

Next, an overview of the processing of the search processing device 10 shown in FIG. 1 will be explained.

The search processing bag [10] shown in FIG.

(1) Take a keyword string as input, apply the BMW for each keyword, set a bit on the HBM that is numbered according to the result obtained, and at the same time,
The result obtained by applying the HPH to each keyword is ANDed with the data on the HPM that is assigned an address according to the result obtained by applying the BMW, and the result is again used for the address. A set operation to store the data in the HPM.

and.

(2) When inputting a search key string and outputting the result of applying the HBH to a search key whose keyword string may contain data in the search key string, apply the BMW to each search key. Check whether a bit is set on the HBM addressed according to the result obtained by applying the HPH to each data in the keyword string using the HPM to apply the HPH in case a conflict occurs. A search operation that applies the HPH and checks whether each data of the search key string is included in the keyword string.

This is to realize the following.

By using the search processing device 10, efficient search processing can be realized, and specifically, the number of times of check processing using a hash function on the main storage device can be reduced.

FIGS. 2(a) to 2(d) are flowcharts illustrating the internal operation of the search processing device 10 that implements the above search operation and set operation. The second (a) shows the search processing 200 of the search processing device 10, and the second (b) shows the initialization 210 of the search processing device 10.

(Q) indicates a set operation, and (d) indicates a search operation.

First, the notation used to explain the flowchart will be described. The keys input to the search processing device 10 are KBO, K.
B2, HGO, HG are the hash functions applied to each key.
I, HBM for hashed bitmap, HPM for hashed product storage, V for logical sum, △ for logical product, 7 for negation.
, assign ←.

Let equal comparison be = and truth be T.

In the following description, search processing 200. Initialization 210, set operation 220, and search operation 230 will be described in this order using the above method.

The search process 200 performs eof (end offil) indicating whether the set operation or search operation ends.
e) Check whether the flag is specified using the action specification 191 (Fig. 2(a) Process 201)
, if not specified, keys KBO and KBI
Then, the key is set using the key operators 193 and 194 (202).

Next, in process 203, a hash function HGO is determined in advance for the input key.

Applying HGI 6 Next, it is checked whether the set operation 220 or the search operation 230 is specified in the operation specification 191 (steps 204 and 205).

If a set operation is specified, the process goes to 220, if a search operation is specified, the process goes to 230; otherwise, the process ends.

Regarding the set operation 220 and search operation 230,
This will be explained later. Initialization 210 is performed before a series of set and search operations are started. Specifically, the reset of the registers in the search processing bag [10 (FIG. 2(b) processing 211) and the above-mentioned HBM and HP M
(212).

Next, the setting operation 220 will be explained based on FIG. 2(c). First, a hash function HGO is applied to the key KBO, and as a result, if a bit is not set on the HBM addressed by the hash function evaluation value, a bit is set on the HBM addressed by the relevant hash function evaluation value (processing 221
→222).

Otherwise, the process proceeds to step 223, where the logic between the result of applying the hash function HGI to the key KBI, the application of the hash function HGO to the key KBO, and the data on the HPM that is addressed by the hash function evaluation value. Take the product,
The result is logically ANDed with the data on the HPM assigned that address again, and the result is
The set operation 220 is then stored in PM and ends.

Next, the search operation 230 will be explained based on FIG. 2(d). First, the hash function HGO is applied to the key KBO, and if as a result, no bit is set on the HBM addressed by the hash function evaluation value (process 231), the search operation is ended. Otherwise,
The result of applying the hash function HGI to the key KBI and the hash function HGO to the key KBO are logically ORed with the negation of the data on the HPM addressed by the hash function evaluation value, and the result is If it is not true, the search operation 230 is ended (232).

Otherwise, after setting the hash function evaluation value obtained by applying the hash function HGO to the key KBO in the output address register OAR100.

Output using address output 192 and search operation 230
(processing 233).

Next, the configuration of the search processing device 10 will be described in detail with reference to FIG. The search processing device 10 specifies the operation specification 191. Address output 192, key power 193,1
94. Operation specification 191 is a search processing bag! 110 is for specifying whether to perform a set operation or a search operation. Address output 192, key human power 193°1 in search operation
The hash function is used to transfer the evaluation value of the hash function that has been determined to have a possibility of matching the input key as a result of applying the hash function to the key input via 94.

Key operators 193 and 194 are used to transfer keys to search processing device 10. Output address register 1
00 is the search processing device I! Used to transfer 10 keys. The output address register 100 is for latching the evaluation value of the hash function that is determined to have a possibility of matching the input key as a result of applying the hash function to the key in the search operation.

Key registers (KR) = 110 and 111 are for latching keys that are input in the set operation and search operation, respectively.

Hash generators (HG) 120 and 121 are for inputting the key registers 110 and 111, respectively, and outputting a result of applying an appropriate hash function. A memory address register (MAR) 130 is for addressing the HBM 170 and the 1-IPM 1 80.

A control circuit (CL) 140 receives the above operation information and the H
It has a function of inputting BM bit information and outputting setting information to the HBM.

Memory data register (MDR) 160. 161 is
It is used to read and write the HBM 170 and HPM 180, respectively. HBM170 and HP M1
80 is used to set the evaluation result of the hash function.

The arithmetic unit (ALU) 150 is for performing comparison operations. The decoder (DEC) 190 has an operation specification 191
It is used to interpret the control information specified by and set the function codes of the control circuit 140 and the arithmetic unit 150.

Also, the calculation result flag of the calculation unit 150 is as follows.

Used to control input to output address register 100.

Next, using specific examples, Figure 1 and Figures 2 (a) to (d)
A comprehensive explanation will be provided based on the following.

As an example.

(1) The search processing device 10 uses the keyword string (43, 17
, 8, 24, 55) by a set operation and the search key string (17, 77) is processed by a set operation.
(12), 17(4,10).

24 (14, 16), 54 (3a)> is input with the set operation, search key string (17 (4)
), 24(10)) are processed by a search operation.

However, in the keyword string and search key string, for example, the notation r17(4,10)J.

It is shown that key 4 and key 10 have a parent-child relationship with respect to key 17. Further, the HBM has an 8-bit configuration, and the HPM has a 4-bit data length configuration,
It is assumed that the key length is an 8-bit data length configuration.

The following hash function is used.

HGO=(nty r12.n3 ” n6y n
s+ nt)HG 1” (not n2e n4y
nQ* ” nts n3* nls+ nt) However, HGO, HGI: 1st bit of the 8-tush function
) has been executed.

Description of operation (Step O: In the initial state, registers are reset. HBM 170 and HPM 180 are also cleared.

The control signal specifying the set operation is the operation specification 191
is instructed to the decoder 190 via.

The control signal is interpreted by the decoder 190 and set in the control circuit 140 and the arithmetic unit 150 in a form converted into one function code.

Step 1: The keyword "43" is set in the key registers 110 and 111 via the key inputs 193 and 194, respectively (
Fig. 2(a) Processing 201°202), hash function HG
Applying O, HGI, HGO(43)=7. HG1(4
3) = 8 (203), and the above "7" is set in the memory address register 130.

Since this is a set operation, process 204. →Go to 220,
The HBM 170 is read, and the control circuit 140
As a result of the bit comparison, set the bit in HBM170 (
FIG. 2 (Q) Processing 221→222).

HBM=(0,0,0,0,0,0,0,1) Also, H
PM(7)=8 (223).

Step 2: The keyword (17) is set in the key registers 110 and 111 via the key inputs 193 and 194, respectively (
Fig. 2(a) Processing 201°202), hash function HG
Applying O, HGI, HGO(17)=6. HGI (1
7)=6 (203), and the above "6" is set in the memory address register 130.

Since this is a set operation, the process proceeds to steps 204→220, the HBM 170 is read, and the control circuit 140 sets a bit in the HBM 170 as a result of the bit comparison (FIG. 2(c) steps 221→222).

HBM=(0', O, O, O, O, 0, 1, 1) Also,
HPM(6)=6 (223).

Step 3: Keyword (8) through key human power 193,194,
The hash function HGO is set in the key registers 110 and 111 (processes 201 and 202 in FIG. 2(a)), respectively.
, HGI is applied, HGO(8)=O, HGO(8)=
2 (203), the memory address register 130
The above [0] is set.

Since this is a set operation, the process advances to steps 204→220. The HBM 170 is read and the control circuit 140 sets a bit in the HBM 170 as a result of bit comparison (FIG. 2(c) process 221→222).

HBM=(1,0,0,0,0,0,1,1) Also, H
PM(0)=2 (223).

Step 4: The keyword (24) is set in the key registers 110 and 111 via the key inputs 193 and 194, respectively (
Fig. 2(a) Process 201→202), hash function HG
Applying O, HGI, HGO(24)=2. HGI(2
4) = 6 (203) Memory address register 1
30 is set to the above "2".

Since it is a set operation, the process proceeds to steps 204→220, the HBM 170 is read, and the control circuit 14-O performs the following steps:
As a result of the bit comparison, set the bit in HBM170 (
FIG. 2 (Q) Processing 221→222).

HBM=(1,0,1,0,0,0,1,1) Also, H
PM(2)=6 (223).

Step 5: The keyword (55) is set in the key registers 110 and 111 via the key inputs 193 and 194, respectively (
FIG. 2(a) Processing 201→202). hash function HG
Applying O,HGI, HGO(55)=O,HGI(5
5) = 15 (203), and the above-mentioned "0" is set in the memory address register 130.

Since this is a set operation, the process proceeds to steps 204→220, the HBM 170 is read, and the control circuit 140 sets a bit in the HBM 170 as a result of the bit comparison (FIG. 2(c) steps 221→222).

HBM=(1,0,1,0,0,0,1,1) Also, H
PM(0)=2 (223).

Step 6: Below, the search operation begins.

The search key RL 17J requires 193°194
are set in the key registers 110 and 111 respectively (FIG. 2(a) process 201→202). Applying hash functions HGO and HGI, HGO(17)=6. H
GI(17)=6 (203).

Since this is a search operation, the process proceeds to steps 204→205→230. After reading out the HBM 170, the control circuit 140 compares the bits and finds that the bit has already been set in the HBM 170.

In FIG. 2(d), the process proceeds from 231 to 232.

Further, the HPM 180 is read out, and the arithmetic unit 150 is used to perform negation and logical sum operations. This result is true, so
"6" is set in the output address register 100 and outputted via the address output 192 (FIG. 2(d) process 232→233).

Step 7: Search key "77" is set in key registers 110 and 111 via key inputs 193 and 194, respectively (
FIG. 2(a) Processing 201° 202). hash function HG
Applying O,HGI, HGO(77)=O,HGI(7
7) = 10 (203).

Since this is a search operation, the process proceeds to steps 204→205→230. The HBM 170 is read and the control circuit 140 reads the result of bit comparison.

Since the bit has already been set in the HBM 170, the process proceeds from steps 231 to 232 in FIG. 2(d). ``Also, the HPM 180 is read out, and the arithmetic unit 150 is used to perform negation and logical sum operations. This result is not true, so
End (232).

According to the above explanation, the keyword string (43, 17,
8, 24, 55) is included in search key [17], but search key r77
” is not included.

Explanation of operation (n) In this explanation, the keyword string (43 (12), 17 (4, 1G), 8 (20° 24 (
14, 16), 54 (33)) are subjected to the same processing process as the set operation in the above operation description (1), and the processing results are stored in the HBM 170 and HPM 180.

However, in the set operation, for each keyword in the keyword string, data corresponding to the parent in the parent-child relationship uses the key human power 193, and data corresponding to the child uses the key human power 19.
4 is used. This makes it possible to extend search processing to data that has a parent-child relationship.

The HBM 170 and HPM 180 are configured as shown below.

HBM=(1,1,1,0,0,0,1,1)HPM=
(4,0,6,0,0,0,13,5) Step 0: Search key r17 (4) J is entered into the key register 110 via the key input 193, and "17" is entered into the key register via the key input 194. 111r4J are respectively set (FIG. 2(a) process 201→202). hash function HGO
, HGI is applied, HGO(17)=4. HGI(4)
= 4 (203).

Since this is a search operation, the process proceeds to steps 204→205→230. The HBM 170 is read and the bits are compared in the control circuit 140. Since the bit has already been set in the HBM 170, the process 231→23 is performed in FIG. 2(d).
Proceed with 2.

Further, the HPM 180 is read out, and the arithmetic unit 150 is used to perform negation and logical sum operations. Since this result is true, "6" is set in the output address register 100 and outputted via the address output 192 (FIG. 2(d) process 232→233).

Step 1: The search key r24(10)J is set to "24" in the key register 110 via the key input 193, and the key register 1llrl is set via the key input 194 (
FIG. 2(a) Processing 201→202). hash function HG
Applying O, HG1, HGO(24)=2. HGI (1
0)=4 (203).

Since this is a search operation, the process proceeds to steps 204→205→230, and the HB M 1' 70 is read.

In the control circuit 140, as a result of the bit comparison, HBM1
Since the bit has already been set in 70, the process proceeds from steps 231 to 232 in FIG. 2(d).

Further, the HPM 180 is read out, and the arithmetic unit 150 is used to perform negation and logical sum operations. Since this conclusion is not true, the process ends (232).

According to the above explanation, the keyword string (43(12)
, 17(4,10), 8(20).

(24 (14, 16), 54 (33)) search key r
17(4)J is included, but search key r24(10)
It can be seen that J is not included.

As described above, according to this embodiment, the data to be searched may be a single value or may have a parent-child relationship. Also, if you prepare multiple HPMs above,
Even for data that has multiple children in a parent-child relationship,
Search processing is possible.

According to this embodiment, a hashed bitmap.

Using hash product storage and the corresponding hash function, it is now possible to check whether a keyword string contains a search key string, and data that has a parent-child relationship with the above data can also be searched. Is possible.

FIG. 3 shows another embodiment of the invention. 4 is a configuration diagram of a join processing device f (join processor: JP) 45. FIG.

Furthermore, FIG. 4 shows an overview of an example of a database system to which this embodiment is applied.

The database system includes an input/output control unit 20 that inputs queries input from users and the like and outputs results of processing the queries, and a make-up process.

0MS2 manages the entire database including resource management
1, and a database 22 that stores data to be processed by the database.

In addition to managing and controlling the entire system, the DBMS 21 includes a system control unit 23 that manages input and output, a timing analysis unit 24A that performs syntactic and semantic analysis of queries, and an optimal system that generates appropriate internal processing procedures. a logic processing unit 24 consisting of a code generation logic processing execution unit 24G that generates an internal code and interprets and executes the internal code;
It is composed of a physical processing unit 25 that executes physical processing of the database such as management of a database buffer 26 that stores data to be processed by the MS 21.

FIG. 5 shows an example of a hardware configuration to which this embodiment is applied. Specifically, a central processing unit (CPU) 30 that holds a main memory (MM) 34, a channel unit (CHU) 31 that performs input/output control, and selection and control in database processing.

An intelligent file controller 11 (IFC) 32 and a disk (DK) 3 that hold a local memory (LM) 35 for setting data to be subjected to projection and join operations.
Consider a system consisting of 3.

The intelligent file control section ff! 32 and the disk 33, relative block addresses, input data, etc. are transferred, and a path 38 is used. Furthermore, filtering processing results, search condition information, etc. are transferred between the channel device 31 and the intelligent file control device 32, and a path 37 is used.

Furthermore, the path 36 is connected to the central processing unit i! ! 30 and channel device 31 for data transfer and control information exchange. Both routes 36 and 37 are, respectively,
For example, it does not matter if a plurality of channel devices 31 and intelligent file control devices 32 are connected.
A configuration in which the channel device i31 and the intelligent file control device 32 are integrated may also be considered as a modification.

In the following description, the central processing unit 30. A system will be discussed in which each of the channel device 31, intelligent file controller 32, and disk 33 is configured as one unit.

This configuration differs from the method in which all database processing is executed by the DBM 21 implemented on the central processing unit 3o. That is, among the database processing, the intelligent file control device 32 performs relational algebraic operations corresponding to the internal processing code generated by the code generation logic processing execution unit 24C of the DBMS 21, that is, various operations such as 1 selection/constraint, projection, and combination. This is to share functions.

This configuration corresponds to a so-called crowd counting device with various calculation functions (for example, a filtering processing device) added thereto. This basic operation is shown below.

(1) The DBMS 21 analyzes the query input by the user and determines the arithmetic processing procedure.

(2) In accordance with the above procedure, a relational algebra operation is extracted and a processing request is issued to the intelligent file control device 32.

(3) Intelligent file control device 32.

It interprets the processing request and issues a physical input/output request to the disk 33. The database 22 is a base relay controller, and pages, which are management units of the database, are read in synchronization with the rotation of the disk 33 and set in the local storage 35.

(4) Perform operations such as selection/constraint, projection, and combination on the data in the local storage 35, and perform filtering processing.

(5) Set this processing result in the database buffer 36 of the DBMS 21.

(6) DBMS'21 processes the processing results.

Output the adjustment result to the user.

The feature of this embodiment is that while the above processes (3), (4), and (5) are executed in an overlapping manner, a hash function with the minimum conflict is determined, and at the same time a hash function based on the hash function is Create.

Also, join processing is performed based on the hash table.

However, this embodiment is a method of selecting a hash function with the minimum conflict from among a plurality of hash functions prepared in advance, and does not generate a hash function that minimizes the conflict.

FIG. 6 is a diagram showing a block configuration of the intelligent file control device 32. As shown in FIG. Intelligent file controller 32 is coupled to channel device 31 via path 37 and to disk 33 via path 38. The intelligent file control device 32 includes a projection processor (PRP) 40 and a channel processor (C:HP) that shares the interface with the channel device 31.
41, and a local memory (LM) that stores data to be processed by the intelligent file control device 32.
42, a file control processor (FCP) 43 which shares the interface with the disk 33, a selection/constraint processor (R8P) 44 which shares the selection/constraint calculation, etc.
It is composed of a combination processor (JP) 45 that performs combination operations, and the HBM 46.

Path 47A connects the shooting processor 40. Channel processor 41. Local memory 42. File control processor 439 connects selection/constraint processor 44 and combination processor 45 and is a transfer path for data to be filtered.

Path 47B connects the projection processor 40. It connects the channel processor 41 and the file control processor 43 and is a transfer path for data that is not subject to filtering processing.

Details of the combination processor 45 that performs combination operations will be described below.

The combination processor 45 takes the join key extracted from the selection/control processor 44, that is, the column value input for the combination operation, and performs a filtering operation (hereinafter referred to as "filtering mode") that returns an address on the HBM 48.
A check operation (hereinafter referred to as "check function") takes the join key as input, applies multiple hash functions set for each data type, and selects the hash function that minimizes the conflict as a result of creating an HBM. mode) and a combination of the above two operations.

Specifically, in filtering mode, the selected
As a result of applying a preset hash function to the join key extracted from the constraint processor 44, it is checked whether the contents of the addressed address on the HBM 46 have already been set, and the relevant If it is determined that there is a possibility of matching the join key, the hash evaluation value of the join key is output. This results in
If the join processor 45 is used, matching processing and matching processing on the main storage device can be performed using a hash function that minimizes the conflict regarding the join key that is the target of the joining operation.

FIGS. 7(a) to 7(d) are flowcharts illustrating the internal operation of the combination processor 45 that combines the above-mentioned filtering operation and checking operation.

7(a) shows the overall operation 500 of the combination processor 45, FIG. 7(b) shows the initialization 510, FIG. 7(c) shows the filtering operation 520, and FIG. 7(d) shows the checking operation 530. ing.

First, the notation used to explain the flowchart will be described. The join key input to the join processor 45 is
KRO, JKRI, conflict counter CCNT+
(i=O~3), radix counter CARDJ (j=
O~3), the hash function is hk (k=o~3), and
The hashed bitmap is HB M +n (m ”
O~3).

In the following description, it is assumed that the filtering operation and the checking operation operate simultaneously. That is, in step 506 of the processing flow of the overall operation 500, the hash function (temporarily assumed) that minimizes the conflict has been determined, and the hash function is. The HBM obtained as a result of applying has already been set to 1-IBMO.

The overall operation 500 shown in FIG. 7(a) shows whether the filtering operation and checking operation are completed.
Check whether the f (end of file) flag has been specified from the selection/constraint processor 44 using the path 48 (process 501). If the eof flag is not specified, join key register JKR
Set a join key to O and JKRI (501
→502). The join key register JKRO is set with the join key to be processed by the filtering operation, and the join key register JKRI is set with the join key to be processed by the checking operation.
A hash function to be applied to the input join key is determined (503). Hash function applied to join key JKRO, which is the target of filtering operation. has already been decided. Further, the hash function hk (k=1 to 3) applied to the join key JKRI, which is the target of the check operation, is determined by the selection/constraint processor 44.

The filtering operation 504 and the checking operation 505 in this embodiment are processes that are processed in parallel. A filtering operation 504 applies the hash function ho to the join keys JKRO for possible matches.

The evaluation value of the hash function is output.

Further, in the check operation 505, the eight-tush function hk
(k=1 to 3) is applied to the join key JKRI, a comparison process is performed with the corresponding value in the hashed bitmap HBMm (m=1 to 3), and the conflict counter or radix counter is updated. Also, processing 5
In 01.

If the eof is detected, a hash function that minimizes the conflict is determined using the conflict counter and radix counter of the HBM that was the object of processing in the check operation (process 506).

The hash function selected here is applied in a filtering operation to another join key column.

Initialization 510 shown in FIG. 7(b) is performed before a series of filtering operations and checking operations are started. Specifically, it consists of resetting counters such as a conflict counter and a radix counter, and clearing the hashed bitmap (processing 511→512).

Next, based on FIG. 7(C), filtering operation 5
04 will be explained.

Join key J evaluated in filtering operation 504
The above-mentioned hash function is provided to KRO. is applied to generate an evaluation value for the process (process 521). Hashified bitmap HB Mo that is addressed by the hash function evaluation value
If the bit above is set, the hash function evaluation value is output to the selection/constraint processor 44 (522 →
523).

Otherwise, the filtering operation ends.

Finally, based on FIG. 7(d), check operation 505
Explain.

A hash function hk (k=1 to 3) is applied to the join key JKR1 evaluated in the check operation.

The hash function evaluation value is generated (process 531).

Next, it is checked whether or not a bit is set on the hashed bit map HBMm (m=1 to 3) addressed by the hash function evaluation value (step 532). If the bit is set at the address, the conflict counter CCNT+ (i = 1 to 3) of the corresponding hash function
1 increment (533).

If the bit is not set at the address, the radix counter of the corresponding hash function CA RD J (j=1~
3) is incremented and 534) is added to the bit position of the address.
Set g I II (same 535).

This completes the check operation.

The above filtering operation and checking operation can be performed in parallel, and in reality, the join processor 45
Now, they work in parallel.

Next, the configuration of the combination processor 45 will be explained in detail.

The combination processor 45 includes the selection/constraint processor 44
and the local storage 42 and the path 47 via the path 48
A and the hashed bitmap 46 and a path 49, respectively.

The path 48 is used to transfer the join key extracted from the selection/constraint processor 44 and the evaluation value of the hash function that is determined to have a possibility of matching as a result of applying the hash function to the join key. . route 4
7A is used to transfer the join key string.

The output address register (OAR) 100 is for latching the evaluation value of the hash function that is determined to have a possibility of matching as a result of applying the hash function to the join key in the filtering operation. JKRO, JKRI) 112 and 113 are for latching join keys to be processed in the filtering operation and checking operation, respectively.

Hash Generation Box (HGB) 122. It is for inputting the join key registers 112 and 113 and outputting an appropriate hash function and an appropriate result.
is used to address the hashed bitmap.

The control circuits (CL, , i=o to 3) 141 to 144 process the above operation information and the hashed bit map (HB
M,, i=o~3) 181~184 bit information is input, setting information for the hashed bitmap, conflict counter (CCNT!, i=o~3) 161~1
64 and radix counter (CARDl, i=o~3) 1
It has a function of outputting update request information for 65 to 168.

Here, the conflict counters 161 to 164 are
It is used to count the number of conflicts of the corresponding hash function during the checking operation.

The radix counters 165 to 168 are for counting the radix of the corresponding hash function during the checking operation. Memory data register (MDR) 171~
174 is used to read and write the HBM.

HBMs 181 to 184 are used to set hash function evaluation results. In addition, a comparison circuit (C
MP) 151 is the conflict counter 161-1.
64 and the radix counters 165 to 168, and outputs hash function information with the smallest conflict.

The decoder (DEC) 190 interprets the control information specified by the selection/constraint processor 44 and outputs the hash generating box (HGB) 122 and the control circuit 141.
This is for setting function codes etc. of 144 to 144.

The hash generating box 122 is a hash function generating section, and is comprised of a selection circuit and a hash function evaluation circuit for interpreting the function code specified by the decoder 190 and applying an appropriate hash function.

Note that the above selection circuit can be realized by a combination of conventionally well-known multiplexers.

Also, if the hash function evaluation circuit generates a join key remainder, it is a barrel shifter.

Any combination of subtractors and registers that produces a join key convolution! It can be realized by a combination of exclusive OR circuits and registers.

The control circuits 141 to 144 interpret the function code specified by the decoder 190, and input 1 bit information indicating that the filtering operation or check operation is specified, and 1 bit information of the HB M (1 bit I-). and outputs 1 bit each of the various counter setting request information, control information for setting the hash evaluation value to the output address counter 100, and 1 bit of the HBM bit setting information. The main control circuits 141 to 144 are
It can be realized with three two-man power AND gates and two inverters.

The comparison circuit (CMP) 151 is for performing comparison processing of conflict counters related to the hash function to be checked.

Next, a comprehensive explanation will be given using a specific example.

Examples include a case where the join processor 45 performs the check operation on the join key string (43, 17, 8, 24, 55), and a case where the join processor 45 performs the check operation on the join key string (43, 17, 8, 24, 55), The case where filtering operation is performed is shown.

However, the HBM has an 8-bit bridge structure and the join key length is 8 bits. Furthermore, the hash function uses remainder and exclusive OR.

Specifically, ho=(nt* n4t n6w ” n3+
new nt) hx=(nt+ n2y n
3* ” n6* n1ip nt) h2:N
Set it to mod 7. However, hk: hash function (k=0 to 2) N rainbow ink - nl: first pit of join key +: exclusive OR mod: remainder operation. Initialization 51 is a preliminary step to the explanation of step 1 below.
0 is already executed.

Description of check operation Step O: In the initial state, the counters and registers are reset, and I(8M181-184) are also cleared.

Control signals specifying check operations are directed from the selection and constraint processor 44 to the decoder 190 via path 48. The control signal is interpreted by the decoder 190 and sent to hash generation box 122 and control circuits 141-14.
4, converted into a function code.

Step 1: Join key [43] is set in join key register 112 from 1 selection/control processor 44 via path 48 (processing 500→501→502). Apply the hash function, h.

(43)=7, ht(43)=7, h2(
43) = 1 (processing 503 → 505), and the memory address register 1 is set to r7j at 31 and r7 to 132.
, 133 is set to "1".

88M181, 182, 183 are read and control circuit 14
As a result of bit comparison in 1 to 143, 1 is added to each of the radix counters 165 to 167 (processing 5
32 → 534).

The hashed bitmap is as follows (535
).

HBMo=(0-0,0,0,0,0,0,1)HBM
t=(0,0,0,0,0,0,0,1)HBM2=(
0, 1, 0, 0, 0, 0, 0, 0) Step 2; Join key "17" is set in the join key register 112 from the selection/constraint processor 44 via the path 48 (process 500→ 501→502). Apply the hash function, h.

(17)” 6, hx(17)=6, h2(
17) = 3 (processing 503 → 505), the memory address register 131 is set to "6", the memory address register 132 is set to "6",
Set "3" to 133.

, 88M181, 182, 183 are read and bit comparison is performed in the control circuits 141 to 143, and as a result, 1 is added to each of the radix counters 165 to 167 (
Processing 532→534).

The hashed bitmap is as follows (535
).

HBMo=(0,0,0,0,0,0,1,1)HB
M x = (0, O, O, O, O-0, 1, 1) HB
M2=(0,1,0,1,0,0,0,0 Step 3: Join key "8" is set in the join key register 112 from the selection/constraint processor 44 via the path 48 (processing 500→501→502), apply the hash function, ho(8)=2, bt(8)=O,
hsi(8) = 1 (processing 503 → 505), and "
0'' and 133[1''.

Next, the HBMs 181, 182, and 183 are read, and as a result of bit comparison in the control circuits 141 to 143, 1 is added to each of the radix counters 165 and 166 (from 532 to 534), and 1 is added to the conflict counter 163 (from 532 to 534). 532→533). The hashed bitmap is as follows (535).

1 (BMO=(0,0,1,0,0,0,1,1)HB
M1=(1,0,0,0',O,0,1,1)HBM2
= (0, 1, 0, 1, 0, 0, 0, 0) Step 4: Join key "24" is set in join key register 112 from selection/constraint processor 44 via path 48 (processing 500→501→502). Apply the hash function, h.

(24)=1. hz(24)=6. hz (24) = 2 (processing 503 → 505), the memory address register 131 rlJ 132 is set to ``6'', and the memory address register 133 is set to ``2''.
”.

Next, the HBMs 181, 182, and 183 are read out, and the control circuits 141 to 143 perform a pitch comparison, and as a result,
Add 1 to each of the radix counters 165 and 167 (
532 → 534), 1 in conflict counter 162
(532 → 533). The hashed bitmap is as follows (535).

HBMO=(0,1,1,0,0,0,1,1)I-I
BM1=(1,0,0,0,0,0,1,1)HBM2
= (0, 1, 1, 1, 0, 0, 0, 0) Step 5: Join key "55" is set in join key register 112 from selection/constraint processor 44 via path 48 (processing 500→501→502). Apply the hash function, ha(55) = O, ht(55) = O
-hsi(55) = 6 (processing 503 → 505)
, the memory address register 131rOJ 132 is set to "0", and the memory address register 133 is set to "6".

Next, the HBMs 181, 182, and 183 are read, and as a result of bit comparison in the control circuits 141 to 143, 1 is added to each of the radix counters 165 and 167 (from 532 to 534), and 1 is added to the conflict counter 162 (from 532 to 534). 532→533), and the hashed bitmap is as follows (535).

HBMO=(1,1,1,0,0,0,1,1)HBM
1=(1,0,0,0,0,0,1,1)HBM2=(
0, 1, 1, 1, 0, 0, 1, 0) Step: The selection/constraint processor 44 specifies eof to the decoder 190 via the path 48, and the hash function that minimizes the conflict is determined. determined (processing 5
00→501→506). Up to the above steps, the conflict counters 161, 162, and 163 are as follows.

CCNTO=O CCNT1=1 CCN T2 = 2 The comparison result in the comparison circuit 151 is a hash function.

is selected and the hash function information is forwarded to the selection and constraint processor 44 via path 48. Also, H.B.
MI81 is not cleared and is saved for the next filtering operation. The other 88M182 and 183 are cleared.

Next, an example will be shown in which the filtering operation is executed while the HBMI 81 is held.

Description of Filtering Operations Step 0: Control signals specifying filtering operations are directed from selection and control processor 44 to decoder 190 via path 48. The control signal is interpreted by a decoder 190 and sent to the hash generator box 122 and the control circuit 14.
1, converted into a function code.

Step 1: The join key "24" is connected to the selection/control processor 4
4 via path 48 to join key register 113
(Fig. 7(a) Processing 500 → 501 → 50
2), applying the hash function ho, ho(24)=1 (step 503→504→process 521 in FIG. 7(c)),
The memory address register (MAR+) 131
Set 1 to .

The HBM 181 is read out, and the result of bit comparison in the control circuit 141 (seventh (C) process 522) is
Since the bit has already been set at address 1 of 181, it is considered that there is a possibility that it matches the join key "24". The evaluation value "1" of the hash function hO mentioned above is 1.
Set in the output address register 100 by the control signal output from the control circuit 141 (523)
, the address information is transferred via path 48 to the selection and control processor 44 .

Step 2: From the selection and constraint processor 44 via path 48,
A designation indicating eof is made to the decoder 190 (processing 501→506 in FIG. 7(a)).

The processing procedure has been explained above according to the flowchart.

As mentioned above, the above checking operation and filtering operation can be performed in parallel.

According to this embodiment, by using the address information obtained as a result of applying the hash function that minimizes the conflict, it is possible to efficiently implement join processing on the main storage device, thereby achieving speeding up of join operations, etc. can.

The technique shown in this embodiment can also be applied to matching processing, which is a basic operation in join processing using the hash technique.

〔Effect of the invention〕

As described above, according to the present invention, the following effects can be obtained.

(1) By using HBM, HP M, and multiple hash functions, we can realize a search processing device that can perform search processing based on the hashish method at low cost. (2) Filtering and join processing also based on the hashish method To realize a low-cost join processing device that can accomplish matching processing in 0(1) by using a processor dedicated to the processing.

[Brief explanation of drawings]

FIG. 1 is a configuration diagram of a search processing device which is an embodiment of the present invention, and FIGS. 2(a) to (d) are its operation flowcharts.
FIG. 3 is a block diagram of a join processing device according to another embodiment of the present invention, and FIG. 4 is a diagram showing a database system to which this embodiment is applied. FIG. 5 is a diagram showing the hardware configuration thereof, FIG. 6 is a diagram showing the detailed configuration of the intelligent file control device which is the main part thereof, and FIGS. 7(a) to (d) are operational flowcharts. 10: search processing device, 120, 121: hash generator, 170: hashed bitmap, 180: hashed product storage, 200: search processing, 210: initialization, 220: set operation. 230: Search operation, 45: Join processing device. 30: Central processing unit, 31: Channel control device, 32:
Intelligent file controller, 44: Selection/constraint processor, 46: Hashing bitmap. Introduction Figure 7 Rattan 2nd eye (9) 212Il (L) uL) ψth evening eye Figure 2 1 To K33 7th eye (E) Figure 7

Claims (1)

[Claims] 1. Sequentially inputting the keyword string and search key string to check whether one data string (keyword string) includes the other data string (search key string). means for outputting an evaluation value obtained by applying a hash function prepared in advance to the search key string; means for storing a hashed bitmap to be subjected to hashed bitmap calculation processing; A data processing device having a means for storing a logical product of evaluation values obtained by applying hash functions prepared in advance, the data processing device being characterized in that a plurality of hash functions are prepared and each of the following means is provided. Place. (1) Apply one hash function prepared in advance to the keyword string to create a hashed bitmap, and at the same time apply the other hash function prepared in advance to the keyword string to store hash products. Means for Creating (2) Using a hashed bitmap in which one of the hash functions prepared in advance is applied to the keyword string, the hash is assigned an address by the evaluation value obtained by applying the hash function to the search key string. (3) means for checking whether the contents of the address on the bitmap have already been set; means 2 for performing a logical OR between the negation of the hash product storage and the result of applying the other hash function prepared in advance to each data in the search key string, each data in the keyword string and the search key string; has a parent-child relationship, and the means for sequentially inputting the keyword string and the search key string includes a means for inputting a parent key and a means for inputting a child key, and the means for inputting the keyword string and the search key string in sequence includes a means for inputting a parent key and a means for inputting a child key. 2. The data processing apparatus according to claim 1, wherein the means for outputting the evaluation value obtained by applying the hash function is a means for outputting the evaluation value of the parent key. 3. means for storing a plurality of hashed bitmaps to be subjected to bitmap calculation processing; means for sequentially inputting join keys to which a plurality of pre-prepared hash functions are applied; 1. A data processing device comprising means for outputting an evaluation value obtained by applying a plurality of hash functions. (1) Means for determining a hash function that minimizes conflict as a result of applying a plurality of hash functions to create the hashed bitmap. (2) The hash function determined above and hashed bits based on the hash function. Using a map, it is checked whether the contents of the address on the hashed bitmap that is numbered by the hash function evaluation value has already been set, and based on the check result, there is a possibility that it matches the join key. If it is determined that the hash function evaluation value of the join key is output.