KR100319761B1 - Frame-partitioned parallel processing method for database retrieval using signature file - Google Patents

Abstract

The present invention relates to a frame division parallel processing method in a database retrieval system using a signature file that can create an index structure of a bit string called a signature and process the same by using a hashing function to speed up the retrieval of multimedia data. A first step of dividing the signature using the decompression hashing and the frame partitioning scheme and transferring the signature to each processing node to build a storage structure in parallel; A second step of generating a query signature from the query, comparing the query signature with the storage structure, generating a peer key by passing the same amount of the peer key to each processing node when the query of the search is given to the constructed storage structure; Each processing node that has received the same key finds a frame indicated by the same key, determines whether it matches the query signature, and transmits the result of the determination to a neighboring processing node; And a fourth step in which the processing node receiving the determination result searches for and determines a matched frame using the match result, and transfers to the next processing node to terminate the search when the last frame is reached. .

Description

FRAME-PARTITIONED PARALLEL PROCESSING METHOD FOR DATABASE RETRIEVAL USING SIGNATURE FILE}

The present invention relates to a frame division parallel processing method in a database retrieval system using a signature file, and to a computer readable recording medium having recorded thereon a program for realizing the method. In particular, data in the form of a bit string A computer-readable recording method of frame division parallel processing in a database retrieval system using a signature file and a program for realizing the method to speed up a search for a query in a database storing It relates to a recording medium.

Such a parallel processing method performs faster retrieval by using a hashing technique, which is one of index methods for efficient data management, when storing massive multimedia data in a database. It is for. To this end, in the present invention, by applying a hashing technique to the multimedia data to obtain a shortened bit string per record, it is stored in an index structure. In this case, the abbreviated bit string is called a record signature. In addition, the query to be searched is made into the same query signature for searching. The multimedia data is in a dynamic environment that is added, deleted, and updated from time to time.

In addition, the environment of a computer system in which such parallel processing is performed is performed in a parallel processing computer having a shared nothing structure. The SN structure has its own memory and disk in each microprocessor, and is connected to each other in a high speed network. Here, a collection of processors, memory, and disks is called a processing node.

On the other hand, the conventional method and apparatus for distributing code for parallel processing is to develop a method and apparatus for distributing data stored in a code form in a parallel computer to a processor, and evenly distribute similar codes for query codes in a parallel processor. In order to do this, we split the code into processors that are folded. This can increase the efficiency of parallel processing.

Here, the homogeneous code means all bit string codes that match the query. For example, when the query code is '0101', the similarity codes are '0101', '0111', '1101', and '1111'.

The conventional 'hamming filter' method uses a Hamming code proposed in 1973 for data transmission, and creates a check matrix and multiplies the code with a resultant value to determine and process the processor number. to be. When the number of rows in the distributed matrix is m and the number of columns is n, the number P of processors is obtained by Equation 1 below.

P = 2 ^m = n + 1

For example, when the number of processors is 8, a dispersion matrix of (3X7) is obtained.

However, the Hamming filter has a disadvantage that good dispersion can be achieved only when the processor is determined by Equation 1.

In addition, conventional techniques classify according to key values of a hashing scheme and transmit the signatures to processing nodes in order to process the signatures in the form of bit strings in parallel. That is, the signatures are classified in the direction of distributing homogeneous keys generated from the hashing keys by the hashing function evenly among possible processing nodes. Here, the same key has the same meaning as the above same code. In the conventional code distribution method and apparatus for parallel processing, a folding method is used to evenly distribute similar keys, and the conventional Hamming filter method is a distributed matrix among records of a bit string like the above distribution method. The number of processors was determined by calculating the correct size.

Therefore, in the related art, even though the parallel key is distributed evenly, the efficiency of parallel processing is improved. However, since there is no method for evenly distributing the same key in all cases, it is difficult to achieve the necessary execution equality for parallel processing.

The present invention has been made to solve the above-described problems, the signature file using a hashing function to speed up the search for multimedia data to create an index structure of a bit string called a signature and to process them in parallel It is an object of the present invention to provide a frame division parallel processing method in a database retrieval system using a computer and a computer readable recording medium recording a program for realizing the method.

1 is an exemplary configuration diagram of a database search system to which the present invention is applied.

2 is an explanatory diagram showing a parallel storage structure of a frame division parallel processing method in a database retrieval system using a signature file according to the present invention;

3 is a flowchart illustrating a method for parallel processing of frame division in a database search system using a signature file according to the present invention.

* Explanation of symbols for the main parts of the drawings

11: record 12: database

13: record signature 14: parallel search

15: query signature

In order to achieve the above object, the present invention provides a frame division parallel processing method applied to a database retrieval system using a signature file, wherein the signature is partitioned using decompression hashing and frame division schemes, and transferred to each processing node and stored in parallel. A first step of building the structure; Given a query of a search in the constructed storage structure, a query signature is generated from the query, the query signature is compared with the storage structure, and a co-key is generated to deliver the same amount of co-key to each processing node. A second step of doing; A third step of each processing node that has received the same key finds a frame indicated by the same key to determine whether it matches the query signature, and transmits the result of the determination to a neighboring processing node; And a fourth step in which the processing node that has received the determination result searches for and determines a matched frame using the match result, and transfers to the next processing node to terminate the search when the last frame is reached. .

The present invention also provides a database retrieval system having a processor, comprising: a first function of dividing a signature by using decompression hashing and a frame dividing scheme and transferring the signature to each processing node to build a storage structure in parallel; Given a query of a search in the constructed storage structure, a query signature is generated from the query, the query signature is compared with the storage structure, and a co-key is generated to deliver the same amount of co-key to each processing node. A second function of doing; A third function for each processing node receiving the same key to find a frame indicated by the same key to determine whether it matches a query signature, and to transmit a result of the determination to a neighboring processing node; And a computer for recording a program for realizing a fourth function of searching for and determining a matched frame using the match result, and transmitting the result to the next processing node to end the search when the processing node receives the determination result. It provides a recording medium that can be read by.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary configuration diagram of a database search system to which the present invention is applied.

As shown in Fig. 1, the record 11 is stored according to the storage structure of the database 12, and insertion and deletion frequently occur. Searching for a query in such a multimedia database means that all records containing data existing in the query are retrieved. Therefore, in order to speed up the search, first, the record signature 13 is extracted from the record 11 and stored, the query signature 15 is extracted from the given query, and the query signature 15 is compared with the record signature 13. To find the matching record signature. It is checked whether the actual record corresponding to the record signature 13 thus obtained includes a query. Herein, the query signature 15 matches the record signature 13 means that the record signature 13 includes a bit string of the query signature 15.

In order to support a dynamic environment where record insertion and deletion occur frequently in a computer system, the parallel processing method of the present invention uses extensible hashing. However, in the related art, the parallel processing is performed based on the key of the hashing function, whereas in the present invention, the parallel processing is performed based on the frame of the signature for equality of execution.

The frame is a signature fragment of the same size obtained by dividing the signature. The number of frames of the signature is set to an integer multiple of the processing node. This way, you can achieve quantitative equality and equality of execution per processing node. The divided frames are delivered to each processing node and stored.

2 is an explanatory diagram illustrating a parallel storage structure in a frame division parallel processing method in a database retrieval system using a signature file according to the present invention.

As shown in FIG. 2, after the signature is divided and transferred to each processing node and stored in parallel using decompression hashing and frame division methods, a storage structure as shown in FIG. 2 is generated. This will be described in detail below.

First, the signature is partitioned using the decompression hashing and the frame partitioning method and passed to each processing node to form the storage structure in parallel.

Secondly, given a query of a search in a parallel stored structure, a query signature is created from the query, the query signature is compared with the parallel stored structure, and a concurrency key is generated for each processing node. Pass the same amount of concurrency keys.

Third, each processing node that has received the same key finds a frame indicated by the same key, determines whether it matches the query signature, and delivers the matched determination result to the neighboring processing node.

Fourth, the processing node that has received the determination result retrieves only the matched frame using the received match result to obtain the determination result again, and transfers to the next processing node to end the search when reaching the last frame.

One of the processing nodes present in the computer system acts as a main processing node, where records are inserted into the database, making them record signatures, and determining where to store them by hashing functions. The frame signature is divided into frames for storing the record signature, and the frame is transmitted to each processing node. The main processing node may be a node participating in parallel processing or may be a separate processing node. The main processing node owns hashing keys of decompression hashing and is responsible for performing insertion, deletion and retrieval of records.

As a result of dividing the signature and transferring it to each processing node to form a storage structure in parallel, a storage structure as shown in FIG. . The number of concurrent keys is divided by the number of processing nodes so that the same amount of similar keys is delivered to each processing node. In FIG. 2, since the hashing key is '1010', four concurrency keys '1010', '1011', '1110', and '1111' are generated, of which '1010' is in processing node P1 and '1011' is processing node P2. Is transferred to processing node P3 and '1111' to processing node P4 and the search begins. So far, this is done at the main processing node.

From the third process described above, it is performed at each processing node. Frame '0010 1100' of the query signature is compared with the frames of the record signature at processing node P1. For parallelism, P1 retrieves the top two blocks of its own block, P2 searches the second two top blocks, and Pn searches the last block simultaneously. The block containing the record signature frame is called a frame block.

In this case, the frame search is performed according to the search order of the query signature frame. Since P1 has a greater number of '1's than' 0010 'in P1, searching first may find the unmatched record signature frame faster. In FIG. 2, it can be seen that no record signature frame matches '1100', and the next query frame '0010' does not need to be searched.

In this way, the searched result is delivered to P2, and P1 receives the result performed at Pn and searches the lowest block. In P2, the same procedure can be seen that both signatures match. The query signature frame '0000' matches all frames, so frame search is not performed to reduce block access. By doing this, you can speed up the search. The matched result is passed to P3, which receives the result of P1 and searches for the highest block.

The above steps are completed when returning back to the first execution position at each processing node, and the matched result is passed to the main processing node to integrate the match result.

3 is a flowchart illustrating a method for parallel processing of frame division in a database search system using a signature file according to the present invention.

As shown in FIG. 3, a query signature is generated from the user query (301), and a similar key set (EK) is calculated using the query signature (302).

The same amount of concurrency keys is then passed to each processing node to separate into each processing node (303). At this time, this process is performed in the main processing node.

In this way, each processing node that has received the same amount of like keys initializes the search state of the frame group for each like key (304). Here, a frame group means a set of frame blocks indicated by one concatenation key, and a search state has a value of a processing node's number as a value to know in which processing node a search is performed for one frame group.

Next, the match state of the frames is initialized to '1' (305). Here, the match state indicates whether or not the signature matches the query signature for signatures in the frame group. Therefore, the match state requires as many bits as the number of record signatures.

In addition, the search state flag of the frame group is initialized to 'False' (306). The reason is to indicate that there is a record signature frame that matches the frame block. If no matching frame is found, the search status is changed to '-1' to perform a quick search because it is no longer necessary to search the frame group containing the frame block.

On the other hand, it is searched whether there is a similar key which has not been searched yet (307).

As a result, if the same key does not exist, the last matched signature is output (308), and if the same key exists, the number of signatures of the frame block is repeated (309), and then it is determined whether the query frame matches the record frame (310). ).

As a result of the search, if the same key does not exist, the last matched signature is output (308). If the same key exists, the number of signatures of the frame block (309) is repeatedly determined whether the query frame matches the record frame (310). Here, repeating in step 309 means repeating all parts corresponding to steps 310, 311, and 312.

As a result, if the query frame matches the record frame, the search state is changed to 'true' (311). If the query frame does not match the record frame, the search state of the frame is changed to '0' (312). Here, the reason for changing the search state to '0' is that since the signature is divided into several frames, it is assumed that once a match is made, and if any part does not match, the entire signature does not match.

Subsequently, it is checked whether the state flag of the searched frame block is 'True' (313).

As a result, if the status flag of the searched frame block is 'True', the search status and match status are transmitted to the next processing node (314). If the status flag of the searched frame block is not 'False', the search status is '-1'. '315 and delivers only the search status (315).

That is, from 313 to 315, the process of checking the search state for the frame block and passing the match result (match state and search state) to the next processing node. In other words, if the search status flag is 'True', there is a matching frame, so both the search status and the match status are passed to the next processing node. In case of False, however, since there is no matching signature, it is not necessary to search from the next block, so only the search status with '-1' is passed.

Subsequently, the process is repeatedly performed in step 307 of searching for whether there is a similar key that has not been searched yet, and after searching all the same key, a finally matching signature is output.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes can be made in the art without departing from the technical spirit of the present invention. It will be apparent to those of ordinary knowledge.

The present invention as described above has the effect of equalizing the execution which could not be achieved by the conventional method by dividing and storing the signature into frames for parallel processing.

In addition, according to the present invention, a matching key is generated by a matching characteristic of a signature. By uniformly distributing the matching key to parallel processing nodes, it is possible to minimize the bias on data and the execution on parallel processing. It is effective to perform quick search by effectively using nodes.

In addition, the present invention utilizes a method of dividing a signature into frames, which does not impair the structure of decompression hashing, so that it can be efficiently processed even in a dynamic environment in which data insertion and deletion frequently occur.

Claims (5)

In the frame division parallel processing method applied to a database retrieval system using a signature file, A first step of dividing the signature using the decompression hashing and the frame division scheme and passing the signature to each processing node to build a storage structure in parallel; Given a query of a search in the constructed storage structure, a query signature is generated from the query, the query signature is compared with the storage structure, and a co-key is generated to deliver the same amount of co-key to each processing node. A second step of doing; A third step of each processing node that has received the same key finds a frame indicated by the same key to determine whether it matches the query signature, and transmits the result of the determination to a neighboring processing node; And A fourth step in which the processing node receiving the determination result searches for and determines a matched frame by using the match result, and transfers to the next processing node to terminate the search when the last frame is reached. Frame division parallel processing method comprising a. The method of claim 1, In the third step, A fifth step of initializing a retrieval state of frames for each concurrent key at each processing node, and initializing a match state of frames and a retrieval state flag of a frame group; A sixth step of determining whether the query frame matches the record frame by repeating the number of signatures of the frame block according to the search result of the same key; And A seventh step of transmitting a search state and a match state or a search state to a neighboring processing node according to the determination result; Frame division parallel processing method using a signature file comprising a. The method of claim 2, The frame group, And a set of frame blocks indicated by one concatenation key. The method of claim 2 or 3, The match state is, And indicating whether or not the signature in the frame group matches the query signature. In a database search system with a processor, A first function of dividing a signature using decompression hashing and a frame partitioning scheme and passing it to each processing node to build a storage structure in parallel; Given a query of a search in the constructed storage structure, a query signature is generated from the query, the query signature is compared with the storage structure, and a co-key is generated to deliver the same amount of co-key to each processing node. A second function of doing; A third function for each processing node receiving the same key to find a frame indicated by the same key to determine whether it matches a query signature, and to transmit the result of the determination to a neighboring processing node; And A fourth function of searching for and determining a matched frame using the match result by the processing node receiving the determination result, transferring the result to the next processing node, and ending the search when the last frame is reached. A computer-readable recording medium having recorded thereon a program for realizing this.