JP5244559B2 - Distributed index join method and system - Google Patents

Description

  The present invention relates to a distributed index joining method and system, and more particularly to a distributed index joining system, method and program for joining indexes of relational databases in a distributed system on a network. More specifically, the present invention relates to a distributed index combination method and system in a distributed system in which indexes of a relational database are connected to each other via a network that is slower than a bus transfer speed inside a computer.

  Conventionally, as an algorithm for performing a join operation between relational databases, a nested loop join in which records of two databases are sequentially matched and a result record is generated for those satisfying the join condition or two databases are used. Are sorted by join conditions, merge join that sequentially matches from the beginning and merge, or hash that uses a hash or index to find records that satisfy the join condition by expanding records from one database into a hash table Bonding and the like are known. An example of this is shown in FIG.

  Step 10) Initialization is performed so that the pointers pointing to the records of indexes A and B start.

  Step 11) The key value of the record pointed to by the pointer of index A is transmitted to machine B.

  Step 12) The index B is searched with the key value of the record pointed to by the pointer of the index A.

  Step 13) It is determined whether or not a record satisfying the search condition exists in the index B. If there is a record, the process proceeds to Step 14, and if not, the process proceeds to Step 15.

  Step 14) The records of indexes A and B are sent to the machine J, and both records are combined and output by the machine J.

  Step 15) The index B is searched for the minimum value that exceeds the key value of the record pointed to by the pointer of the index A.

  Step 16) It is determined whether or not a record satisfying the search condition exists in the index B. If there is a record, the process proceeds to Step 17, and if not, the process ends.

  Step 17) The index B pointer is advanced to the corresponding record.

  Step 18) The key value of the record pointed to by the pointer of index B is sent to machine A.

  Step 19) The index A is searched with the key value of the record pointed to by the pointer of the index B.

  Step 20) It is determined whether or not a record satisfying the search condition exists in the index A. If there is a record, the process proceeds to Step 21, and if not, the process proceeds to Step 22.

  Step 21) The records of indexes A and B are sent to machine J, and both records are combined and output by machine J.

  Step 22) The index A is searched for the minimum value that exceeds the key value of the record pointed to by the pointer of the index B.

  Step 23) It is determined whether or not there is a record satisfying the search condition in the index A. If there is a record, the process proceeds to Step 24, and if not, the process ends.

  Step 24) The pointer of index A is advanced to the corresponding record, and the process proceeds to step 11.

It also separates join operations into separate jobs and optimally schedules them across multiple processors, thereby allowing two database databases on a common field in a parallel relational database environment in the presence of data skew. There is a technology that merges relations. This technique can minimize the total execution time for the natural combination of two relations by scheduling the current set of jobs between processors using minimum make-span optimization techniques (eg, patents). Reference 1).
JP-A-3-126169

  However, if the algorithm shown in FIG. 10 is applied to a system distributed by a network as it is, there is a problem that the number of communications increases and performance does not appear.

  In addition, the merge-joining technology using the minimum make-span optimization technique assumes a multiprocessor system, and communication between distributed systems is premised on interconnects that are faster than the network. No consideration is given to a distributed system connected by a network that is slower than the bus transfer speed of the network.

  The present invention has been made in view of the above points. In a distributed system in which indexes are connected to each other via a network that is slower than the bus transfer speed inside the computer, the merge connection between the indexes is accelerated, and communication during processing is performed. It is an object of the present invention to provide a distributed index combining system, method and program capable of reducing the number of times.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (claim 1), in the distributed network system in which a plurality of processors in the network are connected to a coupling method of distributed index merging index each other via the network,
A storage means A is provided for storing an index A composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index A, and a key value list composed of key values of records corresponding to the pointers. Processor A;
There is provided storage means B for storing an index B composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index B, and a key value list composed of key values of records corresponding to the pointers Processor B;
In a system in which a record of index A of processor A and a processor J that combines records of index B of processor B are connected via a network,
In processor A,
From the processor B, a receiving A step of receiving a search flag list B a true false value is set, the key value list B consisting of key value of the record corresponding to the pointer of the N content (Step 1),
When the i-th (i ≧ 1) flag extracted from the search flag list B is set, the search flag processing A step of storing the record pointed to by the i-th corresponding pointer in the pointer list of the index A in the transmission buffer A (Step 2)
The search flag list A is initialized, the index A is searched from the storage means A with the i-th key value extracted from the key value list B, and if there is a record that satisfies the search condition, the record is stored in the transmission buffer A. A search A step for setting the i-th flag in the search flag list A;
A key value list A generation step (step 3) for generating N key value lists A in order from the smallest value exceeding the Nth value of the key value list B acquired from the processor B;
A list transmission A step (step 4) for transmitting the key value list A and the search flag list A to the processor B;
A record transmission A step of transmitting a record of the buffer to the processor J when the storage amount of the record stored in the buffer A is equal to or greater than a predetermined threshold;
And
In processor B,
A reception B step for receiving a search flag list A in which a true / false value is set, and a key value list A composed of key values of records corresponding to N pointers;
When the i-th (i ≧ 1) flag extracted from the search flag list A is set, the search flag processing B step for storing the record pointed to by the i-th corresponding pointer in the pointer list of the index B in the transmission buffer B When,
The search flag list B is initialized, the index B is searched from the storage means B with the i-th key value extracted from the key value list A, and if there is a record that satisfies the search condition, the record is stored in the transmission buffer B. A search B step for setting the i-th flag in the search flag list B;
A key value list B generating step (step 5) for generating N key value lists B in order from the smallest value exceeding the Nth value of the key value list A;
A list transmission B step (step 6) for transmitting the search flag list B and the key value list B to the processor A ;
A record transmission B step of transmitting a record of the buffer to the processor J when the storage amount of the record stored in the buffer B exceeds a predetermined threshold;
And
Processor J is
Records received from the processor A are stored in the processor A buffer, records received from the processor B are stored in the processor B buffer,
The records stored in the processor A buffer and the processor B buffer are combined and output .

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 2), in the distributed network system in which a plurality of processors in the network are connected to a coupling system distributed index merging index each other via the network,
A storage unit 130A is provided for storing an index A composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index A, and a key value list composed of key values of records corresponding to the pointers. Processor A;
The storage unit 130B stores an index B composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index B, and a key value list composed of key values of records corresponding to the pointers. Processor B;
A processor J that combines the record of index A of processor A and the record of index B of processor B ;
Processor A,
Receiving A means for receiving a search flag list B in which true / false values are set and a key value list B consisting of key values of records corresponding to N pointers from the processor B;
When the i-th (i ≧ 1) flag taken out from the search flag list B is set, the search flag processing A means for storing in the transmission buffer A the record pointed to by the i-th corresponding pointer in the pointer list of the index A When,
The search flag list A is initialized, the index A is searched from the storage means A with the i-th key value extracted from the key value list B, and if there is a record that satisfies the search condition, the record is stored in the transmission buffer A. A search A means 110A for setting the i-th flag in the search flag list A;
Key value list A generation means 120A for generating N key value lists A in order from the smallest value exceeding the Nth value of the key value list B acquired from the processor B;
List transmission A means 141A for transmitting the key value list A and the search flag list A to the processor B;
A record transmission A means for transmitting the record of the buffer to the processor J when the storage amount of the record stored in the buffer A exceeds a predetermined threshold;
Have
Processor B is,
Receiving B means for receiving, from the processor A, a search flag list A in which true / false values are set, and a key value list A composed of key values of records corresponding to N pointers;
When the i-th (i ≧ 1) flag taken out from the search flag list A is set, the search flag processing B means for storing the record pointed to by the i-th corresponding pointer in the pointer list of the index B in the transmission buffer B 110B,
The search flag list B is initialized, the index B is searched from the storage means B with the i-th key value extracted from the key value list A, and if there is a record that satisfies the search condition, the record is stored in the transmission buffer B. And a search B means for setting the i-th flag in the search flag list B ,
Key value list B generation means 120B for generating N key value lists B in order from the smallest value exceeding the Nth value of the key value list A acquired from the processor A;
A list transmission B means for transmitting the search flag list B and the key value list B to the processor A;
A record transmission B means 141B for transmitting the record of the buffer to the processor J when the storage amount of the record stored in the buffer B exceeds a predetermined threshold;
Have
Processor J is
A processor A buffer for storing records received from the processor A;
A buffer for processor B for storing a record received from processor B;
And a record combining unit 220 that combines and outputs the records stored in the processor A buffer and the processor B buffer .

Further, according to the present invention (Claim 6), in the processor A,
A record A transmission unit 141A for transmitting a record of the buffer A to the processor J when the storage amount of the record stored in the buffer A exceeds a predetermined threshold;
In processor B,
A record B transmission unit 141B for transmitting the record of the buffer B to the processor J when the storage amount of the record stored in the buffer B exceeds a predetermined threshold;
Processor J is
A processor A buffer for storing records received from the processor A;
A buffer for processor B for storing a record received from processor B;
And a record combining unit 220 that combines and outputs the records stored in the processor A buffer and the processor B buffer.

Further, according to the present invention (Claim 7), in the search B means 110B of the processor B,
Means for storing in the transmission buffer B the record pointed to by the i-th pointer in the pointer list of the index B when the i-th (i ≧ 1) flag taken out from the search flag list A is set;
The index B is searched from the storage means B with the i-th key value extracted from the key value list. If there is a record satisfying the search condition, the record is stored in the transmission buffer B, and the i-th key in the search flag list B is stored. Means for flagging.

Further, according to the present invention (claim 8), in the search A means 110A of the processor A,
Means for storing in the transmission buffer A the record pointed to by the i-th pointer in the pointer list of the index A when the i-th (i ≧ 1) flag taken out from the search flag list B is set;
The index A is searched from the storage means A with the i-th key value extracted from the key value list. If there is a record satisfying the search condition, the record is stored in the transmission buffer A, and the i-th key in the search flag list A is stored. Means for flagging.

  As described above, according to the present invention, each processor has N pointers for scanning an index, and the key of the record corresponding to the pointers for N pointers in units of a pointer list composed of N pointers between the processors. By transmitting only the value (key value list), the processing load is reduced. This is because the processing overhead associated with starting / stopping data transmission / reception is large over a network, so even if the total amount of data to be transmitted / received is the same, the processing time can be shortened by reducing the number of times required for that exchange. The

  Furthermore, if only the key value is sent, it will not be possible to know whether the key value sent by the other processor has been searched or not, so the search flag list that records what was "true" as a result of searching the index Will be sent at the same time. Since the search flag has only a true / false value, the amount of information is sufficiently small compared to transmitting the entire record to the other processor, resulting in a reduction in communication amount.

  Further, when performing communication between the processor A or the processor B and the processor J, processing overhead related to the start / stop of data transmission / reception in communication via the network is performed by buffering and processing the transmission targets. And the communication efficiency can be improved.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 shows the configuration of a distributed index combining system according to an embodiment of the present invention.

  The distributed index combination system shown in FIG. 1 is a processor A that has an index A, a processor B that has an index B, and a processor J that controls the combination of the records of the processor A and the processor B and triggers the operations described below. Consists of

  The processor A and the processor B have the same configuration, and are respectively an input / output unit 100, a search unit 110, a key value list processing unit 120, a key value list storage unit 130, a communication unit 140, a search flag list processing unit 150, and a buffer. 170. Further, although not shown, a search flag list described below is stored in a memory.

  As shown in FIG. 4, the key value list storage unit 130 stores a pointer list and a key value list. In the following description, the “pointer” is a pointer indicating where the index is searched, and a list of a plurality of pointers is referred to as a “pointer list”. The key value list is a list of only the key values of the index records pointed to by the pointer list. In the example of FIG. 4, the top of the pointer list points to the top record of the index, and the key value “0034” of the record is set in the key value list. The second pointer list points to the nth record in the index, and the key value “0305” of the record is set in the key value list.

  Note that the entire record is transmitted to the processor J.

  The processor J is a processor that supervises merge join between indexes, which is different from the processor A having the index A and the processor B having the index B, and is a processor that triggers the entire processing represented by the following flowchart. is there. The processor J includes a merge processing unit 210, a record combining unit 220, a communication unit 230, a processor B buffer 240, and a processor A buffer 250. The merge processing unit 210 is connected to the search condition input device 310 and records combining. The unit 220 is connected to the combined result output device 320.

  The merge processing unit 210 outputs the search condition input from the search condition input device 310 to the processors A and B to the processors A and B.

  The record combining unit 220 combines the corresponding records of the index search results transmitted from the communication units 140 of the processor A and the processor B and stored in the buffers 240 and 250, and outputs the combined records to the combination result output device 320. Execution of this processing indicates that there is a record having a key value common to both indexes.

  The operation in the above configuration will be described below.

  FIG. 5 is a sequence chart of the entire operation in the embodiment of the present invention.

  Step 51) When the search condition is input from the search condition input device, the processor J sends the search condition to the processors A and B to instruct the start of the index A and B combination processing.

  Step 52) The processor A generates N key value lists A in order from the top of the index A in the key list processing unit 120A.

  Step 53) The search flag list processing unit 150A initializes the search flag list A of the index A.

  Step 54) The key value list A and the search flag list A are transmitted via the communication unit 140A.

  Step 55) The processor B initializes the search flag list B of the index B.

  Step 56) The search flag list A received in step 54 is confirmed, and the record of index B corresponding to the record of index A searched by the processor A is stored in the buffer B. In the case of the first time, since the search flag list A initialized in step 53 is referred to, there is no record to be stored in the buffer B.

  Step 57) The index B is searched using the key value list A as a key. If there is a searched record, it is stored in the buffer B, and the flag “true” is set in the search flag list B.

  Step 58) The communication unit 140B determines whether the storage amount of the record of the index B stored in the buffer B exceeds a predetermined buffer storage threshold, and if so, the record group stored in the buffer B is selected. To processor J. If not, the process proceeds to step 61.

  Processor 59) In the processor J, the record group received from the processor B is stored in the buffer B240.

  Step 60) The record combining unit 220 of the processor J determines whether the record of the index A stored in the buffer A250 and the record of the index B stored in the buffer B240 can be combined. Combine.

  Step 61) The key value list processing unit 120B creates N key value lists B in order from the smallest value exceeding the Nth value (maximum value in the list) of the key value list A in the index B.

  Step 62) The key value list B and the search flag list B are transmitted from the communication unit 140B to the processor A.

  Step 63) In the processor A, the search flag list processing unit 150A initializes the search flag list A of the index A.

  Step 64) The search unit 110A of the processor A confirms the search flag list B, and stores the record of the index A corresponding to the record of the index B searched by the processor B in the buffer 170A.

  Step 65) The search unit 110A searches the index A using the key value list B as a key, and if there is a searched record, stores it in the buffer 170A, and the search flag list processing unit 150A sets the corresponding search flag list A Set to true.

  Step 66) The communication unit 140A determines whether the storage amount of the index A record stored in the buffer 170A exceeds a predetermined buffer storage threshold, and if so, the record group stored in the buffer 170A is selected. To processor J. If not, the process proceeds to step 69.

  Step 67) In the processor J, the record group received from the processor A is stored in the buffer A250.

  Step 68) The record combining unit 220 of the processor J determines whether the record of the index A stored in the buffer A250 and the record of the index B stored in the buffer B240 can be combined. Combine.

  Step 69) The key value list processing unit 120A creates N key value lists in order from the smallest value exceeding the Nth value (maximum value in the list) of the key value list B in the index A.

  Step 70) The processor A transmits the key value list and the search flag list A to the processor B from the communication unit 140A.

  Thereafter, the processing after step 55 is repeated.

  Next, the merge join process will be described.

  FIG. 6 is a flowchart of merge join in one embodiment of the present invention.

  Step 110) When N pointer lists indicating records are input via the input / output unit 100B in the processor B, the key value list processing unit 120B causes the key value so that the pointer points to the first record of the index B. The pointer list of N indexes B in the list storage unit 130B is initialized.

  Step 116) When N pointers pointing to the record of index A are input through the input / output unit 100A in the processor A, the key value list processing unit 120A records N records in order from the first record of the index A. N pointers are initialized so as to point to, and a list of N pointers pointing to the records of the index A that has been initialized is output.

  Step 117) Among the key value list sent from the processor B, the search flag list for recording the index value of “true” as a result of searching the index A is initialized, and the index A that has been initialized is searched. Output flag list.

  Step 120) In the key value list processing unit 120A, the N pointer lists indicating the records of the index A are read from the key value list storage unit 130A, and the key values of the records of the index A indicated by the N pointer lists are acquired. Are stored in the key value list storage unit 130A as a key value list and transmitted to the processor B via the communication unit 140A.

  Step 122) At this time, the search flag list A of the index A in which the result of searching the index A is “true” is also transmitted to the processor B.

  Step 125) When the key value list is received by the communication unit 140B of the processor B, the key value list processing unit 120B stores the key value list in the key value list area of the processor A in a memory (not shown). Then, the search unit 110B searches the index value B from the key value list storage unit 130B using the key values of the N key value list records of the index A acquired from the processor A as search keys, and points to the record of the index B. Obtain N pointer lists. Details of this step will be described later.

  Step 130) The key value list processing unit 120B acquires the key value of each record of the index B indicated by the N pointer lists indicating the acquired record of the index B, and stores it as a key value list in the key value list storage unit 130B. At the same time, the data is transmitted to the processor A via the communication unit 140B.

  Step 132) At this time, the processor B also transmits to the processor A the search flag list B of the index B in which the index B is searched for “true”.

  Step 135) When the key value list is obtained from the processor B via the communication unit 140 of the processor A, the key value list processing unit 120 stores the key value list in a memory (not shown). The retrieval unit 110A retrieves the index A from the key value list storage unit 130A by using the key values of the N key value list records of the index B acquired as the retrieval key, and N retrievals indicating the index B records. Get the pointer list. Details will be described later. Further, the search flag list processing unit 150A outputs the search flag list of the index A in which the index “A” as a result of searching the index A is recorded.

  Next, the index B search process in step 125 will be described in detail.

  FIG. 7 is a flowchart of the index search process according to the embodiment of the present invention.

  Step 205) The search flag list processing unit 150B of the processor B uses the search flag list B for recording the key value list sent from the processor A that is “true” as a result of searching the index B. initialize.

  Step 210) In the key value list processing unit 120B of the processor B, a loop counter j is provided in a memory (not shown) for processing all at a time, and is initialized as j = 1.

  Step 211) The search flag list processing unit 150B extracts a search flag corresponding to the counter j from the search flag list A of the index A.

  Step 212) The search flag list processing unit 150B confirms whether or not the jth search flag of the index A is set in the search flag A. If standing, the process proceeds to step 213, and if not standing, the process proceeds to step 216.

  Step 213) The fact that the j-th search flag of index A was set in step 212 means that the key value of the record pointed to by the j-th pointer recorded in step 251 when the index B was searched last time. It means that the search is made with the index A. Therefore, the search unit 110B performs the following processing.

  1. The jth pointer is taken out from the list of N pointers indicating the record of index B.

  2. From the key value list storage unit 130B. The record of the index B pointed to by the pointer acquired in (1) is acquired.

  3.2. Is stored in the buffer 170B for sending to the processor J.

  Step 215) The key value list processing unit 120B extracts the key value corresponding to the counter j (jth key value in the key value list of the index A) from the N key value lists of the index A transmitted from the processor A. .

  Step 225) The retrieval unit 110B retrieves the index B from the key value list storage unit 130B using the j-th key value in the key value list of the index A extracted in step 215. When the search result is “true”, a pointer indicating the record corresponding to the search result of index B is output.

  Step 230) The search unit 110B determines whether or not a record satisfying the search condition exists in the index B of the key value list storage unit 130B. If the search result in step 225 is “true”, the process proceeds to step 232, and if “false”, the process proceeds to step 265.

  Step 232) The following processing is performed.

  1. The record corresponding to the search result of the index B indicated by the pointer obtained in step 225 is acquired.

  2.1. Is stored in the buffer 170B for sending to the processor J. This processing means that there is a record having a key value common to both indexes.

  Step 233) The search flag list processing unit 150B sets the jth flag in the search flag list of the index B to notify the processor A that the jth key in the key value list of the index A has been searched.

  Step 265) It is confirmed whether the counter j for the loop does not exceed N. If N is not exceeded, that is, N loops are not completed, the process proceeds to step 270. If N is exceeded, that is, N loops are completed, the process proceeds to step 238.

  Step 270) The counter j is incremented by 1, and the process proceeds to Step 215.

  Step 238) The communication unit 140B confirms whether the storage amount of the buffer 170B to be sent to the processor J does not exceed a predetermined threshold value. If not, the process proceeds to step 240. 239.

  Step 239) All the records corresponding to the search result of the index B stored in the buffer 170B are transmitted to the processor J. The operation of the processor J that acquired the record corresponding to the search result will be described in detail with reference to FIG.

  Step 240) The search unit 110B uses the N-th key value in the key value list of the index A extracted in Step 215 by the conditional branch of Step 265, from the index B of the key value list storage unit 130B. Search for “minimum value over key value”. If the search result is “true”, a pointer indicating the record corresponding to the search result of index B is output.

  Step 245) In Step 240, it is confirmed whether or not a record satisfying the search condition given from the processor J exists in the index B. If it exists, the process proceeds to Step 251, and if it does not exist, the process proceeds to Step 252.

  Step 251) The key value list processing unit 120B records in the N pointer lists indicating the records of the index B so as to indicate the N records in order from the record corresponding to the search result of the index B. Thereafter, the index B search process is terminated, and the process proceeds to step 130 in FIG.

  Step 252) In Step 245, if there is no record satisfying the search condition in the index B, it is confirmed whether one or more flags in the search flag list of the index B are set. (If any key in the key value list of index A is searched in step 232, even if it is determined in step 245 that the search of index B has been completed, the processing is continued and processor A is (It is necessary to convey the search result on the processor B side). If standing, the “index B search process” is terminated, and the process proceeds to step 130 in FIG. If not standing, the process is terminated.

  FIG. 8 is a flowchart of the index A search process according to the embodiment of the present invention.

  Step 305) Of the key value list sent from the processor B, a search flag list for recording a list of “true” as a result of searching the index A is initialized.

  Step 310) In the key value list processing unit 120A of the processor A, a loop counter i is provided in a memory (not shown) for processing N pieces collectively, and i = 1 is set.

  Step 311) The search flag corresponding to the counter i is extracted from the search flag list of the index B.

  Step 312) It is confirmed whether or not the i-th search flag of the index B is set. If it is set, the process goes to Step 313. If it is not set, the process goes to Step 316.

  Step 313) The fact that the i-th search flag of index B was set in step 312 indicates that the key value of the record pointed to by the i-th recorded pointer in step 351 is index B when the index A was previously searched. It means that it was searched by. Therefore, the search unit 110A performs the following processing.

  1. The i-th pointer is extracted from the N pointer list indicating the record of index A from the key value list storage unit 130A.

  2.1. The record of the index A pointed to by the pointer acquired in (1) is acquired.

  3.2. Is stored in the buffer 170A for sending to the processor J.

  Step 315) The key value list processing unit 120A extracts the key value corresponding to the counter i (the i-th key value in the key value list of the index B) from the N key value lists transmitted from the processor B.

  Step 326) The retrieval unit 110A retrieves the index A from the key value list storage unit 130A using the j-th key value in the key value list of the index A extracted in Step 315. If the search result is “true”, a pointer indicating the record corresponding to the search result of index A is output.

  Step 331) The search unit 110A determines whether or not a record satisfying the search condition exists in the index A of the key value list storage unit 130A. If the search result is “true”, the process proceeds to step 332, and if it is “false”, the process proceeds to step 365.

  Step 332) The retrieval unit 110A performs the following processing.

  1. The record corresponding to the search result of index A pointed to by the pointer obtained in step 326 is acquired.

  2.1. Is stored in the buffer 170A for sending to the processor J (this processing means that there is a record having a key value common to both indexes).

  Step 333) The search flag list processing unit 150A sets the i-th flag in the search flag list of the index A to notify the processor B that the i-th key in the key value list of the index B has been searched.

  Step 365) Check if the loop counter i does not exceed N. If N is not exceeded, that is, if N loops are not completed, the process proceeds to Step 370, where N is exceeded. If N loops have been completed, the process proceeds to step 338.

  Step 370) The counter i is incremented by 1, and the process proceeds to Step 315.

  Step 338) It is confirmed whether or not the storage amount of the buffer 170A to be sent to the processor J exceeds a predetermined threshold value. If it does not exceed, the process proceeds to Step 341. If it exceeds, the process proceeds to Step 339.

  Step 339) The search record corresponding to the index A stored in the buffer 170A is sent to the processor J.

  Step 341) The search unit 110A uses the N-th key value in the key value list of the index B extracted in Step 315 by the conditional branch in Step 365 to search for “key” from the index A of the key value list storage unit 130A. Search for “minimum value over value”. If the search result is “true”, a pointer indicating the record corresponding to the search result of index A is output.

  Step 346) In Step 341, it is determined whether or not a record satisfying the search condition given by the processor J exists in the index A. If there is a record, the process proceeds to Step 351. If not, the process proceeds to Step 352.

  Step 351) The key value list processing unit 120A records in the N pointer lists indicating the records of the index A so as to indicate N records in order from the corresponding record of the search result of the index A. Thereafter, the “index A search process” is terminated, and the process proceeds to step 120 in FIG.

  Step 352) It is confirmed whether or not one or more flags in the search flag list of index A are set (if any key in the key value list of index B is searched in step 332), step Even if it is determined in 346 that the search for the index A has been completed, it is necessary to continue the processing and transmit the search result on the processor A side to the processor B). If standing, “index A search processing” is terminated, and the process proceeds to step 120 in FIG. If not standing, the process is terminated.

  Next, the operation of the processor J that has acquired the corresponding records of the search results of the index A and index B from the processors A and B will be described.

  FIG. 9 is a flowchart of the index A and B combining process by the processor J in the embodiment of the invention.

  Step 410) The communication unit 230 waits for communication until the record corresponding to the search result from the processor A or the processor B arrives.

  Step 415) Upon receiving the search result corresponding record from the processor A or the processor B, the communication unit 230 checks whether or not the transmission source of the search result corresponding record is the processor A. Otherwise, go to step 440.

  Step 420) The communication unit 230 stores the record corresponding to the search result received from the processor A in the buffer 250 for the processor A.

  Step 425) It is confirmed whether the maximum value of the key value of the record stored in the buffer for the processor A is equal to or greater than the maximum value of the key value of the record stored in the buffer for the processor B (index A, B). In order for the processor J to perform the combination, it is necessary to perform both of the indexes A and B. In the algorithms shown in FIGS. Therefore, the maximum value is confirmed here, and if it is equal to or greater than the other maximum value, it is possible to combine records having the same key value).

  Step 430) If it is equal to or greater than the maximum value, the process proceeds to step 435, and if not, the process proceeds to step 440.

  Step 435) The record combining unit 220 extracts and combines the records having the same key value from the buffer 250 for the processor A and the buffer 240 for the processor B, and combines them to obtain a combined result output device as a result of combining the indexes A and B. To 320.

  Step 440) It is confirmed whether or not the transmission source of the record corresponding to the search result is the processor B. If received from the processor B, the process proceeds to step 445; otherwise, the process returns to step 410.

  Step 445) The communication unit 230 stores the record corresponding to the search result received from the processor B in the buffer 240 for the processor B.

  Step 450) The record combining unit 220 determines that the maximum value of the key value of the record stored in the processor B buffer 240 is equal to or greater than the maximum value of the key value of the record stored in the processor A buffer 250. (In order to combine the indexes A and B with the processor J, it is necessary to perform both of the index key values together. In the algorithms shown in FIGS. 7 and 8, the key value is small.) Since the processing is performed in order from the value, the maximum value is confirmed here, and if it is equal to or larger than the other maximum value, it is possible to combine records having the same key value).

  Step 455) When the maximum value of the key value of the record stored in the buffer 240 for the processor B is equal to or larger than the maximum value of the key value of the record stored in the buffer 250 for the processor A, the process proceeds to step 460. Otherwise, return to step 410.

  Step 460) The record combining unit 220 extracts the records having the same key value from the buffer 250 for the processor A and the buffer 240 for the processor B, combines them, and outputs them to the combination result output device 320.

  FIG. 10 shows comparison data when 5 million pieces of data are merged and joined by the method according to the present invention and the method according to the conventional technique. Due to the mock-up test, there is a difference in index reading processing and the like, and the index reading processing time cannot be compared fairly, but it was confirmed that the network transfer processing time was reduced by the present invention.

  Although the present invention has been described based on the configuration of the processor shown in FIG. 3, the present invention is not limited to this example, and the operation of each processor component is constructed as a program, installed in each processor (computer), and executed. Or can be distributed via a network.

  In addition, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to DBMS (Data Base Management System) and RDBMS (Relational Date Base Management System).

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the distributed index coupling | bonding system in one embodiment of this invention. It is a figure for demonstrating the record list memory | storage part in one embodiment of this invention. It is a flowchart of the whole operation | movement in one embodiment of this invention. It is a flowchart of the merge connection in one embodiment of this invention. It is a flowchart of the index B search process in one embodiment of this invention. It is a flowchart of the index A search process in one embodiment of this invention. It is a flowchart of the index A and B combining process by the processor J in one embodiment of this invention. It is a comparison result of the method of the present invention and the conventional method. It is a flowchart of the conventional merge join.

Explanation of symbols

100 Input / output unit 110 Search unit, search unit 120 Key value list generation unit, key value list processing unit 130 Storage unit, key value list storage unit 140 Communication unit 141 Transmission unit 150 Search flag list processing unit 151 Search flag list B initialization Means 152 Search Flag List 210 Merge Processing Unit 220 Record Combining Unit, Record Combining Unit 230 Communication Unit 240 Buffer B
250 Buffer A
310 Search condition input device 320 Combined result output device

Claims (2)

In a distributed network system to network multiple processors are connected to a coupling method of distributed index merging index each other via the network,
A storage means A is provided for storing an index A composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index A, and a key value list composed of key values of records corresponding to the pointers. Processor A;
There is provided storage means B for storing an index B composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index B, and a key value list composed of key values of records corresponding to the pointers Processor B;
In the system in which the processor A that combines the record of the index A of the processor A and the record of the index B of the processor B is connected via the network,
In the processor A,
From the processor B, and the search flag list B a true false value is set, and receiving A step of receiving a key value list B consisting of key value of the record corresponding to the pointer of the N content,
When the i-th (i ≧ 1) flag taken out from the search flag list B is set, the search flag processing A for storing the record pointed to by the i-th pointer in the pointer list of the index A in the transmission buffer A Steps,
The search flag list A is initialized, the index A is searched from the storage means A with the i-th key value extracted from the key value list B, and if there is a record satisfying the search condition, the record is stored in the transmission buffer. A search A step of storing in A and flagging the i-th flag in the search flag list A;
A key value list A generation step of generating N key value lists A in order from the smallest value exceeding the Nth value of the key value list B acquired from the processor B;
A list transmission A step of transmitting the key value list A and the search flag list A to the processor B;
A record transmission A step of transmitting a record of the buffer to the processor J when the storage amount of the record stored in the buffer A is equal to or greater than a predetermined threshold;
And
In the processor B,
A reception B step for receiving from the processor A a search flag list A in which true / false values are set, and a key value list A consisting of key values of records corresponding to N pointers;
When the i-th (i ≧ 1) flag taken out from the search flag list A is set, the search flag processing B for storing the record pointed to by the i-th corresponding pointer in the pointer list of the index B in the transmission buffer B Steps,
The search flag list B is initialized, the index B is searched from the storage means B with the i-th key value extracted from the key value list A, and if there is a record that satisfies the search condition, the record is stored in the transmission buffer. A search B step for storing in B and flagging the i th flag in the search flag list B;
A key value list B generating step for generating N key value lists B in order from the smallest value exceeding the Nth value of the key value list A;
A list transmission B step of transmitting the search flag list B and the key value list B to the processor A;
A record transmission B step of transmitting a record of the buffer to the processor J when the storage amount of the record stored in the buffer B exceeds a predetermined threshold;
The stomach line,
The processor J is
The record received from the processor A is stored in the processor A buffer, the record received from the processor B is stored in the processor B buffer,
A method of combining distributed indexes, wherein the records stored in the processor A buffer and the processor B buffer are combined and output . In a distributed network system in which a plurality of processors are connected to a network, a coupling system of distributed index merging index each other via the network,
A storage means A is provided for storing an index A composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index A, and a key value list composed of key values of records corresponding to the pointers. Processor A;
There is provided storage means B for storing an index B composed of indexes of a plurality of records, a pointer list composed of N pointers for scanning the index B, and a key value list composed of key values of records corresponding to the pointers Processor B;
A processor J for combining the record of the index A of the processor A and the record of the index B of the processor B ;
The processor A,
Receiving A means for receiving from the processor B a search flag list B in which true / false values are set, and a key value list B consisting of key values of records corresponding to N pointers;
When the i-th (i ≧ 1) flag taken out from the search flag list B is set, the search flag processing A for storing the record pointed to by the i-th pointer in the pointer list of the index A in the transmission buffer A Means,
The search flag list A is initialized, the index A is searched from the storage means A with the i-th key value extracted from the key value list B, and if there is a record satisfying the search condition, the record is stored in the transmission buffer. Search A means for storing in A and flagging the i-th flag in the search flag list A;
And key value list A generating means for generating a N-th key value list A of N content from the minimum value in the order exceeds the value of the key value list B acquired from the processor B,
List transmission A means for transmitting the key value list A and the search flag list A to the processor B;
A record transmission A means for transmitting a record of the buffer to the processor J when the storage amount of the record stored in the buffer A is equal to or greater than a predetermined threshold;
Have
The processor B is
Receiving B means for receiving, from the processor A, a search flag list A in which true / false values are set, and a key value list A consisting of key values of records corresponding to N pointers;
When the i-th (i ≧ 1) flag taken out from the search flag list A is set, the search flag processing B for storing the record pointed to by the i-th corresponding pointer in the pointer list of the index B in the transmission buffer B Means,
The search flag list B is initialized, the index B is searched from the storage means B with the i-th key value extracted from the key value list A, and if there is a record that satisfies the search condition, the record is stored in the transmission buffer. Search B means stored in B and flagging the i-th flag in the search flag list B ;
Key value list B generating means for generating N key value lists B in order from the smallest value exceeding the Nth value of the key value list A acquired from the processor A;
List transmission B means for transmitting the search flag list B and the key value list B to the processor A;
A record transmission B means for transmitting a record in the buffer to the processor J when the storage amount of the record stored in the buffer B exceeds a predetermined threshold;
Have
The processor J is
A processor A buffer for storing records received from the processor A;
A processor B buffer for storing records received from the processor B;
Record combining means for combining and outputting records stored in the processor A buffer and the processor B buffer;
A distributed index combining system comprising:

Images