JP2012150562A - Method and device and program for compressing n-branch tree internal node - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the size of one comparison key (the numerical value of branch determination set in the nodes of an N-branch tree).SOLUTION: With respect to input M pieces of integral columns, N-branch tree creation is performed, and the N-branch tree is decomposed into partial tress of every height H from a ROOT node, and the partial tress are rearranged with width given priority, and the expression space of a comparison key in the partial trees is converted so as to be degenerated from 8×S bits to 8×T bits, and a block including the converted comparison key is stored in storage means, and the block including the ROOT node is read from the storage means, and conversion processing which is similar to conversion processing performed by initialization means is performed to an input search key, and processing of comparing the converted search key with the comparison key of the read block is repeated until a leaf node is reached, and when there exists the same comparison key as the search key in the leaf node, position information in an integral column corresponding to the comparison key is output.

Description

  The present invention relates to an N-ary tree internal node compression method, apparatus, and program, and in particular, an N-minute structured to speed up a process of searching for an arbitrary integer (search key) from M integer strings. The present invention relates to an N-ary tree internal node compression method, apparatus, and program for reducing the data size of an internal node of a tree.

  The M integer strings to be searched are arranged in ascending order, and it is determined whether there is an integer having the same key value as the search key from the integer string. The search process repeats the comparison process of the comparison key and the search key in the node in order from the ROOT node located at the highest node, passes through the BRANCH node existing at the intermediate position, and corresponds to the search key (position to the integer string) The LEAF node (which holds the information) is calculated, and finally the presence or absence of the search key in the integer string is determined. In the example of FIG. 9, since the search key is “323”, first, the search moves to the BRANCH nodes (“201”, “393”) indicating the leftmost arrow of the ROOT node. Since “201” <search key (“323”) <“393” ”, it moves to the LEAF node ahead of the BRANCH node middle arrow. Since the same comparison key as the search key in the LEAF node finally exists, this search process is determined to be successful, and the position information “323” in the integer string is returned.

  There are two N-ary tree data structures for speeding up the search processing of M integer strings: “using a pointer” (FIG. 10) and “using an array” (FIG. 11). As shown in FIG. 10, the former “using a pointer” includes (N−1) comparison keys for the BRANCH node to specify the target position of the search key, and N numbers for the next BRANCH node. It consists of a pointer (see, for example, Patent Document 1). On the other hand, as shown in FIG. 11, the latter “using an array” is composed of only (N−1) comparison keys for the BRANCH node (see, for example, Non-Patent Documents 1 and 2). From the comparison result at the node, the number of offsets to the next BRANCH node is calculated and moved (see, for example, Patent Document 2).

Japanese Patent No. 04351530 JP 2005-235209 A

Changkyu K. et al .: FAST: fast architecture sensitive tree search on modern CPUs and GPUs, Proceedings of the 2010 international conference on Management of data (SIGMOD'10), 2010. Benjamin S. et al .: k-ary search on modern processors, Proceedings of the 19th International Workshop on Data Management on New Hardware, 2009.

  If an N-ary tree is constructed for M integer sequences, the height of the N-ary tree is

で表され、N分木内部ノード内の総比較キー数（N_key）は初項（N-1）で、公比Nの等比級数となり、

The total number of comparison keys (N _key ) in the N-branch internal node is the first term (N-1), which is a geometric series of the common ratio N,

になる。N分木内部ノードの比較キーのデータサイズは、前述の従来技術の非特許文献１，２、特許文献２においては、N分木を構成する整数列と同じデータサイズ（このデータサイズをSバイトとする）になるため、整数列のデータサイズが４バイト（S=4）の場合は、内部ノードの総サイズはN_key×４バイトになり、８バイト（S=8）の場合は、N_key×８バイトになる。このN分木の総ノードサイズは、整数列が少ない（Mが小）場合には問題にならないが、センサ情報や、アクセスログデータなど大規模データ内の整数列に対するN分木を構成する際、N分木の内部ノードサイズが非常に大きくなる問題がある。

become. The data size of the comparison key of the N-tree internal node is the same data size as the integer string constituting the N-tree in the above-mentioned prior art Non-Patent Documents 1 and 2 and Patent Document 2 (this data size is S bytes). Therefore, if the integer column data size is 4 bytes (S = 4), the total size of the internal node is N _key × 4 bytes, and if it is 8 bytes (S = 8), N _key × 8 bytes. The total node size of this N-ary tree is not a problem when the integer string is small (M is small), but when configuring an N-ary tree for integer strings in large-scale data such as sensor information and access log data The internal node size of the N-ary tree becomes very large.

  The present invention has been made in view of the above points, and a compression method of an N-ary tree internal node that can reduce the size of one comparison key (a numerical value for branch determination set in an N-ary tree node). And an apparatus and a program.

In order to solve the above problem, the present invention reduces the data size of the internal node of the structured N-ary tree in order to speed up the process of searching for an arbitrary integer (search key) from M integer strings. An N-ary tree internal node compression device for
N (N is a system parameter) branch tree is created for the M integer strings that have been input. An initialization unit that performs conversion to degenerate a comparison key expression space that is a numerical value of branch determination, and stores a block including the converted comparison key in a storage unit;
The block including the ROOT node is read from the storage means, and the input search key is subjected to a conversion process similar to the conversion process performed by the initialization means, and the converted search key and the read block Search key processing that repeats the process of comparing the comparison keys until a leaf node is reached, and outputs the position information in the integer string corresponding to the comparison key when the same comparison key as the search key exists in the leaf node Means.

Also, the initialization means described above is
The N-ary tree is decomposed into subtrees of height H from the ROOT node, rearranged in the priority order, and the expression space of the comparison key in the subtree is changed from 8 × S bits to 8 × T bits (where T is arbitrary) Parameter, S> T).

The search key processing means is
Means for converting the search key into a value that can be compared for each block, using the information used to convert the comparison key converted by the initialization means;
Means for searching for the target position while moving to the next subtree by determining the next block to be searched by comparing the search key and the comparison key.

The search key processing means is
Using the comparison key in the leaf node, it is determined whether or not the leaf node that is searched first is a correct leaf node corresponding to the search key. Means for correcting.

As described above, since the single comparison key in the N-ary tree internal node can be expressed from 8 × S bits to 8 × T bits, the total internal node size of the N- _ary tree can be expressed as N _key × S bytes to N _key. X T bytes, and the total size is about T / S (if the metadata area is relatively small, it is ignored). More specifically, assuming that M integer strings are 4 bytes (S = 4) and T is 1, the total internal node size of the N-ary tree is about 25% before conversion.

It is an apparatus block diagram in one embodiment of this invention. It is a figure which shows the decomposition | disassembly to the subtree in the N-ary tree initialization part in one embodiment of this invention, and width priority rearrangement. It is a specific example of comparison key degeneracy conversion processing using four arithmetic operations in one embodiment of the present invention. It is a flowchart as one of the specific examples of the comparison key conversion processing module in the subtree in the embodiment of the present invention. It is a figure which shows the problem (T = 1) of the gap of the search process in one embodiment of this invention. It is a figure which shows the correction process to the correct position in one embodiment of this invention. It is a flowchart of the process of the N branch tree initial stock in one embodiment of this invention. It is a flowchart of the process of the search key process part in one embodiment of this invention. It is a block diagram of N-ary trees. This is a data structure of an N-ary tree using a pointer. This is a data structure of N-ary trees using arrays.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows an apparatus configuration according to an embodiment of the present invention.

  The apparatus shown in FIG. 1 includes an N-ary tree initialization unit 10, a search key processing unit 20, and an N-ary tree storage unit 30, and an integer sequence for inputting M integer sequences to the N-ary tree initialization unit 10. The search key input device 2 for inputting a search key and the result output device 3 are connected to the search key processing unit 20.

  The process in the above configuration will be described below.

  The N-ary tree initialization unit 10 creates an N-ary tree (N is a system parameter) for M integer strings input from the integer string input device 1 in order to perform high-speed processing of search keys. The N-ary tree initialization unit 10 creates a conventional N-ary tree from the ascending-ordered N-ary tree target integer string input from the integer string input device 1. After that, as shown in FIG. 2, the N-branch tree is decomposed into subtrees for each height H, the subtrees are rearranged with priority given to the width, and the internal comparison keys are arranged in ascending order to arrange the data structure of the N-branch tree. (Block). When converting to a block, the expression space of the comparison key for each block is degenerated from 8 × S bits to 8 × T bits (T is an arbitrary parameter in the present invention, and S> T). It is a feature. Details will be described in detail with reference to FIGS. As a specific example of the comparison key conversion process in the subtree, an example of the comparison key degeneration conversion process using four arithmetic operations is shown in FIG.

  The search key processing unit 20 determines whether or not the search key input from the search key input device 2 is included in the integer string input from the integer string input device 1 and returns its position information. When searching, the metadata included in each block to be compared (information used to convert the comparison key in the block) is used to make the search key comparable to each block. After conversion, the comparison with the comparison key in the block is performed. The next block to be searched is determined, and the search process to the target position is performed while moving to the next subtree. In the search process, the problem is that different comparison keys generated when compressing the comparison key in the subtree are converted to the same value (a specific example using the flowchart of FIG. 4 is used to explain this problem in FIG. 5). Need to deal with (shown)) (penalty for degenerating the expression space). More specifically, when different comparison keys are converted to the same value, “deviation” from the subtree to be originally searched for occurs. As shown in FIG. 6, this shift is added to the process of correcting the shift by comparing the search key with the internal comparison key when it arrives at the LEAF node.

  Next, detailed processing of the N-ary tree initialization unit 10 will be described.

  FIG. 7 is a flowchart of the process of the N-ary tree initialization unit in one embodiment of the present invention.

  Step 500) M integer strings in ascending order are input from the integer string input device 1.

  Step 505) An N-ary tree is created for the input M integer values by a conventional method to obtain a normal N-ary tree data structure.

  Step 510) The N-ary tree obtained in step 505 is decomposed into subtrees of every height H in order from the ROOT node as shown in FIG.

  Step 515) Initialize i with the value 1.

  Step 520) The i-th subtree is extracted from the S subtrees obtained by decomposing the N-ary tree at every height H.

  Step 525) Using the “comparison key conversion process in subtree” (for example, the flowchart of FIG. 4) module, the comparison key included in the i-th subtree is converted into a block. In this conversion process, an arbitrary conversion process for degenerating the expression space of each sub-tree comparison key from 8 × S bits to 8 × T bits is performed. As a specific example of the degenerate conversion processing, there is a conversion method using four arithmetic operations. This method will be described later.

  Step 530) When the block which has been converted and compressed in Step 525 is input, the input block is stored in a FIFO buffer (not shown).

  Step 535) Add 1 to i.

  Step 540) All blocks in the FIFO buffer (not shown) are sequentially read and stored in the N-ary tree storage unit 30.

  Hereinafter, the degeneracy conversion process in step 525 will be described with reference to FIG.

  Step 600) When the comparison key set of the subtree is input in the N-ary tree initialization unit 10, the comparison keys of the subtree are rearranged in ascending order as follows.

V ₁ (V _min ), V ₂ , ..., V _N-1 (V _max )
Step 605) All the comparison keys in the comparison key set are subtracted by the minimum value V _min in the comparison key, and x (correction term) is added. The correction term x exists to associate the minimum value in the subtree with “1”. As shown in FIG. 3, after the comparison key is converted (assuming “greater than” comparison processing), a search for the leftmost subtree is performed. enable.

Step 610) Divide all comparison keys (V _n −V _min + x) in the set of comparison keys by (V _max −V _min + x).

Step 615) Multiply all comparison keys in the set of comparison keys by ^{28 T} (T is a system parameter) and round off the decimal part.

  The comparison key in the subtree up to the above steps 605, 610, 615 is

で変換（T<Ｓ， Ｖ*_nは右辺を四捨五入した値

(T <S, V * _n is the value obtained by rounding off the right side.

する。

To do.

Step 620) The ascending comparison key after the conversion process with the metadata V _min and V _{max added at the} head is output as a block.

  Next, the process of the search key process part 20 is demonstrated along the flowchart of FIG.

  Step 700) The search key processing unit 20 receives a search key from the search key input device 2.

  Step 705) The block of the ROOT node is read from the N-ary tree storage unit 30.

  Step 710) The metadata contained in the read block (information used for converting the comparison key in the block) is read, and the comparison key in the block is subjected to the same processing as the converted key using the search key. The converted search key is output.

  Step 715) Using the converted search key, for example, using the method shown in “URL: http://www.geocities.jp/m_hiroi/light/abcruby13.html”, the comparison key in the block And the comparison process. Thereafter, the next block to be read is determined, and the block to be searched next is read from the N-ary tree storage unit 30. If the next block to be read is a LEAF node, the process proceeds to step 720. If it is not a LEAF block, the process proceeds to step 710.

  Step 720) The block information including the LEAF node is acquired and searched first to cope with the problem of “shift” in the search position caused by the conversion process of the comparison key of the N-ary tree internal node (FIG. 5). Whether the LEAF node is the correct LEAF node corresponding to the search key is determined by using the comparison key in the LEAF node. If it is not correct, the LEAF node position to be originally searched is corrected as shown in FIG. 6 by reading the LEAF nodes in order.

  Step 725) It is checked whether there is any LEAF node having the same comparison key as the search key. If there is, a pointer in the integer string corresponding to the comparison key is output to the result output device 3. If there is not, the integer string is output as having no value as the search key.

As described above, according to the present invention, when a subtree including a ROOT node is used as a head, a subtree is rearranged into a block having the internal data structure of an N-ary tree from the sequence in which the subtree is arranged in priority (step 520 to step 535) The process is characterized by performing a process of degenerating the expression space of the comparison key for each block (step 525). Note that the case where H (height) = 1 and the comparison key conversion process in the subtree in step 525 is not performed corresponds to the conventional technique. By performing the comparison key conversion as in the present invention, the expression space (0-2 ^S -1 to 0-2 ^T-1 -1) is degenerated in a state comparable to the search key, and the comparison key is configured. Allows node compression of N-ary trees.

  The operations of the components of the N-branch internal node compression device shown in FIG. 1 are constructed as a program and installed and executed on a computer used as the N-branch internal node compression device, or via a network. Can be distributed.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made within the scope of the claims.

1 Integer String Input Device 2 Search Key Input Device 3 Result Output Device 10 N-ary Tree Initialization Unit 20 Search Key Processing Unit 30 N-ary Tree Storage Unit

Claims (9)

N-tree internal node compression device for reducing the data size of the internal node of the N-tree structured to speed up the process of searching for an arbitrary integer (search key) from M integer strings. There,
N (N is a system parameter) branch tree is created for the M integer strings that have been input. An initialization unit that performs conversion to degenerate a comparison key expression space that is a numerical value of branch determination, and stores a block including the converted comparison key in a storage unit;
The block including the ROOT node is read from the storage means, and the input search key is subjected to a conversion process similar to the conversion process performed by the initialization means, and the converted search key and the read block Search key processing that repeats the process of comparing the comparison keys until a leaf node is reached, and outputs the position information in the integer string corresponding to the comparison key when the same comparison key as the search key exists in the leaf node Means,
An N-ary tree internal node compression apparatus characterized by comprising: The initialization means includes
The N-ary tree is decomposed into subtrees of height H from the ROOT node, rearranged in the priority order, and the expression space of the comparison key in the subtree is changed from 8 × S bits to 8 × T bits (where T is arbitrary) The apparatus for compressing an N-ary tree internal node according to claim 1, further comprising means for degenerating the parameter: S> T). The search key processing means includes
Means for converting the search key into a value that can be compared for each block, using the information used to convert the comparison key converted by the initialization means;
Means for searching to the target position while moving the process of determining the next block to be searched by comparing the search key and the comparison key to the next subtree;
The compression apparatus for an N-ary tree internal node according to claim 1. The search key processing means includes
Using the comparison key in the leaf node, it is determined whether or not the leaf node that is searched first is a correct leaf node corresponding to the search key. 4. A compression apparatus for an N-ary tree internal node according to claim 1, further comprising means for correcting. N-tree internal node compression method to reduce the data size of the internal node of the structured N-ary tree to speed up the process of searching for an arbitrary integer (search key) from M integer strings. There,
Initialization means creates N (N is a system parameter) branch tree for the input M integer strings, and starts with the subtree including the ROOT node and sorts the subtree in width-first order. An initialization step of performing a conversion to degenerate an expression space of a comparison key that is a numerical value of a branch decision in a subtree of the tree, and storing a block including the converted comparison key in a storage unit;
A search key processing unit reads the block including the ROOT node from the storage unit, performs a conversion process similar to the conversion process performed by the initialization unit on the input search key, and converts the converted search key and The process of comparing the comparison key of the read block is repeated until a leaf node is reached, and if the same comparison key as the search key exists in the leaf node, position information in the integer string corresponding to the comparison key Search key processing step for outputting
A method for compressing N-ary tree internal nodes. In the initialization step,
The N-ary tree is decomposed into subtrees of height H from the ROOT node, rearranged in the priority order, and the expression space of the comparison key in the subtree is changed from 8 × S bits to 8 × T bits (where T is arbitrary) 6. The method of compressing an N-ary tree internal node according to claim 5, wherein the compression is reduced to a parameter of S> T). In the search key processing step,
Using the information used to convert the comparison key converted by the initialization means, the search key is converted into a value that can be compared for each block,
6. The method of compressing an N-ary tree internal node according to claim 5, wherein a process for determining a next block to be searched by comparing the search key and the comparison key is performed to move to a next subtree and search to a target position. . In the search key processing step,
Using the comparison key in the leaf node, it is determined whether or not the leaf node that is searched first is a correct leaf node corresponding to the search key. The method of compressing an N-ary tree internal node according to claim 5 or 7, wherein correction is performed.   The program for functioning a computer as each means which comprises the compression apparatus of the N branch tree internal node of any one of Claims 1 thru | or 4.

Images