JP2011233016A - Management program, management device, and management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase efficiency of search by a Bloom filter.SOLUTION: A second transposed Bloom filter row tBF(p)s, tBFa(p)s are divided per word into words W1-1 to W1-12 and W2-1 to W2-12, and the words W1-1 to W1-12 and W2-1 to W2-12 are grouped in accordance with the identical arrangement position. Specifically, words having the same # in the words W1-# and W2-# are arranged in order of the word arrangement. For example, since both of the word W1-1 and the word W2-1 have 1 at the tail, the words W1-1 and W2-1 are arranged in this order. As a result, an original first transposed Bloom filter tbf(p-1) is restored. The other words are transposed in the same way, so that a first transposed Bloom filter row new-tBF(p) is restored.

Description

  The present invention relates to a management program, a management apparatus, and a management method for managing a Bloom filter.

  Conventionally, when large-scale data is managed in a tree structure, a relatively large amount of data is managed in a data structure called a B-tree. Compared to a simple binary tree, the B-tree stores a plurality of data entries in one block, and therefore has an advantage that the range in which the change in the shape of the tree structure spreads can be narrowed even if data entries are added. is there. For this reason, the B-tree is often used as a data management method for a disk such as a hard disk.

  However, when retrieving data managed in a tree structure on the disk, it is necessary to actually read a plurality of data blocks. In general, I / O (input / output) for a disk is slower than memory access, and thus searching for data on the disk may take time and effort.

  For this reason, recently, in order to avoid a search delay due to disk I / O, a countermeasure such as having a tree structure in the memory has been considered. However, in the B-tree, when the number of data entries increases, there is a possibility that the required memory amount increases accordingly. For this reason, a method using a method (cache) in which only the most read portion of the tree structure is stored in the memory is also considered.

  On the other hand, a data structure called a Bloom filter has recently been known. The Bloom filter is a method for efficiently checking whether an entry belongs to an existing set. In addition, there is also disclosed a group process in which two pulse speed bits and even / odd bits are provided in the dial pulse in the dial pulse process of the electronic exchange, and the bits are fetched. Also disclosed is a technique for repeatedly performing transposition and substitution in a data agitation circuit applicable to encryption and authentication.

  Also disclosed is a technique for shortening the processing time by combining a “user index” for each user, a “group index” that can be used by a plurality of users, and a “system shared index” that can be used by all users. Also disclosed is a technique for securing a free area by adding a variable-length index to a fixed-length area and thinning the key frame information from the index when it is determined that the area overflows.

JP 2007-52698 A Japanese Patent Laid-Open No. 4-18895 JP 7-177139 A JP 2003-289495 A

  As described above, since the B-tree can handle a large amount of data, it is possible to reduce disk I / O if a cache is appropriately mounted. However, the number of times cannot be reduced beyond a certain level. Further, when the tree structure changes due to the addition of the data entry, I / O for managing the tree structure may be required. In addition, the Bloom filter can only be used for data management because it knows only the existence of a data entry.

  In addition, if the index is thinned out when it overflows from the free space, the bit string in the Bloom filter changes, and there is an error that there is no data in the retrieved data block even though it was originally registered at the time of retrieval. Judgment will be made. Moreover, although it is not originally registered, an erroneous determination is made that data exists in the retrieved data block, and the probability of occurrence of false positives increases.

  An object of the present invention is to provide a management program, a management apparatus, and a management method capable of improving the efficiency of data retrieval when using a Bloom filter in order to solve the above-described problems caused by the conventional technology. .

  In order to solve the above-described problems and achieve the object, a data block set including a registered data group for each data block, and n number of bits in which m bits indicating negative in a predetermined number of data blocks are arranged If it is possible to access j first transposed Bloom filter sequences in which m n-bit first transposed Bloom filters in which bit sequences in the Bloom filter are collected by bits at the same position are arranged, the j pieces The first transposed Bloom filter string storage request is received, and when the storage request is received, the bit sequence of each first transposed Bloom filter is divided into n divisors for each first transposed Bloom filter By dividing by the number c, c words of n / c bits are generated (c × m × j), and the divided words are divided for each of the first transposed Bloom filter columns. By combining the first transposed Bloom filters at the same position in the first transposed Bloom filters, a second transposed Bloom filter array in which c second transposed Bloom filters of {(n / c) × m} bits are arranged. Transpose and store the transposed j second transposed Bloom filter strings in the storage device, accept a restoration instruction to the first transposed Bloom filter string, and if the restoration instruction is accepted, the second By dividing the second transposed Bloom filter string by the division number c for each transposed Bloom filter string, m words of n / c bits are generated for each second transposed Bloom filter, In j second transposed Bloom filter rows, the second transposed blue is obtained by arranging m words divided for each of the second transposed Bloom filters at the same arrangement position. The j second transposed Bloom filter trains are transposed into a third transposed Bloom filter train in which the j first transposed Bloom filter trains are arranged in one row by combining them in the order of filters. It is a requirement to do.

  According to the management program, management device, and management method of the present invention, there is an effect that it is possible to improve the efficiency of data search when using the Bloom filter.

It is a block diagram which shows the hardware structural example of the management apparatus concerning embodiment. It is a block diagram which shows the structural example of the management apparatus concerning this Embodiment. It is explanatory drawing which shows an example of a hash table group. It is explanatory drawing which shows an example of a hierarchical Bloom filter. It is explanatory drawing which shows the learning process example of the hierarchical Bloom filter by a registration process part. It is explanatory drawing which shows the example of a search process of the hierarchical Bloom filter by a search process part. It is explanatory drawing which shows the transposition example of the Bloom filter row | line | column of the p-th stage in a hierarchical Bloom filter. It is explanatory drawing which shows the search processing example of the hierarchical transposed Bloom filter by a search process part. It is a block diagram which shows the functional structural example of a search process part. It is a flowchart which shows the learning processing procedure of the hierarchical Bloom filter by a registration process part. It is a flowchart (the 1) which shows the search processing procedure by a search process part. It is a flowchart (the 2) which shows the search processing procedure by a search process part. It is explanatory drawing which shows the learning process example of the hierarchical transposition Bloom filter tBF by a registration process part. It is a flowchart which shows the learning processing procedure of the hierarchical Bloom filter BF by a registration process part. It is explanatory drawing which shows the re-transposition and preservation | save example of transposition Bloom filter row | line | column tBF (p). It is explanatory drawing which shows the example of an update of 2nd transposed Bloom filter row | line | column tBF (p) s. FIG. 3 is a block diagram illustrating a detailed functional configuration example of a save / restore processing unit illustrated in FIG. 2. It is a flowchart which shows the preservation | save processing procedure of 1st hierarchical transposition Bloom filter tBF by a preservation | save / restoration process part. It is a flowchart which shows the detailed process sequence of all the preservation | save processes (step S1803) of FIG. It is a flowchart which shows the detailed process sequence of the partial preservation | save process (step S1804) of FIG. It is a flowchart which shows the restoration process procedure by a preservation | save / restoration process part. It is explanatory drawing which shows the restoration example (the 1) in case two or more 2nd transposed Bloom filter row | line | columns tBF (p) s are preserve | saved. It is explanatory drawing which shows the integrated restoration example of 2nd transposed Bloom filter row | line | columns tBF (p) s and tBFa (p) s. It is a flowchart which shows the integrated restoration process procedure by a preservation | save / restoration process part.

  Exemplary embodiments of a management program, a management device, and a management method according to the present invention will be described below in detail with reference to the accompanying drawings. First, a configuration example of the management apparatus according to the present embodiment will be described.

<Hardware configuration example of management device>
FIG. 1 is a block diagram of a hardware configuration example of the management apparatus according to the embodiment. In FIG. 1, the search device includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a magnetic disk drive 104, a magnetic disk 105, and an optical disk drive 106. An optical disk 107, a display 108, an I / F (Interface) 109, a keyboard 110, a mouse 111, a scanner 112, and a printer 113. Each component is connected by a bus 100.

  Here, the CPU 101 controls the entire management apparatus. The ROM 102 stores a program such as a boot program. The RAM 103 is used as a work area for the CPU 101. The magnetic disk drive 104 controls reading / writing of data with respect to the magnetic disk 105 according to the control of the CPU 101. The magnetic disk 105 stores data written under the control of the magnetic disk drive 104.

  The optical disk drive 106 controls reading / writing of data with respect to the optical disk 107 according to the control of the CPU 101. The optical disc 107 stores data written under the control of the optical disc drive 106, and causes the computer to read data stored on the optical disc 107.

  The display 108 displays data such as a document, an image, and function information as well as a cursor, an icon, or a tool box. As this display 108, for example, a CRT, a TFT liquid crystal display, a plasma display, or the like can be adopted.

  An interface (hereinafter abbreviated as “I / F”) 109 is connected to a network 114 such as a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet through a communication line. Connected to other devices. The I / F 109 controls an internal interface with the network 114 and controls data input / output from an external device. For example, a modem or a LAN adapter may be employed as the I / F 109.

  The keyboard 110 includes keys for inputting characters, numbers, various instructions, and the like, and inputs data. Moreover, a touch panel type input pad or a numeric keypad may be used. The mouse 111 performs cursor movement, range selection, window movement, size change, and the like. A trackball or a joystick may be used as long as they have the same function as a pointing device.

  The scanner 112 optically reads an image and takes in the image data into the management apparatus. The scanner 112 may have an OCR (Optical Character Reader) function. The printer 113 prints image data and document data. For example, a laser printer or an ink jet printer can be employed as the printer 113.

<Functional configuration example of management device>
FIG. 2 is a block diagram illustrating a configuration example of the management apparatus according to the present embodiment. The management device 200 includes a data block set db, a hash table group HTs, a hierarchical Bloom filter BF, a hierarchical transposed Bloom filter tBF, a registration processing unit 201, a search processing unit 202, and a save / restore processing unit 203. And.

  The data block set db has a plurality of data blocks, and each data block holds registered data. Each data block is represented as db #. # Is a number indicating the block number. The block number # corresponds to the bit position of the data block db #.

  The hash table group HTs is a set of hash tables corresponding to each data block db # in the data block set db. Each hash table is represented as HT #. # Is a number and matches the block number of the data block db #. The hash table HT # is a table in which a hash value when data is given to a specific hash function is associated with data that is a generation source of the hash value (data itself or a pointer to the data).

  FIG. 3 is an explanatory diagram showing an example of the hash table group HTs. In FIG. 3, SHA-1 is used as the hash function. In addition, what hash function is used may be set in advance.

  Returning to FIG. 2, the hierarchical Bloom filter BF is index information in which the Bloom filter has a hierarchical structure. The hierarchical Bloom filter BF will be described later. The Bloom filter is index information in which a plurality of bits indicating false positives (also called false positives or false positives) / negatives in a predetermined number of data blocks are arranged. When the Bloom filter bit is ON, it indicates false positive, and when it is OFF, it indicates negative. The bit value may be set to 1 and 0 may be set to OFF. Conversely, the bit value may be set to 0 and 1 may be set to OFF. In this embodiment, the bit value is set to 1 and 0 is set to OFF.

  The hierarchical transposed Bloom filter tBF is index information obtained by transposing the hierarchical Bloom filter BF. The hierarchical transposed Bloom filter tBF is generated by the search processing unit 202. Details of the hierarchical transposed blue filter tBF will be described later.

  The data block set db, the hash table group HTs, and the hierarchical Bloom filter BF are stored in a storage device such as the ROM 102, the RAM 103, and the magnetic disk 105 shown in FIG. In FIG. 2, the data block set db, the hash table group HTs, and the hierarchical Bloom filter BF are stored in the management device 200, but may be stored in an external device outside the management device 200. In this case, the management apparatus 200 reads out from or writes to the external apparatus via the network.

  When the data to be registered is entered in the data block set db, the registration processing unit 201 registers in the free area of the data block set db. When registering data, a hash value is obtained by a hash function, and the data (or its pointer) and the hash value are added to the hash table HT # corresponding to the data block db # of the registration destination. In addition, the registration processing unit 201 updates the hierarchical Bloom filter BF so that the hierarchical Bloom filter BF learns that data has been newly registered in the registration destination data block db #.

  When the search target data is input, the search processing unit 202 refers to the hierarchical Bloom filter BF and specifies the data block db # in which the search target data will exist. When the data block db # that may contain the search target data is not specified, the search target data does not exist in any data block db # (negative). On the other hand, even when the data block db # that may contain the search target data is specified, the specified data block db # does not necessarily include the search target data (false positive).

  Whether the false positive becomes positive or negative depends on the search result in the hash table HT # corresponding to the data block db # finally specified by the search processing unit 202. For example, in the hash table HT # corresponding to the data block db # finally specified by the search processing unit 202, it is positive if the hash value of the search target data is hit, and is negative if it is not hit.

  The save / restore processing unit 203 saves and restores the hierarchical Bloom filter BF and the hierarchical transposed Bloom filter tBF described later, details of which will be described later. The storage destination is, for example, a storage device such as the ROM 102, RAM 103, magnetic disk 105, and optical disk 107 shown in FIG. Further, the storage destination may be a storage area in the management device 200 or a storage device outside the management device 200.

  Note that the registration processing unit 201 to the storage / restoration processing unit 203 described above specifically, for example, stores a program stored in a storage device such as the ROM 102, the RAM 103, the magnetic disk 105, and the optical disk 107 illustrated in FIG. This function is realized by executing this function.

<Hierarchical Bloom Filter BF>
FIG. 4 is an explanatory diagram showing an example of the hierarchical Bloom filter BF. The hierarchical Bloom filter BF is configured by a memory area of h stages × s bits width. The s bit width corresponds to the bit width of the data block set db. The bit length s of each stage is divided based on the division number d of the h-th stage that is the uppermost stage. Each of the divided parts is a Bloom filter and forms a Bloom filter row in each stage. The number of divisions d is basically an integer of 2 or more, but d = 1 may be used when the h-th stage as the uppermost stage is a single Bloom filter.

When an arbitrary stage is p, the bit width m of the Bloom filter bf (p) constituting the p-th Bloom filter row BF (p) is m = s / d ^{[h− (p−1)].} . In FIG. 4, d = 2. The number n of arrangement of the Bloom filters bf (p) in the p-th Bloom filter row BF (p) is n = d ^{[h− (p−1)]} .

  Therefore, in the hierarchical Bloom filter BF, the number of arrangements of the Bloom filters bf (p) in the p-th Bloom filter row BF (p) increases as the level decreases (h decreases). Note that the number of arrangement of the bloom filters bf (1) in the lowermost (first) Bloom filter row Bf (1) is the same as the number of data blocks db #.

  As a result, the Bloom filter bf (1) bitted when reaching the first level has a one-to-one correspondence with the data block db #. The number of stages h of the hierarchical Bloom filter BF is basically a plurality of stages, but may be one stage (h = 1). However, in this case, d ≠ 1.

<Example of learning process of hierarchical Bloom filter BF>
FIG. 5 is an explanatory diagram showing an example of learning processing of the hierarchical Bloom filter BF by the registration processing unit 201. In FIG. 5, for the sake of explanation, it is assumed that the total bit width s = 4096 bits, the number of stages h = 3, and the number of divisions d = 2 at the h-th stage.

Accordingly, the first-stage Bloom filter row BF (1), which is the lowermost stage, is divided into 8 (= d ^{[h- (p-1)]} = 2 ³ ) and Bloom filters bf (1-1) to bf ( 1-8). The second-stage Bloom filter row BF (2) is divided into 4 (= d ^{[h- (p-1)]} = 2 ² ) to obtain Bloom filters bf (2-1) to bf (2-4). Consists of. The third-stage Bloom filter row BF (3), which is the uppermost stage, is divided into 2 (= d ^{[h- (p-1)]} = 2 ¹ ) and Bloom filters bf (3-1) to bf ( 3-2).

  Further, the number k of hash functions to which the registered target data D is given is set to k = 3. Here, the hash functions H1 (), H2 (), and H3 () are used. A hash function to be registered in the hash table is H1 ().

First, it is assumed that the target data D is registered in the data block db3 in the data block set db. As an example, the hash values when the target data D is given to the hash functions H1 (), H2 (), and H3 () are as follows.
H1 (D) = 1234567
H2 (D) = 3984012
H3 (D) = 9803323

  In the learning process of the hierarchical Bloom filter BF, a specific bit in the Bloom filter to be updated is turned on, but if the specific bit is already turned on, it is left as it is.

  Here, the registration processing unit 201 creates a hash table entry E3 for the hash table HT3 of the block number 3 of the data block db3 of the registration destination. Then, the registration processing unit 201 additionally registers the created hash table entry E3 in the hash table HT3.

  Next, the registration processing unit 201 specifies a Bloom filter to be updated from the first-stage Bloom filter row BF (1). In the bottom row, the Bloom filter bf (1-3) having the same array number as the block number 3 of the data block db3 of the registration destination corresponds. Therefore, the Bloom filter bf (1-3) is the update target. The Bloom filter bf (1-3) is a 512-bit bit string.

  Also, the registration processing unit 201 divides each hash value by 512 that is the bit width of the first-stage Bloom filter bf (1), and calculates a remainder value. It is assumed that the remainder value of the hash value H1 (D) is “135”, the remainder value of the hash value H2 (D) is “140”, and the remainder value of the hash value H3 (D) is “59”.

  Next, the registration processing unit 201 turns on the bit at the position corresponding to the remainder value in the Bloom filter to be updated. If the remainder value is 0, the last bit of the Bloom filter to be updated is turned ON. In the example of FIG. 5, the Bloom filter bf (1-3) has 512 bits, so the 135th bit from the beginning is turned ON for the remainder value “135”. Similarly, for the remainder value “140”, the 140th bit from the beginning is turned ON. For the remainder value “59”, the 59th bit from the beginning is turned ON. This completes the learning process in the first stage.

  Next, the learning process of the second stage is started. The registration processing unit 201 identifies the Bloom filter to be updated from the second-stage Bloom filter row BF (2). Specifically, the Bloom filter including the bit position of the Bloom filter bf (1-3) to be updated in the first stage is specified from the Bloom filter string BF (2) in the second stage. In this example, the Bloom filter bf (2-2) is obtained. More specifically, the array number 3 of the Bloom filter bf (1-3) to be updated in the first stage of the previous stage is divided by the division number d (= 2), and the fraction is rounded up to round up the update target. The sequence number is 2. Therefore, the Bloom filter bf (2-2) is specified.

  Then, the registration processing unit 201 divides each hash value by 1024 which is the bit width of the second-stage Bloom filter bf (2), and calculates a remainder value. It is assumed that the remainder value of the hash value H1 (D) is “647”, the remainder value of the hash value H2 (D) is “652”, and the remainder value of the hash value H3 (D) is “571”.

  Next, the registration processing unit 201 turns on the bit at the position corresponding to the remainder value in the Bloom filter to be updated. If the remainder value is 0, the last bit of the Bloom filter to be updated is turned ON. In the example of FIG. 5, the Bloom filter bf (2-2) has 1024 bits, so the 647th bit from the beginning is turned ON for the remainder value “647”. Similarly, for the remainder value “652,” the 652nd bit from the beginning is turned ON. For the remainder value “571”, the 571st bit from the beginning is turned ON. Thereby, the learning process in the second stage is completed.

  Next, the process moves to the third-stage learning process, which is the uppermost stage. The registration processing unit 201 specifies the Bloom filter to be updated from the third-stage Bloom filter row BF (3). Specifically, the Bloom filter including the bit position of the Bloom filter bf (2-2) to be updated in the second stage is specified from the third stage Bloom filter string BF (3). In this example, the Bloom filter bf (3-1) is obtained. More specifically, by dividing the array number 2 of the Bloom filter bf (2-2) to be updated in the second stage of the previous stage by the division number d (= 2), the array number to be updated is 1 It becomes. Therefore, the Bloom filter bf (3-1) is specified.

  Then, the registration processing unit 201 divides each hash value by 2048 which is the bit width of the third-stage Bloom filter bf (3) to calculate a remainder value. It is assumed that the remainder value of the hash value H1 (D) is “1671”, the remainder value of the hash value H2 (D) is “652”, and the remainder value of the hash value H3 (D) is “1595”.

  Next, the registration processing unit 201 turns on the bit at the position corresponding to the remainder value in the Bloom filter to be updated. If the remainder value is 0, the last bit of the Bloom filter to be updated is turned ON. In the example of FIG. 5, the Bloom filter bf (3-1) has 2048 bits, so the 1671th bit from the beginning is turned ON for the remainder value “1671”. Similarly, for the remainder value “652,” the 652nd bit from the beginning is turned ON. For the remainder value “1595”, the 1595th bit from the beginning is turned ON. Thereby, the learning process in the third stage is finished.

  With this procedure, the registration processing unit 201 causes the hierarchical Bloom filter BF to learn data entries.

<Example of search processing using hierarchical Bloom filter BF>
FIG. 6 is an explanatory diagram showing an example of search processing of the hierarchical Bloom filter BF by the search processing unit 202. 6 will be described using the same hierarchical Bloom filter BF as in FIG. Note that FIG. 6 illustrates an example in which the target data D registered in FIG. 5 is searched as search target data.

  In the learning process shown in FIG. 5, processing is performed from the first stage at the bottom, but in the search process, processing is performed from the top (the third stage in the case of FIG. 6). First, the search processing unit 202 divides the three hash values for the target data D by the bit widths 2048 of the third-stage Bloom filters bf (3), resulting in residual values “1671”, “652”, “1595” is obtained.

  Next, the search processing unit 202 specifies the Bloom filter to be filtered from the third-stage Bloom filter row BF (3). Since the third stage is the top stage, all the Bloom filters bf (3-1) and bf (3-2) in the third stage are specified unconditionally.

  Then, the search processing unit 202 specifies a Bloom filter in which all the bits at the position corresponding to the calculated remainder value are ON from the Bloom filter group specified as the filtering target. In the case of the third stage, it is assumed that all of the bits at the position corresponding to the calculated remainder value are ON in the Bloom filters bf (3-1) and bf (3-2). Thereby, the filtering process in the third stage is finished.

  Next, the process proceeds to the second-stage filtering process. The search processing unit 202 divides the three hash values of the target data D by the bit width 1024 of each second-stage Bloom filter bf (2), and the remainder values “647”, “652”, “571 "

  Next, the search processing unit 202 specifies the Bloom filter to be filtered from the second-stage Bloom filter row BF (2). In the case of the remaining stages excluding the uppermost stage, the Bloom filter bf (p + 1) in which all the bits in the position corresponding to the calculated remainder value are turned on is searched in the upper stage, and the Bloom filter bf (p + 1). The Bloom filter bf (p) included in the bit positions of is the filtering target.

  In the case of the second stage, the blooms included in the bit positions of the Bloom filters bf (3-1) and bf (3-2) in which all the bits at the positions corresponding to the remainder values calculated in the third stage are ON. Filters bf (2-1) to bf (2-4) are targeted for filtering.

  Then, the search processing unit 202 specifies a Bloom filter in which all the bits at the position corresponding to the calculated remainder value are ON from the Bloom filter group specified as the filtering target. In the case of the second stage, the Bloom filters bf (2-2) and bf (2-3) are assumed to have all the bits at the positions corresponding to the calculated remainder values turned on. On the other hand, for the Bloom filters bf (2-1) and bf (2-4), it is assumed that one of the bits at the position corresponding to the calculated remainder value is OFF.

  Therefore, the lower-stage Bloom filters bf (1-1), bf (1-2), bf (1-7) included in the bit positions of the Bloom filters bf (2-1) and bf (2-4). , Bf (1-8) are excluded from filtering, and the data block db # in which the target data D exists is a data block included in the bit positions of the Bloom filters bf (2-2), bf (2-3). It is narrowed down to db #. Thereby, the filtering process in the second stage is completed.

  Next, the process proceeds to the first-stage filtering process which is the lowest stage. The search processing unit 202 divides the three hash values for the target data D by the bit width 512 of each Bloom filter bf (1) in the first stage, resulting in residual values “135”, “140”, “59 "

  Next, the search processing unit 202 specifies a Bloom filter to be filtered from the first-stage Bloom filter row BF (1). In the case of the first stage, the bloom included in the bit positions of the Bloom filters bf (2-2) and bf (2-3) in which all the bits at the positions corresponding to the remainder values calculated in the second stage are ON. Filters bf (1-3) to bf (1-6) are targeted for filtering.

  Then, the search processing unit 202 specifies a Bloom filter in which all the bits at the position corresponding to the calculated remainder value are ON from the Bloom filter group specified as the filtering target. In the case of the first stage, the Bloom filters bf (1-3) and bf (1-6) are assumed to have all the bits at positions corresponding to the calculated remainder values turned ON. On the other hand, for the Bloom filters bf (1-4) and bf (1-5), it is assumed that one of the bits at the position corresponding to the calculated remainder value is OFF.

  At the bottom level, there are no further lower levels, so the Bloom filters bf (1-3) and bf (1-6) have false positives. The search processing unit 202 determines whether or not the hash value H1 (D) is registered in the hash table HT3 corresponding to the array number “3” of the identified Bloom filter bf (1-3). Since the entry E3 is registered in the hash table HT3, it is found that the target data D is registered in the data block db3 corresponding to the hash table HT3.

  On the other hand, the search processing unit 202 also determines whether or not the hash value H1 (D) is registered in the hash table HT6 corresponding to the array number “6” of the identified Bloom filter bf (1-6). In the hash table HT6, since the hash value H1 (D) = 1234567 is not registered, it is found that the target data D is not registered in the data block db6 corresponding to the hash table HT6. Thereby, the search process is terminated.

  By such a procedure, the search processing unit 202 can specify the data block in which the target data D exists using the hierarchical Bloom filter BF.

  Next, the influence of the false positive of the Bloom filter will be described.

  The Bloom filter false-positive occurrence probability FPR is as follows when the number of data registrations is N (N <m) and the number of hash functions is k when there are h Bloom filters with a bit length of m. It can be expressed as equation (2).

FPR = {1- (1-1 / m ) kN} k ≒ {1-e (-kN / m))} k ... (1)

  In this case, by changing k, m, and N, the false positive occurrence probability FPR can be made very small. That is, in the present embodiment, the false positive occurrence probability FPR can be set to a value (substantially 0) that is much smaller than 1 depending on the setting of k, m, and N. Therefore, in the example of FIG. 5, the Bloom filter bf (1-6) is almost never selected.

In the present embodiment, since the data block number Ndb is d ^h , the height step number h can be expressed by the following equation (2).

  h = log (Ndb) / log (d) +1 (2)

  The above formula (2) is based on the assumption that log (Ndb) / log (d) is divisible, but if not, h can be determined by changing the value of d from other stages depending on the stage. it can.

  Further, in the above-described search process, it is necessary to perform collation as many times as the number of hash values (k times (constant)), and the number of filtering targets per search is at most d. Therefore, the memory access count MA by search is at most about the level expressed by the following equation (3).

  MA = k * d * log (Ndb) / log (d) (3)

  That is, the number of stages h (= memory amount) can be reduced by increasing the number of divisions d, while the number of searches is in a trade-off relationship that increases as the number of divisions d increases. Therefore, by considering this relationship, an appropriate memory operation can be performed.

<Hierarchical transposed Bloom filter>
Next, the hierarchical transposed Bloom filter will be described. In the above description, the registration process and the search process of the hierarchical Bloom filter BF have been described. However, the hierarchical Bloom filter BF is transposed in order to increase the search speed.

  FIG. 7 is an explanatory diagram showing a transposition example of the p-th Bloom filter row bf (p) in the hierarchical Bloom filter BF. (A) shows the Bloom filter row BF (p). Here, the Bloom filter row BF (p) shows Bloom filters bf (p−1) to bf (p−4) divided into four as an example. That is, the Bloom filter string BF (p) is a bit string of 10 bits × 4 filters. In the case of transposition, the bit string is 4 bits × 10 filters.

  (B) shows transposition of the Bloom filter row BF (p). When transposing, the bits at the same position of each of the Bloom filters bf (p-1) to bf (p-4) are collected, and the bit strings collected at the same positions are arranged in the order of the bit positions.

  Specifically, the first bits of each of the Bloom filters bf (p-1) to bf (p-4) are grouped in the order of the array number to form a bit string {0110}. From the left, the first bit “0” is the first bit of the Bloom filter bf (p−1), the second bit “1” is the first bit of the Bloom filter bf (p−2), and the third bit “1” is the Bloom filter The leading bit and trailing bit “0” of bf (p−3) are the leading bit of the Bloom filter bf (p−4).

  This bit string {0110} is referred to as a transposed Bloom filter tbf (p−1). The transposed Bloom filters tbf (p-1) to tbf (p-10) are obtained by collecting the second to last bit positions in the same manner. The index information in which the transposed Bloom filters tbf (p-1) to tbf (p-10) are arranged in the order of bit positions is referred to as a transposed Bloom filter row tBF (p). By generating the transposed Bloom filter row tBF (p) at all stages, the hierarchical transposed Bloom filter tBF is obtained.

  (C) shows a search comparison example between the Bloom filter row BF (p) and the transposed Bloom filter row tBF (p). Here, two hash values of the target data D are obtained by two types of hash functions, and a remainder value obtained by dividing by the bit width 10 of the Bloom filter bf (p) constituting the Bloom filter string BF (p) is “4”. And “8”.

  When searching with the Bloom filter column BF (p), the Bloom filter bf (p) in which the bit positions “4” and “8” having the remainder values “4” and “8” are all ON is selected as the Bloom filter column BF. Search from (p). In this case, the Bloom filter bf (p-2) is applicable.

  On the other hand, when the transposed Bloom filter row tBF (p) is used, the Bloom filter bf (p) in which the bit positions “4” and “8” are all ON, such as the Bloom filter row BF (p), is searched. First, transposed Bloom filters tbf (p-4) and tbf (p-8) having the same sequence numbers as the remainder values “4” and “8” are extracted. Then, an AND operation is performed on the extracted transposed Bloom filters tbf (p−4) and tbf (p−8) to identify the bit position “2” that is both ON.

  In the case of the Bloom filter string BF (p), since the fourth and eighth bits in the four Bloom filters bf (p-1) to bf (p-4) are referred to, 8 (= 4 × 2) Memory access is required. On the other hand, the transposed Bloom filter row tBF (p) is index information folded for each bit position of the Bloom filters bf (p-1) to bf (p-4) before transposition. Therefore, it is possible to make a determination by two memory accesses of extracting the transposed Bloom filters tbf (p-4) and tbf (p-8) and their AND operation. Therefore, the memory access frequency is reduced, and the search speed is increased.

<Example of search processing using hierarchical transposed Bloom filter>
FIG. 8 is an explanatory diagram showing a search process example of the hierarchical transposed Bloom filter by the search processing unit 202. In FIG. 8, for the sake of explanation, the total bit width s = 64 bits, the number of stages h = 3, and the number of divisions d = 2 at the h-th stage.

Since the bit width of the Bloom filter constituting the first-stage Bloom filter column BF (1) which is the lowest stage is 8 (= s / d ^h = 64/2 ³ ) bits, the first stage which is the lowest stage The transposed Bloom filter row tBF (1) includes 8 (= s / d ^h = 64/2 ³ ) transposed Bloom filters tbf (1-1) to tbf (1-8).

Further, since the bit width of the Bloom filter constituting the second-stage transposed Bloom filter string tBF (2) is 16 (= s / d ^h = 64/2 ² ) bits, the second-stage transposed Bloom filter string tBF (2) is composed of 16 (= s / d ^h = 64/2 ² ) transposed Bloom filters tbf (2-1) to tbf (2-16).

The bit width of the Bloom filter constituting the third-stage Bloom filter column BF (3), which is the uppermost stage, is 32 (= s / d ^h = 64/2 ¹ ) bits, so The three-stage transposed Bloom filter row tBF (3) includes 32 (= s / d ^h = 64/2 ¹ ) transposed Bloom filters tbf (3-1) to tbf (3-32).

  In FIG. 8, for the purpose of explanation, the transposed Bloom filter rows tBF (1) to tBF (3) and the pre-transposed Bloom filter rows BF (1) to BF (3) are also shown for comparison.

  First, the search processing unit 202 divides the three hash values of the hash functions H1 () to H3 () for the search target data Dx by the number of transposed Bloom filters 32 in the third stage “2” ”,“ 19 ”,“ 27 ”.

  Next, the search processing unit 202 identifies the transposed Bloom filter to be filtered from the third-stage transposed Bloom filter string tBF (3). Specifically, the transposed Bloom filters tbf (3-2), tbf (3-19), and tbf (3-27) that are in the same bit position as the remainder value (or the end position when the remainder value is 0) are specified. . Then, an AND operation is performed on each bit string {10}, {11}, {10} of the identified transposed Bloom filter tbf (3-2), tbf (3-19), tbf (3-27). The AND result is {10}.

  When “1” is not included in the AND result, the search processing unit 202 determines that the search target data Dx does not exist in the data block set db. On the other hand, when “1” is included in the AND result, the search target data Dx may be registered, and the process moves to the next lower level.

  Also in the second stage, first, the search processing unit 202 divides the three hash values for the search target data Dx by the number of transposed Bloom filters 16 in the second stage, “8” and “11”. , “13” is obtained.

  Next, the search processing unit 202 specifies the transposed Bloom filter to be filtered from the second-stage transposed Bloom filter row tBF (2). Specifically, the transposed Bloom filters tbf (2-8), tbf (2-11), and tbf (2-13) that have the same bit position as the remainder value (or the end position when the remainder value is 0) are specified. . Then, an AND operation is performed on each bit string {0110}, {0100}, {0110} of the identified transposed Bloom filters tbf (2-8), tbf (2-11), and tbf (2-13). The AND result is {0100}.

  When “1” is not included in the AND result, the search processing unit 202 determines that the search target data Dx does not exist in the data block set db. On the other hand, when “1” is included in the AND result, the search target data Dx may be registered, and the process moves to the next lower level.

  Even in the first level, which is the lowest level, first, the search processing unit 202 divides the three hash values for the search target data Dx by the number of transposed Bloom filters of the first level “8”. , “5”, “7” are obtained.

  Next, the search processing unit 202 identifies the transposed Bloom filter to be filtered from the first-stage transposed Bloom filter string tBF (1). Specifically, the transposed Bloom filters tbf (1-2), tbf (1-5), and tbf (1-7) that are in the same bit position as the remainder value (or the end position when the remainder value is 0) are specified. . Then, an AND operation is performed on each bit string {00110110}, {10011010}, and {00110111} of the identified transposed Bloom filters tbf (1-2), tbf (1-5), and tbf (1-7). The AND result is {00010010}.

  Since there is no lower stage, there is a possibility that the search target data Dx exists in the data blocks db4 and db7 corresponding to the bit positions 4 and 7 whose AND result {00010010} is “1” due to false positive.

  In this case, it is assumed that the data block db4 is hit and the data block db7 is not hit by searching the hash tables HT4 and HT7 using the hash value of the hash function H1 () as a key. Thereby, it is found that the search target data Dx is registered in the data block db4. Thereby, the search process is terminated.

  By such a procedure, the search processing unit 202 can search for target data at a higher speed than the hierarchical Bloom filter BF by using the hierarchical transposed Bloom filter.

<Detailed Functional Configuration Example of Search Processing Unit 202>
Next, a detailed functional configuration example of the search processing unit 202 will be described.

  FIG. 9 is a block diagram illustrating a functional configuration example of the search processing unit 202. The search processing unit 202 includes a receiving unit 901, a transposing unit 902, a converting unit 903, a first specifying unit 904, a second specifying unit 905, a determining unit 906, a determining unit 907, and an extracting unit 908. And an output unit 909.

  The receiving unit 901 has a function of receiving a transposition request for the Bloom filter row BF (p). For example, a transposition request from the hierarchical Bloom filter BF to the hierarchical transposed Bloom filter tBF is accepted.

  Here, the transposition request is a request for transposing the Bloom filter string BF (p) to the transposed Bloom filter string tBF (p). For example, the activation completion notification that the management apparatus 200 is activated and the activation is completed is used as the transposition request. Thereby, when the management apparatus 200 is activated, the hierarchical Bloom filter BF is transposed to the hierarchical transposed Bloom filter tBF. In this way, if the entire transposed Bloom filter tBF is transposed at the time of activation, the transposed Bloom filter tBF can be used at all times until shutdown.

  The search request for the data block set db may be a transposition request. Thereby, the search processing unit 202 stands by until a search request is made, and when there is a search request, the Bloom filter string BF (p) used in the search processing unit 202 is transposed among the hierarchical Bloom filters BF. It will be. Thereby, the transposed Bloom filter string tBF (p) increases as the search frequency increases. Therefore, since it is gradually transposed from the Bloom filter row BF (p) necessary for the search, it is possible to reduce useless transposition processing of the Bloom filter row that was not used until the shutdown.

  The transposition unit 902 has a function of transposing the Bloom filter string BF (p) to the transposed Bloom filter string tBF (p) when the transposition request is accepted by the accepting unit 901. Specifically, the transposition unit 902 transposes the Bloom filter row BF (p) to the transposed Bloom filter row tBF (p) by the method shown in FIG.

For example, the Bloom filter row BF (p) is index information in which n (= d ^{[h− (p−1)]} ) Bloom filters bf (p−1) to bf (pn) are arranged. . The bit width of each of the Bloom filters bf (p−1) to bf (p−n) is m (= s / n) bits.

When the Bloom filter row BF (p) is transposed according to FIG. 7, the transposed Bloom filter row tBF (p) has m (= s / n) transposed Bloom filters tbf (p−1) to tbf (p−m). Becomes the index information arranged. The bit width of each transposed Bloom filter tbf (p-1) to tbf (pm) is n (= d ^{[h- (p-1)]} ) bits. In this way, the number of arrays and the bit width are interchanged by transposition.

  The conversion unit 903 has a function of converting the search target data into position information representing the array position of the transposed Bloom filter for each hash function based on a plurality of types (number of types k) of hash functions. Specifically, for example, k hash values are obtained by providing search target data to k different hash functions H1 () to Hk ().

  In the entry to the Bloom filter column BF (p) shown in FIG. 5, the hash value is divided by the bit width m (= s / n) of the Bloom filter constituting the Bloom filter column BF (p), and the remainder is obtained. The corresponding bit position was set to ON.

  Therefore, when k hash values are obtained, the conversion unit 903 divides the hash value by the number m (= s / n) of transposed Bloom filter arrays constituting the transposed Bloom filter sequence tBF (p), Calculate the remainder value. The k remainder values are position information representing the arrangement position of the transposed Bloom filter. Note that the position information when the calculated remainder value is 0 is m.

  The first specifying unit 904 has a function of specifying, for each positional information, a transposed Bloom filter corresponding to the positional information converted by the converting unit 903 from the transposed Bloom filter row tBF (p). Specifically, a transposed Bloom filter whose array number matches the position information obtained from the conversion unit 903 is specified from the transposed Bloom filter row tBF (p).

  In the example illustrated in FIG. 8, in the third-stage transposed Bloom filter row tBF (3), the remainder values (position information) are “2”, “19”, and “27”, so the array element number is “2”. , “19”, “27” are specified as transposed Bloom filters tbf (3-2), tbf (3-19), tbf (3-27).

  The second specifying unit 905 generates the Bloom filter bf (p) corresponding to the position information common to the plurality of transposed Bloom filters tbf (p) specified by the first specifying unit 904, and the Bloom filter row BF (p). It has a function to identify from. Specifically, an AND operation is performed on the bit strings of the plurality of transposed Bloom filters tbf (p) specified by the first specifying unit 904. The bit position where “1” appears by the AND operation is a Bloom filter having a bit that would have been turned ON at the time of entry in the Bloom filter string BF (p) before transposition.

  In the example shown in FIG. 8, when the transposed Bloom filters tbf (3-2), tbf (3-19), and tbf (3-27) are ANDed in the third stage, the leading bit position in the AND result is “1”. " Therefore, it can be seen that any bit of the Bloom filter bf (3-1) in the Bloom filter row BF (3) before transposition may have been turned ON at the time of entry of search target data.

  As described above, the first specifying unit 904 and the second specifying unit 905 reduce the number of accesses to the transposed Bloom filter row tBF (p) than that before the transposed Bloom filter row BF (p). Therefore, as the number of stages p of the transposed Bloom filter tbf (p) increases, the number of accesses is significantly suppressed compared to the pre-transposed Bloom filter string BF (p), and the search speed can be increased. .

  The determination unit 906 has a function of determining whether there is a Bloom filter row having the Bloom filter specified by the second specifying unit 905 as an array element. Specifically, it is only necessary to determine whether or not there is a transposed Bloom filter row one level below the transposed Bloom filter row tBF (p). More specifically, it is only necessary to check whether p = 1.

  When the transposed Bloom filter row of the next lower stage exists (p ≠ 1), the first specifying unit 904 newly converts the transposed Bloom filter row of the lower stage to the transposed Bloom filter row tBF (p). And On the other hand, when there is no lower-stage transposed Bloom filter row (p = 1), the transposed Bloom filter row tBF (p) becomes the first-stage transposed Bloom filter row tBF (1), which is the lowest stage. .

  If the determination unit 906 determines that the data does not exist, the determination unit 907 includes the data block db # corresponding to the Bloom filter bf (p) specified by the second specification unit 905 in the data block set db. It has a function of determining the presence or absence of search target data. When p = 1, since the Bloom filter bf (1) specified by the second specifying unit 905 has a one-to-one correspondence with the data block, the bit that is “1” in the AND result of the second specifying unit 905 The data block db # having the block number # as the position is determined as a determination target. The determination unit 907 determines whether the determination target data block db # is positive or negative and therefore is positive or negative.

  Then, with reference to the hash table HT # of the determination target data block db #, false positive / negative of the search target data is determined. In this case, similarly to the search process using the hierarchical Bloom filter BF, data (or a pointer thereof) from the hash table HT # using a hash value of search target data by a specific hash function (for example, H1 ()) as a key. Determine the presence or absence. If it does not exist, it means that a miss has occurred due to a false positive.

  The extraction unit 908 has a function of extracting search target data from the determination target data block db # when the determination unit 907 determines that the data is positive. Specifically, for example, since the data storage position is determined by the hash table HT # corresponding to the determination target data block, the extraction unit 908 obtains the search target data and the data related to the search target data from the storage position. Extract.

  For example, the search target data itself is extracted from the determination target data block db #. If it can be extracted, it is found that the search target data is registered. If the search target data is a file number, file data related to the file number is extracted. In addition, when the search target data is a dictionary or a term headword, the term explanation data of the headword is extracted.

  The output unit 909 outputs the determination result by the determination unit 907 and the data extracted by the extraction unit 908. For example, the determination result such as positive or negative of the search target data and the extracted data when it is positive are output. Examples of the output format include display on the display 108, audio output, print output, transmission to another apparatus, and the like.

<Learning Processing Procedure of Hierarchical Bloom Filter BF by Registration Processing Unit 201>
FIG. 10 is a flowchart showing the learning processing procedure of the hierarchical Bloom filter BF by the registration processing unit 201. First, the registration processing unit 201 determines whether there is data to be registered (target data D) (step S1001). When there is target data D (step S1001: Yes), the registration processing unit 201 sets the number of stages p to p = 1 (step S1002), and p> h (h is the uppermost stage of the hierarchical Bloom filter BF). Whether or not (step S1003). If p> h is not satisfied (step S1003: NO), the registration processing unit 201 calculates k hash values of the target data using k types of hash functions (step S1004). Then, k hash values are divided by the bit width of the Bloom filter string BF (p) to calculate k remainder values (step S1005).

  Next, the registration processing unit 201 identifies the bloom filter bf (p) r that is the registration destination from the p-th Bloom filter row BF (p) (step S1006). When p = 1, the bloom filter bf (1- #) corresponding to the block number # of the data block db # of the storage destination is set as the bloom filter bf (p) r of the registration destination.

  Specifically, as shown in FIG. 5, when registering the target data D in the data block db3, the first-stage Bloom filter bf (1-3) having the same array number as the block number 3 of the data block db3. ) Is the bloom filter bf (1) r of the registration destination.

  If p ≠ 1, the Bloom filter corresponding to the bit position of the (p−1) -th registered Bloom filter bf (p−1) r from the p-th Bloom filter string BF (p) Let bf (p) be the newly registered Bloom filter bf (p) r.

  Specifically, as shown in FIG. 5, when P = 2, the Bloom filter bf (1) r of the first registration destination is selected from the second-stage Bloom filter row BF (2). The Bloom filter bf (2-2) including the bit position of the Bloom filter bf (1-3) is set as a new registration destination Bloom filter bf (2) r.

  Then, the k remainder values calculated in step S1005 are entered in the registered Bloom filter bf (p) r (step S1007). That is, the bit at the same bit position as the remainder value is turned ON. When the remainder value is 0, the tail bit is turned ON. Then, the stage number p is incremented (step S1008), and the process returns to step S1003.

  If p> h in step S1003 (step S1003: Yes), a hash table entry of the target data is added (step S1009). Specifically, for example, as shown in FIG. 4, the hash table entry E3 is additionally registered in the hash table HT3.

  Then, the process returns to step S1001. In step S1001, if there is no target data D (step S1001: No), the learning process of the hierarchical Bloom filter BF by the registration processing unit 201 is terminated. By such a processing procedure, the hierarchical Bloom filter BF is constructed.

<Search Processing Procedure by Search Processing Unit 202>
FIG. 11 and FIG. 12 are flowcharts showing a search processing procedure by the search processing unit 202. Note that the Bloom filter used by the search processing unit 202 may be a hierarchical transposed Bloom filter tBF that has already been transposed, or a hierarchical Bloom filter BF. When the hierarchical Bloom filter BF is used, a transposition process is performed each time an untransposed Bloom filter is specified.

  That is, the hierarchical transposed Bloom filter tBF is constructed while searching. 11 and 12, the repair procedure when the hierarchical transposed Bloom filter tBF is constructed while searching will be described. In addition, when batch transposition has been completed, it is only necessary to omit the determination of whether transposition has been performed (step S1104) and the transposition processing (step S1105).

  First, in FIG. 11, the search processing unit 202 waits for the search target data Dx by the receiving unit 901 (step S1101: No), and when the search target data Dx is received (step S1101: Yes), the search processing unit 202. The conversion unit 903 gives the search object data Dx to k types of hash functions to calculate k hash values (step S1102).

  Then, the search processing unit 202 sets p = h, that is, sets the number of stages p to the maximum number of stages h (step S1103), and determines whether or not the p-th Bloom filter string BF (p) has been transposed. (Step S1104). When it has been transposed (step S1104: Yes), the process proceeds to step S1106. On the other hand, if it is not transposed (step S1104: No), the search processing unit 202 transposes the p-th Bloom filter row BF (p) by the transposing unit 902 (step S1105), and proceeds to step S1106. .

  In step S1106, the search processing unit 202 uses the conversion unit 903 to divide k hash values by the number of arrays of transposed Bloom filters to calculate k remainder values (step S1106). Then, the search processing unit 202 uses the first specifying unit 904 to extract k transposed Bloom filters tbf (p) r corresponding to k remainder values from the p-th transposed Bloom filter sequence tBF (p). Specify (step S1107).

  Then, the search processing unit 202 performs an AND operation on the k transposed Bloom filters tbf (p) r by the second specifying unit 905 (step S1108), and proceeds to step S1201 in FIG.

  In FIG. 12, the search processing unit 202 uses the second specifying unit 905 to set the first bit of the AND result as the target bit (step S1201), and determines whether the target bit is ON (step S1202). . If not ON, the search processing unit 202 uses the determination unit 907 to determine whether the target bit can be shifted (step S1203). Specifically, it is determined whether or not the target bit is the tail bit.

  When the shift is possible (step S1203: Yes), the search processing unit 202 shifts the target bit toward the end of one bit (step S1204), and returns to step S1202. On the other hand, if the shift is not possible in step S1203 (step S1203: No), the search processing unit 202 determines the search result (negative) by the determination unit 907 and outputs it from the output unit 909 (step S1205). Thereby, the processing procedure when the search result is negative is terminated.

  On the other hand, if the target bit is ON in step S1202 (step S1202: Yes), the search processing unit 202 determines whether the current stage number p is p = 1 by the determination unit 906 (step S1202). S1206). When p is not 1 (step S1206: No), p is decremented (step S1207), and the process returns to step S1104.

  On the other hand, if p = 1 (step S1206: Yes), the search processing unit 202 uses the determination unit 907 to search for a hash table corresponding to the bit position of the target bit (step S1208). Then, it is determined whether or not the search target data Dx exists (step S1209).

  When the search target data does not exist (step S1209: No), the process returns to step S1203 to determine whether the target bit can be shifted. On the other hand, when search target data exists (step S1209: Yes), the search processing unit 202 outputs a search result (positive) (step S1210). The search processing unit 202 extracts related data by the extraction unit 908 as necessary and outputs it as a search result. Thereby, the processing procedure when the search result is negative is terminated.

  As described above, according to the present embodiment, transposing the Bloom filter string BF (p) to the transposed Bloom filter string tBF (p) reduces the memory access and increases the search speed. There is an effect. In particular, by using the hierarchical transposed Bloom filter tBF, memory access at each stage is reduced, so that a search can be performed at a higher speed.

  Moreover, when the management apparatus 200 is activated by using the activation completion notification as a transposition request, the hierarchical Bloom filter BF is transposed to the hierarchical transposed Bloom filter tBF. In this way, if the transposition to the hierarchical transposed Bloom filter tBF is collectively performed at the time of startup, the hierarchical transposed Bloom filter tBF can be used at all times until the shutdown.

  Also, by making the search request for the data block set db a transposition request, when there is a search request, the Bloom filter string BF (p) used in the search processing unit 202 in the hierarchical Bloom filter BF is transposed. . Thereby, the transposed Bloom filter string tBF (p) increases as the search frequency increases. Therefore, since it is gradually transposed from the Bloom filter row BF (p) necessary for the search, it is possible to reduce useless transposition processing of the Bloom filter row that was not used until the shutdown.

  Further, in the case of the hierarchical transposed Bloom filter tBF transposed from the hierarchical Bloom filter BF, until the bottom level is reached, whether the AND result is ON or not is determined whether it is false positive or negative, and the AND result is If there is even one ON bit, it is determined as a false positive and the process proceeds to the next lower stage. Thus, by using the AND result, it can be easily determined whether or not the result is negative, and it can be identified early that the search target data does not exist.

<Example of learning process of hierarchical transposed Bloom filter tBF>
FIG. 13 is an explanatory diagram illustrating a learning process example of the hierarchical transposed Bloom filter tBF by the registration processing unit 201. Here, the hierarchical transposed Bloom filter tBF shown in FIG. 8 will be described as an example. Further, FIG. 8 illustrates an example in which the search target data Dx is searched and hit, but in FIG. 13, the search target data Dx (referred to as target data Dx in FIG. 13) is still registered in the data block set db. It will be described as not being done.

  Further, the number k of hash functions to which the registered target data D is given is set to k = 3. Here, the hash functions H1 (), H2 (), and H3 () are used. A hash function to be registered in the hash table is H1 ().

First, it is assumed that the target data Dx is registered in the data block db4 in the data block set db. As an example, the hash value when the target data Dx is given to each hash function H1 (), H2 (), H3 () is as follows.
H1 (Dx) = x1
H2 (Dx) = x2
H3 (Dx) = x3

  Further, in the learning process of the hierarchical transposed Bloom filter tBF, a specific bit in the transposed Bloom filter tbf (p) to be updated is turned ON, but if the specific bit is already ON, it is left as it is. To do.

  Here, the registration processing unit 201 creates a hash table entry E4 for the hash table HT4 of the block number 4 of the registration destination data block db4. Then, the registration processing unit 201 additionally registers the created hash table entry E4 in the hash table HT4.

  Then, the process proceeds to the first stage learning process. The registration processing unit 201 identifies the transposed Bloom filter tbf (1) to be updated from the first-stage transposed Bloom filter row tBF (1). Specifically, the registration processing unit 201 divides each hash value x1 to x3 by 8, which is the number of arrays of the first-stage transposed Bloom filter string tBF (1), and calculates a remainder value. It is assumed that the remainder value of the hash value x1 is “2”, the remainder value of the hash value x2 is “5”, and the remainder value of the hash value x3 is “7”. Therefore, the transposed Bloom filter tbf (1) to be updated in the first stage is the transposed Bloom filter tbf (1-2), tbf (1-5), tbf (1-7).

  In the lowest level, the bit position corresponding to the block number 4 of the registration-destination data block db4 is set as the update target bit. Therefore, the fourth bit from the beginning of the transposed Bloom filter to be updated is turned ON. Thereby, the learning process of the first-stage transposed Bloom filter row tBF (1) is completed.

  Next, the learning process of the second stage is started. The registration processing unit 201 identifies the transposed Bloom filter tbf (2) to be updated from the second-stage transposed Bloom filter row tBF (2). Specifically, the registration processing unit 201 divides each hash value x1 to x3 by 16, which is the number of arrays of the second-stage transposed Bloom filter string tBF (2), and calculates a remainder value. It is assumed that the remainder value of the hash value x1 is “8”, the remainder value of the hash value x2 is “11”, and the remainder value of the hash value x3 is “13”. Therefore, the transposed Bloom filter tbf (2) to be updated in the second stage is the transposed Bloom filter tbf (2-8), tbf (2-11), tbf (2-13).

Next, the bit position of the bit in the transposed Bloom filter tbf (2-8), tbf (2-11), tbf (2-13) will be described. In the hierarchical Bloom filter BF before transposition, the number of divisions is d, and each Bloom filter row BF (p) is divided into n (= d ^{[h− (p−1)]} ). As a result, the bit width of each Bloom filter column BF (p) is m (= s / n) bits.

  For this reason, in the learning process of the hierarchical Bloom filter BF, the Bloom filter bf (p) including the bit position of the Bloom filter bf ((p-1)-#) to be updated in the (p-1) -th stage is used. The p-th Bloom filter row BF (p) is specified.

  For example, in the example of FIG. 5, in the second stage, the array number 3 of the Bloom filter bf (1-3) to be updated in the first stage of the previous stage is divided by the division number d (= 2), and the fraction is calculated. By rounding up, the array number to be updated becomes 2. Therefore, the Bloom filter bf (2-2) is specified.

  On the other hand, in the hierarchical transposed Bloom filter tBF, the array number n and the bit width m are interchanged, so the array of Bloom filters bf ((p-1)-#) to be updated in the (p-1) stage. Instead of the number #, the bit position to be updated at the (p−1) -th stage is divided by the division number d, and the fraction is rounded up.

  In the case of the second stage, the update target bit in the first stage is the fourth bit from the head, and the transposed Bloom filters tbf (1-2), tbf (1-5), tbf (1-7) The 4th bit was turned ON. Therefore, since the update target bit in the second stage is d = 2, the bit of 4 / d = 2 bit from the head is set as the update target bit. In this example, the second bit from the beginning of the transposed Bloom filters tbf (2-8), tbf (2-11), and tbf (2-13) is turned ON. Thereby, the learning process of the second-stage transposed Bloom filter row tBF (2) is completed.

  Next, the process moves to the third stage learning process. The registration processing unit 201 identifies the transposed Bloom filter tbf (3) to be updated from the third-stage transposed Bloom filter row tBF (3). Specifically, the registration processing unit 201 divides each hash value x1 to x3 by 32, which is the number of arrays of the third-stage transposed Bloom filter string tBF (3), and calculates a remainder value. It is assumed that the remainder value of the hash value x1 is “2”, the remainder value of the hash value x2 is “19”, and the remainder value of the hash value x3 is “27”. Therefore, the transposed Bloom filter tbf (3) to be updated in the third stage is the transposed Bloom filter tbf (3-2), tbf (3-19), tbf (3-27).

  Next, the update target bits in the transposed Bloom filters tbf (3-2), tbf (3-19), and tbf (3-27) are determined. Similar to the second stage, not the array number # of the Bloom filter bf ((p-1)-#) to be updated in the (p-1) stage, but the bit position to be updated in the (p-1) stage. Divide by the division number d and round up the fraction.

  In the case of the third stage, the update target bit in the second stage, which is the previous stage, is the second bit from the beginning, and is a transposed Bloom filter tbf (2-8), tbf (2-11), tbf (2- The second bit of 13) was turned ON. Accordingly, since the update target bit in the third stage is d = 2, the bit of 2 / d = 1 bit from the head is set as the update target bit. In this example, the first bit of the transposed Bloom filter tbf (3-2), tbf (3-19), tbf (3-27) is turned ON. Thereby, the learning process of the third-stage transposed Bloom filter row tBF (3) is completed.

<Learning Processing Procedure of Hierarchical Transposed Bloom Filter tBF by Registration Processing Unit 201>
FIG. 14 is a flowchart showing the learning processing procedure of the hierarchical Bloom filter BF by the registration processing unit 201. First, the registration processing unit 201 determines whether there is data to be registered (target data Dx) (step S1401). When there is target data Dx (step S1401: Yes), the registration processing unit 201 sets the number of stages p to p = 1 (step S1402), and p> h (h is the uppermost stage of the hierarchical transposed Bloom filter tBF). It is determined whether or not there is (step S1403). If p> h is not satisfied (step S1403: NO), the registration processing unit 201 calculates k hash values of the target data Dx using k types of hash functions (step S1404).

  Next, the registration processing unit 201 divides the k hash values by the number of arrays of the p-th transposed Bloom filter tBF (p) to calculate k remainder values (step S1405). Then, the registration processing unit 201 identifies k transposed Bloom filters tbf (p) r having the same array number as the k remainder values (step S1406).

  Thereafter, it is determined whether or not p = 1 (step S1407). If p = 1 (step S1407: Yes), the registration processing unit 201 determines k transposed Bloom filters tbf (p ) In r, the block number # of the data block db # to which the target data Dx belongs is entered (step S1408). That is, the block number # of the data block db # to which the target data Dx belongs is set as the update target bit position #, and the bit at the update target bit position # of the identified k transposed Bloom filters tbf (p) r is turned ON. To do. Then, control goes to a step S1410.

  On the other hand, if p ≠ 1 in step S1407 (step S1407: No), the k-th transposed Bloom filter tbf (p) r is updated in the (p−1) th stage of the target data Dx. A quotient (rounded up) obtained by dividing the bit position by the division number d is entered (step S1409). That is, the quotient (rounded up) obtained by dividing the update target bit position in the (p−1) -th stage of the target data Dx by the division number d is set as the update target bit position, and the identified k transposed Bloom filters tbf (P) The bit at the update target bit position of r is turned ON. Then, control goes to a step S1410.

  In step S1410, the registration processing unit 201 increments the number of stages p (step S1410) and returns to step S1403. Thereby, the update target bit can be turned ON from the lowest level to the highest level.

  On the other hand, in step S1403, when p> h (step S1403: Yes), the registration processing unit 201 adds a hash table entry of the target data (step S1411), and returns to step S1401. If there is no target data Dx (step S1401: No), the learning process of the hierarchical transposed Bloom filter tBF by the registration processing unit 201 is terminated.

  By such a procedure, the registration processing unit 201 causes the hierarchical transposed Bloom filter tBF to learn the data entry. That is, even when data is registered after transposition, it is not necessary to return from the hierarchical transposed Bloom filter tBF to the hierarchical Bloom filter BF, and learning can be performed with the transposed hierarchical transposed Bloom filter tBF. Therefore, useless processing such as returning before transposition is eliminated, and search efficiency can be improved.

<Retransposition and storage of transposed Bloom filter row tBF (p)>
Next, re-transposition and storage of the transposed Bloom filter row tBF (p) will be described. The transposition and storage of the transposed Bloom filter row tBF (p) is performed by the storage / restoration processing unit 203 shown in FIG. Although the transposed Bloom filter string tBF (p) can be stored as it is, it is desirable that the index information can be stored in the storage device in consideration of the time from the restart of the management apparatus 200 to the start of the service. When the transposed Bloom filter row tBF (p) is stored in the memory arrangement, there is no locality in updating, and partial updating cannot be performed.

  Here, locality means that the target bit group is gathered in one filter. If the target bit group spans multiple filters, there is no locality.

  That is, in each transposed Bloom filter tbf (p) in the transposed Bloom filter row tBF (p), the arrangement number n and the bit width m are interchanged. Therefore, the bit arrangement in the transposed Bloom filter tbf (p) becomes the arrangement of the Bloom filter bf (p) before transposition, and the arrangement of the transposed Bloom filter tbf (p) becomes in the Bloom filter bf (p) before transposition. It becomes a bit array. For this reason, each bit in the transposed Bloom filter tbf (p) straddles all the Bloom filters bf (p) before transposition, thereby preventing a partial update.

Therefore, in order to improve the search speed by the transposed Bloom filter string tBF (p) and to ensure the locality at the time of storage, each transposed Bloom filter string tbf (p) in the transposed Bloom filter string tBF (p) is set to a predetermined word. (Bit width) to divide divided bit strings at the same word position. The bit width w of the word is determined by the division number c. For example, the bit width n (n = d ^{[h− (p−1)]} ) of the transposed Bloom filter tbf (p) is divided by the division number c (w = n / c).

  As a result, for each word position, a plurality of Bloom filters bf (p) before transposition are re-transposed into Bloom filters in which the number of divisions c is collected. For example, when n = 15 bits, c = 3 or c = 5. When c = 3, each transposed Bloom filter tbf (p) is divided into three equal parts. When c = 5, the transposed Bloom filter tbf (p) is divided into five equal parts. The division number c is set in advance.

  The division number c is a divisor of n (except c = 1 and n). In the case of c = 1, it is excluded because it is not divided. In the case of c = n, it is returned to the Bloom filter row BF (p) before transposition. In this case, access is required for the total number of bits, and it is excluded because re-transposition and storage processing take time. This will be described below with reference to the drawings.

<Re-transposition and storage example of transposed Bloom filter row tBF (p)>
FIG. 15 is an explanatory diagram illustrating an example of re-transposition and storage of the transposed Bloom filter row tBF (p). In FIG. 15, the Bloom filter row BF (p) and the transposed Bloom filter row tBF (p) shown in FIG. 7 will be described as an example. In FIG. 15, it is assumed that the bit width n of the transposed Bloom filter tbf (p) is n = 4, the division number c is c = 2, and the word bit width w = 4/2 = 2.

  (A) is the transposition example (1st transposition) shown in FIG. Description of (A) is the same as FIG. (B) shows re-transposition (second transposition). Thereafter, the transposed Bloom filter row tBF (p) by the first transposition is “first transposed Bloom filter row tBF (p)”, and the transposed Bloom filter tbf (p) is “first transposed Bloom filter tbf (p)”. Is written. Further, the hierarchical transposed Bloom filter tBF composed of the first transposed Bloom filter rows tBF (1) to tBF (h) from the first stage to the hth stage is expressed as “first hierarchical transposed Bloom filter tBF”. To do.

  In addition, the Bloom filter row transposed by the second transposition from the first transposed Bloom filter row tBF (p) is referred to as “second transposed Bloom filter row tBF (p) s”, and the second transposed Bloom filter row tBF ( p) The Bloom filter of the division number c constituting s is expressed as “second transposed Bloom filter tbf (p−1) s to tbf (p−c) s”.

  An arbitrary second transposed Bloom filter is denoted as “tbf (p) s”. Further, the hierarchical transposed Bloom filter composed of the first to h-th second transposed Bloom filter rows tBF (1) s to tBF (h) s is referred to as “second hierarchical transposed Bloom filter tBFs”. write.

  In the case of (B), since the division number c = 2, the second transposed Bloom filter row tBF (p) s has c = 2 transposed Bloom filters tbf (p−1) s, tbf (p−2). ) S.

  In the second transposition, the first transposed Bloom filter tbf (p) is divided into c pieces with the word bit width w. In the example of (B), since the first transposed Bloom filter tbf (p) has a bit width of 4 bits, a 2-bit word is divided into two. In this example, the first word of the first transposed Bloom filter tbf (p) is the first word, and the next word is the second word. The first word is a bit string of bit numbers 1 and 2, and the second word is a bit string of bit numbers 3 and 4.

  Then, the first words of the first transposed Bloom filter tbf (p) are grouped in the order of arrangement of the first transposed Bloom filter tbf (p) to be a second transposed Bloom filter tbf (p−1) s.

  Similarly, the second words of the first transposed Bloom filter tbf (p) are grouped in the order of arrangement of the first transposed Bloom filter tbf (p) to be the second transposed Bloom filter tbf (p−2) s. . Then, the second transposed Bloom filter string tbf (p−1) s and tbf (p−2) s are arranged in the order of word arrangement to be converted into the second transposed Bloom filter string tBF (p) s.

  By transposing in this way, the second transposed Bloom filter tbf (p) s collects the same word, so that bit numbers indicating bit positions are consecutive. When the bit positions are close, the data block bd # is also close.

  As described above, the second transposed Bloom filter tbf (p) s has a bit arrangement in which a plurality of adjacent Bloom filters bf (p) (for the number of divisions c) are collected, and thus locality can be maintained.

  Next, the restoration from the second transposed Bloom filter row tBF (p) s to the first transposed Bloom filter row tBF (p) will be described. In the second transposed Bloom filter row tBF (p) s, the second transposed Bloom filter tbf (p) s corresponding to the division number c is arranged in the order of arrangement in the first transposed Bloom filter tbf (p).

  The second transposed Bloom filter tbf (p−1) s is the bit at the same bit position of the Bloom filters bf (p−1) and bf (p−2) before the first transposition, and the Bloom filter bf (p− The index information is arranged in the order of 1) and bf (p-2). Similarly, the second transposed Bloom filter tbf (p-2) s is a bit at the same bit position of the Bloom filters bf (p-3) and bf (p-4) before the first transposition. The index information is arranged in the order of (p-3) and bf (p-4).

  In the case of FIG. 15, since the second transposed Bloom filter tbf (p-1) s is a head filter, the first division position of the first transposed Bloom filters tbf (p-1) to tbf (p-10). Are arranged in the order of arrangement of the first transposed Bloom filters tbf (p-1) to tbf (p-10).

  Therefore, when restoring, a memory area corresponding to the bit width of the second transposed Bloom filter row tBF (p) s is secured. In this memory area, the first transposed Bloom filters tbf (p-1) to tbf (p-10) are restored in order to restore the first transposed Bloom filters tbf (p-1) to tbf (p-10). An area is set for each. Then, the second transposed Bloom filter tbf (p) s is divided by the number of arrangement of the first transposed Bloom filter tbf (p) for each bit width corresponding to the division number c.

  In FIG. 16, for example, in the case of the second transposed Bloom filter row tBF (p−1) s, dividing by the bit width w (= 2) of the word results in 10 divisions. Then, the bit string having the bit width of each word is written in an area prepared for each of the first transposed Bloom filters tbf (p−1) to tbf (p−10). Specifically, the first word {01} of the second transposed Bloom filter tbf (p−1) s is written in the restoration area of the first transposed Bloom filter tbf (p−1).

  The second word {00} of the second transposed Bloom filter tbf (p-1) s is written into the restoration area of the first transposed Bloom filter tbf (p-2). Thus, the sequential writing is performed, and the tenth word {10} of the second transposed Bloom filter tbf (p-1) s is written in the restoration area of the first transposed Bloom filter tbf (p-10). . The second transposed Bloom filter tbf (p-2) s is similarly written. In this case, the arrangement order is maintained by writing to a bit string that has already been written.

  By transposing in this way, the second transposed Bloom filter row tBF (p) s is restored to the first transposed Bloom filter row tBF (p).

  Since restoration returns to the first transposed Bloom filter row tBF (p), it is possible to maintain an increase in search speed. The second transposition shown in FIG. 15 is executed by the first to h-th transposed Bloom filter rows tBF (1) to tBF (h), so that the first hierarchical transposed Bloom filter tBF is obtained. Transposed to the second hierarchical transposed Bloom filter tBFs.

<Update example of second transposed Bloom filter row tBF (p) s>
Next, a case where data is newly registered in the data block set db or registered data is updated (hereinafter collectively referred to as “update”) will be described. When the data is updated, as shown in FIGS. 13 and 14, a part of the first hierarchical transposed Bloom filter tBF is updated.

  In the example of FIG. 13, in the first stage, the fourth bit of the first transposed Bloom filter tbf (1-2), tbf (1-5), tbf (1-7) is updated. In the second stage, the second bit of the first transposed Bloom filter tbf (2-8), tbf (2-11), tbf (2-13) is updated. In the third stage, the first bit of the first transposed Bloom filter tbf (3-2), tbf (3-19), tbf (3-27) is updated.

  Thus, when the first transposed Bloom filter tbf (p) is updated, the bit at the same bit position in the stage p (the first stage is the fourth bit, the second stage is the second bit, the third bit The first bit in the stage is updated. In this case, instead of transposing the first transposed Bloom filter row tBF (p) and generating the updated second transposed Bloom filter tBF (p) s, the first transposed Bloom filter tbf (p ) To update the second transposed Bloom filter tbf (p) s including the bits involved in the update.

  Therefore, the transposition to the second hierarchical transposed Bloom filter tBFs may be performed only once, and thereafter, if the second transposed Bloom filter tbf (p) s corresponding to the update is partially updated. Good. In this way, by partially updating, the performance degradation of the management device 200 can be suppressed.

  FIG. 16 is an explanatory diagram illustrating an update example of the second transposed Bloom filter row tBF (p) s. FIG. 16 will be described using the first transposed Bloom filter row tBF (p) and the second transposed Bloom filter row tBF (p) s shown in FIG.

  When k types of hash functions (here, k = 3) are used in the learning process according to FIGS. 13 and 14, k first transposed Bloom filters are used in the first transposed Bloom filter string tBF (p). tBf (p) is specified. In FIG. 16, it is assumed that the first transposed Bloom filters tbf (p-1), tbf (p-4), and tbf (p-9) are specified.

  Then, in the identified first transposed Bloom filters tbf (p−1), tbf (p−4), and tbf (p−9), the bit at the same bit position is updated and turned ON. In FIG. 16, in each of the identified first transposed Bloom filters tbf (p-1), tbf (p-4), and tbf (p-9), the third bit from the beginning is turned ON. To do.

  When the above update is seen in the Bloom filter row BF (p), it is the same as that the first, fourth, and ninth bits from the top of the Bloom filter bf (p-3) at the third array position are updated. It is.

  When the first transposed Bloom filter row tBF (p) is updated, the second transposed Bloom filter tbf (p) s is specified from the updated bit position. Specifically, the second transposed Bloom filter tbf (p) s including the updated bit position is specified with reference to the bit position table BT.

  In the example of FIG. 16, the third bit from the beginning is turned ON in each of the identified first transposed Bloom filters tbf (p-1), tbf (p-4), and tbf (p-9). Yes. Therefore, the second transposed Bloom filter tbf (p−2) s including the bit position “3” is specified.

  Thereafter, another data is updated, and the fourth bit of the first transposed Bloom filter tbf (p-3), tbf (p-8), tbf (p-10) is specified. . Similarly, in this case, the fourth bit from the head is updated to ON in each of the specified first transposed Bloom filters tbf (p-3), tbf (p-8), and tbf (p-10). The Therefore, the second transposed Bloom filter tbf (p−2) s including the bit position “4” is specified.

  As described above, in the two data updates described above, a part of the first transposed Bloom filters tbf (p−1) to tbf (p−10) constituting the first transposed Bloom filter row tBF (p). The third and fourth bits are updated. Therefore, the second transposed Bloom filter tbf (p) s to be updated is the second transposed Bloom filter tbf (p-2) s.

  Thereby, the second transposed Bloom filter tbf (p-2) s is updated. On the other hand, the second transposed Bloom filter tbf (p−1) s is not updated at this time, and thus does not need to be updated. In this way, the update process can be completed in a short time by partially updating only the updated filter.

<Detailed Functional Configuration Example of Save / Restore Processing Unit 203>
Next, a detailed functional configuration example of the save / restore processing unit 203 will be described.

  FIG. 17 is a block diagram illustrating a detailed functional configuration example of the save / restore processing unit 203 illustrated in FIG. In FIG. 17, the storage / restoration processing unit 203 includes a reception unit 1701, a transposition unit 1703, a storage unit 1704, an update detection unit 1705, and a specification unit 1706.

  The accepting unit 1701 has a function of accepting a storage request and a restoration request for the first transposed Bloom filter row tBF (p). Specifically, for example, a storage request or a restoration request is accepted by a user operation input. In addition, when the management apparatus 200 is shut down, the storage request for the first transposed Bloom filter row may be requested. Furthermore, a restoration request for the first transposed Bloom filter string tBF (p) may be made when the activation of the management apparatus 200 is completed.

  When the storage unit 1701 receives a storage request, the dividing unit 1702 divides the bit string of each first transposed Bloom filter tbf (p) by a division number c that is a divisor of n, thereby obtaining n / c bits. Has the function of generating c words. Specifically, for example, in the example of FIG. 15, since n = 4 and c = 2, the first transposed Bloom filter tbf (p) having the bit width n = 4 is extracted from the bit string at the bit positions 1 and 2. And a second word consisting of a bit string at bit positions 3 and 4.

  Since the dividing unit 1702 performs division for each first transposed Bloom filter tbf (p), the total number of words is c × m (m is the number of arrays of the first transposed Bloom filter tbf (p)). In the case of FIG. 15, since m = 10, the total number of words in the first transposed Bloom filter row tBF (p) is 20.

  In addition, when the restoration instruction is received, the dividing unit 1702 divides the second transposed Bloom filter string tBF (p) s by the division number c, thereby obtaining each second transposed Bloom filter tbf (p) s. Next, m words of n / c bits are generated. Specifically, for example, in the example of FIG. 15, in the second transposed Bloom filter tbf (p−1) s, n / c = 4/2 = 2 bits are divided and m = 10 words. Is generated. Similarly, the second transposed Bloom filter tbf (p−2) s also divides n / c = 4/2 = 2 bit units to generate m = 10 words.

  The transposition unit 1703 has a function of transposing the first hierarchical transposed Bloom filter tBF to the second hierarchical transposed Bloom filter tBFs. Specifically, for example, when a storage request is accepted, the first hierarchical transposed Bloom filter tBF is transposed to the second hierarchical transposed Bloom filter tBFs. This transposition corresponds to the first transposition shown in FIG.

  In the first transposition shown in FIG. 15, the transposing unit 1703 arranges the words at the same position in the order of arrangement of the first transposed Bloom filter tbf (p) to obtain the second transposed Bloom filter tbf (p) s. At the time of transposition, the transposition destination of the word is written in the bit position table. For example, in the example of FIG. 15, the second transposed Bloom filter tbf (p-1) s is used for the first word, and the second transposed Bloom filter tbf (p-2) s is used for the second word. .

  Then, the first words of the first transposed Bloom filters tbf (p-1) to tbf (p-10) are arranged in the order of the first transposed Bloom filters tbf (p-1) to tbf (p-10). , A second transposed Bloom filter tbf (p−1) s.

  Similarly, the second word of each first transposed Bloom filter tbf (p-1) to tbf (p-10) is replaced with the first transposed Bloom filter tbf (p-1) to tbf (p-10) in this order. These are arranged as a second transposed Bloom filter tbf (p-2) s.

  The transposition unit 1703 has a function of transposing the second hierarchical transposed Bloom filter tBFs to the first hierarchical transposed Bloom filter tBF. Specifically, for example, when a restoration request is accepted, the second hierarchical Bloom filter tBFs stored in the storage device is transposed to the first hierarchical transposed Bloom filter tBF. Thereby, the first hierarchical transposed Bloom filter tBF is restored.

  Specifically, for example, the second transposed Bloom filter tbf (p) between the m words divided by the dividing unit 1702 for each second transposed Bloom filter tbf (p) s and having the same arrangement position. These are arranged in the order of the arrangement of s. In the example of FIG. 15, the dividing unit 1702 generates 10 words for each second transposed Bloom filter tbf (p) s. The words having the same arrangement position of the ten words, that is, the first word, the words at the second arrangement position,..., The words at the arrangement position at the end position (right end) are collected. At the time of grouping, the second transposed Bloom filter tbf (p) s is arranged in the order of arrangement. As a result, the first transposed Bloom filter tbf (p) is restored.

  The storage unit 1704 has a function of storing the second hierarchical Bloom filter tBFs transposed by the transposition unit 1703 in a storage device. Specifically, for example, as shown in FIG. 15, the second hierarchical transposed Bloom filter tBFs in which the second hierarchical Bloom filter tBFs has a hierarchical structure is stored in the storage device. If there is an update, the updated second transposed Bloom filter tbf (p) s is overwritten and saved.

  The update detection unit 1705 has a function of detecting an update bit position of the updated first transposed Bloom filter tbf (p) from the first transposed Bloom filter string tBF (p). Specifically, for example, as shown in FIG. 16, the third bit of the first transposed Bloom filter tbf (p-1), tbf (p-4), tbf (p-9) is updated. If it does, bit position “3” is detected.

  The specifying unit 1706 has a function of specifying, from the first transposed Bloom filter string tBF (p), a word group having the same arrangement position as the word including the update bit position detected by the update detection unit 1705. Specifically, for example, in the example of FIG. 16, the bit numbers corresponding to the first word are 1 and 2, and the bit numbers corresponding to the second word are 3 and 4. Therefore, a word whose update bit position includes “3” can be specified as the second word.

  Thereby, the transposing unit 1703 collects all the second words from the first transposed Bloom filter row tBF (p) and arranges them in the order of arrangement in the first transposed Bloom filter tbf (p). A second transposed Bloom filter tbf (p) s is generated. Thereafter, the updated second transposed Bloom filter tbf (p) s is stored by the storage unit 1704.

<Saving Process of First Hierarchical Transposed Bloom Filter tBF by Saving / Restoring Processing Unit 203>
FIG. 18 is a flowchart showing a storage processing procedure of the first hierarchical transposed Bloom filter tBF by the storage / restoration processing unit 203.

  First, the save / restore processing unit 203 waits for a save instruction by the accepting unit 1701 (step S1801: No). When the save instruction is accepted (step S1801: Yes), the save / restore process unit 203 has already saved. It is determined whether it has been completed (step S1802). If it has not been saved (step S1802: No), since it corresponds to a new save, the save / restore processing unit 203 executes a full save process for the first hierarchical transposed Bloom filter tBF (step S1803). The all storage process is a process for collectively storing all the first transposed Bloom filter strings tBF (p) constituting the first hierarchical transposed Bloom filter tBF. Thereby, the storage process of the first hierarchical transposed Bloom filter tBF by the storage / restoration processing unit 203 ends.

  On the other hand, if the data has been saved in step S1802 (step S1802: Yes), the save / restore processing unit 203 executes a partial save process (step S1804). The partial saving process is a process of overwriting and saving the second transposed Bloom filter tbf (p) s to be updated among the saved second hierarchical transposed Bloom filter tBFs. Thereby, the storage process of the first hierarchical transposed Bloom filter tBF by the storage / restoration processing unit 203 ends.

<All save processing procedure>
FIG. 19 is a flowchart showing a detailed processing procedure of the entire storage process (step S1803) in FIG. First, the save / restore processing unit 203 sets the number of stages p to p = 1 (step S1901), and determines whether the number of stages p is p> h (h is the uppermost stage) (step S1902).

  If p> h is not satisfied (step S1902: NO), the storage / restoration processing unit 203 secures a storage area for the p-th second transposed Bloom filter tbf (p) s (step S1903). Specifically, since the bit width s (= n × m) and the number of divisions c of the second transposed Bloom filter row tBF (p) at the p-th stage are known, c pieces having the bit width s / c are obtained. Reserve a storage area.

  Then, the storage / restoration processing unit 203 uses the dividing unit 1702 to divide each first transposed Bloom filter tbf (p) at the p-th stage into c pieces in units of words (step S1904). Then, the storage / restoration processing unit 203 is identical to the storage area in the arrangement order of the first transposed Bloom filter tbf (p) for each word at the same position after the division of the first transposed Bloom filter tbf (p). The word group at the position is arranged (step S1905). As a result, the second transposed Bloom filter row tBF (p) s is generated.

  Thereafter, the stage number p is incremented (step S1906), and the process returns to step S1902. Thereby, the second transposed Bloom filter tBf (p) s can be generated from the first stage to the h-th stage.

  In step S1902, if p> h (step S1902: Yes), the save / restore processing unit 203 saves the second hierarchical transposed Bloom filter tBFs in the save area in the storage device (step S1907). . As a result, the entire storage process (step S1803) is terminated.

<Partial storage processing procedure>
FIG. 20 is a flowchart showing a detailed processing procedure of the partial storage processing (step S1804) of FIG. First, the save / restore processing unit 203 sets the stage number p to p = 1 (step S2001), and determines whether the stage number p is p> h (h is the top stage) (step S2002).

  When p> h is not satisfied (step S2002: No), the storage / restoration processing unit 203 uses the update detection unit 1705 to detect the updated bit position of the updated first transposed Bloom filter tbf (p) (step S2003). .

  Then, the specifying unit 1706 specifies the word at the same arrangement position as the word including the update bit position from the first transposed Bloom filter tbf (p) (step S2004). Then, a storage area is secured (step S2005). In this case, the bit width corresponding to the second transposed Bloom filter tbf (p) s to which the word including the update bit position belongs is secured. Specifically, since the bit width w of the word is w = n / c and the number of arrays of the first transposed Bloom filter tbf (p) is m, (n / c) × m = s / c A bit width should be secured.

  Next, the transposition unit 1703 generates the second transposed Bloom filter tbf (p) s including the update bit by arranging the specified word group at the same arrangement position in the storage area (step S2006). Thereafter, the stage number p is incremented (step S2007), and the process returns to step S2002. Thereby, it is possible to update from the first stage to the h-th stage.

  In step S2002, if p> h (step S2002: Yes), the save / restore processing unit 203 overwrites and saves the second transposed Bloom filter tbf (p) s in the save area in the storage device ( Step S2008). That is, the second transposed Bloom filter tbf (p) s before update in the storage device is overwritten and saved. Thereby, the partial storage process (step S1804) is terminated.

<Restoration Process by Save / Restore Processing Unit 203>
FIG. 21 is a flowchart showing a restoration processing procedure by the save / restore processing unit 203. First, the save / restore processing unit 203 waits for a restore instruction by the accepting unit 1701 (step S2101: No). When the restore instruction is accepted (step S2101: Yes), the save / restore process unit 203 sets the number of steps p. p = 1 is set (step S2102), and it is determined whether or not the number of stages p is p> h (h is the uppermost stage) (step S2103).

  If p> h is not satisfied (step S2103: NO), the storage / restoration processing unit 203 secures a restoration area (step S2104). The restoration region is the bit width of the first transposed Bloom filter row tBF (p) and is divided for each first transposed Bloom filter tbf (p).

  Then, the storage / restoration processing unit 203 determines whether there is an unselected second transposed Bloom filter tbf (p) s (step S2105). When there is an unselected second transposed Bloom filter tbf (p) s (step S2105: Yes), the save / restore processing unit 203 selects the second transposed Bloom filter tbf (p) that has not been selected from the top of the array position. s is selected (step S2106). In the example of FIG. 15, the second transposed Bloom filter tbf (p−1) s and tbf (p) s are selected in this order.

  Thereafter, the saving / restoring processing unit 203 divides the selected second transposed Bloom filter tbf (p) s by word unit by the dividing unit 1702 (step S2107). Then, the storage / restoration processing unit 203 writes the divided word group in the restoration area for each first transposed Bloom filter tbf (p) by the transposing unit 1703 (step S2108). Thereafter, the process returns to step S2105.

  In step S2105, when there is no unselected second transposed Bloom filter tbf (p) s (step S2105: No), the save / restore processing unit 203 increments the number of stages p (step S2109) and returns to step S2102. . Thereby, it is possible to restore from the first stage to the h-th stage. In step S2103, when p> h is satisfied (step S2103: Yes), the restoration process is terminated.

  Thus, according to the present embodiment, the number of memory accesses is reduced because transposition and storage are performed in units of words. That is, when the first transposed Bloom filter row tBF (p) is returned to the original Bloom filter row BF (p), it must be transposed bit by bit, which increases memory access and takes time. In the embodiment, the storage process can be speeded up.

  Further, by grouping together in units of words, adjacent ones of the first transposed Bloom filter row tBF (p) are stored together. Therefore, when there is an update in the adjacent data block, only the first transposed Bloom filter tbf (p) including the update bit needs to be updated in the first transposed Bloom filter row tBF (p). As described above, the first hierarchical transposed Bloom filter tBF is not transposed and stored, but only the updated portion is partially transposed and re-saved, so that the efficiency of the saving process can be improved.

<Example of restoration when a plurality of second transposed Bloom filter rows tBF (p) s are stored>
Next, a restoration example when a plurality of second transposed Bloom filter rows tBF (p) s are stored will be described. When the number of data registrations in the data block set db exceeds a certain level, the Bloom filter search accuracy decreases. Therefore, when the number of registered data exceeds a certain level, a hierarchical Bloom filter BF having the same size is newly set.

  The newly set hierarchical Bloom filter BF is also transposed as described above to become the first hierarchical transposed Bloom filter tBF. Here, in order to distinguish the new first hierarchical transposed Bloom filter tBF from the already registered first hierarchical transposed Bloom filter tBF, it is referred to as “first hierarchical transposed Bloom filter tBFa”.

  Further, the p-th stage transposed Bloom filter row constituting the first hierarchical transposed Bloom filter tBFa is referred to as “first transposed Bloom filter row tBFa (p)”. Further, the transposed Bloom filter constituting the first transposed Bloom filter row tBFa (p) is denoted as “first transposed Bloom filter tbfa (p)”.

  When the first hierarchical transposed Bloom filter tBFa is generated, the number of first hierarchical transposed Bloom filters is two. When searching for the target data D, the search is first performed using the first hierarchical transposed Bloom filter tBF, and then the search is performed using the first hierarchical transposed Bloom filter tBFa.

  When the first hierarchical transposed Bloom filters tBF and tBFa are stored, the “second transposition” is once performed separately and stored in the storage device. Thereby, the first hierarchical transposed Bloom filter tBF is stored in the storage device as the second hierarchical transposed Bloom filter tBFs. On the other hand, the data structure of the first hierarchical transposed Bloom filter tBFa is the same as that of the first hierarchical transposed Bloom filter tBF, and is thus stored in the same manner.

  Here, the second hierarchical transposed Bloom filter for the first hierarchical transposed Bloom filter tBFa is referred to as “second hierarchical transposed Bloom filter tBFas”. Further, the p-th stage transposed Bloom filter row constituting the second hierarchical transposed Bloom filter tBFas is denoted as “second transposed Bloom filter row tBFa (p) s”. Further, the transposed Bloom filter constituting the second transposed Bloom filter row tBFa (p) s is denoted as “second transposed Bloom filter tbfa (p) s”.

  FIG. 22 is an explanatory diagram of a restoration example (part 1) when a plurality of second transposed Bloom filter rows tBF (p) s are stored.

  In FIG. 22, the left side shows an existing Bloom filter BF (p), a first transposed Bloom filter tBF (p), and a second transposed Bloom filter tBF (p) s. The right side is a newly added Bloom filter BFa (p), a first transposed Bloom filter tBFa (p), and a second transposed Bloom filter tBFa (p) s. The Bloom filter BFa (p) is a Bloom filter row that is a transposition source of the first transposed Bloom filter tBFa (p) in the “first transposition”.

  In FIG. 22, as an example, the bit width s of the Bloom filter columns BF (p) and BFa (p) is s = 24 bits. In addition, the arrangement number n of Bloom filters constituting each Bloom filter row BF (p), BFa (p) is n = 4. Further, the bit width m of the Bloom filters bf (p−1) to bf (p−8) is set to m = 6.

  Further, the division number c is set to c = 2. Therefore, in the first transposed Bloom filters tBF (p) and tBFa (p), c (= 2) words having a bit width n / c (= 4/2 = 2) are generated. Of the generated words, the word at the head position is designated as word W1- #, and the word at the next array position is designated as word W2- #.

  For example, the word W1-1 is the first word of the first transposed Bloom filter tbf (p-1), and the word W2-1 is the word W1-1 of the first transposed Bloom filter tbf (p-1). The following word.

  Similarly, the word W1-7 is the first word of the first transposed Bloom filter tbfa (p-7), and the word W2-7 is the word W1-7 of the first transposed Bloom filter tbfa (p-7). It is the word that follows.

  In FIG. 22, the first transposed Bloom filter sequence tBF (p), tBFa (p) is converted into the second transposed Bloom filter sequence tBF (p) s, tBFa ( p) transposed to s and stored in storage.

  Further, the second transposed Bloom filter string tBF (p) s, tBFa (p) s is converted into the first transposed Bloom filter string tBF (p), tBFa (p by the restoration process by the storage / restoration processing unit 203, respectively. ) And restored from the storage device.

  Thus, in the example shown in FIG. 22, since the first transposed Bloom filter string tBF (p), tBFa (p) exists separately, when performing a data search, first, the search processing unit 202 First, a search is performed using the first transposed Bloom filter string tBF (p), and then the search is performed using the first transposed Bloom filter string tBFa (p).

  In the example shown in FIG. 22, two Bloom filters of the first transposed Bloom filter string tBF (p) and tBFa (p) are separately used for the search, but the number of registered data increases. As a result, the number of Bloom filters increases, and the search efficiency decreases. Therefore, in the following example, when restoring from the state stored separately as the first transposed Bloom filter row tBF (p) and tBFa (p), the first transposed Bloom filter row tBF (p) is integrated. To do. Thereby, the search efficiency is improved.

  Also, when the integrated first transposed Bloom filter row tBF (p) is stored, the second transposed Bloom filter row tBF (p) s is obtained by the “second transposition”. Thus, each time a Bloom filter is added, the transposed Bloom filter is integrated, and the first transposed Bloom filter row tBF (p) is expanded.

  When searching with the first transposed Bloom filter string tBF (p) after the integration, the bit width of the transposed Bloom filter string tBF (p) at each stage is doubled before the integration, but the search is performed for each stage. By doing so, the narrowing down doubles.

  For example, if there is data that does not hit in the first transposed Bloom filter string tBF (p) and hits in the first transposed Bloom filter string tBFa (p) before the integration, the first transposed Bloom filter string tBF The search using (p) is wasted, and it takes time until the hit.

  On the other hand, after integration, since it can be narrowed down only by the first transposed Bloom filter row tBF (p), useless searches can be reduced as compared with the state before integration, and search efficiency can be improved. .

<Example of Integrated Restoration of Second Transposed Bloom Filter Sequence tBF (p) s, tBFa (p) s>
FIG. 23 is an explanatory diagram illustrating an example of integrated restoration of the second transposed Bloom filter row tBF (p) s, tBFa (p) s. In FIG. 23, the second transposed Bloom filter row tBF (p) s, tBFa (p) s is divided into words W1-1 to W1-12 and W2-1 to W2-12 in units of words.

  Then, the words W1-1 to W1-12 and W2-1 to W2-12 are grouped at the same arrangement position. Specifically, the same word is arranged according to the word arrangement in the words W1- # and W2- #. For example, since the word W1-1 and the word 2-1 both end with 1, the words W1-1 and W2-1 are arranged in this order.

  As a result, the original first transposed Bloom filter tbf (p−1) is restored. The other words are similarly transposed to be integrated into the first transposed Bloom filter row tBF (p). Note that the first transposed Bloom filter string tBF (p) after integration is referred to as “first transposed Bloom filter string new-tBF (p)”.

  In the integration of this example, since the two filters of the second transposed Bloom filter sequence tBF (p) s and tBFa (p) s are integrated, the bit width of the first transposed Bloom filter sequence new-tBF (p) is s × 2 = n × m × 2 = 48 bits. When the number of filters to be integrated is generalized as j, the bit width after integration is n × m × j bits.

  The first transposed Bloom filter string new-tBF (p) has the same data structure as the first transposed Bloom filter string tBF (p) before integration, except for the bit width, the number of filters, and the bit value. It is. Therefore, the first transposed Bloom filter row tBF (p) can be transposed and stored in the same manner. The second transposed Bloom filter row tBF (p) s transposed from the first transposed Bloom filter row new-tBF (p) is expressed as “second transposed Bloom filter row new-tBF (p) s”. .

  The second transposed Bloom filter row new-tBF (p) s is configured by arranging second transposed Bloom filters new-tBF (p-1) s and new-tBF (p-2) s. The second transposed Bloom filter new-tBF (p-1) s is configured by arranging words W1-1 to W1-12.

  That is, the second transposed Bloom filter new-tBF (p-1) s is a transposed Bloom filter in which the second transposed Bloom filters tBF (p-1) s and tBFa (p-1) s are integrated. Similarly, the second transposed Bloom filter new-tBF (p-2) s is configured by arranging words W2-1 to W2-12. That is, the second transposed Bloom filter new-tBF (p-2) s is a transposed Bloom filter in which the second transposed Bloom filters tBF (p-2) s and tBFa (p-2) s are integrated.

<Integrated Restoration Processing Procedure by Save / Restore Processing Unit 203>
FIG. 24 is a flowchart showing an integrated restoration processing procedure by the storage / restoration processing unit 203. In FIG. 24, the storage / restoration processing unit 203 waits for a restoration instruction by the accepting unit 1701 (step S2401: No). When the restoration instruction is accepted (step S2401: Yes), the second hierarchical transposed Bloom filter The stored number j of tBFs is detected (step S2402).

  In the example shown in FIG. 23, since the second transposed Bloom filter row tBF (p) s, tBFa (p) s is saved, the number j to be saved is j = 2.

  Then, the save / restore processing unit 203 sets the number of stages p to p = 1 (step S2403), and reserves a restore area for the number j of saves in the p-th stage in the memory area in the management apparatus 200 (step S2404). ). Then, the storage / restoration processing unit 203 determines whether or not there is an unselected second transposed Bloom filter row tBF (p) s among the j second transposed Bloom filter rows tBF (p) s. (Step S2405).

  If there is an unselected second transposed Bloom filter string tBF (p) s (step S2405: Yes), the save / restore processing unit 203 selects the unselected second transposed Bloom filter string tBF (p) s. Is selected (step S2406). When there is no unselected second transposed Bloom filter row tBF (p) s (step S2405: No), the process proceeds to step S2409.

  In the example shown in FIG. 23, j = 2 and there are second transposed Bloom filter rows tBF (p) s and tBFa (p) s. If both the second transposed Bloom filter sequence tBF (p) s and tBFa (p) s are not selected, the storage / restoration processing unit 203 firstly stores the second transposed Bloom filter sequence tBF (p) s. Select. Thereafter, when the process proceeds to step S2405 again, the storage / restoration processing unit 203 selects the second transposed Bloom filter string tBFa (p) s. Next, when the process proceeds to step S2405, there is no unselected one.

  After selecting the unselected second transposed Bloom filter string tBF (p) s, the storage / restoration processing unit 203 uses the dividing unit 1702 to select the selected second transposed Bloom filter string tBF (p) s. Is divided into words (step S2407).

  In the example shown in FIG. 23, when the second transposed Bloom filter string tBF (p) s is selected, the save / restore processing unit 203 converts the second transposed Bloom filter tBF (p) s into the word W1- 1 to W1-6, W2-1 to W2-6. When the second transposed Bloom filter tBFa (p) s is selected, the storage / restoration processing unit 203 converts the second transposed Bloom filter tBFa (p) s into the words W1-7 to W1-12, W2-7. Divide into ~ W2-12.

  Then, the transposition unit 1703 causes the storage / restoration processing unit 203 to arrange the divided word groups with the same word arrangement position and write them in the restoration area (step S2408).

  In the example shown in FIG. 23, the save / restore processing unit 203 uses the same word arrangement positions for the words W1-1 to W1-6 and W2-1 to W2-6, that is, the end numbers are the same. Put things together. That is, {W1-1, W2-1}, {W1-2, W2-2},..., {W1-6, W2-6} are collectively written.

  Similarly, the save / restore processing unit 203 also collects words W1-7 to W1-12 and W2-7 to W2-12 that have the same word arrangement position, that is, the same end numbers. That is, {W1-7, W2-7}, {W1-8, W2-8},..., {W1-12, W2-12} are written together.

  Then, the process returns to step S2405. If there is no unselected second transposed Bloom filter string tBF (p) in step S2405 (step S2405: No), the save / restore processing unit 203 increments the number of stages p (step S2409), and p> h It is determined whether or not (step S2410). If p> h is not satisfied (step S2410: NO), the process returns to step S2404. On the other hand, if p> h (step S2410: YES), the integrated restoration process by the save / restore processing unit 203 is terminated.

  As described above, according to the present embodiment, a plurality of second hierarchical transposed Bloom filters tBFs (for example, the second hierarchical transposed Bloom filters tBFs and tBFas) are stored by repeatedly storing and restoring. Even in such a case, it will fit in one memory block, and the data structure is optimal for retrieval. In this way, by expanding the second transposed Bloom filter tBFs, data can be indexed as long as the memory area continues.

  From the above, it is optimized by repeating storage and restoration, and it is possible to suppress degradation of search time. That is, instead of using a large memory area from the beginning, it is small at the beginning and can be scaled up gradually as the number of registered data increases.

  In the above description, once integrated and restored, the next storage process is stored as a single hierarchical transposed Bloom filter tBFs. However, in the storage process, it remains divided as in the state before the integration. It may be stored. In this case, the first transposed Bloom filter string new-tBF (p) after the integration is divided by the division number c, thereby transposing each first transposed Bloom filter string tBF (p) ("second transposition"). ) And stored in the storage device as the second transposed Bloom filter row tBF (p) s.

  As described above, even if the data is divided and stored, it can be restored to the first transposed Bloom filter string new-tBF (p) if reintegrated. Moreover, by storing separately, the storage processing of each first transposed Bloom filter row tBF (p) can be executed in parallel, and the storage time can be shortened.

  The search method and management method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. The search program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The search program may be distributed via a network such as the Internet.

200 Management Device 201 Registration Processing Unit 202 Search Processing Unit 203 Save / Restore Processing Unit 901 Reception Unit 902 Transposition Unit 903 Conversion Unit 904 First Specification Unit 905 Second Specification Unit 906 Determination Unit 907 Determination Unit 908 Extraction Unit 909 Output Unit 1701 Reception unit 1702 Division unit 1703 Transposition unit 1704 Storage unit 1705 Update detection unit 1706 Specific unit BF Hierarchical Bloom filter db Data block set HTs Hash table group tBF (first) hierarchical transposition Bloom filter tBFs Second hierarchical transposition Bloom filter

Claims (5)

A data block set including a registered data group for each data block and a bit string in n Bloom filters in which m bits indicating negative in a predetermined number of data blocks are arranged together by bits at the same position And a computer having access to j first transposed Bloom filter arrays in which m n-bit first transposed Bloom filters are arranged,
A first receiving step of receiving a storage request for the j first transposed Bloom filter rows;
When the storage request is accepted by the first acceptance step, by dividing the bit string of each first transposed Bloom filter by a division number c that is a divisor of n for each first transposed Bloom filter , A first dividing step for generating (c × m × j) n / c-bit words;
For each first transposed Bloom filter string, {(n / c) × m} bits are obtained by grouping the words divided by the first dividing step at the same position in each first transposed Bloom filter. A first transposing step of transposing a second transposed Bloom filter array of c second transposed Bloom filters;
A storage step of storing in the storage device the j second transposed Bloom filter trains transposed by the first transposing step;
A second receiving step of receiving a restoration instruction to the first transposed Bloom filter row;
When the restoration instruction is accepted by the second accepting step, the second transposed Bloom filter row is divided by the division number c for each of the second transposed Bloom filter rows, so that the second A second splitting step for generating m n / c-bit words for each transposed Bloom filter;
In the j second transposed Bloom filter rows, the second transposition is performed between m words divided by the second transposing step for each of the second transposed Bloom filters in common arrangement positions. The j second transposed Bloom filter rows are combined into a third transposed Bloom filter row in which the j first transposed Bloom filter rows are arranged in one row by combining the Bloom filters in the order of arrangement. A second transposition step to transpose;
A management program characterized by causing The first receiving step includes
Accepting a save request for the third transposed Bloom filter row;
The first dividing step includes
When the storage request for the third transposed Bloom filter string is accepted in the first accepting step, the bit number of each of the first transposed Bloom filter strings constituting the third transposed Bloom filter string is the division number c. To generate c n / c-bit words for each of the first transposed Bloom filters,
The first transposing step includes
The second transposed Bloom filter of {(n / c) × m × j} bits is represented by c by grouping the words divided in the first dividing step at the same position in each first transposed Bloom filter. Transpose to the fourth array of transposed Bloom filter rows,
The storage step includes
The management program according to claim 1, wherein the fourth transposed Bloom filter row transposed by the first transposing step is stored in the storage device. The first receiving step includes
Accepting a save request for the third transposed Bloom filter row;
The first dividing step includes
When the storage request for the third transposed Bloom filter row is accepted by the first accepting step, the third transposed Bloom filter row is changed to c first transposed Bloom filter rows by the division number c. In each of the first transposed Bloom filter sequences, the first transposed Bloom filter bit sequence is divided by the division number c, so that n words / c bits of c words are converted into the first transposed Bloom sequence. Generate for each Bloom filter,
The first transposing step includes
In each of the first transposed Bloom filter rows, the c pieces of the second transposed Bloom are collected by grouping the words divided by the first dividing step at the same position in each of the first transposed Bloom filters. Transpose to filter column,
The storage step includes
2. The management program according to claim 1, wherein the c second transposed Bloom filter trains transposed by the first transposing step are stored in the storage device. A data block set including a registered data group for each data block and a bit string in n Bloom filters in which m bits indicating negative in a predetermined number of data blocks are arranged together by bits at the same position A j-th transposed Bloom filter array in which m n-bit first transposed Bloom filters are arranged;
First receiving means for receiving a storage request for the j first transposed Bloom filter rows;
When the storage request is received by the first receiving means, by dividing the bit string of each first transposed Bloom filter by a division number c that is a divisor of n for each first transposed Bloom filter , First dividing means for generating (c × m × j) n / c-bit words;
For each first transposed Bloom filter string, {(n / c) × m} bits are obtained by grouping the words divided by the first dividing means at the same position in each first transposed Bloom filter. First transposing means for transposing a second transposed Bloom filter array in which c second transposed Bloom filters are arranged;
Storage means for storing the j second transposed Bloom filter trains transposed by the first transposing means in a storage device;
Second receiving means for receiving a restoration instruction to the first transposed Bloom filter row;
When the restoration instruction is accepted by the second accepting unit, the second transposed Bloom filter row is divided by the division number c for each of the second transposed Bloom filter rows, so that the second A second dividing means for generating m n / c-bit words for each transposed Bloom filter;
In the j second transposed Bloom filter trains, the second transposing is performed by using the m words divided by the second dividing unit for each of the second transposed Bloom filters in common arrangement positions. The j second transposed Bloom filter rows are combined into a third transposed Bloom filter row in which the j first transposed Bloom filter rows are arranged in one row by combining the Bloom filters in the order of arrangement. A second transposing means for transposing;
A management apparatus comprising: A data block set including a registered data group for each data block and a bit string in n Bloom filters in which m bits indicating negative in a predetermined number of data blocks are arranged together by bits at the same position And a computer having access to j first transposed Bloom filter rows in which m n-bit first transposed Bloom filters are arranged,
A first receiving step of receiving a storage request for the j first transposed Bloom filter rows;
When the storage request is accepted by the first acceptance step, by dividing the bit string of each first transposed Bloom filter by a division number c that is a divisor of n for each first transposed Bloom filter , A first dividing step for generating (c × m × j) n / c-bit words;
For each first transposed Bloom filter string, {(n / c) × m} bits are obtained by grouping the words divided by the first dividing step at the same position in each first transposed Bloom filter. A first transposing step of transposing a second transposed Bloom filter array of c second transposed Bloom filters;
A storage step of storing in the storage device the j second transposed Bloom filter trains transposed by the first transposing step;
A second receiving step of receiving a restoration instruction to the first transposed Bloom filter row;
When the restoration instruction is accepted by the second accepting step, the second transposed Bloom filter row is divided by the division number c for each of the second transposed Bloom filter rows, so that the second A second splitting step for generating m n / c-bit words for each transposed Bloom filter;
In the j second transposed Bloom filter rows, the second transposition is performed between m words divided by the second transposing step for each of the second transposed Bloom filters in common arrangement positions. The j second transposed Bloom filter rows are combined into a third transposed Bloom filter row in which the j first transposed Bloom filter rows are arranged in one row by combining the Bloom filters in the order of arrangement. A second transposition step to transpose;
The management method characterized by performing.

Images