CN112699323A - Cloud caching system and cloud caching method based on double bloom filters - Google Patents
Cloud caching system and cloud caching method based on double bloom filters Download PDFInfo
- Publication number
- CN112699323A CN112699323A CN202110018383.5A CN202110018383A CN112699323A CN 112699323 A CN112699323 A CN 112699323A CN 202110018383 A CN202110018383 A CN 202110018383A CN 112699323 A CN112699323 A CN 112699323A
- Authority
- CN
- China
- Prior art keywords
- elements
- bit array
- array
- hash function
- bloom filter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000013507 mapping Methods 0.000 claims description 12
- 230000009977 dual effect Effects 0.000 claims description 9
- 238000012217 deletion Methods 0.000 claims description 7
- 230000037430 deletion Effects 0.000 claims description 7
- 238000003491 array Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a cloud caching system based on a double bloom filter, which comprises: a first bloom filter and a second bloom filter; the first bloom filter includes: a first hash function, the first hash function comprising: a first bit array; the second bloom filter includes: a second hash function, the second hash function comprising: a first bit array; wherein the initial value of each bit of the first bit array and the second bit array is 0. The invention also discloses a cloud caching method based on the double bloom filters. The invention adopts two hash algorithms corresponding to the arrays of the two bloom filters, which can effectively reduce the misjudgment, and the misjudgment rate is below 0.01%; meanwhile, short integer data is stored in the bloom filter array, if two elements correspond to the same position in the bit array, the median value of the bloom filter is 2, only one element is 1, and the median value is 0 when the element does not exist.
Description
Technical Field
The invention relates to the technical field of cloud caching, in particular to a cloud caching system and a cloud caching method based on double bloom filters.
Background
With the increase of internet users, the access concurrency of the system is larger and larger. The access performance of the database becomes the bottleneck of the system performance, and the problem is well solved by the appearance of the caching technology.
In the prior art, a single bloom filter is generally used and a corresponding array of bloom filters stores bit-type data, so that the following disadvantages exist: when the hash value is subjected to modulo operation, collision of arrays can occur (for example, the modulo values of the hash value 11 and the hash value 6 on the array with the length of 5 are both 1), so that certain misjudgment rate exists; in addition, because the bloom filter stores bit type data, if the value in the array in the filter corresponds to 1, there may be multiple pieces of data, so the bloom filter cannot support the delete operation.
Therefore, the existing caching method has the problem that the consistency between the cached data and the database data is poor when the cached data is ensured. In addition, there are two drawbacks to using conventional bloom filters: firstly, misjudgment can exist; and secondly, element deletion is not supported.
Disclosure of Invention
Aiming at the problems, the invention provides a cloud caching system and a cloud caching method based on double bloom filters.
In order to achieve the object of the present invention, a cloud caching system based on a dual bloom filter is provided, the cloud caching system comprising: a first bloom filter and a second bloom filter; the first bloom filter includes: a first hash function, the first hash function comprising: a first bit array; the second bloom filter includes: a second hash function, the second hash function comprising: a second bit array; wherein the first bit array and the second bit array are both short types, and the initial value of each bit of the first bit array and the second bit array is 0.
The invention also provides a cloud caching method based on the double bloom filters, and the element adding process comprises the following steps: a user sends a request for adding an element; mapping the elements by using a first hash function to obtain the positions of the elements on the first digit array; increasing the value corresponding to the position of the element on the first digit array by 1; mapping the elements by using a second hash function to obtain the positions of the elements on the second bit array; increasing the value corresponding to the position of the element on the second bit array by 1; newly adding the added elements into the cloud cache; and updating the newly added elements in the cloud cache into the database to complete the operation.
Further, in the cloud caching method, the process of querying the element includes: the user sends a request for inquiring the element according to the unique identifier of the element; the first hash function calculates and obtains a first hash value corresponding to the unique identifier according to the unique identifier of the element; performing modular operation on the first digit array by using the first hash value, and obtaining the position of the element on the first digit array; judging whether the value corresponding to the position of the element on the first digit array is greater than 0;
the second hash function calculates and obtains a second hash value corresponding to the unique identifier according to the unique identifier of the element; performing modular operation on the second bit array by using the second hash value, and obtaining the position of the element on the second bit array; judging whether the value corresponding to the position of the element on the second digit group is greater than 0;
if the value of the element on the first bit array and the value on the second bit array satisfy both more than 0,
then the element is present; executing the operation of the query element in the cloud cache; if the query element in the cloud cache fails, continuing to execute the operation of the query element in the database and returning a query result; if the query element in the cloud cache is successful, directly returning a query result;
otherwise, the element does not exist, and the query operation is directly finished.
Further, in the cloud caching method, the process of deleting an element includes: a user sends a request for deleting an element; mapping the elements by using a first hash function to obtain the positions of the elements on the first digit array; subtracting 1 from the value corresponding to the position of the element on the first digit array; mapping the elements by using a second hash function to obtain the positions of the elements on the second bit array;
subtracting 1 from the value corresponding to the position of the element on the second bit array;
executing deletion operation on the elements in the cloud cache;
and executing deletion operation on the elements in the database.
Compared with the prior art, the invention has the following beneficial technical effects:
the invention adopts two hash algorithms corresponding to the bit arrays of the two bloom filters, which can effectively reduce the misjudgment, and the misjudgment rate is below 0.01%; meanwhile, short integer data is stored in the bit array of the bloom filter, if two elements correspond to the same position in the bit array, the median value of the bloom filter is 2, only one element is 1, and the median value is 0 when the element does not exist.
Drawings
FIG. 1 is a schematic flow diagram illustrating adding elements in a dual bloom filter based cloud caching method, according to an embodiment;
FIG. 2 is a schematic flow diagram of a query element in a dual bloom filter based cloud caching method, under an embodiment;
FIG. 3 is a schematic flow diagram for deleting elements in a dual bloom filter based cloud caching method, under an embodiment;
FIG. 4 is a schematic diagram of a bloom filter of an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The embodiment provides a cloud caching system based on a dual bloom filter, and the cloud caching system comprises: a first bloom filter and a second bloom filter; the first bloom filter includes: a first hash function, the first hash function comprising: a first bit array; the second bloom filter includes: a second hash function, the second hash function comprising: a second bit array; wherein the first bit array and the second bit array are both short types, and the initial value of each bit of the first bit array and the second bit array is 0.
The embodiment also provides a cloud caching method based on a dual bloom filter, and as shown in fig. 1, the process of adding elements includes:
a user sends a request for adding an element; mapping the elements by using a first hash function to obtain the positions of the elements on the first digit array; increasing the value corresponding to the position of the element on the first digit array by 1; mapping the elements by using a second hash function to obtain the positions of the elements on the second bit array; increasing the value corresponding to the position of the element on the second bit array by 1; newly adding the added elements into the cloud cache; and updating the newly added elements in the cloud cache into the database to complete the operation.
In one embodiment, as shown in fig. 2, in the cloud caching method, the process of querying the element includes:
the user sends a request for inquiring the element according to the unique identifier of the element;
the first hash function calculates and obtains a first hash value corresponding to the unique identifier according to the unique identifier of the element; performing modular operation on the first digit array by using the first hash value, and obtaining the position of the element on the first digit array; judging whether the value corresponding to the position of the element on the first digit array is greater than 0;
the second hash function calculates and obtains a second hash value corresponding to the unique identifier according to the unique identifier of the element; performing modular operation on the second bit array by using the second hash value, and obtaining the position of the element on the second bit array; judging whether the value corresponding to the position of the element on the second digit group is greater than 0;
if the value of the element on the first bit array and the value on the second bit array satisfy both more than 0,
then the element is present; executing the operation of the query element in the cloud cache; if the query element in the cloud cache fails, continuing to execute the operation of the query element in the database and returning a query result, and simultaneously replanting the queried element in the database to the cloud cache; if the query element in the cloud cache is successful, directly returning a query result;
otherwise, the element does not exist, and the query operation is directly finished.
In one embodiment, as shown in fig. 3, in the cloud caching method, the process of deleting an element includes:
a user sends a request for deleting an element; mapping the elements by using a first hash function to obtain the positions of the elements on the first digit array; subtracting 1 from the value corresponding to the position of the element on the first digit array; mapping the elements by using a second hash function to obtain the positions of the elements on the second bit array; subtracting 1 from the value corresponding to the position of the element on the second bit array; executing deletion operation on the elements in the cloud cache; and executing deletion operation on the elements in the database.
As shown in fig. 4, which is a schematic diagram of a bloom filter of an embodiment, the bloom filter has data A, B, C, D, E, and the corresponding location is as shown in fig. 4, where a value of 1 at the corresponding location of the filter is obtained when querying E indicates that there is data, and values of 0 at the corresponding locations are obtained when querying F and G indicates that there is no data.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
It should be noted that the terms "first \ second \ third" referred to in the embodiments of the present application merely distinguish similar objects, and do not represent a specific ordering for the objects, and it should be understood that "first \ second \ third" may exchange a specific order or sequence when allowed. It should be understood that "first \ second \ third" distinct objects may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented in an order other than those illustrated or described herein.
The terms "comprising" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that comprises a list of steps or modules is not limited to the listed steps or modules but may alternatively include other steps or modules not listed or inherent to such process, method, product, or device.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (4)
1. A cloud caching system based on double bloom filters is characterized in that,
the cloud caching system comprises: a first bloom filter and a second bloom filter;
the first bloom filter includes: a first hash function, the first hash function comprising: a first bit array;
the second bloom filter includes: a second hash function, the second hash function comprising: a second bit array;
wherein the first bit array and the second bit array are both short types, and the initial value of each bit of the first bit array and the second bit array is 0.
2. A cloud caching method based on a dual bloom filter is characterized in that the process of adding elements comprises the following steps:
a user sends a request for adding an element;
mapping the elements by using a first hash function to obtain the positions of the elements on the first digit array;
increasing the value corresponding to the position of the element on the first digit array by 1;
mapping the elements by using a second hash function to obtain the positions of the elements on the second bit array;
increasing the value corresponding to the position of the element on the second bit array by 1;
newly adding the added elements into the cloud cache;
and updating the newly added elements in the cloud cache into the database to complete the operation.
3. The dual bloom filter based cloud caching method according to claim 2, wherein the process of querying the element comprises:
the user sends a request for inquiring the element according to the unique identifier of the element;
the first hash function calculates and obtains a first hash value corresponding to the unique identifier according to the unique identifier of the element;
performing modular operation on the first digit array by using the first hash value, and obtaining the position of the element on the first digit array;
judging whether the value corresponding to the position of the element on the first digit array is greater than 0;
the second hash function calculates and obtains a second hash value corresponding to the unique identifier according to the unique identifier of the element;
performing modular operation on the second bit array by using the second hash value, and obtaining the position of the element on the second bit array;
judging whether the value corresponding to the position of the element on the second digit group is greater than 0;
if the value of the element on the first bit array and the value on the second bit array satisfy both more than 0,
if the query element in the cloud cache fails, continuing to execute the operation of the query element in the database and returning a query result, and if the query element in the cloud cache succeeds, directly returning the query result;
otherwise, the element does not exist, and the query operation is directly finished.
4. The dual bloom filter based cloud caching method according to claim 2, wherein the process of deleting an element comprises:
a user sends a request for deleting an element;
mapping the elements by using a first hash function to obtain the positions of the elements on the first digit array;
subtracting 1 from the value corresponding to the position of the element on the first digit array;
mapping the elements by using a second hash function to obtain the positions of the elements on the second bit array;
subtracting 1 from the value corresponding to the position of the element on the second bit array;
executing deletion operation on the elements in the cloud cache;
and executing deletion operation on the elements in the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110018383.5A CN112699323A (en) | 2021-01-07 | 2021-01-07 | Cloud caching system and cloud caching method based on double bloom filters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110018383.5A CN112699323A (en) | 2021-01-07 | 2021-01-07 | Cloud caching system and cloud caching method based on double bloom filters |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112699323A true CN112699323A (en) | 2021-04-23 |
Family
ID=75515011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110018383.5A Pending CN112699323A (en) | 2021-01-07 | 2021-01-07 | Cloud caching system and cloud caching method based on double bloom filters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699323A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103116599A (en) * | 2012-11-30 | 2013-05-22 | 浙江工商大学 | Urban mass data flow fast redundancy elimination method based on improved Bloom filter structure |
| CN105260429A (en) * | 2015-09-30 | 2016-01-20 | 河南科技大学 | ICN network information name searching method based on multiple Bloom filters |
| CN108287840A (en) * | 2017-01-09 | 2018-07-17 | 北京大学 | A kind of data storage and query method based on matrix Hash |
| CN109684231A (en) * | 2017-10-19 | 2019-04-26 | 三星电子株式会社 | The system and method for dsc data in solid-state disk and stream for identification |
| CN111563199A (en) * | 2020-04-26 | 2020-08-21 | 北京奇艺世纪科技有限公司 | Data processing method and device |
| CN112052398A (en) * | 2020-10-13 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Media information recommendation method and device, electronic equipment and storage medium |
| CN112068958A (en) * | 2020-08-31 | 2020-12-11 | 常州微亿智造科技有限公司 | Bloom filter and data processing method |
-
2021
- 2021-01-07 CN CN202110018383.5A patent/CN112699323A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103116599A (en) * | 2012-11-30 | 2013-05-22 | 浙江工商大学 | Urban mass data flow fast redundancy elimination method based on improved Bloom filter structure |
| CN105260429A (en) * | 2015-09-30 | 2016-01-20 | 河南科技大学 | ICN network information name searching method based on multiple Bloom filters |
| CN108287840A (en) * | 2017-01-09 | 2018-07-17 | 北京大学 | A kind of data storage and query method based on matrix Hash |
| CN109684231A (en) * | 2017-10-19 | 2019-04-26 | 三星电子株式会社 | The system and method for dsc data in solid-state disk and stream for identification |
| CN111563199A (en) * | 2020-04-26 | 2020-08-21 | 北京奇艺世纪科技有限公司 | Data processing method and device |
| CN112068958A (en) * | 2020-08-31 | 2020-12-11 | 常州微亿智造科技有限公司 | Bloom filter and data processing method |
| CN112052398A (en) * | 2020-10-13 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Media information recommendation method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 齐开诚: "IPv4/IPv6网络攻击溯源系统的研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI717866B (en) | Cross-chain certificate deposit method and access method, device and electronic equipment | |
| CN102682037B (en) | A kind of data capture method, system and device | |
| CN102739720B (en) | Distributed cache server system and application method thereof, cache clients and cache server terminals | |
| CN110191428B (en) | Data distribution method based on intelligent cloud platform | |
| JP6103037B2 (en) | Computer system | |
| CN104346345B (en) | The storage method and device of data | |
| CN102122285A (en) | Data cache system and data inquiry method | |
| US11599503B2 (en) | Path name cache for notifications of file changes | |
| CN105989076A (en) | Data statistical method and device | |
| CN107197359B (en) | Video file caching method and device | |
| CN102867070A (en) | Method for updating cache of key-value distributed memory system | |
| Xiao et al. | Using parallel bloom filters for multiattribute representation on network services | |
| CN111723073B (en) | Data storage processing method, device, processing system and storage medium | |
| CN112015820A (en) | Method, system, electronic device and storage medium for implementing distributed graph database | |
| CN111400334B (en) | Data processing method, data processing device, storage medium and electronic device | |
| CN109299111A (en) | A kind of metadata query method, apparatus, equipment and computer readable storage medium | |
| CN104933054B (en) | The URL storage methods and device of cache resource file, cache server | |
| CN111782659A (en) | Database index creation method and device, computer equipment and storage medium | |
| CN114969061A (en) | Distributed storage method and device for industrial time sequence data | |
| CN111198886A (en) | Method for constructing Hbase secondary index table | |
| CN108920631B (en) | File query method, device, equipment and readable storage medium | |
| CN112039993B (en) | Long connection address processing method and device | |
| US11947553B2 (en) | Distributed data processing | |
| CN112699323A (en) | Cloud caching system and cloud caching method based on double bloom filters | |
| CN105610921B (en) | Correcting and eleting codes archiving method based on data buffer storage under a kind of cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210423 |