CN113904999B - Data expansion method and programmable switch - Google Patents
Data expansion method and programmable switch Download PDFInfo
- Publication number
- CN113904999B CN113904999B CN202111273580.8A CN202111273580A CN113904999B CN 113904999 B CN113904999 B CN 113904999B CN 202111273580 A CN202111273580 A CN 202111273580A CN 113904999 B CN113904999 B CN 113904999B
- Authority
- CN
- China
- Prior art keywords
- data
- matching
- sub
- bitmap
- hash values
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/45—Arrangements for providing or supporting expansion
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a data expansion method and a programmable switch, and relates to the field of data expansion. The method is applied to the programmable switch, and the method is characterized in that the bloom filter bitmap is divided into n sub bitmaps and is respectively stored in n matching units, each matching unit sequentially judges whether data to be stored hits or not, so that the bloom filter is applied to the programmable switch and the data is filtered, and the programmable switch can support millions or even tens of millions of table items. Compared with the prior art, the method has the characteristics of low cost and high throughput.
Description
Technical Field
The invention relates to the field of data expansion, in particular to a data expansion method and a programmable switch.
Background
The programmable switch is an emerging technology, has the characteristics of low delay, low power consumption and high flexibility, but the number of the supported table items is limited because the hardware resources of the programmable switch chip are limited, and the programmable switch can not meet the requirement of a large number of the table items.
In the prior art, a plurality of programmable switches are connected in series, and different table entries are configured by different programmable switches to improve the capacity of the programmable switches, however, the method has high cost and throughput which is only half that of a single programmable switch.
Disclosure of Invention
The invention aims to provide a data expansion method and a programmable switch, which can at least partially solve the problems.
In order to achieve the above object, the technical scheme adopted by the embodiment of the invention is as follows:
in one aspect, a data capacity expansion method is applied to a programmable switch, the programmable switch includes a first processor, a first memory and a programmable chip, the programmable chip includes a processing unit and n matching units, the processing unit is respectively connected with the first processor, the first memory and n matching units, the n matching units are sequentially connected, n is greater than or equal to 1, n is a natural number, and the first processor is connected with the first memory, the method includes:
the first processor sends an original list to the processing unit;
the processing unit constructs a bloom filter bitmap according to the original list, divides the bloom filter bitmap into n sub-bitmaps, and stores the n sub-bitmaps into n matching units respectively;
n matching units obtain n sub-bitmaps;
the processing unit receives data to be stored, judges whether the data to be stored hits or not in sequence through the sub-bitmaps of the n matching units, obtains hit data if the data hit, stores the hit data into the first memory, and discards the data to be stored if the data hit does not hit.
Optionally, after the step of storing the hit data in the first memory, the method further includes:
the first processor writes the original list into a data structure, generates an original list structure table, matches the hit data with the original list structure table, obtains accurate data if the matching is successful, stores the accurate data into the first memory, and discards the hit data if the matching is not successful.
Optionally, the step of receiving the data to be stored by the processing unit and determining, by n matching units, whether the data to be stored hits includes:
the processing unit receives data to be stored, calculates the data to be stored through m hash functions to obtain m hash values, and sends the m hash values to the first matching unit;
part or all of the matching units sequentially judge whether m hash values exist in the corresponding sub-bitmap, if so, the corresponding position values are taken out, and if the corresponding position values are 1, the matching result of the hash values is set to be 1;
the processing unit judges whether the matching results of m hash values are all 1;
if yes, judging that the data to be stored hits.
Optionally, the step of sequentially determining, by the matching unit, whether m hash values exist in the corresponding sub-bitmap includes:
the first matching unit sequentially judges whether m hash values exist in the sub-bitmap of the first matching unit, sends the hash value which does not exist to the next matching unit for judgment until all n matching units judge to be finished or m hash values exist in the sub-bitmap, and judges to be finished.
Optionally, the step of the first matching unit sequentially determining whether m hash values exist in the sub-bitmap of the first matching unit includes:
subtracting the m hash values respectivelyObtaining m calculated values;
when the calculated value is greater than or equal to 0 and less thanWhen the hash value corresponding to the calculated value is judged to exist in the sub-bitmap of the first matching unit;
where x is the number of bits of the bloom filter bitmap.
Optionally, the step of obtaining n sub-bitmaps by the n matching units includes:
and setting a default matching table item in the n matching units respectively, and obtaining the sub-bitmap after the matching unit performs processing action according to the default matching table item.
On the other hand, the programmable switch comprises a first processor, a first memory and a programmable chip, wherein the programmable chip comprises a processing unit and n matching units, the processing unit is respectively connected with the first processor, the first memory and the n matching units, the n matching units are sequentially connected, n is more than or equal to 1, n is a natural number, and the first processor is connected with the first memory;
the first processor is used for sending an original list to the processing unit;
the processing unit is used for constructing a bloom filter bitmap according to the original list, dividing the bloom filter bitmap into n sub-bitmaps, and respectively storing the n sub-bitmaps into n matching units;
the n matching units are used for obtaining n sub-bitmaps;
the processing unit is further configured to receive data to be stored, sequentially determine whether the data to be stored hits through the sub-bitmaps of the n matching units, obtain hit data if hit, and store the hit data to the first memory, otherwise discard the data to be stored.
Optionally, the first processor is further configured to:
writing the original list into a data structure, generating an original list structure table, matching the hit data with the original list structure table, obtaining accurate data if the matching is successful, storing the accurate data into the first memory, and otherwise discarding the hit data.
Optionally, the data structure includes a hash table or a tree structure.
Optionally, the programmable switch further includes a second processor and a second memory, where the second processor is connected to the first processor, the programmable chip, and the second memory, respectively;
the first processor is used for sending an original list to the second processor;
the programmable chip is used for sending the hit data to the second processor;
the second processor is configured to store the original list into a data structure, generate an original list structure table, match the hit data with the original list structure table, obtain accurate data if the matching is successful, and store the accurate data into the second memory, otherwise discard the hit data.
Compared with the prior art, the invention has the following beneficial effects:
the method divides a bloom filter bitmap into n sub bitmaps and stores the n sub bitmaps into n matching units respectively, and each matching unit judges whether data to be stored hits or not in sequence, so that bloom filters are applied to the programmable switch and the data are filtered, and the programmable switch can support millions or even tens of millions of table items. Compared with the prior art, the method has the characteristics of low cost and high throughput.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is one of the block diagrams of a programmable switch architecture provided in an embodiment of the present application;
FIG. 2 is a flowchart of a data expansion method according to an embodiment of the present disclosure;
FIG. 3 is a second flowchart of a data expansion method according to an embodiment of the present disclosure;
FIG. 4 is a third flowchart of a data expansion method according to an embodiment of the present disclosure;
fig. 5 is a second block diagram of a programmable switch structure according to an embodiment of the present application.
Description of main reference numerals: 100-programmable switches; 110-a first memory; 120-a first processor; 130-a programmable chip; 140-a second processor; 150-a second memory.
Detailed Description
As described in the background art, the prior art often adopts a mode of connecting a plurality of programmable switches in series, and different programmable switches configure different entries to improve the capacity of the programmable switches, but the method has high cost and the throughput is only half of that of a single programmable switch.
The problems of the prior art are the results obtained by the inventor after practice and careful study, and therefore, the discovery process of the problems and the solutions proposed by the embodiments of the present invention below for the problems should be all contributions from the inventor in the inventive process.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
In the description of the present invention, it should be noted that, if the terms "upper", "lower", "inner", "outer", and the like indicate an azimuth or a positional relationship based on the azimuth or the positional relationship shown in the drawings, or the azimuth or the positional relationship in which the inventive product is conventionally put in use, it is merely for convenience of describing the present invention and simplifying the description, and it is not indicated or implied that the apparatus or element referred to must have a specific azimuth, be configured and operated in a specific azimuth, and thus it should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, if any, are used merely for distinguishing between descriptions and not for indicating or implying a relative importance.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Some embodiments of the present invention are described in detail below with reference to the accompanying drawings. It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.
In order to better understand the technical solutions of the present application, the following may further describe the technical terms involved:
P4A programmable model of a network data plane, the programmable model of P4 is relative to a traditional fixed pipeline exchange chip, and the architecture of the programmable model is PISA (Protocol Independent Switch Arch) full pipeline programmable architecture.
Tofino: the Tofino of Intel is a first end user programmable Ethernet switch in the world, is constructed by adopting a switch architecture (PISA) independent of a protocol, has P4 programmability and can achieve a forwarding throughput rate of 6.5Tbit/s.
Bloom filter: (English: bloom Filter) was proposed by Bloom in 1970. It is in effect a very long binary vector and a series of random mapping functions. Bloom filters may be used to retrieve whether an element is in a collection. Its advantages are high space efficiency and inquiry time, and high error rate and deletion difficulty. If it is desired to determine whether an element is in a collection, it is generally contemplated that all elements in the collection be saved and then determined by comparison. Data structures such as linked lists, trees, hash tables (also called Hash tables), and the like are all such ideas. But as the number of elements in the collection increases, the memory space we need is getting larger. And the search speed is also getting slower and slower. The principle of bloom filters is that when an element is added to a set, it is mapped to K points in a bit array by K hash functions, setting them to 1. In search, we only look at whether these points are all 1's or not and know (about) if there is any point in the collection: if any of these points is 0, the inspected element must not be present; if both are 1, the inspected element is likely to be present. This is the basic idea of bloom filters.
It should be noted that, due to the limited hardware resources of the programmable switch and the larger bitmap of the bloom filter, the bloom filter cannot be directly applied to the programmable switch to realize the filtering of the data, so as to improve the capacity of the data.
Taking the programmable switch chip Tofino as an example, the Tofino chip contains multiple pipelines, each Pipeline can run different table look-up logic (i.e. different P4 programs), each Pipeline contains 12 MAUs (Match-Action units), and outgoing/incoming pipelines share these MAUs. Each MAU corresponds to one Stage in the Pipeline, and each Stage supports a plurality of concurrent searches (no dependence among tables), so that the concurrent search performance can be improved, and the MAU can be searched sequentially. The resources of a single MAU are very limited, its SRAM is only 10Mbits, and the SRAM is a localized resource, so MAU cannot use the SRAM resources of other MAUs. How to implement bloom filters (phase inversion implementation) on a Tofino chip and how to fully utilize the resources of the Tofino chip is a difficulty.
Therefore, how to apply bloom filter to programmable switch is also one of the core invention points of the present application.
Based on the above analysis, please refer to fig. 1 and fig. 2 in combination, the present application provides a data capacity expansion method, which is applied to a programmable switch 100, and can effectively improve the data capacity of the programmable switch 100.
The programmable switch 100 includes a first processor 120, a first memory 110 and a programmable chip 130, the programmable chip 130 includes a processing unit and n matching units, the processing unit is respectively connected with the first processor 120, the first memory 110 and the n matching units, the n matching units are sequentially connected, wherein n is greater than or equal to 1, n is a natural number, and the first processor 120 is connected with the first memory 110, the method includes:
step 201: the first processor 120 sends the original list to the processing unit.
Step 202: the processing unit constructs a bloom filter bitmap according to the original list, divides the bloom filter bitmap into n sub-bitmaps, and stores the n sub-bitmaps into n matching units respectively.
Step 203: the n matching units obtain n sub-bitmaps.
Step 204: the processing unit receives the data to be stored, sequentially judges whether the data to be stored hits or not through the sub-bitmaps of the n matching units, obtains hit data if the data hit, stores the hit data into the first memory 110, and discards the data to be stored if the data hit does not hit.
By the method, the bloom filter can be applied to the programmable switch, the data to be stored is filtered through the bloom filter, the data capacity of the programmable switch is effectively improved, and compared with the method for improving the capacity by connecting the programmable switches in series in the prior art, the method has the advantages that the use cost is low, only one programmable switch is needed, the throughput is high, and the throughput is close to the total throughput of a single programmable switch under the condition that the filtered flow is smaller.
In another alternative embodiment, after the step of storing hit data in the first memory 110 in the step 204, the method further includes:
step 205: the first processor 120 writes the original list into the data structure, generates an original list structure table, matches the hit data with the original list structure table, obtains the accurate data if the matching is successful, and stores the accurate data into the first memory 110, otherwise discards the hit data.
Through the step 205, the hit data is accurately filtered again, so that the problem of low filtering precision of the bloom filter can be solved, and the data filtering effect is better improved.
In another possible embodiment, referring to fig. 3, the step of the processing unit receiving the data to be stored and determining whether the data to be stored hits by n matching units in the step 204 includes:
step 301: the processing unit receives data to be stored, calculates the data to be stored through m hash functions to obtain m hash values, and sends the m hash values to the first matching unit.
Step 302: and part or all of the matching units sequentially judge whether m hash values exist in the corresponding sub-bitmap, if so, the corresponding position values are taken out, and if the corresponding position values are 1, the matching result of the hash values is set to be 1.
Step 303: the processing unit judges whether the matching results of the m hash values are all 1, if yes, the data to be stored is judged to be hit.
It should be noted that, first, the first matching unit in step 301 refers to a matching unit storing the first sub-bitmap.
For example, assuming a bloom filter bitmap having 24 bits (for ease of understanding, the present embodiment limits the bloom filter bitmap number to 24, but in practical applications, the number of bits may be millions or even tens of millions), when the programmable chip 130 has 12 matching units, we divide the bloom filter bitmap into 12 sub-bitmaps, with the first sub-bitmap for bits 1-2 of the Ying Bulong filter bitmap, the second sub-bitmap for bits 2-3 of the Ying Bulong filter bitmap, and so on.
In step 303, it is determined that the data to be stored is hit only when the matching result of all the hash values is 1, otherwise, even if one matching result is not 1, the data to be stored should be discarded.
In another alternative embodiment, the step of determining, in sequence, whether the m hash values exist in the corresponding sub-bitmap by using part or all of the matching units in step 302 includes:
the first matching unit sequentially judges whether m hash values exist in the sub-bitmap of the first matching unit, sends the hash value which does not exist to the next matching unit for judgment until the n matching units are judged to be finished or the m hash values are all exist in the sub-bitmap, and then the judgment is finished.
It should be noted that, in this embodiment, from the first matching unit to the nth matching unit, it is sequentially determined whether m hash values exist in the corresponding sub-bitmap, and in the determination process, the following cases may occur: m hash values just exist in the sub-bitmaps corresponding to n matching units; m hash values only exist in the sub-bitmap corresponding to part of the matching units; the partial hash value does not exist in the sub-bitmap corresponding to the n matching units, i.e., the partial hash value does not exist in the bloom filter bitmap.
Optionally, referring to fig. 4 in combination, the step of the first matching unit sequentially determining whether m hash values exist in the sub-bitmap of the first matching unit includes:
step 401: respectively subtracting m hash valuesM calculated values are obtained.
Step 402: when the calculated value is greater than or equal to 0 and less thanAnd if so, judging that the hash value corresponding to the calculated value exists in the sub-bitmap of the first matching unit.
Where x is the number of bits of the bloom filter bitmap.
It should be noted that, the processing procedure that the subsequent matching unit and the first matching unit determine whether the hash value exists in the sub-bitmap corresponding to the matching unit is consistent, which is not described herein.
In this embodiment, the calculation amount can be effectively reduced and the calculation efficiency can be improved by subtracting the corresponding numerical value from the hash value.
In another possible implementation manner, in the step 203, the step of obtaining n sub-bitmaps by n matching units includes:
and setting a default matching table item in the n matching units respectively, and obtaining the sub-bitmap after the matching units perform processing actions according to the default matching table item.
It should be noted that, according to the characteristics of the hardware structure of the programmable switch, the matching unit cannot immediately obtain the sub-bitmap after storing the sub-bitmap, and a certain Action is required to obtain the sub-bitmap, after setting the default matching table entry, the matching unit processes an Action, and the Action data is the sub-bitmap to be obtained.
The embodiment of the application further provides a programmable switch 100, where the programmable switch 100 includes a first processor 120, a first memory 110 and a programmable chip 130, the programmable chip 130 includes a processing unit and n matching units, the processing unit is respectively connected with the first processor 120, the first memory 110 and the n matching units, the n matching units are sequentially connected, n is greater than or equal to 1, n is a natural number, and the first processor 120 is connected with the first memory 110.
The first processor 120 is configured to send the original list to the processing unit.
The processing unit is used for constructing a bloom filter bitmap according to the original list, dividing the bloom filter bitmap into n sub-bitmaps, and storing the n sub-bitmaps into n matching units respectively.
The n matching units are used for obtaining n sub-bitmaps.
The processing unit is further configured to receive the data to be stored, sequentially determine whether the data to be stored hits through the sub-bitmaps of the n matching units, obtain hit data if hit, and store the hit data to the first memory 110, otherwise discard the data to be stored.
The programmable switch 100 provided by the embodiment of the application realizes the filtration of the data to be stored by applying the bloom filter, effectively improves the data capacity of the programmable switch 100, and has the characteristics of low cost and high throughput.
In another possible implementation, the first processor 120 is further configured to write the original list into the data structure, generate an original list structure table, match the hit data with the original list structure table, obtain the exact data if the matching is successful, and store the exact data into the first memory 110, otherwise discard the hit data.
Optionally, the data structure comprises a hash table or a tree structure.
In another possible embodiment, referring to fig. 5 in combination, the programmable switch 100 further includes a second processor 140 and a second memory 150, where the second processor 140 is connected to the first processor 120, the programmable chip 130, and the second memory 150, respectively.
The first processor 120 is configured to send the original list to the second processor 140.
The programmable chip 130 is used to send hit data to the second processor 140.
The second processor 140 is configured to store the original list in the data structure, generate an original list structure table, match the hit data with the original list structure table, obtain the accurate data if the match is successful, store the accurate data in the second memory 150, and discard the hit data if the match is not successful.
By providing the second processor 140 and the second memory 150, the data capacity of the programmable switch 100 can be effectively increased.
It should be noted that, when the throughput of the data filtered by the bloom filter is within the maximum bandwidth between the first processor 120 and the programmable chip 130 and within the processing capability of the first processor, only the scheme consisting of the first processor 120, the programmable chip 130 and the first memory 110 may be selected.
Optionally, the second memory 150 is a mass memory.
In order to better understand the technical solutions provided by the embodiments of the present application, the embodiments of the present application are described below with reference to specific chips.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present invention should be included in the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (7)
1. The utility model provides a data expansion method which characterized in that is applied to programmable switch, the programmable switch includes first treater, first memory and programmable chip, the programmable chip includes processing unit and n pieces of matching unit, processing unit respectively with first treater, first memory and n pieces of matching unit connect, n pieces of matching unit connect gradually, n is greater than or equal to 1, and n is natural number, first treater with first memory connects, the method includes:
the first processor sends an original list to the processing unit;
the processing unit constructs a bloom filter bitmap according to the original list, divides the bloom filter bitmap into n sub-bitmaps, and stores the n sub-bitmaps into n matching units respectively;
n matching units obtain n sub-bitmaps;
the processing unit receives data to be stored, judges whether the data to be stored hits or not in sequence through the sub-bitmaps of the n matching units, obtains hit data if the data hit, stores the hit data into the first memory, and discards the data to be stored if the data hit does not hit;
the step of receiving data to be stored by the processing unit and judging whether the data to be stored hits or not by n matching units comprises the following steps: the processing unit receives data to be stored, calculates the data to be stored through m hash functions to obtain m hash values, and sends the m hash values to the first matching unit; part or all of the matching units sequentially judge whether m hash values exist in the corresponding sub-bitmap, if so, the corresponding position values are taken out, and if the corresponding position values are 1, the matching result of the hash values is set to be 1; the processing unit judges whether the matching results of m hash values are all 1; if yes, judging that the data to be stored hits;
the step of sequentially judging whether m hash values exist in the corresponding sub-bitmaps by the matching unit includes: the first matching unit sequentially judges whether m hash values exist in the sub-bitmap of the first matching unit, and sends the hash values which do not exist to the next matching unit for judgment until all n matching units judge to be finished or all m hash values exist in the sub-bitmap, and judging to be finished;
the step of the first matching unit sequentially judging whether m hash values exist in the sub-bitmap of the first matching unit includes: subtracting the m hash values respectivelyObtaining m calculated values; when the calculated value is greater than or equal to 0 and less than +.>When the hash value corresponding to the calculated value is judged to exist in the sub-bitmap of the first matching unit; where x is the number of bits of the bloom filter bitmap.
2. The data expansion method of claim 1, wherein after the step of storing the hit data to the first memory, the method further comprises:
the first processor writes the original list into a data structure, generates an original list structure table, matches the hit data with the original list structure table, obtains accurate data if the matching is successful, stores the accurate data into the first memory, and discards the hit data if the matching is not successful.
3. The data expansion method according to claim 1, wherein the step of obtaining n sub-bitmaps by the n matching units includes:
and setting a default matching table item in the n matching units respectively, and obtaining the sub-bitmap after the matching unit performs processing action according to the default matching table item.
4. The programmable switch is characterized by comprising a first processor, a first memory and a programmable chip, wherein the programmable chip comprises a processing unit and n matching units, the processing unit is respectively connected with the first processor, the first memory and the n matching units, the n matching units are sequentially connected, n is more than or equal to 1, n is a natural number, and the first processor is connected with the first memory;
the first processor is used for sending an original list to the processing unit;
the processing unit is used for constructing a bloom filter bitmap according to the original list, dividing the bloom filter bitmap into n sub-bitmaps, and respectively storing the n sub-bitmaps into n matching units;
the n matching units are used for obtaining n sub-bitmaps;
the processing unit is further configured to receive data to be stored, sequentially determine whether the data to be stored hits through sub-bitmaps of n matching units, obtain hit data if hit, and store the hit data to the first memory, otherwise discard the data to be stored;
the method for receiving data to be stored by the processing unit and judging whether the data to be stored hits or not by n matching units comprises the following steps: the processing unit receives data to be stored, calculates the data to be stored through m hash functions to obtain m hash values, and sends the m hash values to the first matching unit; part or all of the matching units sequentially judge whether m hash values exist in the corresponding sub-bitmap, if so, the corresponding position values are taken out, and if the corresponding position values are 1, the matching result of the hash values is set to be 1; the processing unit judges whether the matching results of m hash values are all 1; if yes, judging that the data to be stored hits;
the method for sequentially judging whether the m hash values exist in the corresponding sub-bitmaps by the matching unit comprises the following steps: the first matching unit sequentially judges whether m hash values exist in the sub-bitmap of the first matching unit, and sends the hash values which do not exist to the next matching unit for judgment until all n matching units judge to be finished or all m hash values exist in the sub-bitmap, and judging to be finished;
the method for the first matching unit to sequentially judge whether the m hash values exist in the sub-bitmap of the first matching unit includes: subtracting the m hash values respectivelyObtaining m calculated values; when the calculated value is greater than or equal to 0 and less than +.>When the hash value corresponding to the calculated value is judged to exist in the sub-bitmap of the first matching unit; where x is the number of bits of the bloom filter bitmap.
5. The programmable switch of claim 4, wherein the first processor is further configured to:
writing the original list into a data structure, generating an original list structure table, matching the hit data with the original list structure table, obtaining accurate data if the matching is successful, storing the accurate data into the first memory, and otherwise discarding the hit data.
6. The programmable switch of claim 5, wherein the data structure comprises a hash table or a tree structure.
7. The programmable switch of claim 4, further comprising a second processor and a second memory, the second processor being coupled to the first processor, the programmable chip, and the second memory, respectively;
the first processor is used for sending an original list to the second processor;
the programmable chip is used for sending the hit data to the second processor;
the second processor is configured to store the original list into a data structure, generate an original list structure table, match the hit data with the original list structure table, obtain accurate data if the matching is successful, and store the accurate data into the second memory, otherwise discard the hit data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111273580.8A CN113904999B (en) | 2021-10-29 | 2021-10-29 | Data expansion method and programmable switch |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111273580.8A CN113904999B (en) | 2021-10-29 | 2021-10-29 | Data expansion method and programmable switch |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113904999A CN113904999A (en) | 2022-01-07 |
| CN113904999B true CN113904999B (en) | 2023-08-08 |
Family
ID=79026936
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111273580.8A Active CN113904999B (en) | 2021-10-29 | 2021-10-29 | Data expansion method and programmable switch |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113904999B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103873371A (en) * | 2014-02-21 | 2014-06-18 | 北京邮电大学 | Name routing fast matching search method and device |
| CN108140050A (en) * | 2016-04-25 | 2018-06-08 | 华为技术有限公司 | A kind of method and device using Bloom filter filtering file |
| CN110083312A (en) * | 2019-04-28 | 2019-08-02 | 联想(北京)有限公司 | Disk expansion method, device and computer equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7930547B2 (en) * | 2007-06-15 | 2011-04-19 | Alcatel-Lucent Usa Inc. | High accuracy bloom filter using partitioned hashing |
| US8817541B2 (en) * | 2012-11-09 | 2014-08-26 | Sandisk Technologies Inc. | Data search using bloom filters and NAND based content addressable memory |
| US9740714B2 (en) * | 2014-02-06 | 2017-08-22 | International Business Machines Corporation | Multilevel filters for cache-efficient access |
| US10565204B2 (en) * | 2016-11-14 | 2020-02-18 | Sap Se | Hash collision tables for relational join operations |
-
2021
- 2021-10-29 CN CN202111273580.8A patent/CN113904999B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103873371A (en) * | 2014-02-21 | 2014-06-18 | 北京邮电大学 | Name routing fast matching search method and device |
| CN108140050A (en) * | 2016-04-25 | 2018-06-08 | 华为技术有限公司 | A kind of method and device using Bloom filter filtering file |
| CN110083312A (en) * | 2019-04-28 | 2019-08-02 | 联想(北京)有限公司 | Disk expansion method, device and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113904999A (en) | 2022-01-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9269411B2 (en) | Organizing data in a hybrid memory for search operations | |
| US8780926B2 (en) | Updating prefix-compressed tries for IP route lookup | |
| US7827182B1 (en) | Searching for a path to identify where to move entries among hash tables with storage for multiple entries per bucket during insert operations | |
| Bando et al. | Flashtrie: Hash-based prefix-compressed trie for IP route lookup beyond 100Gbps | |
| CN112425131B (en) | ACL rule classification method, ACL rule search method and ACL rule classification device | |
| CN107431660B (en) | Search device, search method, and recording medium | |
| EP2220824A1 (en) | Low power ternary content-addressable memory (tcams) for very large forwarding tables | |
| JP2004229163A (en) | Device and method for retrieving fixed length data, computer program, and computer readable recording medium | |
| Zhao et al. | Exploiting graphics processors for high-performance IP lookup in software routers | |
| CN106302172A (en) | Support Hash lookup and the storage of route querying, lookup method and device simultaneously | |
| CN106789733B (en) | Device and method for improving large-scale network flow table searching efficiency | |
| US20140358886A1 (en) | Internal search engines architecture | |
| CN102045412B (en) | Method and equipment for carrying out compressed storage on internet protocol version (IPv)6 address prefix | |
| Jiang et al. | A memory-balanced linear pipeline architecture for trie-based IP lookup | |
| Le et al. | Memory-efficient and scalable virtual routers using FPGA | |
| CN107330094A (en) | The Bloom Filter tree construction and key-value pair storage method of dynamic memory key-value pair | |
| CN113904999B (en) | Data expansion method and programmable switch | |
| Wirth et al. | Scalable and flexible high-performance in-network processing of hash joins in distributed databases | |
| Erdem et al. | Array design for trie-based ip lookup | |
| Afek et al. | Recursive design of hardware priority queues | |
| CN114338529A (en) | Quintuple rule matching method and device | |
| JP3558151B2 (en) | Data search circuit | |
| CN106330721B (en) | IP method for searching route and device | |
| Yang et al. | Architecture and performance models for scalable IP lookup engines on FPGA | |
| Van Tu et al. | A high throughput pipelined hardware architecture for tag sorting in packet fair queuing schedulers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |