CN103473237A - Key value grouping method - Google Patents
Key value grouping method Download PDFInfo
- Publication number
- CN103473237A CN103473237A CN2012101872215A CN201210187221A CN103473237A CN 103473237 A CN103473237 A CN 103473237A CN 2012101872215 A CN2012101872215 A CN 2012101872215A CN 201210187221 A CN201210187221 A CN 201210187221A CN 103473237 A CN103473237 A CN 103473237A
- Authority
- CN
- China
- Prior art keywords
- key assignments
- lookup
- key
- searching
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a key value grouping method which comprises the following steps of firstly, physically dividing a lookup engine into different lookup units, wherein when the lookup engine carries out lookup, each lookup unit can independently output a rule with the highest matching priority; secondly, when key values obtained by splitting the lookup rules are written into a search engine, writing the split key values which can be subjected to multiple matching mutually into different lookup units, so that in the lookup process, even though multiple matching occurs, each lookup unit can still reserve results of the split key values subjected to multiple matching; finally, carrying out combination and priority comparison on the reserved results and matching results of other split key values.
Description
Technical field
The invention belongs to networking technology area, particularly relate to the key assignments group technology in a kind of NetFind.
Background technology
In existing for example, NetFind based on Lookup engine (TCAM), for the part that repeats in storage rule in Lookup engine, lack effective method always and realize to repeating the multiplexing of part, to save the storage space of Lookup engine.In addition, for searching of the overlength list item that will realize in network, lack too the effective decomposition to the overlength list item, to complete searching of overlength list item.
At network processing unit (switch, the equipment that router etc. are processed the data message) in the process of message being processed, need to be with Lookup engine to complete various searching, Lookup engine can return to the regular list item address that finds to read associated data or directly to return to associated data, and then network processing unit can determine how to process this packet literary composition according to the associated data of returning.
In the regular list item of storing at Lookup engine, exist a plurality of regular list items and all comprised same part simultaneously, if do not carry out multiplexing to this partial data, the space that will cost a lot of money in Lookup engine is to complete the storage to these list items, particularly, along with the ipv6 address progressively replaces the ipv4 address, this problem is more and more outstanding.
In addition, for the list item of overlength, search, also lack effective list item at present and split means to complete the merging that splits the list item lookup result.
If the part repeated in regular list item is complete exact matching (will search), the prior art scheme can be to this problem supplying method (United States Patent (USP) that Netlogic company applies at present, the patent No.: US7,694,068 B1, patent name: RE-ENTRANT PROCESSING IN A CONTENT ADDRESSABLE MEMORY), the original purpose of this patent is in order to save the bandwidth between network processing unit and Lookup engine, but objectively also can be used to provide the recycling of exact matching repeating part.The method provided according to this patent, can be split list item, then first the part that repeats of list item searched, then will search the new list item of part reformulation that the result returned and back will search to complete follow-up searching.Only need so once this repeating part of storage in Lookup engine, just can represent all list items that contain this repeating part.In addition, for overlength, search the part that repeats in list item, if institute's repeating part is complete exact matching, also can or repeatedly search by 2 times.
The repeating part list item that it is exact matching that prior art can only complete repeating part is shared, otherwise the problem that can run into multiple coupling of searching to repeating part, for example: regular A is split as to { a0, a1}, rule B is split as { a0, b1}, rule C is split as { c0, c1}, the a0 fractional reuse in regular A and regular B wherein, if carry out list item search in a0, c0 mates (due to a0 with the appropriate section found key value simultaneously, c0 is not exact matching, therefore multiple coupling may occur), the result initially found is two, can't only a result have been returned to follow-up searching.So just can't complete the saving on the Lookup engine storage space.
Similarly, thus still can run into multiple coupling for the fractionation of overlength list item can't complete follow-up problem of searching.
Summary of the invention
In order at least one of to address the above problem, the present invention proposes the key assignments group technology after a kind of Lookup engine rule is split, comprise the following steps:
At first, Lookup engine is divided into to the different unit of searching physically, when Lookup engine is searched, each searches the independently the highest rule of output matching priority of unit;
Secondly, when the key assignments that search rule is split into writes search engine, the fractionation key assignments that multiple coupling can occur between mutual is write to different searching in unit, even multiple coupling has occurred like this when searching, each searches unit still can retain each result that multiple coupling has occurred for these;
Finally, the result of these results of retaining and other fractionation key value match is merged and priority ratio.
Preferably, search the stored data in unit and comprise data segment, the mask section, the beginning flag position, group # and follow-up zone bit, wherein said data segment and mask section mean jointly by the key assignments after splitting, whether the beginning flag position is initial key assignments for indicating this key assignments, group # is for indicating the association between key assignments, and identical group # means the association of these two key assignments, and whether follow-up zone bit has follow-up key assignments for indicating this key assignments.
Preferably, search the stored data in unit and comprise data segment, the mask section, the initial segment and ending segment, wherein said data segment and mask section mean jointly by the key assignments after splitting, described the initial segment is to mean that this paragraph key value is initial key assignments at 0 o'clock, and described ending segment is to mean that this paragraph key value is for finishing key assignments at 0 o'clock, when the initial segment of each key assignments equates to mean the key assignments after they belong to the regular fractionation of same with the ending segment of a upper key assignments.
The accompanying drawing explanation
Above-mentioned and/or additional aspect of the present invention and advantage are from obviously and easily understanding becoming the description of embodiment in conjunction with following accompanying drawing.
The search rule of Fig. 1 embodiment of the present invention splits schematic diagram;
The key value match circuit diagram of Fig. 2 embodiment of the present invention;
A kind of data layout schematic diagram of searching unit that writes of Fig. 3 embodiment of the present invention;
The data layout that utilizes Fig. 3 of Fig. 4 embodiment of the present invention is to splitting the associated schematic diagram of key assignments marshalling;
The another kind of Fig. 5 embodiment of the present invention writes the data layout schematic diagram of searching unit;
The data layout that utilizes Fig. 5 of Fig. 6 embodiment of the present invention is to splitting the associated schematic diagram of key assignments marshalling;
The software flow schematic diagram of Fig. 7 embodiment of the present invention;
Fig. 8 embodiment of the present invention search the implementation schematic diagram;
The hardware configuration schematic diagram of the Hash mapping of Fig. 9 embodiment of the present invention.
Embodiment
Below describe embodiments of the invention in detail, the example of described embodiment is shown in the drawings, and wherein same or similar label means same or similar element or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment be described with reference to the drawings, only for explaining the present invention, and can not be interpreted as limitation of the present invention.It is evident that; described embodiment is a part of embodiment of the present invention but not all; embodiment based in the present invention, those of ordinary skills, not making under the creative work prerequisite the every other embodiment obtained, belong to protection scope of the present invention.
This patent is method in Lookup engine (comprise TCAM etc. search device) and realize explaining with it mainly; in addition; chip and system for integrated Lookup engine also may adopt the described method of this patent, therefore also in the protection domain of this patent.
At first illustrate in conjunction with Fig. 1 the problem that this patent will solve---Lookup engine waste of storage space and overlength rule searching: in Fig. 1, all comprise common part key assignments a0 in key assignments A in rule set, key assignments C, key assignments E, all comprise common part key assignments b0 in key assignments B, key assignments D, key assignments F.If, according to the storage of the key assignments in original rule set key assignments A-F, do not take full advantage of key assignments A, C, E can share the characteristics of key assignments a0, causes the waste of storage space in Lookup engine.The purpose of this patent is to wish that the key assignments A-F that the whole piece in Fig. 1 will be searched is converted into the key assignments after right figure decomposes, and at key assignments a0, in the very long situation of b0, just can greatly save the storage space of Lookup engine.When network processing unit sends look-up command, need Lookup engine that key assignments to be found is split into to the to be found key assignments corresponding with look-up table 0 and look-up table 1 and searched respectively, then according to the joining method of appointment, find out the rule of final coupling.
When processing the overlength rule searching, after an overlength rule being split into to the shorter key assignments that 2 or a plurality of Lookup engine can support, the overlength rule searching still can complete.
Before the solution provided at the specific explanations this patent, at first in conjunction with Fig. 1, explain the multiple coupling usually occurred in lower network, that is to say corresponding same key assignments to be found, key assignments a0 and key assignments b0 in look-up table may mate simultaneously.What the look-up table O of for example take deposited is example as the ipv4 address, and key assignments a0 is 192.168.x.x, and key assignments b0 is 192.168.100.100, and when key assignments to be found is 192.168.100.100, a0 and b0 can mate simultaneously.Now can't determine to retain key assignments a0 or key assignments b0 according to the priority of original key assignments A-F and using the possible option mated as finally, because the matched rule that the rule that in key assignments A-F, low priority is corresponding is likely finally won and searched for this on the contrary due to the coupling of corresponding key assignments in look-up table 1.Therefore no matter be to solve the Lookup engine waste of storage space, or the overlength rule searching, all need to solve the problem of multiple coupling.
For solving multiple matching problem, this patent provides a kind of method the key assignments grouping after the rule fractionation to solve the above problems, and its method is as follows:
At first Lookup engine needs to be divided into physically the different unit of searching, and when Lookup engine is searched, each searches the independently the highest rule of output matching priority of unit.When key assignments search rule split at software writes search engine, the fractionation key assignments that multiple coupling can occur between mutual is write to different searching in unit, even multiple coupling has occurred like this when searching, each searches unit still can retain each result that multiple coupling has occurred for these, then these results can be merged with other results that splits key value match and priority ratio.As shown in Figure 2:
In Fig. 2, p0, p1 represents respectively the key assignments after key assignments to be found splits according to the fractionation form of key assignments A-F, after entering Lookup engine, split circuit through key assignments it is sent into to searching in unit and searched separately, search mated result and will send into the matching result consolidation circuit, thereby obtain final mate and the highest result of priority.Wherein, search stored data in unit and correspond respectively to each key assignments split in Fig. 1, with a0_g, a1_g is example, and its form as shown in Figure 3.
Wherein, the mask that data a0 and a0 are corresponding means jointly by the a0 key assignments after splitting, 1 Bit data 0 after data a0 means that a0 is initial key assignments, gid is used for meaning group #, so that the gid corresponding with key assignments a1 equates to mean the associated of these two key assignments in the matching result consolidation circuit of back, 1 Bit data 1 after gid means that key assignments a0 back also has associated data.Therewith roughly the same, it is follow-up key assignments that 1 after key assignments a1 means, O means after key assignments a1 onrelevant key assignments again.Fig. 4 utilizes multiplexing to associated schematic diagram: a0 splitting the key assignments marshalling and a1 of this form, can mean the association between key assignments with Fig. 5.
In superincumbent form, HeadG means while being O that this paragraph key value is initial key assignments, and TailG means the associated end of this paragraph key value at 0 o'clock.When the HeadG of each key assignments equates with the TailG numerical value of the preceding paragraph data, just indicate that they are the key assignments that belong to after same rule splits, so just can realize flexibly shared after key assignment combination.Fig. 6 is split an associated schematic diagram between key assignments by the form in Fig. 5.
To sum up, the said method that utilizes this patent to provide, even in the situation that split between key assignments multiple coupling may occur, also can effectively realize that duplicate key value shares, and realize the overlength rule searching.
Concrete implementation procedure is divided into software section and hardware components, for software section, single part duplicate key value of take is shared and used the correlating method in Fig. 3 is example, its treatment scheme is mainly: analyze each rule to be added, see whether its split key assignments has been present in the corresponding key assignments of fractionation database; If exist, remember to have existed the gid of key assignments, and this gid is served as a mark to remainder, then add in its corresponding key assignments storehouse, if there is no, get untapped gid, and two parts are all added in corresponding fractionation key assignments database with this gid.Its process flow diagram as shown in Figure 7.
Hardware is realized part, and namely the real search procedure of carrying out is as follows: at first key assignments to be found is divided into to two parts by the fractionation rule set in advance, two parts key assignments divides to be clipped in corresponding separately key assignments storehouse to be searched; Gid in two parts key value match result is compared respectively, if there is the situation that gid is equal, show that these two split the key assignment combination success, can participate in final priority ratio; Finally combined the priority ratio of successful key assignments, obtained having the search rule of limit priority.Its process flow diagram as shown in Figure 8.
The 3rd step in above-mentioned flow process, when the gid in two groups of (many groups) lookup results is compared respectively, if hardware circuit is taked the directly mode of comparison, will produce large numbers of comparers, to hardware, it is very large expending, therefore can select the method for Hash mapping to compare, its concrete relevant hardware configuration as shown in Figure 9.
The foregoing circuit structure can be avoided the direct comparison to gid, and the hash-collision problem that may exist for it can adopt multiple Hash mapping, adopt the modes such as standby comparator circuit.The too much problem of comparator circuit that in addition, also can adopt the modes such as software pre-sets to avoid hash-collision to bring.
The present invention can effectively solve the multiple matching problem in the multiplexing and overlength rule searching of duplicate key value.Prior art can only realize that duplicate key value is multiplexing in the exact matching situation.
In addition, utilize the method for HeadG and TailG docking, the present invention can also realize multiple sharing.
Except being applied to Lookup engine (comprise TCAM/CAM, and other Lookup engines of realizing with algorithm), can also be integrated in the chip or system of Lookup engine at other.
In addition, the key assignments form is done to some simple replacements and also can complete function, but still belong to the scope of this patent on method.
Claims (3)
1. a key assignments group technology, is characterized in that, comprises the following steps:
At first, Lookup engine is divided into to the different unit of searching physically, when Lookup engine is searched, each searches the independently the highest rule of output matching priority of unit;
Secondly, when the key assignments that search rule is split into writes search engine, the fractionation key assignments that multiple coupling can occur between mutual is write to different searching in unit, even multiple coupling has occurred like this when searching, each searches unit still can retain each result that multiple coupling has occurred for these;
Finally, the result of these results of retaining and other fractionation key value match is merged and priority ratio.
2. key assignments group technology according to claim 1, it is characterized in that, the fractionation key assignments that multiple coupling can occur between is mutually write different while searching unit, write the data of searching unit and comprise data segment, the mask section, the beginning flag position, group # and follow-up zone bit, wherein said data segment and mask section mean jointly by the key assignments after splitting, whether the beginning flag position is initial key assignments for indicating this key assignments, group # is for indicating the association between key assignments, identical group # means the association of these two key assignments, whether follow-up zone bit has follow-up key assignments for indicating this key assignments.
3. key assignments group technology according to claim 1, it is characterized in that, the fractionation key assignments that multiple coupling can occur between is mutually write different while searching unit, write the data of searching unit and comprise data segment, the mask section, the initial segment and ending segment, wherein said data segment and mask section mean jointly by the key assignments after splitting, described the initial segment is to mean that this paragraph key value is initial key assignments at 0 o'clock, described ending segment is to mean that this paragraph key value is for finishing key assignments at 0 o'clock, when equating to mean them with the ending segment of a upper key assignments, the initial segment of each key assignments belongs to the key assignments after the same rule splits.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101872215A CN103473237A (en) | 2012-06-08 | 2012-06-08 | Key value grouping method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101872215A CN103473237A (en) | 2012-06-08 | 2012-06-08 | Key value grouping method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103473237A true CN103473237A (en) | 2013-12-25 |
Family
ID=49798093
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012101872215A Pending CN103473237A (en) | 2012-06-08 | 2012-06-08 | Key value grouping method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103473237A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104012053A (en) * | 2014-04-30 | 2014-08-27 | 华为技术有限公司 | Searching device and method |
| WO2016169322A1 (en) * | 2015-04-22 | 2016-10-27 | 中兴通讯股份有限公司 | Query method and device for database, and computer storage medium |
| CN108959209A (en) * | 2018-07-03 | 2018-12-07 | 武汉斗鱼网络科技有限公司 | A kind of processing method, device, terminal and the storage medium of barrage message |
| CN109905322A (en) * | 2019-04-01 | 2019-06-18 | 盛科网络（苏州）有限公司 | A kind of method and device of message matching information pre-processing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7694068B1 (en) * | 2005-12-08 | 2010-04-06 | Netlogic Microsystems, Inc. | Re-entrant processing in a content addressable memory |
| CN102291472A (en) * | 2011-09-09 | 2011-12-21 | 华为数字技术有限公司 | Network address lookup method and device |
-
2012
- 2012-06-08 CN CN2012101872215A patent/CN103473237A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7694068B1 (en) * | 2005-12-08 | 2010-04-06 | Netlogic Microsystems, Inc. | Re-entrant processing in a content addressable memory |
| CN102291472A (en) * | 2011-09-09 | 2011-12-21 | 华为数字技术有限公司 | Network address lookup method and device |
Non-Patent Citations (1)
| Title |
|---|
| 张晓琳等: "基于下推自动机的XML数据流递归查询研究", 《计算机工程与设计》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104012053A (en) * | 2014-04-30 | 2014-08-27 | 华为技术有限公司 | Searching device and method |
| CN104012053B (en) * | 2014-04-30 | 2017-01-25 | 华为技术有限公司 | searching device and method |
| US10812382B2 (en) | 2014-04-30 | 2020-10-20 | Huawei Technologies Co., Ltd. | Search apparatus and method |
| US11606295B2 (en) | 2014-04-30 | 2023-03-14 | Huawei Technologies Co., Ltd. | Search apparatus and method |
| WO2016169322A1 (en) * | 2015-04-22 | 2016-10-27 | 中兴通讯股份有限公司 | Query method and device for database, and computer storage medium |
| CN108959209A (en) * | 2018-07-03 | 2018-12-07 | 武汉斗鱼网络科技有限公司 | A kind of processing method, device, terminal and the storage medium of barrage message |
| CN108959209B (en) * | 2018-07-03 | 2022-06-21 | 武汉斗鱼网络科技有限公司 | Barrage message processing method and device, terminal and storage medium |
| CN109905322A (en) * | 2019-04-01 | 2019-06-18 | 盛科网络（苏州）有限公司 | A kind of method and device of message matching information pre-processing |
| CN109905322B (en) * | 2019-04-01 | 2021-05-07 | 盛科网络（苏州）有限公司 | Message matching information preprocessing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100470550C (en) | Information storing method, information search method and engine device | |
| CN102999522B (en) | A kind of date storage method and device | |
| US9342622B2 (en) | Two-phase construction of data graphs from disparate inputs | |
| CN104820717A (en) | Massive small file storage and management method and system | |
| CN103714097A (en) | Method and device for accessing database | |
| CN103473237A (en) | Key value grouping method | |
| CN103034544A (en) | Management method and device for user mode and kernel mode to share memory | |
| CN105630847A (en) | Data storage method as well as data query method, apparatus and system | |
| CN101021858A (en) | Data storing method and device, and data seeking, adding and deleting method | |
| CN103838770A (en) | Logic data partition method and system | |
| US10733186B2 (en) | N-way hash join | |
| CN101853215A (en) | Memory allocation method and device | |
| CN103942205A (en) | Method, device and system for storing and reading catalogue index | |
| CN103873464A (en) | Message processing method and forwarding equipment | |
| CN105843933A (en) | Index building method for distributed memory columnar database | |
| CN107526762A (en) | Service end, multi-data source searching method and system | |
| CN104462144A (en) | Search method and device of packet classification rules | |
| CN108268216A (en) | Data processing method, device and server | |
| CN101534477A (en) | List item management method and device | |
| CN105069035A (en) | Method for realizing data access permission control | |
| CN104636368A (en) | Data retrieval method and device and server | |
| CN106550006A (en) | Cloud Server resource allocation methods and device | |
| CN106302374B (en) | It is a kind of for improve list item access bandwidth and atomicity operation device and method | |
| CN106446050A (en) | Method and system for subscription by aiming at changed data of database | |
| CN100451974C (en) | Method and apparatus for managing session table |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20131225 |