WO2008119269A1 - Procédé et dispositif de moteur de stockage et de consultation d'informations - Google Patents

Procédé et dispositif de moteur de stockage et de consultation d'informations Download PDF

Info

Publication number
WO2008119269A1
WO2008119269A1 PCT/CN2008/070143 CN2008070143W WO2008119269A1 WO 2008119269 A1 WO2008119269 A1 WO 2008119269A1 CN 2008070143 W CN2008070143 W CN 2008070143W WO 2008119269 A1 WO2008119269 A1 WO 2008119269A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
keyword
hash value
storage unit
memory
Prior art date
Application number
PCT/CN2008/070143
Other languages
English (en)
Chinese (zh)
Inventor
Rui Hu
Shijun Shen
Meng Li
Zhenhua Xu
Original Assignee
Huawei Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co., Ltd. filed Critical Huawei Technologies Co., Ltd.
Publication of WO2008119269A1 publication Critical patent/WO2008119269A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L49/00—Packet switching elements
    • H04L49/35—Switches specially adapted for specific applications
    • H04L49/351—Switches specially adapted for specific applications for local area network [LAN], e.g. Ethernet switches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9014—Indexing; Data structures therefor; Storage structures hash tables
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/74—Address processing for routing
    • H04L45/745—Address table lookup; Address filtering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/74—Address processing for routing
    • H04L45/745—Address table lookup; Address filtering
    • H04L45/7453—Address table lookup; Address filtering using hashing

Definitions

  • the present invention relates to the field of communications and database storage, and in particular, to a method for storing information, searching, and an engine device. Background technique
  • the hash-based search method mainly maps a long-length key (keyword) to a shorter-length space according to a certain mapping relationship (hash function), for example, using a hash function to 32bit.
  • the (bit) IP (Internet Protocol) address is mapped to the 20-bit address space (1M).
  • This method has a wide range of applications in fixed length lookups.
  • the hash value of the key value is obtained to obtain the corresponding hash address, and the corresponding subsidiary information can be obtained.
  • the two pieces of information may need to be stored in an address, which creates a collision.
  • the information corresponding to the key value forms a conflicting entry, which brings great difficulty to information storage and searching.
  • FIG. 1 it is a schematic diagram of a linked list storage method in the prior art.
  • the method uses a linked list storage method to store conflicting entries in a hashed list.
  • the received keyword to be searched is hashed to obtain a hash value for finding the hash table.
  • a pointer to the linked list with the conflicting entry is obtained, and then the pointer is used to access the hashed list, and then the hashed list needs to be traversed to find the required information.
  • FIG. 1 it is a schematic diagram of a linked list storage method in the prior art.
  • the method uses a linked list storage method to store conflicting entries in a hashed list.
  • the received keyword to be searched is hashed to obtain a hash value for finding the hash table.
  • From the hash table a pointer to the linked list with the conflicting entry is obtained, and then the pointer is used to access the hashed list, and then the hashed list needs to be traversed to find the required information.
  • the hash table entry 0 The pointer points to a hash list, the length of the hash list is 1, indicating a conflict-free entry; and the table entry X pointer of the hash table points to a hash list, the length of the hash list is 3, indicating that There are 3 conflicting entries in the entry X.
  • the search information requires multiple memory accesses, and high-speed search cannot be realized.
  • Fig. 2 it is a schematic diagram of the multi-hash function method used in the prior art.
  • the method uses multiple hash functions to hash the keywords to be searched, and then reads the information into the corresponding multiple hash tables for searching.
  • the first hash value is calculated using the first hash function to obtain the first hash value, if the position of the hash table corresponding to the first hash value (which may be referred to as the first hash table) is not
  • the table item that is, when there is no conflict, the corresponding information is stored in the position, and no other hash function is used for calculation; if there is a conflict in the position, the second hash function is used to recalculate the keyword.
  • the second hash value is used.
  • the corresponding information is stored in the location, and the information storage ends; Otherwise, you need to continue to use other hash functions for calculations until there is no conflict.
  • the information search multiple hash functions are used to calculate the keywords, and at the same time, the corresponding information is obtained, and after comparing with the keywords to be searched, the corresponding information is found.
  • the less the hash table needs to be stored the less the table entry information needs to be stored. For example, the second hash table needs to store less entry information than the first hash table needs to store.
  • each hash table can be reduced step by step.
  • This method uses multiple hash functions to offset the randomness of the information distribution, which can effectively reduce the probability of collisions.
  • high memory bandwidth is required, which brings great difficulties to hardware design.
  • FIG. 3 it is a schematic diagram of a CAM-based search in the prior art, and a logic controller can Is a logic device or dedicated CPU (Central Processing Unit) provides the CAM with the keyword to be searched.
  • the CAM searches through the hardware in parallel according to the keyword, outputs the memory address of the corresponding information, and then according to the address
  • the attached external information is returned to the logic controller in the corresponding external memory. This allows the required information to be obtained with only one memory read operation.
  • CAM-based methods are used in a large number of existing communication devices, especially routers.
  • large-capacity CAM chips cannot be fabricated, and a large amount of data information, such as 32b i t information larger than 1M, cannot currently be stored.
  • the cost and power consumption of the CAM chip are greatly increased as the capacity increases.
  • an embodiment of the present invention provides a method for storing information, including: receiving information to be stored, where a keyword of the information to be stored is m bits (m is a natural number); using the first hash function to The keyword to be stored is hashed to obtain first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, And 1 ⁇ 111);
  • the other (m ⁇ n ) bit information in the first mapping information is used as the first compression keyword, together with the to-be-stored
  • the attached information of the information is stored together in a storage unit corresponding to the first hash value in the first memory.
  • the embodiment of the invention further provides an engine device, including:
  • a logic controller configured to receive information to be stored, where the keyword of the information to be stored is m bits (m is a natural number), and the keyword of the information to be stored is hashed by using a first hash function to obtain a first a mapping information, the first mapping information is m bits, and n bits in the first mapping information Information as the first hash value (n is a natural number, and 1 ⁇ 111);
  • a first memory when the storage unit corresponding to the first hash value is idle, used to use other (m - n ) bit information in the first mapping information as the first compression keyword, together with the waiting
  • the attached information storing the information is stored together in the storage unit corresponding to the first hash value.
  • an embodiment of the present invention provides a method for searching for information, including:
  • the keyword is m bits (m is a natural number);
  • Searching the first memory according to the first hash value acquiring keyword information in a storage unit corresponding to the first hash value, and comparing the keyword information with the keyword to determine the When the storage unit corresponding to the first hash value corresponds to the keyword, the information in the storage unit corresponding to the first hash value is acquired.
  • the embodiment of the invention further provides another method for searching for information, including:
  • the keyword is m bits (m is a natural number);
  • the first hash information is obtained by hashing the key row by using a first hash function, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value. (n is a natural number, and n is less than m),
  • an embodiment of the present invention provides an engine apparatus, including:
  • a first memory for storing information
  • a logic controller configured to receive a keyword to be searched, the keyword is m bits (m is a natural number), and the first hash function is used to perform hash operation on the keyword to be searched to obtain first mapping information.
  • the first mapping information is m bits, and the n-bit information in the first mapping information is used as a first hash value (n is a natural number, and n is less than m); searching according to the first hash value.
  • a memory the keyword information in the storage unit corresponding to the first hash value is obtained, and the keyword information is compared with the keyword to determine the storage corresponding to the first hash value And when the unit corresponds to the keyword, acquiring information in the storage unit corresponding to the first hash value.
  • Another embodiment of the present invention provides an engine device, including:
  • a content address storage for storing information
  • a first memory for storing information
  • a logic controller configured to receive a keyword to be searched, the keyword is m bits (m is a natural number), and the content address memory is searched according to the keyword, and the keyword is hashed by using a first hash function Computing, the first mapping information is obtained, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m); Obtaining information in the storage unit corresponding to the keyword when the storage unit corresponding to the keyword is found in the content address memory; when the keyword is not found in the content address memory And searching for the first memory according to the first hash value, acquiring keyword information in the storage unit corresponding to the first hash value, using the keyword information and the keyword And comparing, when determining that the storage unit corresponding to the first hash value corresponds to the keyword, acquiring information in the storage unit corresponding to the first hash value.
  • the solution provided by the embodiment of the present invention does not store the complete keyword, but uses a keyword compression scheme, that is, the keyword is hashed, compressed into fewer bits, and then stored, so that the same bit width memory can be stored.
  • the number of entries increases, which improves space utilization, reduces memory capacity requirements, and saves costs; it also reduces the probability of collisions.
  • the engine device can store more information, that is, increase the capacity of the engine device.
  • the solution provided by the embodiment of the present invention can simplify the process of information searching and information storage, and is convenient for quick searching and information maintenance.
  • the solution of the embodiment of the content address memory can solve the conflict problem caused by the hash operation, and the keyword compression scheme can reduce the capacity requirement of the content address memory and save costs.
  • FIG. 1 is a schematic diagram of a linked list storage method in the prior art
  • FIG. 2 is a schematic diagram of a multi-hash function method in the prior art
  • FIG. 3 is a schematic diagram of a CAM-based search in the prior art
  • FIG. 4 is a schematic structural view of an engine device in an embodiment of the present invention.
  • FIG. 5 is a flow chart of information storage by an engine device in an embodiment of the present invention.
  • Figure 6 is a flow chart showing the information search by the engine device in one embodiment of the present invention. detailed description
  • An embodiment of the present invention provides an engine apparatus, including a logic controller and a first memory, where the logic controller is configured to receive information to be stored, where the key of the information to be stored is m bits (m is a natural number),
  • the first hash information is hashed by using the first hash function to obtain the first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value ( n is a natural number, and n is smaller than m); when the storage unit in the first memory corresponding to the first hash value is idle, the other (m - n ) bit information in the first mapping information is used as the first compression keyword
  • the auxiliary information along with the information to be stored is stored in the storage unit corresponding to the first hash value.
  • the logic controller when performing information search, receives the keyword to be searched, the keyword is m bits (m is a natural number), and hashes the keyword to be searched by using the first hash function.
  • the first memory obtains keyword information in a storage unit corresponding to the first hash value, and compares the keyword information with the keyword to be searched to determine a storage unit corresponding to the first hash value and the to-be-searched When the keyword is corresponding, the information in the storage unit corresponding to the first hash value is obtained.
  • the keyword information is compared with the keyword to be searched to determine whether the storage unit corresponding to the first hash value corresponds to the keyword to be searched, and there are multiple implementation manners, for example:
  • the other (m - n ) bit information in the mapping information is used as the first compression keyword; comparing the keyword information with the first compression keyword, when the two are the same, determining the storage unit and the corresponding to the first hash value
  • the keywords that are found correspond.
  • the keyword information is combined with the first hash value to obtain a complete keyword, and compared with the keyword to be searched. When the two are the same, the storage unit corresponding to the first hash value is determined to be searched. The corresponding keyword.
  • the above logic controller may include: a first hash unit and a logic unit.
  • the first hash unit performs a hash operation on the keyword of the information to be stored, to obtain the first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first
  • the logical unit receives the first hash value, determines whether the storage unit in the first memory corresponding to the first hash value is idle or occupied, and notifies the first when the storage unit corresponding to the first hash value is idle.
  • the memory stores the first compressed keyword and the attached information of the information to be stored, wherein the first compressed keyword is other (m - n ) bit information in the first mapping information.
  • the engine device may further include: a second memory, correspondingly, the logic controller further includes a second hash unit, when the storage unit in the first memory corresponding to the first hash value is occupied
  • the second hash unit performs a hash operation on the keyword to be stored, to obtain second mapping information, where the second mapping information is m bits, and the t-bit information in the second mapping information is used as the second hash value (t Is a natural number, and t is less than m);
  • the logic unit receives the second hash value, and determines whether the memory cell in the second memory corresponding to the second hash value is idle or occupied, when the memory cell corresponding to the second hash value When idle, the second memory is notified to store the second compressed keyword and the attached information to be stored.
  • Generic information wherein the second keyword is other (m - t) bit information in the second mapping information.
  • the engine apparatus further includes a content address memory for storing a key of the information to be stored and to be stored when a storage unit in the first memory corresponding to the first hash value is occupied
  • the attached information of the information is stored in a free storage unit in the content address memory.
  • the logic controller when performing information search, receives the keyword to be searched, the keyword is m bits (m is a natural number), and the content address memory is searched according to the keyword; meanwhile, the first hash function is used to pair the keyword Performing a hash operation to obtain first mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is smaller than m).
  • the storage unit corresponding to the keyword When the storage unit corresponding to the keyword is found in the content address storage, acquiring information in the storage unit corresponding to the keyword; when the storage unit corresponding to the keyword is not found in the content address storage, according to the first
  • the hash value is searched for the first memory, and the keyword information in the storage unit corresponding to the first hash value is obtained, and the keyword information is compared with the keyword to be searched to determine the storage unit corresponding to the first hash value.
  • information in the storage unit corresponding to the first hash value is obtained.
  • using the keyword information to compare with the keyword to be searched determining whether the storage unit corresponding to the first hash value corresponds to the keyword to be searched, there are multiple implementation manners, where No longer.
  • the engine device includes a content address memory, and further includes a second memory.
  • the logic controller further includes a second hash unit.
  • the second hash unit performs a hash operation on the keyword of the information to be stored to obtain second mapping information, where the second mapping information is m bits, the t-bit information in the second mapping information is taken as the second hash value (t is a natural number, and t is less than m); the logic unit receives the second hash value, and determines the second corresponding to the second hash value
  • the storage unit in the memory is idle or occupied.
  • the second memory When the storage unit corresponding to the second hash value is idle, the second memory is notified to store the second compressed keyword and the auxiliary information of the information to be stored, where the second compressed keyword is Other (m - t ) bit information in the second mapping information; and when the memory cell in the second memory corresponding to the second hash value is occupied, the content address memory will be the key of the information to be stored.
  • the words and the attached information of the information to be stored are stored in an idle storage unit.
  • the first memory and the second memory are not limited to the on-chip memory or the off-chip memory.
  • FIG. 4 is a schematic structural diagram of an engine apparatus in an embodiment of the present invention, which comprises a logic controller 120, a CAM unit 123, and an off-chip memory 134.
  • the logic controller 120 is composed of a logic unit 122, a plurality of hash units 121a and 121b, and an on-chip memory 131.
  • the CAM unit 123 includes a CAM chip 133 and a CAM memory 132.
  • the logic controller 120 can be implemented by using an ASIC (Application Specific Integrated IC) or an FPGA (Field Programmable Gatec Array);
  • the external memory 134 can be implemented using a DRAM (Dynamic Random Accs s Memory); the CAM memory 132 can be an SRAM (Sta tic Random Acces s Memory).
  • the information lookup includes the following process:
  • the logic controller 120 receives the key (key) to be searched and distributes it to each hash unit and logic unit through the data bus 111;
  • the logic unit After receiving the keyword to be searched, the logic unit sends the keyword to the CAM unit 123 for searching through the data bus 115. If the CAM unit 123 finds the information corresponding to the keyword, the CAM memory 132 returns the information to the logic unit 122. At this time, the logic unit 122 no longer uses the calculation result of each hash unit to access the corresponding memory; if the CAM unit 123 does not find the required information, the B' J logic unit 122 respectively according to the calculation result of each hash unit.
  • the corresponding keyword and the auxiliary information are read on the memory corresponding to each hash unit, and the information is returned to the logical unit 122 through the data bus 117, and the logical unit 122 compares the keyword in the information with the keyword to be searched, if If the keyword of the information is found to be the same as the keyword to be searched, the auxiliary information is obtained from the memory corresponding to the same keyword as the keyword to be searched and returned to the logical unit.
  • the keyword to be found It is 32 bits
  • the affiliate information is 20 bits
  • the database to be checked contains 1M entry.
  • the data bus 1 11 is 32 bits
  • 112a is 20 bits
  • 114 is 20 bits
  • 112b has a number of bits less than 112a
  • 1 12b can be set to 15 bits.
  • the hash table corresponding to the hash unit 121a is stored in the off-chip memory 134, the memory space of the off-chip memory is 1M, and the data bit width is 128 bits. Since the hash unit uses the hash function to map the keyword information of 32b it to the address space (1M) of 20b it, that is, the hash unit 121a hashes the 32-bit keyword information to obtain a 32-bit.
  • the mapping information, the 20-bit information in the 32-bit mapping information is used as the first hash value, and the other 12-bit information is stored as the compression key in the off-chip memory 134. This process is called a keyword. compression.
  • the keyword information in the off-chip memory 134 can be used to obtain complete keyword information.
  • the logic unit 122 retrieves the 32-bit information, it is only necessary to combine the hash function used to obtain the complete keyword, as compared with the key to be searched, as shown in Table 1.
  • one storage unit can store four entries, as shown in Table 2.
  • Table 2 Table storage structure in a storage unit with a bit width of 128 bits
  • a storage unit can store 4 conflicting entries, so that even if 4 conflicts occur, all the entries can be obtained in one read, and a fast search can be realized. Since the probability of occurrence of more than 4 collisions in a storage space is relatively small, the remaining unstorable rushes The number of items is relatively small, which is much less likely to be compared with the way of storing 4M entries using 4M 32-bit storage. In addition, by keyword compression, the size of information to be stored in an entry is reduced. The 1M entry can be stored using 128M space, and all key information is stored. The same storage mode requires 512M of storage space. , space utilization is much higher.
  • the above expression simply uses a hash function that uses a storage unit to store entries. Because the distribution of information is random, and a hash function is a fixed format, there may be a large number of conflicts in some information distribution situations. If these conflicting entries are stored in the CAM unit, the space of the CAM unit is required. Large, the cost will be 4 ⁇ high. To this end, the multi-hash function method can be used to offset the effects of randomness of information distribution. In order not to increase the number of off-chip memories, it is considered that the number of conflicting entries of the off-chip memory has been relatively small, so the on-chip memory 1 31 in the logic controller 120 is used to store the hash table corresponding to the hash unit 121 b. .
  • 32 ⁇ 32-bit on-chip memory 1 31 can be used to store a hash table, which is available for existing logic devices.
  • the hash function used by the hash unit 121 b is selected as far as possible from the hash function used by the hash unit 121 a to reduce the probability of collision occurring. This can effectively reduce the probability of collisions and effectively utilize the memory space.
  • one off-chip memory and one on-chip memory shown in FIG. 4 are only an example. In the logic controller 120, there may be only one memory, not limited to on-chip or off-chip, or multiple memories. When there are multiple memories, there are multiple hash tables.
  • on-chip memory In some cases, a large amount of on-chip memory is required to implement storage of all conflicting entries, and on-chip memory resources are very limited. Therefore, for example, in the case where the number of conflicting entries is small, a CAM with a smaller capacity can be used. The unit stores the remaining conflicting entries. This effectively reduces the need for on-chip memory and resolves the problems that conflicts pose to lookups.
  • IK X 32-bit CAM chip 1 33 and 1K ⁇ 20-bit CAM memory 1 32 can be used to store keywords and ancillary information, respectively.
  • a method for information storage is provided, the process is as follows: receiving information to be stored, the keyword of the information to be stored is m bits (m is a natural number); using the first hash function to store information The keyword is hashed to get the first map Information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m);
  • the other (m ⁇ n ) bit information in the first mapping information is stored as the first compression key together with the auxiliary information of the information to be stored.
  • Within the first memory is in a memory location corresponding to the first hash value.
  • the keyword of the information to be stored is hashed by using the second hash function, to obtain the second mapping information,
  • the second mapping information is m bits, and the t bit information in the second mapping information is used as the second hash value (t is a natural number, and t is less than m);
  • the second compressed keyword and the auxiliary information of the information to be stored are stored in a storage unit corresponding to the second hash value in the second memory, where the second compressed keyword is the other of the second mapping information ( m - t ) bit information.
  • the second hash function is used to hash the keyword to be stored, to obtain the second mapping information, and the second mapping information is m bits,
  • the t-bit information in the second mapping information is used as the second hash value (t is a natural number, and t is less than m);
  • the second compressed keyword is And the auxiliary information of the information to be stored is stored in a storage unit corresponding to the second hash value in the second memory, where the second compressed keyword is other (m ⁇ t ) bit information in the second mapping information;
  • the stored item is less than 1M item
  • step 51 1 the logic controller 120 receives the store instruction and the information to be stored, and then transmits it to the hash unit 121a through the data bus;
  • step 512 the hash unit 121a performs an operation using the first hash function to obtain a first hash value and sends it to the logic unit 122 via the data bus 112a;
  • step 51 3 the logic unit 122 correspondingly searches for the storage unit in the off-chip memory 134 according to the first hash value, and finds the storage unit corresponding to the first hash value, and takes Table 2 as an example, which is 4 tables. A total of 128 items;
  • step 514 it is determined whether the storage unit corresponding to the first hash value has a free space (collision table entry ⁇ 4). If there is free space, then in step 515, the logic unit 122 is based on the first hash of the hash unit 121a. The function compresses the keyword;
  • step 516 the logic unit 122 puts the compressed keyword and the auxiliary information into the free space in the storage unit corresponding to the first hash value, stores it in the off-chip memory 134, and then proceeds to step 523 to wait for other instructions. .
  • step 514 if there is no free space (collision table entry > 4), then step 517 is entered, that is, the hash unit 121 b receives the storage instruction and the corresponding information, and the second hash value is calculated by the second hash function and sent.
  • the logic unit 122 searches the on-chip memory 1 31 for the memory location (32 bits) corresponding to the second hash value according to the second hash value.
  • step 519 it is determined whether the storage unit has valid entry information. If not, that is, idle, then in step 520, the logic unit 122 compresses the keyword according to the second hash function of the hash unit 121b. In step 521, the logic unit 122 places the compressed keyword and the attached information into the storage unit, and then proceeds to step 523 to wait for other instructions. In step 519, if a conflict occurs, the logic unit stores the keyword information and the affiliate information in the CAM unit 123. At this point, the information storage process ends, and the flow proceeds to step 523 to wait for other instructions.
  • an embodiment of the present invention also provides a method for information searching, and the process is as follows: Receiving the keyword to be searched, the keyword is m bits (m is a natural number);
  • the first hash information is hashed by using the first hash function to obtain first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m);
  • Searching the first memory according to the first hash value acquiring keyword information in the storage unit corresponding to the first hash value, comparing the keyword information with the keyword to be searched, and determining, corresponding to the first hash value, When the storage unit corresponds to the keyword to be searched, the information in the storage unit corresponding to the first hash value is obtained.
  • the keyword information is compared with the keyword to be searched to determine whether the storage unit corresponding to the first hash value corresponds to the keyword to be searched.
  • the process of information lookup is slightly different, including:
  • the keyword to be searched is m bits (m is a natural number);
  • the first hash information is hashed by using the first hash function to obtain the first mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number) And n is less than m); when the storage unit corresponding to the keyword to be searched is found in the content address memory, the information in the storage unit corresponding to the keyword is obtained; when the search is not found in the content address memory Searching for the first storage unit according to the first hash value, obtaining keyword information in the storage unit corresponding to the first hash value, and comparing the keyword to be searched using the keyword information, When it is determined that the storage unit corresponding to the first hash value corresponds to the keyword to be searched, the information in the storage unit corresponding to the first hash value is obtained.
  • FIG. 6 it is a flowchart of the information search by the engine device in one embodiment of the present invention. Referring to the structural diagram of the engine device in FIG. 4, the information search process is as follows:
  • step 611 the logic controller 120 receives the lookup instruction and the to-be-checked key and transmits them to the hash unit 121a, the hash unit 121b, and the CAM unit 123 via the data buses 111 and 115.
  • step 612 after the CAM unit 123 receives the to-be-checked keyword, the CAM chip 133 performs a parallel search according to the keyword.
  • the address is output to the CAM memory 132, and the CAM memory 132 returns the auxiliary information; If the chip 133 does not find the matching information, it does not return any value to the logical unit; in steps 613 and 614, the two hash units perform a hash calculation outputting a hash value according to the to-be-checked keyword, and the hash value passes through the data bus 112a and 112b is output to the logic unit 122;
  • the logic unit 122 collects the CAM search result and the operation result of the two hash units, wherein the CAM unit 123 searches for the result, that is, the return value has the highest priority. If the CAM unit 123 has a return value, the value is used as the return value. Ancillary information. If the CAM unit 123 has no return value, the hash unit is used to calculate the result to the corresponding memory to obtain the keyword information and the attached information. Each hash unit is a parallel lookup, and all hash calculations and memory accesses are done in parallel, with no priority.
  • step 616 the logic unit 122 determines whether the CAM unit 123 has a return value. If so, indicating that the CAM unit finds the information to be checked, then in step 617, the CAM search result is used as the returned affiliate information. At this point, proceed to step 624, waiting for other instructions to arrive;
  • step 616 if the CAM unit 123 has no return value, indicating that the CAM unit has not found the information to be checked, then in step 618, the result of the hash operation is used to obtain the keyword information and the attached information in the corresponding memory.
  • step 619 logic unit 122 compresses the keywords based on the hash functions of all of the hash units.
  • step 620 logic unit 122 compares the computed compression key with the key information retrieved from each memory. In step 621, it is determined whether or not they are consistent. If they match, in step 622, the attached information stored in the storage unit in which the keyword matching the compressed keyword is stored is returned. At this point, the process proceeds to step 624, waiting for other commands to arrive.
  • step 621 if it is inconsistent, it indicates that the keyword information to be searched for does not exist, then the "no match" information is returned in step 623, and then proceeds to step 624, waiting for other instructions to arrive, and the entire search process ends.
  • Corresponding software in the embodiments of the present invention may be stored in a computer readable storage medium.
  • the solution in the embodiment of the present invention has been subjected to multiple simulation tests in a computer, and the verification operation is reliable and feasible.
  • the solution provided by the embodiment of the present invention does not store the complete keyword, but uses a keyword compression scheme, that is, the keyword is hashed, compressed into fewer bits, and then stored, so that the same bit width memory can be stored.
  • the number of entries increases, which improves space utilization, reduces memory capacity requirements, and saves costs; it also reduces the probability of collisions.
  • the engine device can store more information, that is, increase the capacity of the engine device.
  • the solution provided by the embodiment of the present invention can simplify the process of information searching and information storage, and is convenient for quick searching and information maintenance.
  • the solution of the embodiment of the content address memory can solve the conflict problem caused by the hash operation, and the keyword compression scheme can reduce the capacity requirement of the content address memory and save costs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Storage Device Security (AREA)

Abstract

L'invention concerne un procédé de stockage d'informations qui comporte les opérations consistant à recevoir des informations à stocker munies d'une clé et constituée de m bits (où m est un nombre naturel) ; une première fonction de hachage utilisée pour effectuer une opération de hachage de la clé des informations à stocker de façon à conduire à des premières informations cartographiées constituées de m bits , puis n bits d'informations dans les premières informations cartographiées sont utilisés en tant que première valeur de hachage (où n est un nombre naturel, et n < m) ; lorsqu'une unité de stockage d'un premier stockage correspondant à la première valeur de hachage est libre, les (m - n) autres bits d'informations dans les premières informations cartographiées sont utilisés en tant que première clé comprimée, et stocke la première clé comprimée et les informations supplémentaires des informations à stocker dans l'unité de stockage correspondante. L'invention concerne également un procédé pour consulter des informations et un dispositif de moteur.
PCT/CN2008/070143 2007-04-02 2008-01-18 Procédé et dispositif de moteur de stockage et de consultation d'informations WO2008119269A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CNB2007100739263A CN100470550C (zh) 2007-04-02 2007-04-02 一种信息存储的方法、信息查找的方法及引擎装置
CN200710073926.3 2007-04-02

Publications (1)

Publication Number Publication Date
WO2008119269A1 true WO2008119269A1 (fr) 2008-10-09

Family

ID=38730967

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2008/070143 WO2008119269A1 (fr) 2007-04-02 2008-01-18 Procédé et dispositif de moteur de stockage et de consultation d'informations

Country Status (2)

Country Link
CN (1) CN100470550C (fr)
WO (1) WO2008119269A1 (fr)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112269784A (zh) * 2020-10-28 2021-01-26 中科驭数（北京）科技有限公司 一种基于硬件实现的哈希表结构以及插入、查询和删除方法
US20210073410A1 (en) * 2018-05-15 2021-03-11 Microsoft Technology Licensing, Llc Secure dataset management
CN112699416A (zh) * 2021-01-04 2021-04-23 烽火通信科技股份有限公司 文件存储方法、文件验证方法及电子设备
CN113592920A (zh) * 2021-07-28 2021-11-02 南京物间科技有限公司 一种基于Census和哈希链表的快速立体匹配方法
CN113726660A (zh) * 2021-08-27 2021-11-30 西安微电子技术研究所 一种基于完美哈希算法的路由查找器和方法
CN114785396A (zh) * 2022-03-09 2022-07-22 西安电子科技大学 逻辑端口配置、查找映射及流量管理方法、系统及终端
US11636047B2 (en) 2021-08-31 2023-04-25 Micron Technology, Inc. Hash operations in memory for data sharing

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100470550C (zh) * 2007-04-02 2009-03-18 华为技术有限公司 一种信息存储的方法、信息查找的方法及引擎装置
CN101140592A (zh) * 2007-09-30 2008-03-12 华为技术有限公司 关键字存储、查找方法和设备
CN101604337B (zh) * 2009-07-13 2011-08-24 中兴通讯股份有限公司 一种哈希表项存储、查找装置及方法
CN101692651B (zh) * 2009-09-27 2014-12-31 中兴通讯股份有限公司 一种哈希查找表的方法和装置
CN101826107B (zh) * 2010-04-02 2015-08-05 华为技术有限公司 哈希数据处理方法和装置
CN102754394B (zh) 2010-08-19 2015-07-22 华为技术有限公司 一种哈希表存储、查找方法以及装置
CN102187642B (zh) * 2011-04-14 2015-01-07 华为技术有限公司 在哈希表中添加、查找或删除键值的方法及装置
WO2012106916A1 (fr) * 2011-07-22 2012-08-16 华为技术有限公司 Procédé et appareil pour le traitement de calculs de hachage
CN102629234B (zh) * 2012-01-18 2015-01-21 物联微电子(常熟)有限公司 一种单片机片内置Flash数据快速检索方法
CN103064948B (zh) * 2012-12-27 2015-08-19 中国航空工业集团公司第六三一研究所 基于散列链表的内容寻址方法及相应的存储器电路
CN104182393A (zh) * 2013-05-21 2014-12-03 中兴通讯股份有限公司 基于hash表的关键字映射处理方法和装置
CN103309950A (zh) * 2013-05-22 2013-09-18 苏州雄立科技有限公司 一种键值搜索方法
CN103399920B (zh) * 2013-08-02 2017-04-26 苏州雄立科技有限公司 键值搜索方法、键值搜索装置及芯片
CN104301228A (zh) * 2014-09-12 2015-01-21 中国电子科技集团公司第三十二研究所 网络处理器中查找模块的实现方法及系统
CN106033420A (zh) * 2015-03-11 2016-10-19 杭州华三通信技术有限公司 哈希表的处理方法及装置
CN106933749B (zh) * 2015-12-31 2020-10-13 北京国睿中数科技股份有限公司 应用于缓存验证系统的地址随机方法和装置
CN106708438A (zh) * 2016-12-16 2017-05-24 盛科网络（苏州）有限公司 降低Hash冲突概率的方法及装置
CN108111421B (zh) * 2017-11-28 2021-02-09 苏州浪潮智能科技有限公司 一种基于多重哈希的报文分流方法及装置
CN108172260B (zh) * 2017-12-30 2020-06-26 盛科网络（苏州）有限公司 一种ASIC芯片中Hash模块的验证方法及装置
CN108632046B (zh) * 2018-05-14 2021-05-04 深圳市合众万邦科技有限公司 分级卡、读写装置、医护管理系统及方法
CN109245879A (zh) * 2018-09-06 2019-01-18 杭州光芯科技有限公司 一种存储和查找ip地址映射关系的双哈希算法
CN111107068B (zh) * 2019-12-09 2021-03-09 江苏云涌电子科技股份有限公司 一种fpga高效规则匹配方法及终端
CN113297219A (zh) * 2021-05-21 2021-08-24 山东云海国创云计算装备产业创新中心有限公司 一种字段存储方法、装置、设备及存储介质
CN113204556A (zh) * 2021-05-21 2021-08-03 山东英信计算机技术有限公司 基于低位哈希标签的存储方法、系统、终端及存储介质
CN113395615B (zh) * 2021-06-10 2022-05-06 烽火通信科技股份有限公司 一种消息处理方法、装置、电子设备和可读存储介质
CN114936087B (zh) * 2021-09-29 2023-06-02 华为技术有限公司 一种嵌入向量预取的方法、装置、系统及相关设备
CN114726920A (zh) * 2022-06-07 2022-07-08 恒生电子股份有限公司 Tcp数据的处理方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5920900A (en) * 1996-12-30 1999-07-06 Cabletron Systems, Inc. Hash-based translation method and apparatus with multiple level collision resolution
US6247014B1 (en) * 1998-07-01 2001-06-12 Nortel Networks Limited Method and apparatus for performing hash lookups using valid bit tables with pointers
US7116664B2 (en) * 2002-05-13 2006-10-03 International Business Machines Corporation Lookups by collisionless direct tables and CAMs
CN1912870A (zh) * 2006-09-05 2007-02-14 四川南山之桥微电子有限公司 一种哈希表查表方法
CN101034412A (zh) * 2007-04-02 2007-09-12 华为技术有限公司 一种信息存储的方法、信息查找的方法及引擎装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5920900A (en) * 1996-12-30 1999-07-06 Cabletron Systems, Inc. Hash-based translation method and apparatus with multiple level collision resolution
US6247014B1 (en) * 1998-07-01 2001-06-12 Nortel Networks Limited Method and apparatus for performing hash lookups using valid bit tables with pointers
US7116664B2 (en) * 2002-05-13 2006-10-03 International Business Machines Corporation Lookups by collisionless direct tables and CAMs
CN1912870A (zh) * 2006-09-05 2007-02-14 四川南山之桥微电子有限公司 一种哈希表查表方法
CN101034412A (zh) * 2007-04-02 2007-09-12 华为技术有限公司 一种信息存储的方法、信息查找的方法及引擎装置

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210073410A1 (en) * 2018-05-15 2021-03-11 Microsoft Technology Licensing, Llc Secure dataset management
CN112269784A (zh) * 2020-10-28 2021-01-26 中科驭数（北京）科技有限公司 一种基于硬件实现的哈希表结构以及插入、查询和删除方法
CN112699416A (zh) * 2021-01-04 2021-04-23 烽火通信科技股份有限公司 文件存储方法、文件验证方法及电子设备
CN113592920A (zh) * 2021-07-28 2021-11-02 南京物间科技有限公司 一种基于Census和哈希链表的快速立体匹配方法
CN113592920B (zh) * 2021-07-28 2024-05-10 南京物间科技有限公司 一种基于Census和哈希链表的快速立体匹配方法
CN113726660A (zh) * 2021-08-27 2021-11-30 西安微电子技术研究所 一种基于完美哈希算法的路由查找器和方法
US11636047B2 (en) 2021-08-31 2023-04-25 Micron Technology, Inc. Hash operations in memory for data sharing
CN114785396A (zh) * 2022-03-09 2022-07-22 西安电子科技大学 逻辑端口配置、查找映射及流量管理方法、系统及终端
CN114785396B (zh) * 2022-03-09 2024-04-12 西安电子科技大学 逻辑端口配置、查找映射及流量管理方法、系统及终端

Also Published As

Publication number Publication date
CN101034412A (zh) 2007-09-12
CN100470550C (zh) 2009-03-18

Similar Documents

Publication Publication Date Title
WO2008119269A1 (fr) Procédé et dispositif de moteur de stockage et de consultation d&#39;informations
US11102120B2 (en) Storing keys with variable sizes in a multi-bank database
US8484439B1 (en) Scalable hash tables
US7668890B2 (en) Prefix search circuitry and method
WO2008037201A1 (fr) Procédé et appareil permettant de résoudre une collision de hachage
US7167471B2 (en) Network processor with single interface supporting tree search engine and CAM
US6522632B1 (en) Apparatus and method for efficient prefix search
US7930516B1 (en) Linked list traversal with reduced memory accesses
CN111382327B (zh) 一种字符串匹配装置及方法
US20060248095A1 (en) Efficient RAM lookups by means of compressed keys
Bando et al. FlashTrie: beyond 100-Gb/s IP route lookup using hash-based prefix-compressed trie
US20020138648A1 (en) Hash compensation architecture and method for network address lookup
US20090282167A1 (en) Method and apparatus for bridging
WO2012013023A2 (fr) Procédé de recherche par hachage réduisant les conflits de hachage
US20140358886A1 (en) Internal search engines architecture
US20170078200A1 (en) Multi-table hash-based lookups for packet processing
WO2019034136A1 (fr) Mémorisation et interrogation de données d&#39;entrée
US11782895B2 (en) Cuckoo hashing including accessing hash tables using affinity table
WO2016206490A1 (fr) Dispositif et procédé d&#39;amélioration de bande passante d&#39;accès à un article et d&#39;opération atomique
US8503442B2 (en) Transmission information transfer apparatus and method thereof
CN112818185A (zh) 一种基于sram的最长前缀匹配硬件系统查找的方法
US6343289B1 (en) Efficient search and organization of a forwarding database or the like
WO2021008552A1 (fr) Procédé et appareil de lecture de données et support de stockage lisible par ordinateur
WO2023130953A1 (fr) Procédé et appareil de recherche de données, et dispositif de réseau
US8001524B1 (en) Instruction extension for linked list lookup

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08700800

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08700800

Country of ref document: EP

Kind code of ref document: A1