WO2018099107A1 - Procédé et dispositif de gestion de table de hachage, et support de stockage informatique - Google Patents

Procédé et dispositif de gestion de table de hachage, et support de stockage informatique Download PDF

Info

Publication number
WO2018099107A1
WO2018099107A1 PCT/CN2017/094249 CN2017094249W WO2018099107A1 WO 2018099107 A1 WO2018099107 A1 WO 2018099107A1 CN 2017094249 W CN2017094249 W CN 2017094249W WO 2018099107 A1 WO2018099107 A1 WO 2018099107A1
Authority
WO
WIPO (PCT)
Prior art keywords
hash
linked list
hash table
key value
stored
Prior art date
Application number
PCT/CN2017/094249
Other languages
English (en)
Chinese (zh)
Inventor
王晓泾
陈西
刘振伟
包闯
闫振林
Original Assignee
深圳市中兴微电子技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市中兴微电子技术有限公司 filed Critical 深圳市中兴微电子技术有限公司
Publication of WO2018099107A1 publication Critical patent/WO2018099107A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2255—Hash tables

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a method and apparatus for managing a hash table, and a computer storage medium.
  • the network processor is a dedicated instruction processor involved in the field of network applications. It is designed for network packet processing and can make the network system have high performance and high flexibility.
  • the network processor uses a hash table to establish an index relationship of information when performing information search, thereby realizing quick information search.
  • the hash table uses a hash algorithm to establish an index relationship.
  • the hash algorithm is used to calculate the storage address of the hash table item information corresponding to the key value, so that the hash table item information corresponding to the key value can be directly found through the storage address.
  • embodiments of the present invention are expected to provide a method and apparatus for hash table management, and a computer storage medium, which improve the number of hash collisions.
  • the embodiment of the invention provides a method for managing a hash table, including:
  • the linked list is used to store the hash table corresponding to the hash key value of the hash conflict, and the root table is used to store the hash table corresponding to each hash key value and the linked list information item, where the linked list information item is used to indicate the linked list. Whether there is a hash table entry corresponding to each hash key value;
  • the method further includes: performing a hash calculation on the obtained hash key value according to the hash function corresponding to the root table, to obtain a first hash value;
  • the storing the hash table item to be stored into the root table includes:
  • the hash table corresponding to the first hash value exists in the root table, and the hash table entry is read according to the first hash value in the root table, and the obtained hash key value and the root table are read.
  • the hash table corresponding to the first hash value in the root table is updated to the hash table item to be stored; the obtained hash key is obtained. If the value does not match the hash key value in the hash table entry read in the root table, and the storage location of the hash table entry still exists in the root table, insert the to-be-stored in the root table. History entry
  • the hash table entry to be stored is inserted into the root table.
  • the using the linked list to store the hash table corresponding to the hash key value of the hash conflict including:
  • the linked list information item is not included in the read storage data, and the obtained hash key value does not match the hash key value in the hash table entry read in the root table, and the root table does not exist.
  • a linked list is created, and the hash table to be stored is inserted in the created linked list.
  • the method further includes: updating the linked list information item in the root table based on the hash table item inserted in the linked list.
  • the linked list information item is further used to indicate address information of a hash entry existing in the linked list
  • the storing the hash table item to be stored into the linked list includes:
  • the hash table entry is read according to the second hash value in the linked list, and the obtained hash key value and the linked list are read out.
  • the hash table corresponding to the second hash value in the linked list is updated to the hash table to be stored; the hash key value and the read are obtained.
  • the hash table entries to be stored are inserted into the linked list;
  • a hash table entry to be stored is inserted in the linked list.
  • the method includes: updating the linked list information item based on the inserted hash table item.
  • the method further includes:
  • the hash table entry is searched in the root table according to the search key value, and the hash table entry search result is obtained;
  • Hash table entry When the hash table entry search result indicates that the hash table corresponding to the search key value does not exist in the root table, the hash table entry is searched in the linked list according to the search key value, and the linked list is matched with the search key value. Hash table entry.
  • the method further includes: presetting a first storage space and a second storage space, where the capacity of the second storage space is greater than the capacity of the first storage space;
  • the hash table entries in the root table are stored in the first storage space or the second storage space, and the hash table entries in the linked list are stored in the second storage space.
  • the embodiment of the invention further provides a device for managing a hash table, the device comprising: a linked list and a root table, wherein:
  • a linked list configured to store a hash table entry corresponding to a hash key value in which a hash conflict occurs
  • the root table is configured to store a linked list information item and a hash table entry corresponding to each hash key value, where the linked list information item is used to indicate whether a hash table entry corresponding to each hash key value exists in the linked list;
  • the device further includes: an obtaining module and a processing module; wherein
  • Obtaining a module configured to obtain a hash key value and a hash table item to be stored
  • the processing module is configured to: read the stored data in the root table according to the obtained hash key value; and store the hash table item to be stored to the root table when the stored storage data does not include the linked list information item
  • the hash table item to be stored is stored in the linked list when the linked list information item is included in the read storage data.
  • the processing module is further configured to perform hash calculation on the obtained hash key value according to a hash function corresponding to the root table, to obtain a first hash value;
  • the processing module is further configured to: in the root table, a hash table entry corresponding to the first hash value, where the hash table entry is read according to the first hash value, When the obtained hash key value matches the hash key value in the hash table entry read in the root table, the hash table corresponding to the first hash value in the root table is updated to be stored.
  • the processing module is further configured to not include the linked list information item in the read storage data, and the obtained hash key value and the hash key in the hash table item read in the root table If the value does not match, and the storage location of the hash table entry does not exist in the root table, the linked list is created, and the hash table to be stored is inserted in the established linked list.
  • the processing module is further configured to: after inserting the hash table item to be stored in the linked list, update the linked list information item in the root table based on the hash table item inserted in the linked list.
  • the linked list information item is further used to indicate address information of a hash entry existing in the linked list
  • the processing module is further configured to perform a hash calculation on the obtained hash key value according to the hash function corresponding to the linked list, to obtain a second hash value, and obtain the hash in the linked list information item.
  • the address information of the hash table corresponding to the key value is recorded as a third hash value;
  • the processing module is further configured to: when the third hash value and the second hash value are the same, read the hash table entry according to the second hash value in the linked list, and obtain the hash key When the value matches the hash key value in the hash table entry in the linked list, the hash table corresponding to the second hash value in the linked list is updated to the hash table to be stored; When the hash key value does not match the hash key value in the read hash table entry, the hash table entry to be stored is inserted in the linked list; the third hash value and the second hash value are When the hash values are different, the hash table entry to be stored is inserted in the linked list.
  • the processing module is further configured to: after inserting the hash table item to be stored in the linked list, update the linked list information item based on the inserted hash table item.
  • the device further includes: a lookup module
  • the search module is further configured to obtain a lookup key value for performing a hash search; perform a hash table item search in the root table according to the search key value, and obtain a hash table item search result; and search results in the hash table item.
  • the hash table entry corresponding to the search key value exists in the root table
  • the hash table entry corresponding to the search key value in the root table is determined in the hash table entry search result; in the hash table entry If the search result indicates that the hash table corresponding to the search key value does not exist in the root table, the hash table entry is searched in the linked list according to the search key value, and the hash table corresponding to the search key value in the linked list is obtained.
  • the device further includes: a first storage space and a second storage space, where the capacity of the second storage space is greater than the capacity of the first storage space;
  • the storage location of the hash table entry in the root table is the first storage space or the second storage space, and the storage location of the hash table entry in the linked list is the second storage space.
  • Embodiments of the present invention also provide a computer storage medium storing a computer program configured to perform the method of hash table management described above.
  • a method and device for managing hash table according to an embodiment of the present invention uses a linked list to store a hash table entry corresponding to a hash key value in which a hash conflict occurs, and use a root table to store each hash key value corresponding to the hash table.
  • the linked list information item is used to indicate whether there is a hash table entry corresponding to each hash key value in the linked list; obtaining a hash key value and a hash table entry to be stored; Reading the stored data in the root table according to the obtained hash key value; storing the hash table item to be stored in the root table when the stored storage data does not include the linked list information item; When the stored data includes the linked list information item, the hash table item to be stored is stored in the linked list. In this way, the number of hash collisions is increased.
  • FIG. 1 is a flow chart of a first embodiment of a method for managing hash table according to the present invention
  • FIG. 2 is a schematic structural diagram of a hash entry in an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of a linked list information item according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of a second embodiment of a method for managing a hash table according to the present invention.
  • FIG. 5 is a schematic diagram of a first component structure of an apparatus for managing a hash table according to an embodiment of the present invention
  • FIG. 6 is a schematic diagram of a second component structure of an apparatus for managing a hash table according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of an internal structure of a root table lookup module according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of an internal structure of a linked list lookup module according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of an internal structure of a lookup request scheduling module according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of an internal structure of a lookup return scheduling module according to an embodiment of the present invention.
  • FIG. 11 is a schematic diagram of an internal structure of a software entry management module according to an embodiment of the present invention.
  • FIG. 1 is a flowchart of a first embodiment of a method for managing a hash table according to the present invention. As shown in FIG. 1, the method includes:
  • Step 10 The hash table corresponding to the hash key value in which the hash conflict occurs is stored by using the linked list, and the hash table corresponding to each hash key value and the linked list information item are stored by using the root table, and the linked list information item is used. Indicates whether there is a hash table entry corresponding to each hash key value in the linked list.
  • the hash entry includes: a hash table entry identifier, a hash table header information n, and a hash table entry n, n. Take an integer greater than 0.
  • the hash table entry identifier is used to identify that the current data is a hash table entry
  • the hash table entry header information n is used to determine the location of the hash table entry n
  • the hash table entry n is a hash corresponding to the hash key value n. Entry.
  • the size of the read data is determined by the bit width of the transmitted data, and the larger the bit width, the more the hash table information obtained, that is, the more the n value is. Big. On the contrary, the smaller the value of n.
  • FIG. 3 is a schematic structural diagram of a linked list information item according to an embodiment of the present invention.
  • the linked list information item includes: a linked list information item representation identifier, a linked list comparison information, and a linked list address information.
  • the linked list information item identifier is used to identify the linked list information item
  • the linked list comparison information is used to indicate Whether the hash table corresponding to the hash key value is stored in the linked list
  • the linked list address information is used to indicate the storage location of the hash table corresponding to the hash key value in the linked list.
  • the linked list information includes: A, B, and C
  • the linked list address information includes: a, b.
  • the actual storage address of the hash entry corresponding to A and B is a
  • the actual storage address of the hash entry corresponding to C is b.
  • Step 11 Obtain a hash key value and a hash table entry to be stored.
  • Step 12 The stored data in the root table is read according to the obtained hash key value; when the linked data item is not included in the read stored data, the hash table item to be stored is stored in the root table; When the linked list information item is included in the read storage data, the hash table item to be stored is stored in the linked list.
  • the stored data in the root table is read according to the obtained hash key value, and the stored data included includes: hash table entry information corresponding to the hash key value stored in the root table, or the root table Linked information items.
  • the linked list information item indicates whether the identifier is 1, and if yes, determining that the stored data includes a linked list information item, and if not, determining that the stored data does not include the linked list information item.
  • the method further includes: performing hash calculation on the obtained hash key value according to the hash function corresponding to the root table, to obtain a first hash value.
  • the first hash value is used to determine whether the hash table corresponding to the hash key value is stored in the root table.
  • f can be a remainder remainder method kmodA (A ⁇ m), a random number method random (k), and the like.
  • the hash table entry to be stored is stored in the root table, and the hash table corresponding to the first hash value exists in the root table, and the first hash value is used in the root table.
  • Read hash table And updating when the obtained hash key value matches the hash key value in the hash table item read in the root table, the hash table item corresponding to the first hash value in the root table is updated to The hash table entry to be stored; the obtained hash key value does not match the hash key value in the hash table entry read in the root table, and the hash table entry still exists in the root table.
  • a hash table entry to be stored is inserted in the root table.
  • the hash table corresponding to the first hash value does not exist in the root table
  • the hash table to be stored is inserted into the root table.
  • the method before using the linked list to store the hash table corresponding to the hash key value of the hash conflict, the method further includes: establishing the linked list.
  • the linked list information item is not included in the read storage data, and the obtained hash key value does not match the hash key value in the hash table entry read in the root table, and the root table is in the root table.
  • a linked list is created, and the hash table to be stored is inserted in the established linked list.
  • the method further includes: updating the linked list information item in the root table based on the hash table item inserted in the linked list.
  • the linked list information item is further used to indicate address information of a hash entry existing in the linked list
  • storing the hash table item to be stored in the linked list may include: performing hash calculation on the obtained hash key value according to the hash function corresponding to the linked list, to obtain a second hash value;
  • the address information of the hash table entry corresponding to the obtained hash key value in the linked list information item is recorded as the third hash value.
  • the hash function corresponding to the linked list is another hash function different from the root table hash function.
  • the second hash value and the third hash value are both obtained by calculating a hash key value through a hash function corresponding to the linked list.
  • the hash table entry is read according to the second hash value in the linked list, and the obtained hash key value and the linked list are read out.
  • Haha in the hash table entry When the hash key value is matched, the hash table corresponding to the second hash value in the linked list is updated to the hash table to be stored; in the obtained hash key value and the read hash table entry When the hash key values do not match, the hash table entry to be stored is inserted in the linked list.
  • a hash table entry to be stored is inserted in the linked list.
  • the method further includes: updating the linked list information item based on the inserted hash table item.
  • the method when storing the hash table to be stored, the method further includes: presetting a first storage space and a second storage space, where the capacity of the second storage space is greater than a capacity of the first storage space;
  • the hash table entries in the table are stored in the first storage space or the second storage space, and the hash table entries in the linked list are stored in the second storage space.
  • the second storage space may be of any size, so that a large-capacity hash table entry or a hash table entry that generates a hash conflict is stored in the second storage space, and the storage location of the hash table entry is implemented.
  • Flexible management increases the number of hash collisions.
  • the conflicted hash table entry needs to be stored in the linked list, and the linked list information item may be in the format shown in FIG. Stored in the storage unit where the hash conflict exists in the root table, so that when the hash table item with the hash conflict is searched, the linked list information item can be returned, thereby further searching for the conflict hash table item saved in the linked list. .
  • the method may further include searching for a corresponding hash table entry according to the input key value.
  • the method for finding a hash entry may include:
  • the hash table entry is searched in the root table according to the search key value, and the hash table entry search result is obtained;
  • the hash table entry corresponding to the lookup key value does not exist in the hash table entry search result indicating list. Determining, in the hash table entry search result, a hash table entry corresponding to the lookup key value in the root table;
  • the hash table item search is performed in the linked list according to the search key value, and the hash table corresponding to the search key value is obtained. Histology item.
  • a linked list is used to store a hash table entry corresponding to a hash key value in which a hash conflict occurs, and a root table is used to store a hash table entry corresponding to each hash key value and a linked list information item, where the linked list information is used.
  • the item is used to indicate whether there is a hash table entry corresponding to each hash key value in the linked list; obtaining a hash key value and a hash table item to be stored; and reading the storage in the root table according to the obtained hash key value Data; storing the hash table item to be stored in the root table when the stored data is not included in the stored data; and storing the stored information in the stored data includes the linked list information item
  • the hash table entries are stored in the linked list. In this way, the number of hash collisions is increased.
  • FIG. 4 is a flowchart of a second embodiment of a method for managing a hash table according to the present invention, the method includes:
  • Step 40 Enter the hash entry information to be stored.
  • the hash entry information to be stored may include: a hash table to be stored, a key value corresponding to the hash table to be stored, a key type and size, and the like.
  • Step 41 The storage address obtained according to the hash table entry information accesses the root table to read the stored data.
  • the first hash value is obtained by using the first hash function to calculate the key value in the hash table item information, where the first hash value may be the corresponding hash table item in the root table.
  • the storage address can also be an offset address, plus the base address to get the final storage address.
  • the stored data in the root table is accessed through the obtained storage address.
  • the obtained storage data may be: hash table item information stored in the root table, or root The linked list information item in the table.
  • the hash table entry of the same key value can be found in the root table according to the key value corresponding to the hash table to be stored, the hash table entry is directly obtained, and when the hash table entry of the same key value is not found, , get the linked list information item.
  • the purpose of accessing the data in the storage address is to determine whether the currently input hash table item information is used to update the existing item information in the root table or the linked list, or as a new hash table.
  • the item information is inserted into the root or linked list.
  • Step 42 Determine whether the obtained data is a linked list information item. If yes, go to step 43; if no, go to step 44.
  • the purpose of determining whether the obtained data is a linked list information item is: determining whether the input storage information is stored in the root table or in the linked list.
  • whether the obtained data is a linked list information item can be determined by determining whether there is a linked list information item representation identifier in the read data.
  • Step 43 Obtain a second hash value corresponding to the key value, determine whether a third hash value equal to the second hash value is stored in the linked list information item, and if yes, execute step 431; if no, execute step 432.
  • the second hash function is used to calculate the key value in the stored information to obtain the second hash value, and the second hash value is compared with all the third hashes stored in the linked list comparison information in the linked list information item.
  • Value comparison when there is a third hash value equal to the second hash value, indicating that the storage information at this time is used to update the hash table entry in the linked list; when there is no equal value to the second hash value
  • the third hash value indicates that the storage information at this time is inserted into the linked list as a new hash table entry.
  • Step 431 Determine, according to the entry information corresponding to the second hash value in the linked list, whether the key value corresponding to the hash entry is equal to the input key value. If yes, go to step 45; if no, go to step 432.
  • Step 432 Determine whether the linked list is full. If yes, go to step 46; if no, go to step 433.
  • Step 433 Insert the information of the hash table item to be stored into the linked list, and update the linked list information item in the root table.
  • the second hash value corresponding to the key value in the stored information needs to be stored in the linked list comparison information of the linked list information item.
  • the linked list address information is updated at the same time, and the storage address is allocated for the hash table corresponding to the second hash value.
  • step 47 is performed.
  • Step 44 Determine whether the hash table entry read from the root table is valid. If yes, go to step 442; if no, go to step 441.
  • the method for determining whether the hash entry is valid may be: determining whether the valid identifier of the hash entry in the header information of the hash table entry is 1 by using the stored data, and if yes, indicating that the current read is The hashed entry is valid; if not, it means that no data is included in the read data. In this case, you need to insert the information to be stored into the root table.
  • Step 441 After the hash table item information to be stored is inserted into the root table, step 47 is performed.
  • Step 442 Determine whether the key value corresponding to the hash entry is equal to the input key value. If yes, go to step 45; if no, go to step 443.
  • Step 443 Determine whether the root table is full. If yes, go to step 444; if no, go to step 441.
  • Step 444 Apply to establish a linked list for storing input information, and determine whether the application is successful. If yes, go to step 445; if no, go to step 46.
  • Step 445 The input information is stored in the linked list, and the linked list information item in the root table is updated, and step 47 is performed.
  • Step 45 Update the corresponding hash table item information in the table by using the input hash table item information.
  • step 47 is performed.
  • Step 46 The storage fails.
  • Step 47 The storage ends.
  • the hash table entry information to be stored is the same as the root table update and insertion process.
  • the linked list information item in the root table needs to be performed. Update, and when updating the linked list, only need to replace the corresponding hash table item information does not need to update the linked list information item.
  • the user when the information of the hash table item needs to be queried, the user only needs to input the query information, where the query information includes: the key value, the key type and the size of the hash table item to be queried.
  • the query process and the storage process are the same in the process of reading the data and comparing the key values, but deleting the steps of inserting or updating the hash table entries.
  • FIG. 5 is a schematic diagram of a first component structure of the device for managing a hash table according to an embodiment of the present invention, as shown in FIG.
  • the device includes: a linked list and a root table; wherein
  • a linked list configured to store a hash table entry corresponding to a hash key value in which a hash conflict occurs
  • the root table is configured to store a linked list information item and a hash table entry corresponding to each hash key value, where the linked list information item is used to indicate whether a hash table entry corresponding to each hash key value exists in the linked list;
  • the device further includes: an obtaining module 50 and a processing module 51; wherein
  • the obtaining module 50 is configured to obtain a hash key value and a hash table item to be stored.
  • the processing module 51 is configured to: read the stored data in the root table according to the obtained hash key value; and store the hash table item to be stored to the root when the stored storage data does not include the linked list information item In the table, when the linked data item is included in the read storage data, the hash table item to be stored is stored in the linked list.
  • the processing module 51 is further configured to perform a hash calculation on the obtained hash key value according to a hash function corresponding to the root table, to obtain a first hash value;
  • the processing module 51 is further configured to: in the root table, a hash table entry corresponding to the first hash value, where the hash table entry is read according to the first hash value, When the obtained hash key value matches the hash key value in the hash table entry read in the root table, the hash table corresponding to the first hash value in the root table is updated to be stored.
  • Hash table entry the obtained hash key value does not match the hash key value in the hash table entry read in the root table, and the storage location of the hash table entry still exists in the root table Inserting a hash table entry to be stored in the root table; when there is no hash table entry corresponding to the first hash value in the root table, inserting a hash to be stored in the root table Entry.
  • the processing module 51 is further configured to not include the linked list information item in the read storage data, and the obtained hash key value and the hash key in the hash table item read in the root table If the value does not match, and the storage location of the hash table entry does not exist in the root table, the linked list is created, and the hash table to be stored is inserted in the established linked list.
  • the processing module 51 is further configured to: after inserting the hash table item to be stored in the linked list, update the linked list information item in the root table based on the hash table item inserted in the linked list.
  • the linked list information item is further used to indicate address information of a hash entry existing in the linked list
  • the processing module 51 is further configured to perform hash calculation on the obtained hash key value according to the hash function corresponding to the linked list, to obtain a second hash value, and obtain the hash information information item and the obtained hash value.
  • the address information of the hash table corresponding to the hash key value is recorded as a third hash value;
  • the processing module 51 is further configured to: when the third hash value and the second hash value are the same, read the hash table entry according to the second hash value in the linked list, and obtain the hash When the key value matches the hash key value in the hash table entry read in the linked list, the hash table corresponding to the second hash value in the linked list is updated to the hash table to be stored; Get the hash key value and read it out When the hash key value in the hash table item does not match, the hash table item to be stored is inserted in the linked list; when the third hash value is different from the second hash value, in the linked list Insert the hash table entry to be stored.
  • the processing module 51 is further configured to: after inserting the hash table item to be stored in the linked list, update the linked list information item based on the inserted hash table item.
  • the device further includes: a lookup module
  • the search module is further configured to obtain a lookup key value for performing a hash search; perform a hash table item search in the root table according to the search key value, and obtain a hash table item search result; and search results in the hash table item.
  • the hash table entry corresponding to the search key value exists in the root table
  • the hash table entry corresponding to the search key value in the root table is determined in the hash table entry search result; in the hash table entry If the search result indicates that the hash table corresponding to the search key value does not exist in the root table, the hash table entry is searched in the linked list according to the search key value, and the hash table corresponding to the search key value in the linked list is obtained.
  • the device further includes: a first storage space and a second storage space, wherein the capacity of the second storage space is greater than the capacity of the first storage space;
  • the storage location of the hash table entry in the root table is the first storage space or the second storage space, and the storage location of the hash table entry in the linked list is the second storage space.
  • the first storage space may be an on-chip storage unit inside the device
  • the second storage space may be an off-chip storage unit outside the device
  • the storage space of the off-chip storage unit is large and the on-chip storage unit may be All or part of the root table, and all the linked lists are stored in the off-chip storage unit, improving the working efficiency of the device.
  • the obtaining module 50 and the processing module 51 may each be a Central Processing Unit (CPU), a Micro Processor Unit (MPU), and a Digital Signal Processor (Digital Signal Processor) located in the terminal device. DSP), or Field Programmable Gate Array (FPGA) implementation.
  • CPU Central Processing Unit
  • MPU Micro Processor Unit
  • DSP Digital Signal Processor
  • FPGA Field Programmable Gate Array
  • FIG. 6 is a schematic diagram of a second component structure of an apparatus for managing a hash table according to an embodiment of the present invention.
  • the apparatus may include: a user interface 101, a root table lookup module 102, a linked list lookup module 103, a lookup request scheduling module 104, and an external storage management module. 105.
  • the user interface 101, the root table lookup module 102, the linked list lookup module 103, the lookup request scheduling module 104, the external storage management module 105, and the lookup return scheduling module 107 are used for lookup management of the hash table entries.
  • the software entry management module 108 manages the storage of the hash entries.
  • the external storage module 106 and the internal storage module 109 are configured to implement flexible storage of hash entries.
  • FIG. 7 is a schematic diagram of an internal structure of a root table lookup module according to an embodiment of the present invention.
  • the root table lookup module includes: a root table cache unit 201, a root table parsing unit 202, and a root table processing unit 203.
  • the root table lookup module 102 receives the lookup information transmitted by the user interface 101, and the search information includes: a key value, a key type and a size, etc.; the lookup information is placed in the root table cache unit 201, and the key value is sent to the inside of the device.
  • the internal storage module 109 calculates a first hash value according to the key value, and the return data corresponding to the first hash value is sent to the root table processing unit 203; and the root table parsing unit 202 also calculates the key value according to the key value.
  • the first hash value is generated and sent to the lookup request scheduling module 104.
  • the lookup request may include: all the search information, a trigger signal indicating the search for the external storage module 106, and the like.
  • the lookup request scheduling module 104 requests the external storage module to search for the corresponding hash table entry, and sends the return data corresponding to the first hash value to the root table processing unit 203; the root table processing unit 203 reads the lookup from the cache unit 201.
  • the information is matched with the obtained return data. If it matches, indicating that the returned data contains the hash table corresponding to the key value, the return data is directly sent to the lookup return scheduling module 107, and the structure of the returned data is as shown in the figure. 2; if it does not match, it means that the returned data does not contain the hash table corresponding to the key value, and needs to perform the linked list lookup. At this time, the lookup request is initiated to the linked list lookup module 103, and the structure of the linked list information item in the returned data is as shown in FIG. 3 is shown.
  • the list lookup is automatically performed by parsing the linked list information item, thereby improving the search efficiency of the entry.
  • the root table processing unit 203 reads the lookup information from the root table cache unit 201 and matches the obtained return data, including: the key value, the key type and the size in the search information, and the returned data.
  • the information such as the key value, the key type, and the size of the hash table item are compared one by one. When all the information is the same, it is determined that the data matches, and the returned data includes the hash table item to be searched; if at least one item is not The same, to determine the data does not match, the returned data does not contain the hash table item to be found.
  • FIG. 8 is a schematic diagram of an internal structure of a linked list lookup module according to an embodiment of the present invention.
  • the linked list lookup module includes: a linked list cache unit 301, a linked list parsing unit 302, and a linked list processing unit 303.
  • the root table lookup module 102 initiates the lookup request to the linked list lookup module 103
  • the lookup request is stored in the linked list cache unit 301 and the linked list parsing unit 302.
  • the linked list parsing unit 302 parses the lookup request and sends it to the lookup request scheduling module 104 for a lookup request.
  • the scheduling module 104 applies to the external storage management module 105 for a corresponding query operation, and the external storage management module 105 completes the query to the external storage module 106 and returns the query result to the linked list processing unit 303; the linked list processing unit 303 reads from the linked list cache unit 301.
  • the lookup information contained in the lookup request is compared and compared with the obtained query result, and the comparison result is sent to the lookup return scheduling module 107.
  • the hash table corresponding to the key value is found in the linked list
  • the obtained hash table entry is returned; when the hash table corresponding to the key value is not found in the linked list, the prompt information may be returned Find the corresponding hash table entry.
  • FIG. 9 is a schematic diagram of an internal structure of a lookup request scheduling module according to an embodiment of the present invention, as shown in FIG.
  • the lookup request scheduling module 104 includes a root table lookup cache unit 401, a linked list lookup cache unit 402, and a lookup request scheduling unit 403.
  • the root table lookup cache unit 401 and the linked list lookup cache unit 402 cache the lookup requests generated by the root table parsing unit 202 and the linked list parsing unit 302, respectively.
  • the lookup request scheduling unit 403 after reading the lookup request in the two cache units, adds a lookup type flag to the lookup request and sends it to the external storage management module 105.
  • the search type identifier can identify whether the current table content or the linked list content needs to be searched.
  • a search type flag bit can be set to distinguish the current search object.
  • the search type flag bit is 1, the search object is The entry information in the root table; when the lookup type flag is 0, it indicates that the search object is the entry information in the linked list. You can improve the efficiency of finding hash entries by setting the lookup type identifier.
  • the lookup return scheduling module 107 includes a root table lookup result buffering unit 701, a linked list lookup result buffering unit 702, and a lookup returning scheduling unit 703.
  • the root table search result buffer unit 701 and the linked list search result buffer unit 702 respectively cache the lookup return results generated by the root table processing unit 303 and the linked list processing unit 403, and the lookup return scheduling unit 703 reads the search results in the two cache units, and It is returned to the user via the user interface 101.
  • FIG. 11 is a schematic diagram of an internal structure of a software entry management module according to an embodiment of the present invention.
  • the software entry management module 108 may include: a master device 801, a software management unit 802, and a software cache unit 803.
  • the master device 801 When storing the hash table entry, the master device 801 first determines whether the hash table entry is stored in the root table or the linked list by the software management unit 802. After determining the storage location of the hash table entry storage, the software management unit 802 The hash table entry is first cached into the software cache unit 803, and the software cache is interconnected by a peripheral component (Peripheral Component Interconnect Express, The PCIE) interface is written to the external storage module 106.
  • a peripheral component Peripheral Component Interconnect Express, The PCIE
  • the root table and the linked list may be stored in the external storage module; or the smaller hash entries in the root table may be stored in the internal storage module. , store the large hash table entries in the external storage module.
  • the hash table entry management device may adopt a pipeline method when performing hash table item search, for an entry of the same level entry, that is, an entry of the same root table or a linked list. Because the storage location and the discovery process are consistent, the lookup interface can perform sequential lookups for the same level of lookup. For different levels of lookup, that is, the searched entries are in the root table and the linked list, and the lookup interface can also be searched according to the order in which the lookup requests are obtained.
  • the storage management of the hash entry may be implemented at the same time. If an error occurs in the process of storing the entry, the error identifier may be returned to the hash table item management device, and then The error condition is handled by the management device, for example, re-storing the entry or blocking the lookup of the error entry.
  • embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention can take the form of a hardware embodiment, a software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
  • an embodiment of the present invention further provides a computer storage medium, wherein a computer program is configured, and the computer program is configured to perform a hash table management method according to an embodiment of the present invention.
  • the technical solution of the embodiment of the present invention uses a linked list to store a hash table entry corresponding to a hash key value in which a hash conflict occurs, and uses a root table to store a hash table entry corresponding to each hash key value, and a linked list information item.
  • the linked list information item is used to indicate whether there is a hash table corresponding to each hash key value in the linked list; obtaining a hash key value and a hash table item to be stored; and reading the root table according to the obtained hash key value; Storage data; when the stored data is not included in the stored data, the hash table item to be stored is stored in the root table; when the stored data includes the linked list information item, the The hash table items to be stored are stored in the linked list. In this way, the number of hash collisions is increased.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Selon l'invention, un procédé de gestion de table de hachage comprend les étapes suivantes consistant à : utiliser une table de chaînes pour stocker des entrées de table de hachage correspondant à des valeurs clés de hachage au niveau desquelles surviennent des conflits de hachage, et utiliser une table de racines pour stocker une entrée de table de hachage correspondant à chaque valeur clé de hachage et des entrées d'information de table de chaînes, les entrées d'information de table de chaînes servant à indiquer si la table de chaînes a l'entrée de table de hachage correspondant à chaque valeur clé de hachage ; obtenir les valeurs clés de hachage et les entrées de table de hachage à stocker ; lire des données de stockage dans la table de racines selon les valeurs clés de hachage obtenues ; stocker les entrées de table de hachage à stocker dans la table de racines lorsque les données de stockage lues ne comportent pas les entrées d'information de table de chaînes ; et stocker les entrées de table de hachage à stocker dans la table de chaînes lorsque les données de stockage lues comportent les entrées d'information de table de chaînes. La présente invention concerne également un dispositif de gestion de table de hachage.
PCT/CN2017/094249 2016-12-02 2017-07-25 Procédé et dispositif de gestion de table de hachage, et support de stockage informatique WO2018099107A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611102346.8A CN108153757B (zh) 2016-12-02 2016-12-02 一种哈希表管理的方法和装置
CN201611102346.8 2016-12-02

Publications (1)

Publication Number Publication Date
WO2018099107A1 true WO2018099107A1 (fr) 2018-06-07

Family

ID=62241924

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/094249 WO2018099107A1 (fr) 2016-12-02 2017-07-25 Procédé et dispositif de gestion de table de hachage, et support de stockage informatique

Country Status (2)

Country Link
CN (1) CN108153757B (fr)
WO (1) WO2018099107A1 (fr)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109558423A (zh) * 2018-10-31 2019-04-02 深圳壹账通智能科技有限公司 一种基于键值对的数据查找方法、装置及设备
CN109670083A (zh) * 2018-12-13 2019-04-23 武汉中元华电科技股份有限公司 一种基于动态内存分配存储hash链表的fpga实现装置及方法
CN110968267A (zh) * 2019-11-15 2020-04-07 北京乐我无限科技有限责任公司 数据管理方法、装置、服务器及系统
CN111177476A (zh) * 2019-12-05 2020-05-19 北京百度网讯科技有限公司 数据查询方法、装置、电子设备及可读存储介质
CN111221823A (zh) * 2019-12-31 2020-06-02 上海铿诚智能科技有限公司 一种基于链路管理表的数据处理方法及装置
CN111488371A (zh) * 2020-04-07 2020-08-04 中国人民财产保险股份有限公司 一种数据查询方法和装置
CN111756709A (zh) * 2020-06-09 2020-10-09 杭州孝道科技有限公司 一种降低交互式应用检测中漏洞重复检测的方法
CN111767152A (zh) * 2020-07-17 2020-10-13 腾讯科技（深圳）有限公司 探测资源闲置的方法、设备和计算机可读存储介质
CN111813971A (zh) * 2019-04-10 2020-10-23 阿里巴巴集团控股有限公司 哈希表构建及图像匹配方法和装置，存储介质和电子设备
CN112463214A (zh) * 2019-09-09 2021-03-09 北京京东振世信息技术有限公司 数据处理方法及装置、计算机可读存储介质以及电子设备
CN113672524A (zh) * 2021-08-20 2021-11-19 上海哔哩哔哩科技有限公司 基于多级缓存的数据处理方法及系统
CN113726660A (zh) * 2021-08-27 2021-11-30 西安微电子技术研究所 一种基于完美哈希算法的路由查找器和方法
CN114063931A (zh) * 2021-11-26 2022-02-18 重庆科创职业学院 一种基于大数据的数据存储方法
CN114064572A (zh) * 2021-11-12 2022-02-18 苏州慧工云信息科技有限公司 一种基于哈希算法的对象存储方法及系统
CN114153391A (zh) * 2021-11-26 2022-03-08 山东云海国创云计算装备产业创新中心有限公司 一种基于寄存器的数据存储方法、装置、设备及存储介质
CN114500451A (zh) * 2022-01-21 2022-05-13 中数通信息有限公司 一种ipv4/ipv6地址及解析装置
CN114969913A (zh) * 2022-05-24 2022-08-30 国网北京市电力公司 一种三维模型构件实例化方法、系统、设备及介质
CN115576954A (zh) * 2022-11-24 2023-01-06 恒生电子股份有限公司 哈希表的确定方法及装置
CN116401258A (zh) * 2023-06-06 2023-07-07 支付宝(杭州)信息技术有限公司 数据索引方法、数据查询方法及对应装置
CN117112575A (zh) * 2023-10-20 2023-11-24 杭州行芯科技有限公司 电路元素的名称管理方法、系统及查询方法

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109460406B (zh) * 2018-10-15 2021-03-23 咪咕文化科技有限公司 一种数据处理方法及装置
CN110781091B (zh) * 2019-10-31 2023-10-27 北京奇艺世纪科技有限公司 应用程序的测试方法、装置、电子设备及存储介质
CN111259203B (zh) * 2020-01-08 2023-08-25 上海兆芯集成电路股份有限公司 数据压缩器以及数据压缩方法
CN111858586B (zh) * 2020-07-06 2024-04-09 北京天空卫士网络安全技术有限公司 一种数据处理的方法和装置
CN112597195A (zh) * 2020-12-16 2021-04-02 中国建设银行股份有限公司 缓存数据刷新方法及分布式系统
CN112765174B (zh) * 2021-01-20 2024-03-29 上海达梦数据库有限公司 基于哈希连接的探测方法、装置、设备及存储介质
CN113096284B (zh) * 2021-03-19 2022-08-30 福建新大陆通信科技股份有限公司 一种ctid门禁授权信息的核验方法
CN114338529B (zh) * 2021-12-29 2024-03-08 杭州迪普信息技术有限公司 五元组规则匹配方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103064948A (zh) * 2012-12-27 2013-04-24 中国航空工业集团公司第六三一研究所 基于散列链表的内容寻址方法及相应的存储器电路
CN103106144A (zh) * 2011-11-15 2013-05-15 北京新媒传信科技有限公司 一种内存索引压缩方法和装置
CN104572983A (zh) * 2014-12-31 2015-04-29 北京锐安科技有限公司 基于内存的散列表的构建方法、文本查找方法及相应装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101692651B (zh) * 2009-09-27 2014-12-31 中兴通讯股份有限公司 一种哈希查找表的方法和装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103106144A (zh) * 2011-11-15 2013-05-15 北京新媒传信科技有限公司 一种内存索引压缩方法和装置
CN103064948A (zh) * 2012-12-27 2013-04-24 中国航空工业集团公司第六三一研究所 基于散列链表的内容寻址方法及相应的存储器电路
CN104572983A (zh) * 2014-12-31 2015-04-29 北京锐安科技有限公司 基于内存的散列表的构建方法、文本查找方法及相应装置

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109558423A (zh) * 2018-10-31 2019-04-02 深圳壹账通智能科技有限公司 一种基于键值对的数据查找方法、装置及设备
CN109670083A (zh) * 2018-12-13 2019-04-23 武汉中元华电科技股份有限公司 一种基于动态内存分配存储hash链表的fpga实现装置及方法
CN109670083B (zh) * 2018-12-13 2023-03-24 武汉中元华电科技股份有限公司 一种基于动态内存分配存储hash链表的fpga实现装置及方法
CN111813971A (zh) * 2019-04-10 2020-10-23 阿里巴巴集团控股有限公司 哈希表构建及图像匹配方法和装置，存储介质和电子设备
CN111813971B (zh) * 2019-04-10 2024-06-07 阿里巴巴集团控股有限公司 哈希表构建及图像匹配方法和装置，存储介质和电子设备
CN112463214A (zh) * 2019-09-09 2021-03-09 北京京东振世信息技术有限公司 数据处理方法及装置、计算机可读存储介质以及电子设备
CN112463214B (zh) * 2019-09-09 2023-11-03 北京京东振世信息技术有限公司 数据处理方法及装置、计算机可读存储介质以及电子设备
CN110968267B (zh) * 2019-11-15 2024-05-31 北京乐我无限科技有限责任公司 数据管理方法、装置、服务器及系统
CN110968267A (zh) * 2019-11-15 2020-04-07 北京乐我无限科技有限责任公司 数据管理方法、装置、服务器及系统
CN111177476B (zh) * 2019-12-05 2023-08-18 北京百度网讯科技有限公司 数据查询方法、装置、电子设备及可读存储介质
CN111177476A (zh) * 2019-12-05 2020-05-19 北京百度网讯科技有限公司 数据查询方法、装置、电子设备及可读存储介质
CN111221823A (zh) * 2019-12-31 2020-06-02 上海铿诚智能科技有限公司 一种基于链路管理表的数据处理方法及装置
CN111221823B (zh) * 2019-12-31 2023-05-23 上海铿诚智能科技有限公司 一种基于链路管理表的数据处理方法及装置
CN111488371A (zh) * 2020-04-07 2020-08-04 中国人民财产保险股份有限公司 一种数据查询方法和装置
CN111756709A (zh) * 2020-06-09 2020-10-09 杭州孝道科技有限公司 一种降低交互式应用检测中漏洞重复检测的方法
CN111767152A (zh) * 2020-07-17 2020-10-13 腾讯科技（深圳）有限公司 探测资源闲置的方法、设备和计算机可读存储介质
CN113672524A (zh) * 2021-08-20 2021-11-19 上海哔哩哔哩科技有限公司 基于多级缓存的数据处理方法及系统
CN113726660A (zh) * 2021-08-27 2021-11-30 西安微电子技术研究所 一种基于完美哈希算法的路由查找器和方法
CN114064572A (zh) * 2021-11-12 2022-02-18 苏州慧工云信息科技有限公司 一种基于哈希算法的对象存储方法及系统
CN114064572B (zh) * 2021-11-12 2024-03-19 苏州慧工云信息科技有限公司 一种基于哈希算法的对象存储方法及系统
CN114153391A (zh) * 2021-11-26 2022-03-08 山东云海国创云计算装备产业创新中心有限公司 一种基于寄存器的数据存储方法、装置、设备及存储介质
CN114063931A (zh) * 2021-11-26 2022-02-18 重庆科创职业学院 一种基于大数据的数据存储方法
CN114063931B (zh) * 2021-11-26 2023-04-25 重庆科创职业学院 一种基于大数据的数据存储方法
CN114153391B (zh) * 2021-11-26 2024-02-27 山东云海国创云计算装备产业创新中心有限公司 一种基于寄存器的数据存储方法、装置、设备及存储介质
CN114500451A (zh) * 2022-01-21 2022-05-13 中数通信息有限公司 一种ipv4/ipv6地址及解析装置
CN114969913A (zh) * 2022-05-24 2022-08-30 国网北京市电力公司 一种三维模型构件实例化方法、系统、设备及介质
CN114969913B (zh) * 2022-05-24 2024-03-15 国网北京市电力公司 一种三维模型构件实例化方法、系统、设备及介质
CN115576954B (zh) * 2022-11-24 2023-04-07 恒生电子股份有限公司 哈希表的确定方法及装置
CN115576954A (zh) * 2022-11-24 2023-01-06 恒生电子股份有限公司 哈希表的确定方法及装置
CN116401258B (zh) * 2023-06-06 2023-09-22 支付宝(杭州)信息技术有限公司 数据索引方法、数据查询方法及对应装置
CN116401258A (zh) * 2023-06-06 2023-07-07 支付宝(杭州)信息技术有限公司 数据索引方法、数据查询方法及对应装置
CN117112575A (zh) * 2023-10-20 2023-11-24 杭州行芯科技有限公司 电路元素的名称管理方法、系统及查询方法
CN117112575B (zh) * 2023-10-20 2024-02-13 杭州行芯科技有限公司 电路元素的名称管理方法、系统及查询方法

Also Published As

Publication number Publication date
CN108153757B (zh) 2020-04-03
CN108153757A (zh) 2018-06-12

Similar Documents

Publication Publication Date Title
WO2018099107A1 (fr) Procédé et dispositif de gestion de table de hachage, et support de stockage informatique
WO2017114205A1 (fr) Procédé de traitement de lien court, dispositif et serveur
CN109416694B (zh) 包括资源有效索引的键值存储系统
JP5996088B2 (ja) 暗号ハッシュ・データベース
JP6356675B2 (ja) 集約／グループ化動作：ハッシュテーブル法のハードウェア実装
US8745063B2 (en) Hashing with hardware-based reorder using duplicate values
CN109213699B (zh) 一种元数据管理方法、系统、设备及计算机可读存储介质
US11113199B2 (en) Low-overhead index for a flash cache
WO2018036549A1 (fr) Procédé et dispositif d'interrogation de base de données distribuée, et système de gestion
US9928178B1 (en) Memory-efficient management of computer network resources
WO2020177536A1 (fr) Procédé de déduplication de transactions, procédé de construction de transactions, dispositif et support de stockage
Xiao et al. Using parallel bloom filters for multiattribute representation on network services
CN112015820A (zh) 分布式图数据库实现的方法、系统、电子装置和存储介质
WO2013091167A1 (fr) Procédé et système de stockage de fichier journal
CN108460030B (zh) 一种基于改进的布隆过滤器的集合元素判断方法
WO2016029441A1 (fr) Procédé et appareil de balayage de fichier
WO2022156650A1 (fr) Procédé et appareil d'accès à des données
WO2012013023A2 (fr) Procédé de recherche par hachage réduisant les conflits de hachage
WO2020024446A1 (fr) Procédé et appareil de stockage de données, support de stockage et dispositif informatique
WO2019019382A1 (fr) Procédé et dispositif de gestion de cache, dispositif informatique et support de stockage
CN116991855B (zh) 哈希表处理方法、装置、设备、介质、控制器及固态硬盘
US9129033B1 (en) Caching efficiency using a metadata cache
JP6189266B2 (ja) データ処理装置、データ処理方法及びデータ処理プログラム
CN111104435B (zh) 一种元数据组织方法、装置、设备及计算机可读存储介质
US11016933B2 (en) Handling weakening of hash functions by using epochs

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17875955

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17875955

Country of ref document: EP

Kind code of ref document: A1