CN111352931A - Hash collision processing method and device and computer readable storage medium - Google Patents
Hash collision processing method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN111352931A CN111352931A CN201811571387.0A CN201811571387A CN111352931A CN 111352931 A CN111352931 A CN 111352931A CN 201811571387 A CN201811571387 A CN 201811571387A CN 111352931 A CN111352931 A CN 111352931A
- Authority
- CN
- China
- Prior art keywords
- hash
- value
- index
- hash table
- key value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
Abstract
The application discloses a method and a device for processing hash collision and a computer readable storage medium, comprising the following steps: obtaining a key value and a result value to be inserted into the hash table; performing hash operation on key values to be inserted into the hash table by using at least two different hash functions to obtain hash values; determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and the index table, and distributing a target index value which is different from an index value corresponding to the existing key value in the hash table to the key value to be inserted into the hash table; and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address. It can be seen from the embodiments of the present invention that the probability of hash collisions is reduced from different hash functions, and the target index value is used as the address to be inserted into the key value and the result value, which is inevitably different from the existing address of the key value in the hash table, so that hash collisions are reduced without expanding the hash table.
Description
Technical Field
The embodiment of the invention relates to the technical field of network communication, in particular to a hash collision processing method, a hash collision processing device and a computer-readable storage medium.
Background
The hash algorithm maps a key value (key) of a k-bit (bit) width to an index value (index) of an n-bit width, k > n. For example, a binary data whose key is 200 bits wide is to be mapped to a binary data index of 20 bits wide by hashing. Since the sample space for key is obviously much larger than the sample space for index, it may happen during the mapping that for different key values: key1 and key2, the index after hash mapping is the same, which is the hash collision. The hash table is based on a hash algorithm, and is a data structure network which stores corresponding key values and other table item information by taking index as an address, the hash table can be frequently used in equipment for route searching and data forwarding, and when the hash is prominent, the route searching is wrong, and the data forwarding fails.
In the related art, a method for solving hash collision is to expand a hash table, divide a hash table address pointed by one index into M locations (slots), and store different keys in each slot, so that the hash table space of the original N storage units is expanded into M × N storage units, and when hash collision occurs, different keys are stored in different slots in the address pointed by the same index.
However, since the hash table needs to be stored using a memory, when the hash table is enlarged, it means that a larger memory is required to store the hash table, and thus the overhead on the device side is increased; on the other hand, since the memory has a data bit width limitation, when the data bit width of one hash table address is too large, it is necessary to access the memory many times to read out the information of all slots in the space, and thus the lookup speed of the hash table may be reduced.
Disclosure of Invention
In order to solve the foregoing technical problem, embodiments of the present invention provide a method and an apparatus for processing hash collisions, and a computer-readable storage medium, which can reduce hash collisions without enlarging a hash table.
In order to achieve the object of the present invention, an embodiment of the present invention provides a method for processing a hash collision, including:
obtaining a key value and a result value to be inserted into the hash table;
performing hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions to obtain at least two hash values;
determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from a historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
An embodiment of the present invention further provides a hash collision processing apparatus, including:
the obtaining module is used for obtaining a key value and a result value to be inserted into the hash table;
the operation module is used for respectively carrying out hash operation on the key value to be inserted into the hash table by utilizing at least two preset different hash functions to obtain at least two hash values;
the processing module is used for determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from the historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and the adding module is used for adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
An embodiment of the present invention further provides a hash collision processing apparatus, including: a processor and a memory, wherein the memory has stored therein the following instructions executable by the processor:
obtaining a key value and a result value to be inserted into the hash table;
performing hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions to obtain at least two hash values;
determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from a historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
An embodiment of the present invention further provides a computer-readable storage medium, where the storage medium stores computer-executable instructions, and the computer-executable instructions are configured to perform the following steps:
obtaining a key value and a result value to be inserted into the hash table;
performing hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions to obtain at least two hash values;
determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from a historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
Compared with the prior art, the hash value according to which whether the key value to be inserted into the hash table causes the hash collision is determined is obtained based on at least two hash functions, so that the probability of the hash collision is reduced from different hash functions, and after the key value to be inserted into the hash table is determined not to cause the hash collision, the target index values with different index values corresponding to the key values existing in the hash table are allocated to the key value to be inserted into the hash table, so that the target index values are used as the addresses of the key value to be inserted and the result value in the hash table and are inevitably different from the addresses of the key values existing in the hash table, the hash collision is reduced on the premise of not expanding the hash table, the cost in the aspect of equipment is saved, and the searching speed of the hash table is improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
Fig. 1 is a schematic structural diagram of a hash table in the related art;
FIG. 2 is a diagram illustrating a structure of a hash table expanded for handling hash collision in the related art;
fig. 3 is a flowchart illustrating a hash collision processing method according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an index table according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a hash collision processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another hash collision processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Fig. 1 is a schematic structural diagram of a hash table in the related art, and fig. 2 is a schematic structural diagram of a hash table expanded for processing hash collisions in the related art. As shown in fig. 1, the hash table includes N storage units, and as shown in fig. 2, the hash table is obtained by expanding each storage unit of the hash table shown in fig. 1 (that is, expanding each storage unit by 4 slots) on the basis of the hash table shown in fig. 1, and expanding the hash table shown in fig. 1 by a hash table having 4 × N storage units, when a hash collision occurs, different key values are stored in different slots in addresses pointed to by the same index, so that the hash collision is effectively solved. However, since the hash table needs to be stored by using a memory, when the hash table is expanded, it means that a larger capacity memory is needed for storing the hash table, and it is described by taking a Double Data Rate (DDR) Synchronous Dynamic Random Access Memory (SDRAM) as an example for storing the hash table, when the DDR stores the hash table, the larger the hash table is, the more DDR grains are needed, and due to hardware design and cost control, the number of DDR grains used is limited, and the excessively large hash table cannot be satisfied. And due to the data bit width limitation of the DDR, when the data bit width of one hash table address is too large, the DDR needs to be accessed multiple times to read out the information of all slots in the space, which may reduce the hash table lookup speed, and may seriously deteriorate the routing lookup and data forwarding performance in the data forwarding application of the network device, making it unable to reach the line speed.
To this end, an embodiment of the present invention provides a hash collision processing method, as shown in fig. 3, where the method includes:
Note that the result value (action) is action information on how to operate the key value.
102, performing hash operation on key values to be inserted into a hash table respectively by using at least two preset different hash functions to obtain at least two hash values.
It should be noted that the number of the at least two preset hash functions may be two, or may be more than two, and may be specifically set according to the situation. When the number of preset hash functions is larger, the occurrence rate of hash collision can be reduced, but the implementation complexity of the corresponding method and the consumed computing resources are increased.
And 103, determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing the index value, and distributing a target index value different from the historical index value to the key value to be inserted into the hash table.
And the historical index value is an index value corresponding to the existing key value in the hash table.
And 104, writing the key value and the result value to be inserted into the hash table by taking the target index value as an address.
In the method for processing hash collision provided in the embodiment of the present invention, since the hash value according to which it is determined whether the key value to be inserted into the hash table causes hash collision is obtained based on at least two hash functions, the probability of hash collision is reduced from different hash functions, and after it is determined that the key value to be inserted into the hash table does not cause hash collision, a target index value with different index values corresponding to the key value already existing in the hash table is allocated to the key value to be inserted into the hash table, so that the target index value is used as an address where the key value and the result value are to be inserted into the hash table and is inevitably different from the address of the key value already existing in the hash table, thereby reducing hash collision without expanding the hash table, saving the overhead in the aspect of equipment, and increasing the lookup speed of the hash table.
Optionally, the preset at least two different hash functions include: a first hash function and a second hash function.
Performing hash operation on the key value to be inserted into the hash table to obtain at least two hash values, including:
and carrying out hash operation on the key value to be inserted into the hash table by using a first hash function to obtain a first hash value.
And carrying out hash operation on the key value to be inserted into the hash table by using a second hash function to obtain a second hash value.
Optionally, the index table comprises: a number of addresses.
The address includes: a number of locations.
The positions include: a first field, a second field, and a third field; the first field is used for storing a hash value obtained by carrying out hash operation on the key value by using a second hash function; a second field for storing an index value; a third field for storing a flag indicating whether a location is occupied.
And the product of the number of the addresses of the index table and the number of the positions of each address is not less than the number of the addresses of the hash table.
It should be noted that, the index values are in one-to-one correspondence with the hash values, and how many hash values exist in the hash table, how many corresponding index values exist in the index table, so the size of the index table should be considered according to the maximum number of hash values that can be stored in the hash table, where the maximum number of hash values that can be stored in the hash table is the number of addresses of the hash table, and the number of locations that are used for storing the index values in the index table is the location, so the location in the index table cannot be smaller than the address location of the hash table, that is, the product of the number of addresses of the index table and the number of locations of each address is not smaller than the number of addresses of the hash table.
It should be further noted that the number of addresses in the index table is the depth of the index table, and the bit width of the hash value obtained by performing hash operation on the key value by the first hash function determines the depth of the index table, so as to ensure that the address corresponding to the address searched by using the first hash value in the address of the index table can be found, assuming that the bit width of the hash value obtained by performing hash operation on the key value by the first hash function is i, and the depth of the index table is 2iWhen i is 10, the depth of the index table is 210。
It should also be noted that the mark may include: numeric, alphabetic, and symbolic indicia, and the like. When marked as a numeric label, two different numbers may be used to indicate that a location is occupied and that a location is unoccupied; when marked as an alphabetical mark, two different letters can be used to indicate that a location is occupied and that a location is unoccupied; when marked as a symbol mark, two different symbols may be used to represent that a location is occupied and that a location is unoccupied.
Specifically, the pre-established index table may be as shown in fig. 4, where the depth of the index table (i.e. the number of addresses of the index table) is 2iEach address comprises 4 slots, each slot comprising: a flag (flag) field (corresponding to a third field for storing a flag indicating whether or not a position is occupied), a hash value (sig) field (corresponding to a second field for storing an index value), and an index value (index) field (corresponding to a first field for storing a hash value obtained by hashing a key value using a second hash function).
Optionally, before determining, according to the obtained hash value and a pre-established index table for storing index values, that the key value to be inserted into the hash table does not cause hash collision, the method further includes:
and searching the address corresponding to the first hash value in the address of the index table to be used as a target address.
And judging whether each position in the target address is occupied or not.
If all the positions in the target address are not occupied, determining that the key value to be inserted into the hash table cannot cause hash collision.
And if one part of the positions in the target address are occupied and the rest of the positions are not occupied, judging whether the key value to be inserted into the hash table causes hash collision by using the second hash value and the index table.
It should be noted that, if a part of the positions in the destination address are occupied and the rest of the positions are not occupied, there is a possibility of hash collision, and therefore, it is necessary to further determine whether the key value to be inserted into the hash table causes hash collision by using the second hash value and the index table.
Optionally, determining whether a key value to be inserted into the hash table causes a hash collision by using the second hash value and the index table, including:
and acquiring hash values on the first fields of all occupied positions in the target address as hash values to be compared.
And judging whether the second hash value is the same as any one of the hash values to be compared.
And if the second hash value is different from any one of the hash values to be compared, determining that the key value to be inserted into the hash table can not cause hash collision.
Optionally, after allocating a target index value different from the historical index value to the key value to be inserted into the hash table, the method further includes:
a second hash value is added to a first field of a first unoccupied location in the target address.
Optionally, after allocating a target index value different from the historical index value to the key value to be inserted into the hash table, the method further includes:
the target index value is added to a second field of the first unoccupied location in the target address.
Optionally, after allocating a target index value different from the historical index value to the key value to be inserted into the hash table, the method further includes:
a third field that adds a flag indicating that a location is occupied to a first unoccupied location in the target address.
It should be noted that, in the hash collision processing method provided in the embodiment of the present invention, the size of the hash table is consistent with the size of the sample space of the key value, and the hash table is not expanded. In order to reduce the conflict, an index table is introduced, and each key value is allocated with a completely unique index value in the index table, so that the index value cannot be repeated with the index value corresponding to the key value already existing in the hash table, and the hash conflict is avoided. In addition, although the size of the index table should be consistent with the number of key samples theoretically, the bit width of the index table is much smaller than that of the hash table, so that the index table can be enlarged to achieve the purpose of reducing collisions. When the DDR is used for storing the index table, only one read command can be used for reading all the required index table information, and each key has a unique index value, so that the content of the hash table can be read by one-time reading operation as long as the index table is hit, and the query speed is improved.
An embodiment of the present invention provides a hash collision processing apparatus, as shown in fig. 5, where the hash collision processing apparatus 2 includes:
the obtaining module 21 is configured to obtain a key value and a result value to be inserted into the hash table.
The operation module 22 is configured to perform hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions, respectively, to obtain at least two hash values.
The processing module 23 is configured to determine, according to the obtained hash value and a pre-established index table for storing index values, that a key value to be inserted into the hash table does not cause hash collision, and allocate a target index value different from the historical index value to the key value to be inserted into the hash table; and the historical index value is an index value corresponding to the existing key value in the hash table.
And the adding module 24 is configured to add the key value and the result value to be inserted into the hash table to the hash table by using the target index value as an address.
The preset at least two different hash functions comprise: a first hash function and a second hash function. The operation module 22 is specifically configured to:
and carrying out hash operation on the key value to be inserted into the hash table by using a first hash function to obtain a first hash value.
And carrying out hash operation on the key value to be inserted into the hash table by using a second hash function to obtain a second hash value.
Optionally, the index table comprises: a number of addresses;
the address includes: a number of locations;
the positions include: a first field, a second field, and a third field; the first field is used for storing a hash value obtained by carrying out hash operation on the key value by using a second hash function; a second field for storing an index value; a third field for storing a flag indicating whether a location is occupied.
And the product of the number of the addresses of the index table and the number of the positions of each address is not less than the number of the addresses of the hash table.
Optionally, the processing module 23 is further configured to:
judging whether each position in the target address is occupied or not;
if all the positions in the target address are not occupied, determining that the key value to be inserted into the hash table can not cause hash collision;
and if one part of the positions in the target address are occupied and the rest of the positions are not occupied, judging whether the key value to be inserted into the hash table causes hash collision by using the second hash value and the index table.
Optionally, the processing module 23 is specifically configured to:
acquiring hash values on first fields of all occupied positions in a target address, and taking the hash values as hash values to be compared;
judging whether the second hash value is the same as any one of the hash values to be compared;
and if the second hash value is different from any one of the hash values to be compared, determining that the key value to be inserted into the hash table can not cause hash collision.
Optionally, the adding module 24 is further configured to add the second hash value to the first field of the first unoccupied location in the target address.
Optionally, the adding module 24 is further configured to add the target index value to a second field of a first unoccupied location in the target address.
Optionally, the adding module 24 is further configured to add a mark indicating that the position is occupied to a third field of the first unoccupied position in the target address.
According to the processing device for hash collision provided by the embodiment of the invention, since the hash value according to which whether the key value to be inserted into the hash table causes hash collision is determined to be obtained based on at least two hash functions, the probability of hash collision is reduced from different hash functions, and after the key value to be inserted into the hash table does not cause hash collision, the target index values with different index values corresponding to the key values already existing in the hash table are allocated to the key value to be inserted into the hash table, so that the target index values are taken as the addresses of the key value to be inserted into the hash table and the result value and are inevitably different from the addresses of the key values already existing in the hash table, thereby reducing hash collision on the premise of not expanding the hash table, saving the cost in equipment, and improving the lookup speed of the hash table.
In practical applications, the obtaining module 21, the operation module 22, the Processing module 23, and the adding module 24 may be implemented by a Central Processing Unit (CPU), a microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like, which are located in the hash collision Processing apparatus.
An embodiment of the present invention further provides a hash collision processing apparatus, as shown in fig. 6, the apparatus includes:
the hash function module 31 is configured to accept an input key, and there are 2 different hash functions inside the hash function module, which are recorded as hash _ i and hash _ h, and hash the key through the 2 hash functions, and output 2 hash values: a first hash value (Hi) and a second hash value (Hh).
And the hash collision processing module 32 is configured to receive the 2 hash values output by the 101 module, process a hash collision, and perform corresponding read-write operations on the index table and the hash table according to the operations of inserting, querying, and deleting.
When the key is inserted, the action is also input together with the key. Reading data from the index table by taking Hi as an address, then distributing an index for key through hash collision processing, and writing Hh and the index into corresponding positions of Hi address space of the index table. Then, the key and the action are written into the hash table by taking the index as an address.
When the key is inquired, only the key is input, no action is obtained, and the action is obtained by inquiring from the hash table. Reading data from the index table by taking Hi as an address, extracting a hash value of a hash function hash _ h from returned data, wherein the hash value is marked as Hh ', comparing the Hh and the Hh' generated by the current key, selecting the index at the position with the consistent result, and reading the data from the hash table by taking the index as the address, wherein the key and the action which are returned by the hash table are required.
When deleting the key, only the key is input, and no action exists. Reading data from an index table by taking Hi as an address, extracting Hh 'from returned data, comparing the Hh and the Hh' generated by current key, selecting the index at the position with the consistent result, reading the data from a hash table by taking the index as the address, then performing key comparison, wherein the consistency indicates that the deletion is successful, writing 0 to the hash table by taking the index as the address, and resetting the corresponding slot in the Hi address in the index table. An inconsistency indicates a deletion failure.
It is assumed that the established index table is as shown in fig. 5, where when flag is 0, it indicates that the slot is free, and when flag is 1, it indicates that the slot is occupied. The processing procedures of key insertion, key query and key deletion are explained in detail below:
the key insertion process comprises the following steps:
And 2, returning index table data, and extracting 4 slots of flag from the returned data.
And 3, from the slot (1) to the slot (4), finding the first free slot through a flag, and marking the slot as a slot (j), wherein the slot is a slot position to be inserted by key. When 4 slots are all free, taking j equal to 1, determining that the current key can be inserted, and jumping to step 4; when all slots are occupied, the current item can not be inserted, and the step 7 is skipped; otherwise, the process jumps to step 5.
And step 4, setting flag to be 1, sig to be Hh, allocating a new index for the current key, writing the new flag (at the moment, flag to be 1), sig and index into slot (1) position of the Hi address space of the index table, and jumping to step 6.
And 5, extracting sig _ h in the slot with the flag being 1. And then, the Hh is compared with all the extracted sig _ h, if the extracted sig _ h is consistent, the key cannot be inserted, and if the extracted sig _ h is not consistent, the key can be inserted. When the key is inserted, allocating a new index for the current key, writing the new flag, sig and index into the slot (j) position of the Hi address space of the index table, and jumping to the step 6; when the plug-in is not available, the plug-in fails and the step 7 is skipped.
And step 6, writing the key and the action into a hash table by taking the newly allocated index as an address.
And 7, ending the flow.
The processing process of Key query comprises the following steps:
And 2, returning index table data, and extracting 4 slots of flag from the returned data.
And step 3, marking the sig in the slot with the flag being 1 as sig _ h. And then comparing the Hh with all the sig _ h, if the two are consistent, indicating that the index table query is successful, and recording the slot where the sig _ h is the same as the Hh as slot (h), and if the two are not consistent, indicating that the index query is failed. When the index table is successfully queried, extracting the index from slot (h), and jumping to the step 4; and when the index table query fails, jumping to the step 5.
And 4, reading the hash table by taking the extracted index as an address, extracting key and action fields from the returned data, if the extracted key is consistent with the key used for query, the query of the hash table is successful, the action is required query information, and if the extracted key is inconsistent with the key used for query, the query of the hash table is failed, the action is invalid, and the step 5 is skipped.
And 5, ending the flow.
The processing process of Key deletion comprises the following steps:
And 2, returning index table data, and extracting 4 slots of flag from the returned data.
And step 3, marking the sig in the slot with the flag being 1 as sig _ h. And then comparing the Hh with all the sig _ h, if the two are consistent, the index table query is successful, the slot where the sig _ h is the same as the Hh is marked as slot (h), and if the two are not consistent, the index table query is failed. When the index table is successfully queried, extracting the index from slot (h), and jumping to the step 4; and when the index table query fails, jumping to the step 6.
And 4, reading the hash table by taking the extracted index as an address, extracting key and action fields from the returned data, and jumping to the step 5.
And 5, judging whether the read key is consistent with the key used for inquiring, if so, writing all 0 to the hash table by taking the index as an address, and clearing slot (h) in the index table by taking Hi as the address. If not, the deletion is failed. Skipping to step 6;
and 6, ending the process.
The embodiment of the present invention further provides a hash collision processing apparatus, including a memory and a processor, where the memory stores the following instructions executable by the processor:
and acquiring a key value and a result value to be inserted into the hash table.
And respectively carrying out hash operation on the key values to be inserted into the hash table by utilizing at least two preset different hash functions to obtain at least two hash values.
Determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing the index value, and distributing a target index value different from the historical index value to the key value to be inserted into the hash table; and the historical index value is an index value corresponding to the existing key value in the hash table.
And adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
Optionally, the preset at least two different hash functions include: a first hash function and a second hash function. The memory has embodied therein the following instructions executable by the processor:
and carrying out hash operation on the key value to be inserted into the hash table by using a first hash function to obtain a first hash value.
And carrying out hash operation on the key value to be inserted into the hash table by using a second hash function to obtain a second hash value.
Optionally, the index table comprises: a number of addresses.
The address includes: a number of locations.
The positions include: a first field, a second field, and a third field; the first field is used for storing a hash value obtained by carrying out hash operation on the key value by using a second hash function; a second field for storing an index value; a third field for storing a flag indicating whether a location is occupied.
And the product of the number of the addresses of the index table and the number of the positions of each address is not less than the number of the addresses of the hash table.
Optionally, the memory further stores the following instructions executable by the processor:
and searching the address corresponding to the first hash value in the address of the index table to be used as a target address.
And judging whether each position in the target address is occupied or not.
If all the positions in the target address are not occupied, determining that the key value to be inserted into the hash table cannot cause hash collision.
And if one part of the positions in the target address are occupied and the rest of the positions are not occupied, judging whether the key value to be inserted into the hash table causes hash collision by using the second hash value and the index table.
Optionally, the memory further stores the following instructions executable by the processor:
and acquiring hash values on the first fields of all occupied positions in the target address as hash values to be compared.
And judging whether the second hash value is the same as any one of the hash values to be compared.
And if the second hash value is different from any one of the hash values to be compared, determining that the key value to be inserted into the hash table can not cause hash collision.
Optionally, the memory further stores the following instructions executable by the processor:
a second hash value is added to a first field of a first unoccupied location in the target address.
Optionally, the memory further stores the following instructions executable by the processor:
the target index value is added to a second field of the first unoccupied location in the target address.
Optionally, the memory further stores the following instructions executable by the processor:
a third field that adds a flag indicating that a location is occupied to a first unoccupied location in the target address.
An embodiment of the present invention further provides a computer-readable storage medium, where the storage medium stores computer-executable instructions, and the computer-executable instructions are configured to perform the following steps:
and acquiring a key value and a result value to be inserted into the hash table.
And respectively carrying out hash operation on the key values to be inserted into the hash table by utilizing at least two preset different hash functions to obtain at least two hash values.
Determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing the index value, and distributing a target index value different from the historical index value to the key value to be inserted into the hash table; and the historical index value is an index value corresponding to the existing key value in the hash table.
And adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
Optionally, the preset at least two different hash functions include: a first hash function and a second hash function. The computer-executable instructions further perform the steps of:
and carrying out hash operation on the key value to be inserted into the hash table by using a first hash function to obtain a first hash value.
And carrying out hash operation on the key value to be inserted into the hash table by using a second hash function to obtain a second hash value.
Optionally, the index table comprises: a number of addresses.
The address includes: a number of locations.
The positions include: a first field, a second field, and a third field; the first field is used for storing a hash value obtained by carrying out hash operation on the key value by using a second hash function; a second field for storing an index value; a third field for storing a flag indicating whether a location is occupied.
And the product of the number of the addresses of the index table and the number of the positions of each address is not less than the number of the addresses of the hash table.
Optionally, the computer executable instructions further specifically perform the steps of:
and hashing the key value by using a first hashing function to obtain a target first hashing value.
And searching the address corresponding to the first hash value in the address of the index table to be used as a target address.
And judging whether each position in the target address is occupied or not.
If all the positions in the target address are not occupied, determining that the key value to be inserted into the hash table cannot cause hash collision.
And if one part of the positions in the target address are occupied and the rest of the positions are not occupied, judging whether the key value to be inserted into the hash table causes hash collision by using the second hash value and the index table.
Optionally, the computer executable instructions further specifically perform the steps of:
and acquiring hash values on the first fields of all occupied positions in the target address as hash values to be compared.
And judging whether the second hash value is the same as any one of the hash values to be compared.
And if the second hash value is different from any one of the hash values to be compared, determining that the key value to be inserted into the hash table can not cause hash collision.
Optionally, the computer-executable instructions further perform the steps of:
a second hash value is added to a first field of a first unoccupied location in the target address.
Optionally, the computer-executable instructions further perform the steps of:
the target index value is added to a second field of the first unoccupied location in the target address.
Optionally, the computer-executable instructions further perform the steps of:
a third field that adds a flag indicating that a location is occupied to a first unoccupied location in the target address.
Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (11)
1. A method for processing hash collision comprises the following steps:
obtaining a key value and a result value to be inserted into the hash table;
performing hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions to obtain at least two hash values;
determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from a historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
2. The processing method according to claim 1, wherein the preset at least two different hash functions include: a first hash function and a second hash function;
performing hash operation on the key value to be inserted into the hash table to obtain at least two hash values includes:
performing hash operation on the key value to be inserted into the hash table by using the first hash function to obtain a first hash value;
and carrying out hash operation on the key value to be inserted into the hash table by using the second hash function to obtain a second hash value.
3. The processing method of claim 2, wherein the index table comprises: a number of addresses;
the address includes: a number of locations;
the location includes: a first field, a second field, and a third field; the first field is used for storing a hash value obtained by performing hash operation on a key value by using the second hash function; the second field is used for storing an index value; the third field is used for storing a mark for indicating whether the position is occupied or not;
and the product of the number of the addresses of the index table and the number of the positions of each address is not less than the number of the addresses of the hash table.
4. The processing method according to claim 3, wherein before determining, according to the obtained hash value and a pre-established index table for storing index values, that the key value to be inserted into the hash table does not cause hash collision, the method further comprises:
searching an address corresponding to the first hash value in the address of the index table to serve as a target address;
judging whether each position in the target address is occupied or not;
if all the positions in the target address are not occupied, determining that the key value to be inserted into the hash table cannot cause the hash collision;
and if one part of the positions in the target address are occupied and the rest of the positions are not occupied, judging whether the key value to be inserted into the hash table causes the hash collision by using the second hash value and the index table.
5. The processing method according to claim 4, wherein the determining, by using the second hash value and the index table, whether the key value to be inserted into the hash table causes hash collision includes:
acquiring hash values on first fields of all occupied positions in the target address to serve as hash values to be compared;
judging whether the second hash value is the same as any one of the hash values to be compared;
and if the second hash value is different from any one of the hash values to be compared, determining that the key value to be inserted into the hash table does not cause the hash collision.
6. The processing method according to claim 4 or 5, wherein after assigning a target index value different from the historical index value to the key value to be inserted into the hash table, the method further comprises:
adding the second hash value to a first field of a first unoccupied location in the target address.
7. The processing method according to claim 4 or 5, wherein after assigning a target index value different from the historical index value to the key value to be inserted into the hash table, the method further comprises:
adding the target index value to a second field of a first unoccupied location in the target address.
8. The processing method according to claim 4 or 5, wherein after assigning a target index value different from the historical index value to the key value to be inserted into the hash table, the method further comprises:
adding a marker indicating that a location is occupied to a third field of a first unoccupied location in the target address.
9. A hash collision processing apparatus, comprising:
the obtaining module is used for obtaining a key value and a result value to be inserted into the hash table;
the operation module is used for respectively carrying out hash operation on the key value to be inserted into the hash table by utilizing at least two preset different hash functions to obtain at least two hash values;
the processing module is used for determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from the historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and the adding module is used for adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
10. A hash collision processing apparatus, comprising: a processor and a memory, wherein the memory has stored therein the following instructions executable by the processor:
obtaining a key value and a result value to be inserted into the hash table;
performing hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions to obtain at least two hash values;
determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from a historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
11. A computer-readable storage medium having stored thereon computer-executable instructions for performing the steps of:
obtaining a key value and a result value to be inserted into the hash table;
performing hash operation on the key value to be inserted into the hash table by using at least two preset different hash functions to obtain at least two hash values;
determining that the key value to be inserted into the hash table cannot cause hash collision according to the obtained hash value and a pre-established index table for storing index values, and distributing a target index value different from a historical index value to the key value to be inserted into the hash table; the historical index value is an index value corresponding to a key value existing in the hash table;
and adding the key value and the result value to be inserted into the hash table to the hash table by taking the target index value as an address.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811571387.0A CN111352931A (en) | 2018-12-21 | 2018-12-21 | Hash collision processing method and device and computer readable storage medium |
| PCT/CN2019/126860 WO2020125741A1 (en) | 2018-12-21 | 2019-12-20 | Hash collision processing method, apparatus, device, and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811571387.0A CN111352931A (en) | 2018-12-21 | 2018-12-21 | Hash collision processing method and device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111352931A true CN111352931A (en) | 2020-06-30 |
Family
ID=71102527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811571387.0A Pending CN111352931A (en) | 2018-12-21 | 2018-12-21 | Hash collision processing method and device and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111352931A (en) |
| WO (1) | WO2020125741A1 (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112162950A (en) * | 2020-09-11 | 2021-01-01 | 杭州涂鸦信息技术有限公司 | Data processing method and device based on file system and computer equipment |
| CN112202677A (en) * | 2020-09-29 | 2021-01-08 | 中移（杭州）信息技术有限公司 | Hardware acceleration query method, system, electronic equipment and storage medium |
| CN113051302A (en) * | 2021-04-19 | 2021-06-29 | 哈尔滨工业大学 | Overall design-oriented multi-dimensional data matching method and device and computer storage medium |
| CN114244817A (en) * | 2021-11-30 | 2022-03-25 | 慧之安信息技术股份有限公司 | Hash collision processing method and device based on osi protocol stack header field |
| CN115576954A (en) * | 2022-11-24 | 2023-01-06 | 恒生电子股份有限公司 | Hash table determining method and device |
| CN116401258A (en) * | 2023-06-06 | 2023-07-07 | 支付宝(杭州)信息技术有限公司 | Data indexing method, data query method and corresponding devices |
| CN116450656A (en) * | 2023-06-16 | 2023-07-18 | 北京数巅科技有限公司 | Data processing method, device, equipment and storage medium |
| CN116822456A (en) * | 2023-07-03 | 2023-09-29 | 中科驭数（北京）科技有限公司 | Character string encoding method, device, equipment and storage medium |
| CN117390029A (en) * | 2023-12-11 | 2024-01-12 | 格创通信(浙江)有限公司 | Table entry inserting method and device, electronic equipment and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117807277A (en) * | 2024-03-01 | 2024-04-02 | 中国人民解放军国防科技大学 | High-order dynamic image data storage method, device, equipment and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8364909B2 (en) * | 2010-01-25 | 2013-01-29 | Hewlett-Packard Development Company, L.P. | Determining a conflict in accessing shared resources using a reduced number of cycles |
| CN102609509B (en) * | 2010-04-26 | 2015-09-30 | 华为技术有限公司 | Hash data processing method and device |
| CN103577564A (en) * | 2013-10-25 | 2014-02-12 | 盛科网络（苏州）有限公司 | Method and device for reducing HASH collision through software shift |
| CN104158744B (en) * | 2014-07-09 | 2017-07-11 | 中国电子科技集团公司第三十二研究所 | Method for building table and searching for network processor |
| CN108111421B (en) * | 2017-11-28 | 2021-02-09 | 苏州浪潮智能科技有限公司 | Message distribution method and device based on multiple Hash |
-
2018
- 2018-12-21 CN CN201811571387.0A patent/CN111352931A/en active Pending
-
2019
- 2019-12-20 WO PCT/CN2019/126860 patent/WO2020125741A1/en active Application Filing
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112162950A (en) * | 2020-09-11 | 2021-01-01 | 杭州涂鸦信息技术有限公司 | Data processing method and device based on file system and computer equipment |
| CN112202677A (en) * | 2020-09-29 | 2021-01-08 | 中移（杭州）信息技术有限公司 | Hardware acceleration query method, system, electronic equipment and storage medium |
| CN113051302A (en) * | 2021-04-19 | 2021-06-29 | 哈尔滨工业大学 | Overall design-oriented multi-dimensional data matching method and device and computer storage medium |
| CN113051302B (en) * | 2021-04-19 | 2022-04-29 | 哈尔滨工业大学 | Overall design-oriented multi-dimensional data matching method and device and computer storage medium |
| CN114244817A (en) * | 2021-11-30 | 2022-03-25 | 慧之安信息技术股份有限公司 | Hash collision processing method and device based on osi protocol stack header field |
| CN115576954B (en) * | 2022-11-24 | 2023-04-07 | 恒生电子股份有限公司 | Hash table determining method and device |
| CN115576954A (en) * | 2022-11-24 | 2023-01-06 | 恒生电子股份有限公司 | Hash table determining method and device |
| CN116401258A (en) * | 2023-06-06 | 2023-07-07 | 支付宝(杭州)信息技术有限公司 | Data indexing method, data query method and corresponding devices |
| CN116401258B (en) * | 2023-06-06 | 2023-09-22 | 支付宝(杭州)信息技术有限公司 | Data indexing method, data query method and corresponding devices |
| CN116450656A (en) * | 2023-06-16 | 2023-07-18 | 北京数巅科技有限公司 | Data processing method, device, equipment and storage medium |
| CN116450656B (en) * | 2023-06-16 | 2023-08-22 | 北京数巅科技有限公司 | Data processing method, device, equipment and storage medium |
| CN116822456A (en) * | 2023-07-03 | 2023-09-29 | 中科驭数（北京）科技有限公司 | Character string encoding method, device, equipment and storage medium |
| CN117390029A (en) * | 2023-12-11 | 2024-01-12 | 格创通信(浙江)有限公司 | Table entry inserting method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020125741A1 (en) | 2020-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111352931A (en) | Hash collision processing method and device and computer readable storage medium | |
| US10706101B2 (en) | Bucketized hash tables with remap entries | |
| CN100450073C (en) | Method and apparatus for memory efficient fast vlan lookups and inserts in hardware-based packet switches | |
| JP2017182803A (en) | Memory deduplication method and deduplication DRAM memory module | |
| CN106874348B (en) | File storage and index method and device and file reading method | |
| CN101826107B (en) | Hash data processing method and device | |
| US20150113230A1 (en) | Directory storage method and query method, and node controller | |
| US20100228914A1 (en) | Data caching system and method for implementing large capacity cache | |
| US20170083406A1 (en) | Method and apparatus for incremental backup | |
| CN108874987B (en) | Table entry management method and device and exchange chip | |
| CN105446664B (en) | A kind of pair of memory carries out file memory method and storage device | |
| CN108255912B (en) | Method and device for storing and inquiring table data | |
| CN108762915B (en) | Method for caching RDF data in GPU memory | |
| CN114546898A (en) | TLB management method, device, equipment and storage medium | |
| CN113835639A (en) | I/O request processing method, device, equipment and readable storage medium | |
| CN111541617B (en) | Data flow table processing method and device for high-speed large-scale concurrent data flow | |
| CN112463055B (en) | Method, system, equipment and medium for optimizing and using L2P table of solid state disk | |
| US20160203082A1 (en) | Cache-optimized hash table data structure | |
| TWI761440B (en) | memory access method | |
| CN116880746A (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN111190545B (en) | Method for carrying out LPM rule compression storage on trie structure based on software implementation | |
| CN108614879A (en) | Small documents processing method and device | |
| CN116126928A (en) | Information searching system based on variable fingerprint cuckoo filter | |
| CN111881064A (en) | Method, device and equipment for processing access request in full flash memory storage system | |
| CN109284234B (en) | Storage address allocation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |