CN102194002A - Table entry adding, deleting and searching method of hash table and hash table storage device - Google Patents

Table entry adding, deleting and searching method of hash table and hash table storage device Download PDF

Info

Publication number
CN102194002A
CN102194002A CN2011101383407A CN201110138340A CN102194002A CN 102194002 A CN102194002 A CN 102194002A CN 2011101383407 A CN2011101383407 A CN 2011101383407A CN 201110138340 A CN201110138340 A CN 201110138340A CN 102194002 A CN102194002 A CN 102194002A
Authority
CN
China
Prior art keywords
hash
list item
bitmap
sublist
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2011101383407A
Other languages
Chinese (zh)
Other versions
CN102194002B (en
Inventor
李彧
张炜
孙远航
崔玉姣
郑永梅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sanechips Technology Co Ltd
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN201110138340.7A priority Critical patent/CN102194002B/en
Priority claimed from CN201110138340.7A external-priority patent/CN102194002B/en
Publication of CN102194002A publication Critical patent/CN102194002A/en
Priority to PCT/CN2012/072473 priority patent/WO2012159480A1/en
Application granted granted Critical
Publication of CN102194002B publication Critical patent/CN102194002B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • G06F16/2255Hash tables

Abstract

The invention discloses a table entry adding, deleting and searching method of a hash table and a hash table storage device. The table entry searching method of the hash table comprises the following steps of: calculating hash values of table entry key values to be searched in parallel, searching the corresponding bitmaps according to the hash values, comparing the search results, and taking out the table entry corresponding to the address of the bitmap in the hash table when the conflict bit is invalid for the first time; and comparing the table entry with the table entry to be searched, if matching, determining that the table entry is the table entry to be searched, namely the table entry to be searched is found. According to the table entry searching method of the hash table, storage of all table entries and no table entry leave are realized with least search time and high table space utilization rate, the required table entry can be accurately searched by accessing the memory for one time, meanwhile, the interface between the hardware and a memory is conveniently realized, and table space size, search efficiency and hardware implementation can be well balanced.

Description

The list item interpolation of Hash table, deletion, lookup method and Hash table memory storage
Technical field
The present invention relates to communication technical field, relate in particular to a kind of list item interpolation, deletion, lookup method and memory storage thereof of Hash table.
Background technology
In the communications field, exist multiple matching algorithm, for example,, can adopt hash algorithm for searching of accurate coupling.The process of described Hash lookup is exactly with hash function find key value (Key) computing of condensing to input, again with the cryptographic hash addressing in the hash index table that calculates, read the hash index of depositing in the respective items (Hash Index) after finding occurrence, use hash index in the addressing of first level address mapping table at last, obtain required lookup result.
Hash algorithm is a kind of common algorithm of searching fast, and its basic thought is that the key word k with each element in the linear list is an independent variable.By certain functional relation H (k), calculate corresponding functional value and come, and this value is interpreted as the unit of a continuous storage space, this element is stored in this unit.Hash algorithm when searching, does not need comparison with setting up a definite corresponding relation between the memory location of element and its key word, and one time Hash mapping just can be searched required element.
In hash function, may obtain same Hash address to different key words, i.e. key1 ≠ key2, and H (key1)=H (key2), this situation is conflict (Collision).In general, running into conflict is very normal thing, and how key is handled after being to run into conflict, searches purpose to meet the requirements of.Handle conflict, mainly contain following two kinds of methods: the one, select and improve reasonable hash function; The 2nd, select proper list item storage organization.
At present, for the application of this class of route querying, the hash function that adopts CRC (Cyclic Redundancy Checking, CRC) class is relatively to be fit to.By choosing different CRC polynomial expressions, can realize the different hash functions under the same bit wide.
Selecting suitable table storage organization to manage conflict aspect the problem, can adopt multidimensional Hash table, conflict chain, Double Hashing etc.The simulation experiment result shows, use these methods, leave under the precondition that promptly all list items can both deposit at the no list item of assurance, there is the relation of opposition in the number of times that calculates cryptographic hash with the storage list space that needs, that is to say if want to put down all list items, then need to carry out the Hash operation of abundant number of times with enough little storage space.Will cause the repeatedly visit to same table like this, because table has only one, can only carry out in proper order so search, this can have a strong impact on the speed of tabling look-up; On the other hand, if want just can find the storage room with enough few calculation times, will guarantee enough big of storage space so, space availability ratio will reduce greatly.Many Hash tables are parallel to be searched if adopt, and then needs to take a plurality of memory interfaces, is unfavorable for the realization on the hardware.
Summary of the invention
The main technical problem to be solved in the present invention is, a kind of list item interpolation, deletion, lookup method of Hash table is provided, and memory storage, realizes whole list items storages and do not have list item and leave over minimum time cost and the higher table space utilization factor of searching; Only need the list item that visit once just can find accurately to be needed simultaneously; Also be convenient to simultaneously the realization of interface between hardware and the storer, can be in the table space size, search efficiency and hardware realization aspect obtain reasonable equilibrium.
For solving the problems of the technologies described above, the technical solution used in the present invention is as follows:
A kind of list item adding method of Hash table comprises:
Hash table is split as many Hash sublists, the corresponding bitmap of each Zhang Haxi sublist, the corresponding hash function of each Zhang Weitu;
Determine current goal Hash sublist, utilize the corresponding hash function of the pairing bitmap of current goal Hash sublist to calculate the cryptographic hash of the key assignments of list item to be stored, determine whether room, the pairing memory location of current goal Hash sublist according to described cryptographic hash, in this way, the information of described list item to be stored is deposited in this room, otherwise, upgrade current goal Hash sublist until finding the room to store the information of described list item to be stored.
Further, described bitmap has comprised conflict position and significance bit, determine whether room, the pairing memory location of current goal Hash sublist according to described cryptographic hash, in this way, the information of described list item to be stored is deposited in this room, otherwise, upgrade current goal Hash sublist until finding the room to store the step of the information of described list item to be stored, comprising:
As index, search the bitmap of described current goal Hash sublist correspondence with described cryptographic hash;
Whether the significance bit of judging described bitmap is effective, and whether the conflict position of described bitmap is invalid;
If described significance bit is effectively, and described conflict position is invalid, and then definite pairing memory location of described current goal Hash sublist is the room, and then the information with described list item to be stored deposits this room in; Otherwise, upgrade current goal Hash sublist until finding the room to store the information of described list item to be stored.
Further, described with described cryptographic hash as index, search the step of the bitmap of described current goal Hash sublist correspondence, comprising:
Whether the capacity of judging current goal Hash sublist is less than the figure place of the described cryptographic hash that calculates;
If the capacity of described current goal Hash sublist then intercepts described cryptographic hash less than the figure place of the described cryptographic hash that calculates, and the cryptographic hash after will intercepting is searched the bitmap of described current goal Hash sublist correspondence as index.
Further, deposit the information of described list item to be stored in room after, also comprise:
With the active position of described bitmap is invalid, and the key assignments of described list item to be stored is stored in this bitmap.
Further, upgrade current goal Hash sublist, comprising until finding the room to store the step of the information of described list item to be stored:
When the conflict position of described bitmap is invalid, significance bit is invalid, and the list item of then having deposited in the Hash sublist with this bitmap correspondence takes out, and be effective with this collision position, and will take out the key assignments of list item and the key assignments of current list item to be stored deposits in this bitmap;
Judge whether current target Hash sublist is last Hash sublist, if not last Hash sublist, then current goal Hash sublist is updated to next Zhang Haxi sublist, stores the information of described list item to be stored until finding the room.
Further, upgrade current goal Hash sublist, comprising until finding the room to store the step of the information of described list item to be stored:
When the conflict position of this bitmap is that effectively the key assignments of then current list item to be stored directly is deposited in this bitmap;
Judge that whether current goal Hash sublist is last Hash sublist, if not last Hash sublist, then is updated to next Zhang Haxi sublist with current goal Hash sublist, until finding the room to store the information of described list item to be stored.
Further, described method also comprises:
After described list item storage to be stored is finished, upgrade the conflict position information of bitmap and the information of Hash sublist synchronously.
Further, the hash function of described Hash sublist correspondence is the CRC function.
Further, adopt different CRC polynomial expressions between the hash function of described many Hash sublist correspondences.
A kind of list item delet method of Hash table, Hash table comprise many Hash sublists, the corresponding bitmap of each Zhang Haxi sublist, and the corresponding hash function of each Zhang Weitu, and every described bitmap comprises a plurality of clauses and subclauses, then described method comprises:
The cryptographic hash of the clauses and subclauses that every pairing hash function parallel computation of bitmap need be deleted;
According to the cryptographic hash that calculates, parallel searching in each self-corresponding bitmap at it;
The lookup result of each Zhang Weitu is compared in proper order, when for the first time the conflict position to occur be invalid, the list item taking-up in the Hash sublist of the address correspondence of this bitmap;
The key assignments of the list item that takes out is compared with the key assignments that need search,, then described list item is deleted from described Hash sublist if identical.
Further, also comprise after the remove entries;
With the collision position of the bitmap of correspondence is invalid, and active position is effective.
Further, described method also comprises:
All key assignments of being stored in all bitmaps before the bitmap at list item place of deletion are compared;
If the key assignments of being stored in certain bitmap is greater than two, the key assignments that then directly will need to delete deletes; Perhaps,
If the key assignments of being stored in certain bitmap equals two, the key assignments that then will need to delete deletes, and finds the position that another key assignments is stored, and with its taking-up, deposits in the pairing Hash sublist of this bitmap.
A kind of lookup method of Hash table, described Hash table comprise many Hash sublists, every corresponding bitmap of Hash sublist, and every corresponding hash function of bitmap, then described Hash table lookup method comprises:
Utilize the cryptographic hash of key assignments of the hash function parallel computation list item to be found of every Hash sublist correspondence;
According to the described cryptographic hash that calculates parallel searching in the bitmap of correspondence;
The lookup result of each Zhang Weitu is compared in proper order, when for the first time the conflict position to occur be invalid, take out the list item in the Hash sublist of address correspondence of this bitmap;
The key assignments of the list item that takes out is compared with the key assignments of list item to be found, if the key assignments of the list item that takes out and the key assignments of described list item to be found are complementary, then with the information of this list item as final checking result.
Further, described method also comprises: if the key assignments of the list item that takes out and the key assignments of described list item to be found are not complementary, then return and search failure.
Further, according to the described cryptographic hash that calculates parallel step of searching in the bitmap of correspondence, comprising:
Capacity according to the Hash sublist of the described cryptographic hash correspondence that calculates intercepts described cryptographic hash;
Cryptographic hash after the intercepting as index, is walked abreast in its corresponding bitmap and searches.
The corresponding hash function of described Hash sublist was identical when further, the hash function of described bitmap correspondence was with storage item.
A kind of Hash table memory storage, comprise the master control logic module, and the operation circuit and the data circuit that link to each other with described master control logic module, wherein, the described operation circuit of master control logic module controls is finished the interpolation and/or the deletion of list item, and the control data circuit is finished list item and searched, described operation circuit comprises: list item splits module, target Hash sublist determination module and list item memory module, wherein
Described list item splits module, is used for Hash table is split as many Hash sublists and many bitmaps, and every corresponding bitmap of Hash sublist and a hash function;
Described target Hash sublist determination module splits module with described list item and links to each other, and is used for splitting module from described list item and splits definite Hash sublist of many Hash sublists that obtain as current target Hash sublist; And after receiving the update instruction that described list item memory module sends, be used for current target Hash sublist is updated to its next Zhang Haxi sublist;
Described list item memory module, link to each other with described target Hash sublist determination module, be used to utilize the pairing hash function of described current goal Hash sublist, calculate the cryptographic hash of the key assignments of described list item to be stored, and determine according to described cryptographic hash whether the pairing memory location of current goal Hash sublist is the room, in this way, deposit the information of list item to be stored in this room, otherwise send update instruction to described target Hash sublist determination module, store described list item to be stored until finding the room.
Further, described bitmap comprises conflict position and significance bit, and described list item memory module comprises calculating sub module, searches submodule, list item adds submodule, wherein,
Described calculating sub module links to each other with described target Hash sublist determination module, is used to utilize the pairing hash function of described current goal Hash sublist, calculates the cryptographic hash of the key assignments of described list item to be stored;
The described submodule of searching, link to each other with described calculating sub module, whether the figure place that is used to judge the cryptographic hash that described target Hash sublist determination module calculates is more than or equal to the capacity of described current goal Hash sublist, in this way, then intercept described cryptographic hash, and the cryptographic hash after will intercepting is as index, search corresponding bitmap, and whether the significance bit of judging described bitmap is effective, and whether the conflict position of described bitmap be invalid, in this way, judge that then the memory location in the described target Hash sublist is the room, otherwise, then send update instruction, until finding the room to described target Hash sublist determination module;
Described list item adds submodule, links to each other with the described submodule of searching, be used for when the described submodule of searching when described current goal Hash sublist finds the room, the information of described list item to be stored is deposited in the described room.
Further, when described list item adds submodule, after depositing in the information of described list item to be stored in the described room, it is invalid that described list item adds the active position that submodule also is used for described bitmap, and the key assignments of described list item to be stored is stored in the described bitmap.
Further, when described when searching submodule to judge the significance bit of described bitmap being invalid, be used to also judge whether the conflict position of described bitmap is invalid, in this way, then described list item adds submodule and is used for the list item that the corresponding Hash sublist of described bitmap has been stored is taken out, and the key assignments of the list item that takes out and the key assignments of described current list item to be stored are deposited in the described bitmap; When searching submodule to judge the conflict position of described bitmap being effective, the list item of then described list item memory module adds submodule and is used for storing the key assignments of current described list item to be stored into described bitmap when described.
Further, described Hash table comprises many Hash sublists, every corresponding bitmap of Hash sublist and a hash function, and described data circuit comprises the parallel computation module, walks abreast and searches module and comparison module, wherein,
The parallel computation module is used to utilize the hash function of every Hash sublist correspondence, the cryptographic hash of the key assignments of parallel computation list item to be found;
Parallel search module, link to each other with described parallel computation module, be used for cryptographic hash that basis calculates at it each self-corresponding bitmap search concurrently;
Comparison module, be used for the checking result of each bitmap is compared judgement, determine the address of Hash sublist at the list item place of coupling, the information of the list item of this address storage is taken out, and compare with key assignments that needs are searched, whether the list item that judge to take out is complementary with list item to be found, as coupling, then with the information of this list item as final checking result.
The invention has the beneficial effects as follows:
The list item adding method of Hash table of the present invention, by Hash table being split as many Hash sublists, and the corresponding bitmap of each Zhang Haxi sublist and a hash function, utilize this hash function to calculate the cryptographic hash of the key assignments of list item to be stored again, determine the room of storage list item to be stored by this cryptographic hash, promptly because every Hash sublist all is to obtain by Hash table is split, and in bitmap, determined the memory address of coupling list item, thereby with minimum update time cost and higher table space utilization factor realize whole list items storages, do not leave over and there is list item.
Hash table lookup method of the present invention, cryptographic hash by parallel computation item key value to be found, and in corresponding bitmap, search according to this cryptographic hash, and the result that will search compares, when for the first time the conflict position to occur be invalid, take out the list item in the Hash sublist of address correspondence of this bitmap; Itself and band are searched list item to be compared, if be complementary, then this list item is list item to be found, because every Hash sublist all is to obtain by Hash table is split, and in bitmap, determined the memory address of coupling list item, just can find the list item that needs accurately, also be convenient to the realization of interface between hardware and the storer simultaneously thereby only need visit a Hash sublist, can be in the table space size, search efficiency and hardware realization aspect obtain reasonable equilibrium.
Description of drawings
Fig. 1 is the structural representation of an embodiment of Hash table memory storage of the present invention;
Fig. 2 is the structural representation of an embodiment of list item memory module of the present invention;
Fig. 3 is the structural representation of a specific embodiment of Hash table memory storage of the present invention;
Fig. 4 is the process flow diagram of an embodiment of the list item adding method of Hash table of the present invention;
Fig. 5 is the process flow diagram of an embodiment of step S407 of the list item adding method of Hash table of the present invention;
Fig. 6 is the process flow diagram of an embodiment of step S409b of the list item adding method of Hash table of the present invention;
Fig. 7 is the process flow diagram of an embodiment of the list item delet method of Hash table of the present invention;
Fig. 8 is the process flow diagram of an embodiment of Hash table lookup method of the present invention;
Fig. 9 is the building-block of logic of an embodiment of Hash table of the present invention;
Figure 10 is the process flow diagram of the specific embodiment that adds of the list item of Hash table of the present invention;
Figure 11 is a logical organization synoptic diagram of searching an embodiment of Hash table of the present invention.
Embodiment
In conjunction with the accompanying drawings the present invention is described in further detail below by embodiment.
Please refer to Fig. 1, be the structured flowchart of an embodiment of the Hash table memory storage of present embodiment.The Hash table memory storage of present embodiment comprises master control logic module 100, and the operation circuit and the data circuit that link to each other with master control logic module 100, wherein, master control logic module 100 control operation circuits are finished the interpolation and/or the deletion of list item, the control data circuit is finished list item and is searched, wherein operation circuit comprises: list item splits module 101, target Hash sublist determination module 102 and list item memory module 103, wherein, list item splits module 101, target Hash sublist determination module 102 and list item memory module 103, wherein, list item splits module 101, is used for Hash table is split as many Hash sublists and many bitmaps, and every corresponding bitmap of Hash sublist and a hash function; Target Hash sublist determination module 102 splits module 101 with list item and links to each other, be used for that list item is split module 101 and split first Hash sublist of many Hash sublists that obtain as current target Hash sublist, and after receiving the update instruction that list item memory module 103 sends, be used for current target Hash sublist is updated to its next Zhang Haxi sublist; List item memory module 103, link to each other with target Hash sublist determination module 102, be used to utilize the pairing hash function of current goal Hash sublist, calculate the cryptographic hash of the key assignments of list item to be stored, and determine according to this cryptographic hash whether the pairing memory location of current goal Hash sublist is the room, in this way, deposit the information of list item to be stored in this room, otherwise send update instruction to target Hash sublist determination module 102, store list item to be stored until finding the room.
Please refer to Fig. 2, be the structural representation of an embodiment of the list item memory module of present embodiment.Bitmap in the present embodiment comprises conflict position and significance bit.The list item memory module comprises calculating sub module 200, searches submodule 201 in the present embodiment, list item adds submodule 203, wherein, calculating sub module 200, link to each other with target Hash sublist determination module, be used to utilize the pairing hash function of current goal Hash sublist, calculate the cryptographic hash of the key assignments of list item to be stored; Search submodule 201, link to each other with calculating sub module 200, whether the figure place that is used to judge the cryptographic hash that this target Hash sublist determination module calculates is more than or equal to the capacity of current goal Hash sublist, in this way, then intercept this cryptographic hash, and the cryptographic hash after will intercepting is as index, search corresponding bitmap, and whether the significance bit of judging the bitmap find is effective, and whether the conflict position be invalid, in this way, judge that then the memory location in this target Hash sublist is the room, otherwise, then send update instruction, until finding the room to target Hash sublist determination module; List item adds submodule 202, with search submodule 201 and link to each other, be used for when searching submodule 201 when current goal Hash sublist finds the room, the information of list item to be stored is deposited in the room, and be invalid with the active position of this bitmap, the key assignments of list item to be stored is stored in this bitmap, and when searching submodule 201 to judge the significance bit of this bitmap be invalid, be used to also judge whether the conflict position of this bitmap is invalid, in this way, then list item adds submodule and is used for the list item that the corresponding Hash sublist of bitmap has been stored is taken out, and the key assignments of the list item that takes out and the key assignments of current list item to be stored are deposited in the bitmap; When searching submodule 201 to judge the conflict position of bitmap be effective, then list item adds submodule 202 and is used for storing the key assignments of current list item to be stored into this bitmap.
Every bitmap in the present embodiment has comprised a bit swiping position and a significance bit, a plurality of conflict key assignments holding tanks position.The active position of the bitmap in the present embodiment is 0, i.e. expression effectively is changed to 1, and it is invalid promptly to represent; The collision position of bitmap is that 0 expression is invalid, is changed to 1 expression effectively.
Please refer to Fig. 1, the data circuit of the Hash table memory storage in the present embodiment comprises parallel computation module 301, parallel module 302 and the comparison module 303 searched, wherein, parallel computation module 301, be used to utilize the hash function of every Hash sublist correspondence, the cryptographic hash of the key assignments of parallel computation list item to be found; Parallel search module 302, link to each other with parallel computation module 301, be used for cryptographic hash that basis calculates at it each self-corresponding bitmap search concurrently; Comparison module 303, be used for the checking result of each bitmap is compared judgement, determine the address of Hash sublist at the list item place of coupling, the information of the list item of this address storage is taken out, and compare with key assignments that needs are searched, determine whether list item mates, as coupling, then with the information of this list item as final checking result.
When the bitmap of the Hash sublist correspondence when searching list item in the present embodiment and storage item, the bitmap degree of depth of Hash sublist correspondence is identical; And when the hash function of Hash sublist correspondence and storage item, the hash function of this Hash sublist correspondence is identical.When searching list item in the present embodiment, corresponding bitmap only comprises one conflict position.
The Hash table memory storage of present embodiment splits module by list item Hash table is split as many Hash sublists, again by target Hash sublist determination module from wherein determining the mark Hash sublist of opening one's eyes wide, and the cryptographic hash of answering by the key-value pair that the list item memory module utilizes the hash function of target Hash sublist correspondence to calculate list item to be stored, and according to searching the room in this cryptographic hash ability target Hash sublist, thereby list item to be stored is deposited in this room, when not finding the room, then the key assignments with list item to be stored is stored in the corresponding bitmap, and more fresh target Hash sublist until finding the room to store this list item to be stored, thereby with minimum update time cost and higher table space utilization factor realize whole list items storages, do not leave over and there is list item; This device cryptographic hash of coming the key-value pair of parallel computation list item to be found to answer on the other hand by the parallel computation module, searching module by walking abreast again searches in each self-corresponding bitmap according to this cryptographic hash, come the comparison lookup result by comparison module at last, and take out corresponding list item in the Hash sublist address at coupling list item place, if this list item and list item to be found are complementary, then this list item is list item to be found, then it is returned as final lookup result, thereby because every Hash sublist all is to obtain by Hash table is split, and in bitmap, determined the memory address of coupling list item, thereby only need visit a Hash sublist and just can find the list item that needs accurately, also be convenient to simultaneously the realization of interface between hardware and the storer, can be in the table space size, search efficiency and hardware realization aspect obtain reasonable equilibrium.
Embodiment one:
Please refer to Fig. 3, be the structural representation of an embodiment of the Hash table memory storage of present embodiment.The Hash table storage system of searching of present embodiment comprises master control logic module 505, a plurality of computing module 502a~502n, a plurality of module 503a~503n that search, list item adds module 504, statistical module 505, list item splits module 506, a plurality of bitmap module 507a~507n, a plurality of list item memory module 508a~508n, data decision module 509 and abnormality processing module 550.The inside of the Hash table memory storage of present embodiment connects and is divided into data circuit and operation circuit, and on data circuit, master control logic module 501 is coupled to a plurality of computing module 502a~502n, data decision module 509; A plurality of computing module 502a~502n are coupled respectively to a plurality of module 503a~503n that search; A plurality of module 503a~503n that search are coupled respectively to a plurality of bitmap module 507a~507n, and a plurality of bitmap module 507a~507n are coupled to a plurality of list item memory module 508a~508n; On operation circuit, master control logic module 501 is coupled to list item and adds module 504, statistical module 505, list item splits module 506 and abnormality processing module 510, and list item adds module 504, statistical module 505, list item splits module 506 and abnormality processing module 510 is coupled to a plurality of module 503a~503n that search respectively.
Wherein, master control logic module 501 is control assemblies of Hash table memory storage, and the algorithm that is mainly used on control operation circuit and the data circuit starts, and stops the data distribution on task scheduling and the data routing, operations such as collection.
Master control logic module 501 in the present embodiment can realize by hardware logic, can realize the chip of logical function function as ASIC (special IC) or FPGA (field programmable gate array) or other.Certainly the master control logic module 501 in the present embodiment also can realize by the software that operates on the CPU (central authorities handle), under this implementation, module on the operation circuit, comprise that list item adds module 504, statistical module 505, list item splits module 506 and abnormality processing module 550, and all can realize by software.
A plurality of computing module 502a~502n are coupled to master control logic module 501 and a plurality of module 503a~503n that searches, in the list item search procedure, when master control logic module 501 sends to key assignments to be found a plurality of computing module 502a~502n simultaneously, these a plurality of computing modules 502 are used for assigned key assignments is carried out Hash operation simultaneously one time, obtain the index different with other computing modules.
Comprise a hash function different in each computing module 502 in the present embodiment with other computing modules 502, and the hash function in the present embodiment can adopt various known methods to realize, as XOR array or class CRC function, perhaps pass through complex mathematical computing or the like.The criterion of choosing of hash function is to reduce the call number conflict that obtains after the different key assignments process hash function computings as much as possible.
A plurality of module 503a~503n that search are coupled respectively to a plurality of computing module 502a~502n and a plurality of bitmap module 507a~507n, and statistical module 505, and list item splits module 506 and abnormality processing module 110.In the present embodiment each searched module 503 and independently is coupled in a bitmap module 507, and address bus by bitmap module 507 and data bus conduct interviews to the data in bitmap module 507 and the pairing memory module 508 of this bitmap and revise.On data circuit, each is searched module 503 and receives the different index that computing module 502 generates, and converts thereof into the address of bitmap module 507, thereby reads this address corresponding to the data in the memory module 508.After the data that get access to corresponding to this address, search the data value that index value that a plurality of computing module 502a~502n that module 503 will receive calculate and a plurality of bitmap module 507a~507n obtain, send data decision module 509 together to and find key value comparison, determine whether coupling; On operation circuit, a plurality of module 503a~503n receiving and counting modules 505 of searching, the read write command that list item fractionation module 506 and abnormality processing module 509 are sent, convert the address of a plurality of bitmap module 507a~507n to, storage data among a plurality of bitmap module 507a~507n are operated, and then the data of storing among a plurality of memory module 508a~508n are operated.
Search the function that module 503 realizes operation circuit and data circuit simultaneously in the present embodiment, can certainly be respectively be provided with and search module accordingly at operation circuit and data circuit.Can there be the expansion of index value in the present embodiment, computings such as plus-minus side-play amount or intercepting.
A plurality of bitmap module 507a~507n are coupled in a plurality of module 503a~503n that search respectively, and each bitmap module 507 has independently address access bus and independently data bus, for each module, address bus and data bus can be multiplexing, but it is not multiplexing between a plurality of memory modules 508, a plurality of bitmap module 507a~507n by above-mentioned independently address access bus and independently data bus be coupled in a plurality of module 503a~503n that search respectively, a plurality of module 503a~503n that search visit data (the conflict position of storing corresponding to place, transmission address among a plurality of bitmap module 507a~507n by sending the address, significance bit and key assignments), and also respectively corresponding memory module 508a~508n of these a plurality of bitmap module 507a~507n, Hash sublist of each memory module 508 storage, every sublist all is to split by 506 pairs of Hash tables of list item fractionation module to obtain, and in bitmap module 507, determined the memory address of coupling list item, only need memory module 508a~508n of visit, so memory module 508a~508n shares a group address access bus and a data bus.
A plurality of bitmap module 507a~507n are coupled in described a plurality of module 503a~503n that searches respectively.Wherein each bitmap module has independently address access bus and independently data bus, and wherein for each module, address bus and data bus can be multiplexing, but not multiplexing between a plurality of list item memory modules.A plurality of bitmap module 507a~507n by above-mentioned independently address access bus and independently data bus be coupled in a plurality of module 503a~503n that search respectively, a plurality of module 503a~503n that search can visit the data (conflict position, significance bit and key assignments) of storing corresponding to place, transmission address among a plurality of bitmap module 507a~507n by sending the address.The respectively corresponding list item memory module 508a~508n of a plurality of bitmap module 507a~507n, Hash sublist of each list item memory module storage, wherein every sublist all is to obtain by described Hash table is split, because in bitmap, determined the memory address of coupling list item, only need list item memory module 508a~508n of visit, so list item memory module 508a~508n shares a group address access bus and a data bus.
A plurality of bitmap modules 507 in the present embodiment and memory module 508 can realize by the various data storage technologies of current existence.Because bitmap module 507 needs concurrent access and the storage space that takies is smaller, can realize with on-chip SRAM.And for memory module 508, under the application scenarios of delaying time less at needs, can be used as memory module 508a~508n with the individual independently high-speed SRAM chip of n, described high-speed SRAM chip can be ZBT SRAM, QDR SRAM, QDRII SRAM, various sram chips commonly used such as QDRII+SRAM; Under the situation that needs the high capacity list item space, can be used as memory module 508a~508n with the individual independently high-speed DRAM chip of n, described high-speed DRAM chip can be SDRAM, DDR SDRAM, DDRII SDRAM, various SDRAM chips commonly used such as DDRIII SDRAM; Preserve under the data conditions in the needs power down, can be used as memory module 508a~508n with n independent F LASH chip, described FLASH chip can be NAND FLASH, NOR FLASH, NV RAM or other various storage chips commonly used; Under the situation of mass data, memory module 508a~508n can also be with a plurality of independently hard disks, a plurality of independently disk memory arrays, or mass data storage technology such as CD drive realizes, also can choose the realization medium of a plurality of bitmap module 507a~507n and memory module 508a~508n under the scope that does not break away from present embodiment arbitrarily.
Data decision module 509 is coupled to a plurality of module 503a~503n and a plurality of memory module 508a~508n of searching, and finishes the decision operation of lookup result.A plurality of 503a~503n that search, after having obtained a plurality of bitmap module 507a~507n lookup result, the result is sent to data decision module 509, the data decision module is pressed the in proper order relatively lookup result of bitmap module 507a~507n of a~n, find out at first the conflict position and be 0 bitmap module 507, send the bitmap address of finding to a plurality of memory module 508a~508n, the data that get access to from memory module 508 send data decision module 509 again to, the original key assignments that sends according to master control logic module 501, relatively whether these data are list items of coupling, owing to a plurality ofly search a plurality of lookup results that module 103a~103n obtains and come from the conflict position of storing among a plurality of bitmap module 107a~107n, and conflict position has only just can put 0 when it does not conflict with other list items, if once conflict just puts 1 with it, so the conflict position of Chu Xianing is that the data of storing in the list item memory module of 0 address correspondence are unique at first, therefore, the list item that does not have repetition in the middle of this, having only a result who searches memory module 508 is possible correct result, the result who perhaps returns all is not inconsistent with key assignments, this Hash lookup failure.Searching under the correct situation, data decision module 509 returns to master control logic module 501 with the index value in one group of correct data, represents the result of this Hash lookup; Searching under the situation of failure, data decision module 509 will return to 501 1 invalid index values of master control logic module or a look-at-me, and this searches failure notice master control logic module 501.
List item splits module 506 and list item adds module 504 collaborative works, is used for Hash table is split as N (N>=1) Zhang Haxi sublist and is deposited into a plurality of memory modules 508.The corresponding bitmap of every Hash sublist and a hash function, wherein N opens the storage size that Hash sublist and N open bitmap and can select as required.
Hash table in the present embodiment comprises M clauses and subclauses, and (M can be to very big entry number from 1,32 powers as 2), in the Hash sublist generative process, list item fractionation module 506 is chosen in M the clauses and subclauses successively, its target Hash sublist and bitmap are set at first Hash sublist and bitmap, and send these clauses and subclauses to list item interpolation module 504.List item adds the hash function of module 104 according to first Hash sublist correspondence, calculate the index value K of these clauses and subclauses, and attempt depositing these clauses and subclauses in the Hash sublist of the hash function correspondence of being got position corresponding to index value K, collision position 0 in the corresponding index value K position among the bitmap that will be corresponding simultaneously with this Hash sublist, active position 1 deposits original key assignments in this position in.
In the present embodiment, when 506 pairs of a plurality of Hash sublists of list item fractionation module split, might run into the situation of hash-collision, promptly two different key assignments are through calculating the same index value of same Zhang Haxi sublist.In this case, if the bitmap of this position conflict position is 0, significance bit is 1, illustrate that this position put into a list item, the data that need will deposit the list item memory module of this index value correspondence in are taken out, and with the conflict place value 1 among the bitmap of this index value correspondence, the key assignments that current key assignments is corresponding with taking out list item all is placed among the bitmap; If the bitmap of this position conflict position is 1, illustrate to have conflicted in this position to surpass once, current key assignments is put into bitmap.When above-mentioned two kinds of conflict situations occurring, return failed message and split module 106 to list item, list item splits module 506 and judges whether current target Hash sublist is last Hash sublist again, if not then the target sublist of these clauses and subclauses is set to next Zhang Zibiao, pass to list item and add module 504, repeat said process; If be last Hash sublist, then return failed message and give master control logic module 501, call the renewal that abnormality processing module 509 is finished Hash table by master control logic module 501.
Be on the operation circuit though the list item in the present embodiment adds module 504, can read or revise content in bitmap module 507 and the memory module 508 by a plurality of module 503a~503n that search.
Abnormality processing module 110 and statistical module 105 are coupled respectively to a plurality of module 503 and master control logic modules 501 of searching, and finish tasks such as system monitoring and wrong recovery.As each inquiry failure, statistical module 505 all can contribute to it and search failure record.510 of abnormality processing modules are used to respond the Hash table refresh requests that master control logic module 501 is sent, and by a plurality of module 503a~503n that search a plurality of Bitmap module 107a~107n and a plurality of memory module 108a~108n are carried out zero clearing and assign operation again.
Below the Hash table memory storage of present embodiment is split into data circuit and two parts of operation circuit are described in detail.
Wherein, data circuit is by master control logic module 501, a plurality of computing module 502a~502n, a plurality of module 503a~503n that search, a plurality of Bitmap module 507a~507n, a plurality of memory module 508a~508n and data decision module 509 constitute, this data circuit is used for finishing search operation
Operation circuit is by master control logic module 501, and list item adds module 504, and statistical module 505, list item split module 506 and abnormality processing module 510 constitutes.List item adds module 504 and is coupled to a plurality of modules 503 of searching, and is used for conflict situation information and list item data are added into a plurality of bitmap module 507a~507n and a plurality of memory module 508a~508n, and this operation circuit is used for finishing list item to be added and deletion action.
In the Hash table memory storage of present embodiment, N hash function of the Hash sublist correspondence that obtains after the fractionation can be that the CRC function also can be other functions, when adopt the CRC function as present embodiment in during the hash function of Hash sublist correspondence, can adopt different CRC polynomial expressions between the hash function of several Hash sublist correspondences.
Based on above-mentioned Hash table memory storage, present embodiment also provides a kind of list item adding method of Hash table.
Please refer to Fig. 4, be the schematic flow sheet of an embodiment of the list item adding method of the Hash table of present embodiment.The list item adding method of present embodiment Hash table comprises:
S401, Hash table are split as many Hash sublists, the corresponding bitmap of each Zhang Haxi sublist, the corresponding hash function of each Zhang Weitu.
The key influence factor that splits in the present embodiment is that the storage size of every sublist is selected.The selection of can arranging in pairs or groups as required of the storage space of many Hash sublists in the present embodiment, every little table space size is consistent such as being provided with, the also mode that can adopt storage size to successively decrease, still, the fractionation mode is not limited to above-mentioned dual mode.
S403, with first Hash sublist as current target Hash sublist.
S405 gets the key assignments of list item to be stored, utilizes the hash function of current goal Hash sublist correspondence to calculate its cryptographic hash.
S407 according to the cryptographic hash that calculates, determines whether room, the pairing memory location of current goal Hash sublist, in this way, and execution in step S409a then, otherwise, execution in step S409b.
S409a advances to deposit in this room with the information of list item to be stored, and execution in step S4011.
The information of the list item in the present embodiment comprises information such as the key assignments of tabling look-up, checking result.
S409b upgrades current goal Hash sublist and execution in step S405.
S4011 is invalid with the active position in this bitmap, and the key assignments of list item to be stored is stored in this bitmap.
S4013 after list item storage to be stored is finished, upgrades the conflict position information of bitmap and the information of Hash sublist synchronously.
Please refer to Fig. 5, be the process flow diagram of the embodiment of the step S407 of present embodiment.Step S407 in the present embodiment comprises:
Whether S501, the capacity of judging current goal Hash sublist less than the cryptographic hash of calculating, in this way, and execution in step S503a then, otherwise execution in step S503b.
S503a according to the number of bits of the capacity correspondence of Hash sublist, begins to intercept the same bits figure place of the cryptographic hash that calculates from the higher bit position, the cryptographic hash after obtaining intercepting, and the cryptographic hash after will intercepting is as index, execution in step S505.
S503b, directly with the cryptographic hash that calculates as index.
S505 searches the room according to index in target Hash sublist, judge promptly whether the significance bit of this bitmap is 0, and the conflict position of this bitmap is 0, and in this way, promptly the memory location of this target Hash sublist correspondence is the room, then execution in step S409a; Otherwise execution in step S409b.
Please refer to Fig. 6, the step S409b in the present embodiment comprises:
S601 judges whether the conflict position of this bitmap is 0, and whether significance bit is 1, in this way, execution in step S603a then, if the conflict position of this bitmap is 1, execution in step S603b then.
S603a takes out the list item of having stored in the corresponding Hash sublist of this bitmap, and is 1 with the collision position of this bitmap, and execution in step S605.
S603b stores in this bitmap the key assignments of current list item to be stored into execution in step S607.
S605 is deposited in this bitmap taking out the key assignments of list item and the key assignments of current list item to be stored.
S607 judges whether current goal Hash sublist is last Hash sublist, in this way, and execution in step S609a, otherwise execution in step S609b.
S609a returns the storage failed message, end operation.
S609b is updated to next Zhang Haxi sublist with target Hash sublist, and execution in step S405.
In the list item adding method of the Hash table of present embodiment, the hash function of Hash sublist correspondence is the CRC function, and adopts different CRC polynomial expressions between the hash function of many Hash sublist correspondences.
The list item adding method of the Hash table of present embodiment, by Hash table being split as many Hash sublists, and the corresponding bitmap of each Zhang Haxi sublist and a hash function, utilize this hash function to calculate the cryptographic hash of the key assignments of list item to be stored again, determine the room of storage list item to be stored by this cryptographic hash.Because every Hash sublist all is to obtain by Hash table is split, and in bitmap, determined the memory address of coupling list item, thereby with minimum update time cost and higher table space utilization factor realize whole list items storages, do not leave over and there is list item.
Present embodiment also provides a kind of list item delet method of Hash table.Hash table in the present embodiment comprises many Hash sublists, the corresponding bitmap of each Zhang Haxi sublist, and the corresponding hash function of each Zhang Weitu, and every described bitmap comprises a plurality of clauses and subclauses.Please refer to Fig. 7, the list item delet method of the Hash table memory storage of present embodiment comprises:
S701, the cryptographic hash of the clauses and subclauses that every pairing hash function parallel computation of bitmap need be deleted.
S703, according to the cryptographic hash that calculates, parallel searching in each self-corresponding bitmap at it.
S705 compares the lookup result of each Zhang Weitu in proper order, when the conflict position to occur for the first time be invalid, with the list item taking-up in the Hash sublist of the address correspondence of this bitmap.
S707 compares the key assignments of the list item that takes out with the key assignments that need search, if identical, execution in step S709 then.
S709 deletes this list item from the Hash sublist.
S7011 is 0 with the collision position of the bitmap of correspondence, and active position is 0.
S7013, all key assignments of being stored in all bitmaps before the bitmap at list item place of deletion are compared, if the key assignments of being stored in certain bitmap greater than two, execution in step S7015a then, if the key assignments of being stored in certain bitmap equals two, execution in step S7015b then.
S7015a, the key assignments that directly will need to delete deletes, and finishes.
S7015b, the key assignments that needs are deleted deletes, and finds the position that another key assignments is stored.
S7017, the list item that another key-value pair is answered takes out, and deposits in the pairing Hash sublist of this bitmap.
Present embodiment also provides a kind of Hash table lookup method, and the Hash table of searching in this method is made up of several Hash sublists, the selection of can arranging in pairs or groups as required of the storage size of several Hash sublists.Every bitmap has independently address control line, can be simultaneously accessed, and all Hash sublists are shared a group address control line, and each bar list item is searched and is only visited the Hash sublist one time.The corresponding hash function of every Hash sublist, the hash function of every Hash sublist correspondence during with storage item the corresponding hash function of this Hash sublist identical.
See also Fig. 8, this figure is the realization principle flow chart of the Hash table lookup method of present embodiment.The Hash table lookup method of present embodiment mainly may further comprise the steps:
S801, the cryptographic hash of utilizing the key-value pair of a plurality of hash function parallel computations list item to be found of a plurality of Hash sublist correspondences to answer.
S803 is according to the cryptographic hash that calculates parallel searching in the bitmap of correspondence.
S805 compares the lookup result of each Zhang Weitu in proper order, when the conflict position occurring for the first time and be 0, take out this conflict the list item in the Hash sublist of address correspondence of corresponding bitmap.
S807 compares the key assignments of the list item that takes out with the key assignments of list item to be found, if the key assignments of the list item of taking-up and the key assignments of described list item to be found are complementary, and execution in step S809a then, otherwise, execution in step S809b.
S809a, with the information of this list item as final checking result.
S809b returns and searches failure.
Step S805 comprises in the present embodiment: the capacity according to the Hash sublist of the cryptographic hash correspondence that calculates intercepts this cryptographic hash; Cryptographic hash after the intercepting as index, is walked abreast in its corresponding bitmap and Hash sublist and searches.
The Hash table lookup method of present embodiment, cryptographic hash by parallel computation item key value to be found, and in corresponding bitmap, search according to this cryptographic hash, and the result that will search compares, when for the first time the conflict position to occur be invalid, take out the list item in the Hash sublist of address correspondence of this bitmap; Itself and band are searched list item to be compared, if be complementary, then this list item is list item to be found, because every Hash sublist all is to obtain by Hash table is split, and in bitmap, determined the memory address of coupling list item, just can find the list item that needs accurately, also be convenient to the realization of interface between hardware and the storer simultaneously thereby only need visit a Hash sublist, can be in the table space size, search efficiency and hardware realization aspect obtain reasonable equilibrium.
Embodiment two:
See also Fig. 9, this figure is the building-block of logic of an embodiment of the Hash table of present embodiment, each Hash sublist is corresponding to a bitmap and a hash function, wherein importing key assignments is input in the Hash sublist of present embodiment by input interface, calculate the cryptographic hash of input key assignments by the Hash table memory storage, in bitmap and Hash sublist, search the room as index and insert with this cryptographic hash; Wherein bitmap0~9 are that 10 sizes are identical, and storage depth is the state table of m, and Hash sublist 0~9 is that 10 sizes are identical, and storage depth is the storer of m.
Hash table memory storage in the present embodiment is used for finishing whole operations to the input key assignments according to user request, comprises interpolation, searches, monitoring, abnormality processing etc.Please refer to Fig. 3, when Hash key assignments input master control logic module 501, select hash function by master control logic module 501, and utilize this hash function to obtain corresponding cryptographic hash by the Hash calculation logical calculated, add list item logic module 504 and finish the work of adding list item, search logic module 503 and finish specifying the searching work of Hash key assignments.The Hash table memory storage of present embodiment also comprises remove entries logic module and monitoring logic module, does not all draw in the accompanying drawings.Wherein the remove entries logic module is finished the work of remove entries; The monitoring logic module monitors is the space hold situation in the Hash sublist in the process of interpolation and remove entries, and gives master control logic module 501 with this information feedback.
In the present embodiment,, need exception handling logic 510 to handle, at utmost to realize being added in the Hash table of all list items if traveled through all Hash sublists all a certain list item can't be added the abnormality processing situation of Hash table into the time.
See also Figure 10, this figure is the process flow diagram of a specific embodiment of list item adding method of a kind of Hash table of present embodiment, may further comprise the steps:
S1001 utilizes i to open the hash function of Hash sublist correspondence, i.e. CRC16 function calculates the 16bit cryptographic hash of the key assignments of list item to be stored.
The initial value of i is 0 in the present embodiment, and span is 0-9.
S1003 judges table capacity that i opens the Hash sublist whether less than the cryptographic hash of calculating, in this way, and execution in step S1005a then, otherwise execution in step S1005b.
S1005a begins to intercept from the higher bit position to the 16bit cryptographic hash that calculates, and with the intercepting after cryptographic hash as index, execution in step S1007.
S1005b, with the cryptographic hash that calculates as index.
Whether S1007, open to search in the Hash sublist at i according to index value and have vacant position, judge that promptly the conflict position is 0 among the corresponding bitmap, significance bit is 0, if execution in step S1009, otherwise, execution in step S1011.
S1009 puts into the room that finds with pairing all information of list item to be stored, collision position 0 among the bitmap that it is corresponding, active position 1.
So far, this list item calculates, storing process finishes, and begins to store down the bar list item.
S1011, if the conflict position of this bitmap is 0, and significance bit is 1, promptly this address has been stored a list item, then execution in step S1012; If the conflict position of this bitmap is 1, then execution in step S1013.
S1013 takes out the list item of this storage, the item key value of current key assignments and taking-up is deposited among the bitmap, and collision position 1, execution in step S1017.
S1013 deposits current key assignments among the corresponding bitmap in.
S1015, i=i+1 returns step S1001, promptly begins to search the room in next table, till finding the room to store this list item.
See also Figure 11, this figure is a kind of building-block of logic of searching a kind of embodiment of list item in the present embodiment, the Hash key assignments of searching for needs, at first go out cryptographic hash through hash function 0~9 parallel computation, hash function used when each hash function here must be with storage item is identical; Utilize the cryptographic hash that calculates in each self-corresponding bitmap table, to search concurrently respectively then as index value, determine the address of coupling list item, send this address to Hash sublist storer and take out list item, each Hash sublist storer also leaves a port as the CPU visit; At last, with list item that takes out and the key assignments that need search relatively, after determining to mate, the information that this is all return to the user as final checking result, and the process of so far tabling look-up finishes.
Owing to be all 10 bitmap tables to be walked abreast search when searching, so only needing a search cycle and one-period to read Hash sublist storer just can realize all hitting, so just realized guaranteeing under the precondition that no list item is left over, with table space capacity in the smaller sheet and a memory interface, realize Hash lookup fast.
Embodiment three:
See also Fig. 9, this figure is an embodiment building-block of logic of the Hash table in the present embodiment, wherein importing key assignments is input in the Hash sublist of present embodiment by input interface, by the Hash table controller input key assignments is calculated its cryptographic hash, in the Hash sublist, search the room as index and insert with this cryptographic hash; Wherein Hash sublist 0~9 is 10 storeies, represents 10 Hash sublists, and bitmap0~9 are 10 state tables, but preceding 4 storage depth in 10 storeies in the present embodiment is m, and then 6 storage depth is m/2.
The logical organization of Hash table controller in the present embodiment, is searched flow process at the storage of Hash table, and searches among the logical organization of list item and the embodiment two in full accord.
Compare with embodiment two, the space availability ratio of embodiment three is higher.
Above content be in conjunction with concrete embodiment to further describing that the present invention did, can not assert that concrete enforcement of the present invention is confined to these explanations.For the general technical staff of the technical field of the invention, without departing from the inventive concept of the premise, can also make some simple deduction or replace, all should be considered as belonging to protection scope of the present invention.

Claims (21)

1. the list item adding method of a Hash table is characterized in that, comprising:
Hash table is split as many Hash sublists, the corresponding bitmap of each Zhang Haxi sublist, the corresponding hash function of each Zhang Weitu;
Determine current goal Hash sublist, utilize the corresponding hash function of the pairing bitmap of current goal Hash sublist to calculate the cryptographic hash of the key assignments of list item to be stored, determine whether room, the pairing memory location of current goal Hash sublist according to described cryptographic hash, in this way, the information of described list item to be stored is deposited in this room, otherwise, upgrade current goal Hash sublist until finding the room to store the information of described list item to be stored.
2. the method for claim 1, it is characterized in that, described bitmap has comprised conflict position and significance bit, determine whether room, the pairing memory location of current goal Hash sublist according to described cryptographic hash, in this way, the information of described list item to be stored is deposited in this room, otherwise, upgrade current goal Hash sublist until finding the room to store the step of the information of described list item to be stored, comprising:
As index, search the bitmap of described current goal Hash sublist correspondence with described cryptographic hash;
Whether the significance bit of judging described bitmap is effective, and whether the conflict position of described bitmap is invalid;
If described significance bit is effectively, and described conflict position is invalid, and then definite pairing memory location of described current goal Hash sublist is the room, and then the information with described list item to be stored deposits this room in; Otherwise, upgrade current goal Hash sublist until finding the room to store the information of described list item to be stored.
3. method as claimed in claim 2 is characterized in that, described with described cryptographic hash as index, search the step of the bitmap of described current goal Hash sublist correspondence, comprising:
Whether the capacity of judging current goal Hash sublist is less than the figure place of the described cryptographic hash that calculates;
If the capacity of described current goal Hash sublist then intercepts described cryptographic hash less than the figure place of the described cryptographic hash that calculates, and the cryptographic hash after will intercepting is searched the bitmap of described current goal Hash sublist correspondence as index.
4. method as claimed in claim 2 is characterized in that, deposit the information of described list item to be stored in room after, also comprise:
With the active position of described bitmap is invalid, and the key assignments of described list item to be stored is stored in this bitmap.
5. method as claimed in claim 2 is characterized in that, upgrades current goal Hash sublist until finding the room to store the step of the information of described list item to be stored, comprising:
When the conflict position of described bitmap is invalid, significance bit is invalid, and the list item of then having deposited in the Hash sublist with this bitmap correspondence takes out, and be effective with this collision position, and will take out the key assignments of list item and the key assignments of current list item to be stored deposits in this bitmap;
Judge whether current target Hash sublist is last Hash sublist, if not last Hash sublist, then current goal Hash sublist is updated to next Zhang Haxi sublist, stores the information of described list item to be stored until finding the room.
6. method as claimed in claim 2 is characterized in that, upgrades current goal Hash sublist until finding the room to store the step of the information of described list item to be stored, comprising:
When the conflict position of this bitmap is that effectively the key assignments of then current list item to be stored directly is deposited in this bitmap;
Judge that whether current goal Hash sublist is last Hash sublist, if not last Hash sublist, then is updated to next Zhang Haxi sublist with current goal Hash sublist, until finding the room to store the information of described list item to be stored.
7. as any described method in the claim 1 to 6, it is characterized in that, also comprise:
After described list item storage to be stored is finished, upgrade the conflict position information of bitmap and the information of Hash sublist synchronously.
8. as any described method in the claim 1 to 6, it is characterized in that the hash function of described Hash sublist correspondence is the CRC function.
9. as any described method in the claim 1 to 6, it is characterized in that, adopt different CRC polynomial expressions between the hash function of described many Hash sublist correspondences.
10. the list item delet method of a Hash table is characterized in that, Hash table comprises many Hash sublists, the corresponding bitmap of each Zhang Haxi sublist, and the corresponding hash function of each Zhang Weitu, and every described bitmap comprises a plurality of clauses and subclauses, then described method comprises:
The cryptographic hash of the clauses and subclauses that every pairing hash function parallel computation of bitmap need be deleted;
According to the cryptographic hash that calculates, parallel searching in each self-corresponding bitmap at it;
The lookup result of each Zhang Weitu is compared in proper order, when for the first time the conflict position to occur be invalid, the list item taking-up in the Hash sublist of the address correspondence of this bitmap;
The key assignments of the list item that takes out is compared with the key assignments that need search,, then described list item is deleted from described Hash sublist if identical.
11. method as claimed in claim 10 is characterized in that, also comprises after the remove entries;
With the collision position of the bitmap of correspondence is invalid, and active position is effective.
12. method as claimed in claim 10 is characterized in that, also comprises:
All key assignments of being stored in all bitmaps before the bitmap at list item place of deletion are compared;
If the key assignments of being stored in certain bitmap is greater than two, the key assignments that then directly will need to delete deletes; Perhaps,
If the key assignments of being stored in certain bitmap equals two, the key assignments that then will need to delete deletes, and finds the position that another key assignments is stored, and with its taking-up, deposits in the pairing Hash sublist of this bitmap.
13. the lookup method of a Hash table is characterized in that, described Hash table comprises many Hash sublists, every corresponding bitmap of Hash sublist, and every corresponding hash function of bitmap, then described Hash table lookup method comprises:
Utilize the cryptographic hash of key assignments of the hash function parallel computation list item to be found of every Hash sublist correspondence;
According to the described cryptographic hash that calculates parallel searching in the bitmap of correspondence;
The lookup result of each Zhang Weitu is compared in proper order, when for the first time the conflict position to occur be invalid, take out the list item in the Hash sublist of address correspondence of this bitmap;
The key assignments of the list item that takes out is compared with the key assignments of list item to be found, if the key assignments of the list item that takes out and the key assignments of described list item to be found are complementary, then with the information of this list item as final checking result.
14. require 13 described methods as power, it is characterized in that, also comprise:, then return and search failure if the key assignments of the list item that takes out and the key assignments of described list item to be found are not complementary.
15. as claim 13 or 14 described methods, it is characterized in that,, comprising according to the described cryptographic hash that calculates parallel step of searching in the bitmap of correspondence:
Capacity according to the Hash sublist of the described cryptographic hash correspondence that calculates intercepts described cryptographic hash;
Cryptographic hash after the intercepting as index, is walked abreast in its corresponding bitmap and searches.
16. method as claimed in claim 15 is characterized in that, the hash function of described bitmap correspondence during with storage item the corresponding hash function of described Hash sublist identical.
17. Hash table memory storage, comprise the master control logic module, and the operation circuit and the data circuit that link to each other with described master control logic module, wherein, the described operation circuit of master control logic module controls is finished the interpolation and/or the deletion of list item, and the control data circuit is finished list item and is searched, it is characterized in that described operation circuit comprises: list item splits module, target Hash sublist determination module and list item memory module, wherein
Described list item splits module, is used for Hash table is split as many Hash sublists and many bitmaps, and every corresponding bitmap of Hash sublist and a hash function;
Described target Hash sublist determination module splits module with described list item and links to each other, and is used for splitting module from described list item and splits definite Hash sublist of many Hash sublists that obtain as current target Hash sublist; And after receiving the update instruction that described list item memory module sends, be used for current target Hash sublist is updated to its next Zhang Haxi sublist;
Described list item memory module, link to each other with described target Hash sublist determination module, be used to utilize the pairing hash function of described current goal Hash sublist, calculate the cryptographic hash of the key assignments of described list item to be stored, and determine according to described cryptographic hash whether the pairing memory location of current goal Hash sublist is the room, in this way, deposit the information of list item to be stored in this room, otherwise send update instruction to described target Hash sublist determination module, store described list item to be stored until finding the room.
18. device as claimed in claim 17 is characterized in that, described bitmap comprises conflict position and significance bit, and described list item memory module comprises calculating sub module, searches submodule, list item adds submodule, wherein,
Described calculating sub module links to each other with described target Hash sublist determination module, is used to utilize the pairing hash function of described current goal Hash sublist, calculates the cryptographic hash of the key assignments of described list item to be stored;
The described submodule of searching, link to each other with described calculating sub module, whether the figure place that is used to judge the cryptographic hash that described target Hash sublist determination module calculates is more than or equal to the capacity of described current goal Hash sublist, in this way, then intercept described cryptographic hash, and the cryptographic hash after will intercepting is as index, search corresponding bitmap, and whether the significance bit of judging described bitmap is effective, and whether the conflict position of described bitmap be invalid, in this way, judge that then the memory location in the described target Hash sublist is the room, otherwise, then send update instruction, until finding the room to described target Hash sublist determination module;
Described list item adds submodule, links to each other with the described submodule of searching, be used for when the described submodule of searching when described current goal Hash sublist finds the room, the information of described list item to be stored is deposited in the described room.
19. device as claimed in claim 18, it is characterized in that, when described list item adds submodule, after depositing in the information of described list item to be stored in the described room, the active position that described list item interpolation submodule also is used for described bitmap is invalid, and the key assignments of described list item to be stored is stored in the described bitmap.
20. device as claimed in claim 18, it is characterized in that, when described when searching submodule to judge the significance bit of described bitmap being invalid, be used to also judge whether the conflict position of described bitmap is invalid, in this way, then described list item adds submodule and is used for the list item that the corresponding Hash sublist of described bitmap has been stored is taken out, and the key assignments of the list item that takes out and the key assignments of described current list item to be stored are deposited in the described bitmap; When searching submodule to judge the conflict position of described bitmap being effective, the list item of then described list item memory module adds submodule and is used for storing the key assignments of current described list item to be stored into described bitmap when described.
21. device as claimed in claim 17 is characterized in that, described Hash table comprises many Hash sublists, every corresponding bitmap of Hash sublist and a hash function, described data circuit comprises the parallel computation module, parallel module and the comparison module searched, wherein
The parallel computation module is used to utilize the hash function of every Hash sublist correspondence, the cryptographic hash of the key assignments of parallel computation list item to be found;
Parallel search module, link to each other with described parallel computation module, be used for cryptographic hash that basis calculates at it each self-corresponding bitmap search concurrently;
Comparison module, be used for the checking result of each bitmap is compared judgement, determine the address of Hash sublist at the list item place of coupling, the information of the list item of this address storage is taken out, and compare with key assignments that needs are searched, whether the list item that judge to take out is complementary with list item to be found, as coupling, then with the information of this list item as final checking result.
CN201110138340.7A 2011-05-25 2011-05-25 The list item interpolation of Hash table, deletion, lookup method and Hash table storage device Active CN102194002B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201110138340.7A CN102194002B (en) 2011-05-25 The list item interpolation of Hash table, deletion, lookup method and Hash table storage device
PCT/CN2012/072473 WO2012159480A1 (en) 2011-05-25 2012-03-16 Table entry adding, deleting and searching method of hash table and hash table storage device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110138340.7A CN102194002B (en) 2011-05-25 The list item interpolation of Hash table, deletion, lookup method and Hash table storage device

Publications (2)

Publication Number Publication Date
CN102194002A true CN102194002A (en) 2011-09-21
CN102194002B CN102194002B (en) 2016-12-14

Family

ID=

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521228A (en) * 2011-11-01 2012-06-27 浙江省电力试验研究院 Key value mapping method for linear data table
CN102682116A (en) * 2012-05-14 2012-09-19 中兴通讯股份有限公司 Method and device for processing table items based on Hash table
WO2012159480A1 (en) * 2011-05-25 2012-11-29 中兴通讯股份有限公司 Table entry adding, deleting and searching method of hash table and hash table storage device
CN102880724A (en) * 2012-10-23 2013-01-16 盛科网络(苏州)有限公司 Method and system for processing Hash collision
CN102937993A (en) * 2012-11-09 2013-02-20 北京小米科技有限责任公司 Method and device for accessing keywords
WO2013086867A1 (en) * 2011-12-14 2013-06-20 华为技术有限公司 Information processing method and equipment
CN103618733A (en) * 2013-12-06 2014-03-05 北京中创信测科技股份有限公司 Data filtering system and method applied to mobile internet
CN104036054A (en) * 2014-07-08 2014-09-10 携程计算机技术(上海)有限公司 Method and system for searching for destination
CN104378295A (en) * 2013-08-12 2015-02-25 中兴通讯股份有限公司 Table item management device and table item management method
CN104461923A (en) * 2014-10-31 2015-03-25 深圳市锐明视讯技术有限公司 Hard disc physical block management method and system
CN105117489A (en) * 2015-09-21 2015-12-02 北京金山安全软件有限公司 Database management method and device and electronic equipment
CN105516367A (en) * 2016-02-02 2016-04-20 北京百度网讯科技有限公司 Distributed data storage system, method and device
CN105760404A (en) * 2014-12-19 2016-07-13 富士通株式会社 Data integration method and device
WO2016184029A1 (en) * 2015-05-18 2016-11-24 深圳市中兴微电子技术有限公司 Storage and lookup methods and apparatuses supporting hash lookup and routing lookup, and storage medium
WO2017054208A1 (en) * 2015-09-30 2017-04-06 华为技术有限公司 Method and apparatus for determining and indicating air interface resource
CN107315745A (en) * 2016-04-26 2017-11-03 北京京东尚科信息技术有限公司 A kind of personal letter storage method and system
WO2018165963A1 (en) * 2017-03-17 2018-09-20 深圳市秀趣品牌文化传播有限公司 E-commerce data redundancy processing system and method
CN108667836A (en) * 2018-05-08 2018-10-16 众安信息技术服务有限公司 Block chain common recognition method
CN109189792A (en) * 2018-09-12 2019-01-11 天津芯海创科技有限公司 A kind of parallel Hash tables look-up architecture and method
CN109525503A (en) * 2018-10-18 2019-03-26 盛科网络(苏州)有限公司 A kind of compression method and device, computer readable storage medium
CN109558423A (en) * 2018-10-31 2019-04-02 深圳壹账通智能科技有限公司 A kind of data search method based on key-value pair, device and equipment
CN109688237A (en) * 2018-03-26 2019-04-26 新华三技术有限公司 A kind of NAT method, device and NAT device
CN109947762A (en) * 2017-08-16 2019-06-28 深圳市中兴微电子技术有限公司 A kind of Hash table management method and device, computer readable storage medium
CN110543570A (en) * 2019-07-29 2019-12-06 北京邮电大学 Knowledge graph storage method based on Hash addressing
CN111124312A (en) * 2019-12-23 2020-05-08 第四范式(北京)技术有限公司 Data deduplication method and device
CN111858586A (en) * 2020-07-06 2020-10-30 北京天空卫士网络安全技术有限公司 Data processing method and device
CN112597536A (en) * 2020-12-23 2021-04-02 瀚高基础软件股份有限公司 Real-time detection method and device for illegal deletion of database table file
CN113342828A (en) * 2021-07-02 2021-09-03 广东唯审信息科技有限公司 Hash table conflict resolution method based on d-dimensional mapping
CN117349483A (en) * 2023-12-05 2024-01-05 杭州行芯科技有限公司 Parasitic parameter searching method and device, electronic equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7370054B1 (en) * 2003-09-29 2008-05-06 Sun Microsystems, Inc Method and apparatus for indexing a hash table which is organized as a linked list
CN101604337A (en) * 2009-07-13 2009-12-16 中兴通讯股份有限公司 Device and method is stored, searched to a kind of hash table

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7370054B1 (en) * 2003-09-29 2008-05-06 Sun Microsystems, Inc Method and apparatus for indexing a hash table which is organized as a linked list
CN101604337A (en) * 2009-07-13 2009-12-16 中兴通讯股份有限公司 Device and method is stored, searched to a kind of hash table

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
唐铭等: "倒插入分段哈希算法", 《计算机应用》 *

Cited By (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012159480A1 (en) * 2011-05-25 2012-11-29 中兴通讯股份有限公司 Table entry adding, deleting and searching method of hash table and hash table storage device
CN102521228A (en) * 2011-11-01 2012-06-27 浙江省电力试验研究院 Key value mapping method for linear data table
WO2013086867A1 (en) * 2011-12-14 2013-06-20 华为技术有限公司 Information processing method and equipment
CN102682116A (en) * 2012-05-14 2012-09-19 中兴通讯股份有限公司 Method and device for processing table items based on Hash table
CN102682116B (en) * 2012-05-14 2014-06-11 中兴通讯股份有限公司 Method and device for processing table items based on Hash table
CN102880724A (en) * 2012-10-23 2013-01-16 盛科网络(苏州)有限公司 Method and system for processing Hash collision
CN102937993B (en) * 2012-11-09 2016-01-06 小米科技有限责任公司 A kind of method for access key and device
CN102937993A (en) * 2012-11-09 2013-02-20 北京小米科技有限责任公司 Method and device for accessing keywords
CN104378295B (en) * 2013-08-12 2019-03-26 中兴通讯股份有限公司 List item managing device and entry management method
CN104378295A (en) * 2013-08-12 2015-02-25 中兴通讯股份有限公司 Table item management device and table item management method
CN103618733A (en) * 2013-12-06 2014-03-05 北京中创信测科技股份有限公司 Data filtering system and method applied to mobile internet
CN103618733B (en) * 2013-12-06 2017-06-27 北京中创腾锐技术有限公司 A kind of data filtering system and method for being applied to mobile Internet
CN104036054A (en) * 2014-07-08 2014-09-10 携程计算机技术(上海)有限公司 Method and system for searching for destination
CN104461923B (en) * 2014-10-31 2018-07-03 深圳市锐明技术股份有限公司 A kind of hard disc physical block management method and system
CN104461923A (en) * 2014-10-31 2015-03-25 深圳市锐明视讯技术有限公司 Hard disc physical block management method and system
WO2016065695A1 (en) * 2014-10-31 2016-05-06 深圳市锐明视讯技术有限公司 Hard disk physical block management method and system
CN105760404A (en) * 2014-12-19 2016-07-13 富士通株式会社 Data integration method and device
WO2016184029A1 (en) * 2015-05-18 2016-11-24 深圳市中兴微电子技术有限公司 Storage and lookup methods and apparatuses supporting hash lookup and routing lookup, and storage medium
CN105117489B (en) * 2015-09-21 2018-10-19 北京金山安全软件有限公司 Database management method and device and electronic equipment
CN105117489A (en) * 2015-09-21 2015-12-02 北京金山安全软件有限公司 Database management method and device and electronic equipment
WO2017054208A1 (en) * 2015-09-30 2017-04-06 华为技术有限公司 Method and apparatus for determining and indicating air interface resource
US10736115B2 (en) 2015-09-30 2020-08-04 Huawei Technologies Co., Ltd. Air interface resource determining method and apparatus and air interface resource indication method and apparatus
CN105516367B (en) * 2016-02-02 2018-02-13 北京百度网讯科技有限公司 Distributed data-storage system, method and apparatus
CN105516367A (en) * 2016-02-02 2016-04-20 北京百度网讯科技有限公司 Distributed data storage system, method and device
CN107315745A (en) * 2016-04-26 2017-11-03 北京京东尚科信息技术有限公司 A kind of personal letter storage method and system
WO2018165963A1 (en) * 2017-03-17 2018-09-20 深圳市秀趣品牌文化传播有限公司 E-commerce data redundancy processing system and method
CN109947762A (en) * 2017-08-16 2019-06-28 深圳市中兴微电子技术有限公司 A kind of Hash table management method and device, computer readable storage medium
US11201852B2 (en) 2018-03-26 2021-12-14 New H3C Technologies Co., Ltd. Network address translation
CN109688237A (en) * 2018-03-26 2019-04-26 新华三技术有限公司 A kind of NAT method, device and NAT device
CN109688237B (en) * 2018-03-26 2020-05-12 新华三技术有限公司 NAT (network Address translation) conversion method and device and NAT equipment
CN108667836A (en) * 2018-05-08 2018-10-16 众安信息技术服务有限公司 Block chain common recognition method
CN109189792A (en) * 2018-09-12 2019-01-11 天津芯海创科技有限公司 A kind of parallel Hash tables look-up architecture and method
CN109525503A (en) * 2018-10-18 2019-03-26 盛科网络(苏州)有限公司 A kind of compression method and device, computer readable storage medium
WO2020078012A1 (en) * 2018-10-18 2020-04-23 盛科网络(苏州)有限公司 Compression method and device and computer-readable storage medium
CN109525503B (en) * 2018-10-18 2021-12-14 苏州盛科通信股份有限公司 Compression method and device and computer readable storage medium
CN109558423A (en) * 2018-10-31 2019-04-02 深圳壹账通智能科技有限公司 A kind of data search method based on key-value pair, device and equipment
CN110543570B (en) * 2019-07-29 2022-03-11 北京邮电大学 Knowledge graph storage method based on Hash addressing
CN110543570A (en) * 2019-07-29 2019-12-06 北京邮电大学 Knowledge graph storage method based on Hash addressing
CN111124312A (en) * 2019-12-23 2020-05-08 第四范式(北京)技术有限公司 Data deduplication method and device
CN111124312B (en) * 2019-12-23 2023-10-31 第四范式(北京)技术有限公司 Method and device for data deduplication
CN111858586A (en) * 2020-07-06 2020-10-30 北京天空卫士网络安全技术有限公司 Data processing method and device
CN111858586B (en) * 2020-07-06 2024-04-09 北京天空卫士网络安全技术有限公司 Data processing method and device
CN112597536A (en) * 2020-12-23 2021-04-02 瀚高基础软件股份有限公司 Real-time detection method and device for illegal deletion of database table file
CN113342828A (en) * 2021-07-02 2021-09-03 广东唯审信息科技有限公司 Hash table conflict resolution method based on d-dimensional mapping
CN117349483A (en) * 2023-12-05 2024-01-05 杭州行芯科技有限公司 Parasitic parameter searching method and device, electronic equipment and storage medium
CN117349483B (en) * 2023-12-05 2024-04-09 杭州行芯科技有限公司 Parasitic parameter searching method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
WO2012159480A1 (en) 2012-11-29

Similar Documents

Publication Publication Date Title
CN101604337B (en) Apparatus and method for hash table storage, searching
US10990479B2 (en) Efficient packing of compressed data in storage system implementing data striping
US9959062B1 (en) Low latency and reduced overhead data storage system and method for sharing multiple storage devices by high performance computing architectures
CN106066896B (en) Application-aware big data deduplication storage system and method
Jindal et al. Trojan data layouts: right shoes for a running elephant
CN106990915B (en) Storage resource management method based on storage medium type and weighted quota
US20160132541A1 (en) Efficient implementations for mapreduce systems
US20100161564A1 (en) Cluster data management system and method for data recovery using parallel processing in cluster data management system
CN103544261B (en) A kind of magnanimity structuring daily record data global index's management method and device
US20070016555A1 (en) Indexing method of database management system
CN105117355A (en) Memory, memory system and data process method
CN104765575A (en) Information storage processing method
CN103942343A (en) Data storage optimization method for hash joint
CN106570113B (en) Mass vector slice data cloud storage method and system
CN104077423A (en) Consistent hash based structural data storage, inquiry and migration method
CN112799841B (en) Method and device for data object storage management
CN104765574A (en) Data cloud storage method
Ibrahim et al. Intelligent data placement mechanism for replicas distribution in cloud storage systems
CN102662869A (en) Method and device for access to memory of virtual machine and finders
CN107729536A (en) A kind of date storage method and device
CN111176584A (en) Data processing method and device based on hybrid memory
CN107153512A (en) A kind of data migration method and device
CN106547484A (en) It is a kind of that internal storage data reliability method and system realized based on RAID5
CN107948229A (en) The method, apparatus and system of distributed storage
CN104239520A (en) Historical-information-based HDFS (hadoop distributed file system) data block placement strategy

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C41 Transfer of patent application or patent right or utility model
TA01 Transfer of patent application right

Effective date of registration: 20151104

Address after: 518057 Nanshan District Guangdong high tech Industrial Park, South Road, science and technology, ZTE building, Ministry of Justice

Applicant after: ZTE Corp.

Applicant after: SANECHIPS TECHNOLOGY Co.,Ltd.

Address before: 518057 Nanshan District science and Technology Park, Guangdong, South Road, ZTE building, science and Technology Park

Applicant before: ZTE Corp.

C14 Grant of patent or utility model
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20221130

Address after: 518055 Zhongxing Industrial Park, Liuxian Avenue, Xili street, Nanshan District, Shenzhen City, Guangdong Province

Patentee after: SANECHIPS TECHNOLOGY Co.,Ltd.

Address before: 518057 Ministry of justice, Zhongxing building, South Science and technology road, Nanshan District hi tech Industrial Park, Shenzhen, Guangdong

Patentee before: ZTE Corp.

Patentee before: SANECHIPS TECHNOLOGY Co.,Ltd.