CN113434092A - Fingerprint identification method based on hybrid DRAM-NVM - Google Patents
Fingerprint identification method based on hybrid DRAM-NVM Download PDFInfo
- Publication number
- CN113434092A CN113434092A CN202110768156.4A CN202110768156A CN113434092A CN 113434092 A CN113434092 A CN 113434092A CN 202110768156 A CN202110768156 A CN 202110768156A CN 113434092 A CN113434092 A CN 113434092A
- Authority
- CN
- China
- Prior art keywords
- fingerprint
- key
- nvm
- hash
- bucket
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a fingerprint identification method based on a hybrid DRAM-NVM (dynamic random Access memory-non-volatile memory), aiming at solving the problem of high access delay of the key value caused by frequently detecting a hash bucket key value in the existing hash table on the premise of satisfying the condition that data can be recovered when a system crashes. The technical scheme is that an NVM table and a fingerprint table are respectively constructed, the NVM table is stored in the NVM, the fingerprint table is stored in the DRAM, and the two table structures are in one-to-one correspondence; and then determining the address pp of the key value pair to be inquired in the fingerprint bucket, matching the fingerprint of the key x with the fingerprint elements in the fingerprint bucket with the address pp, and obtaining a matching result or 'inquiry failure' information by utilizing the one-to-one correspondence characteristic of the NVM table and the fingerprint table structure. The method solves the problem of high access delay caused by frequently detecting the key value of the hash bucket key value in the hash table query process, filters out a large number of NVM (non-volatile memory) reads, and greatly reduces the access delay.
Description
Technical Field
The invention belongs to the field of computer data storage, and particularly relates to a fingerprint identification method based on a hybrid DRAM-NVM (dynamic random Access memory-non-volatile memory).
Background
In recent years, several new Non-Volatile Memory (NVM) technologies have come into the spotlight, such as Phase Change Memory (PCM), memristor (ReRAM), spin torque magnetic Memory (STT-MRAM), and 3D-XPoint. Compared with the traditional DRAM memory technology, the NVM has the characteristics of power-down non-volatility, high storage density and low power consumption of a magnetic disk, and also has the characteristics of byte-based modification addressing and low read-write delay of the DRAM. Therefore, the NVM technology is used for making up the defects of insufficient memory capacity and high power consumption of the DRAM memory system. However, the current NVM technology still has the disadvantages of asymmetric read/write delay and limited wear times, so the NVM is more suitable for being connected to a memory bus of a computer system together with a DRAM to form a hybrid memory.
The key value pair is the simplest organization form of the database, wherein the key is the index number of the value, the value is the data to be stored in the database, and when a user inquires the data, the corresponding value can be inquired and obtained in the database only by providing the key to be inquired. For example in a telephone number database, the key: pony, value: 18753137777, < horse, 18753137777> is a key-value pair, and the user can obtain his phone number 18753137777 by the key "horse" to be queried. Key-value pairs are currently widely used in various databases, such as Redis, voldemert, Oracle BDB. The hash index is a data structure of a common query key value pair in a database, and is widely applied to various databases and file systems, and currently, with the rise of big data and cloud computing, the requirement for query delay of the memory hash index is higher and higher, which also prompts researchers to research the hash index structure of a novel nonvolatile memory.
Many studies have been made on hash indexing based on non-volatile memory, however most of them are based on a system with only NVM as memory, which does not make good use of the advantages of the hybrid DRAM-NVM memory system. The existing hash index scheme usually uses hash buckets to solve hash collision, and a search thread needs to scan corresponding hash buckets (an array-type data structure for storing key-value pair elements, which is a main component of a hash table) and match key values therein, and find out whether key-value pairs exist by scanning one or more buckets, which brings higher NVM read delay, especially for longer key values. This is the main NVM read overhead for the scanning process, and there is currently no non-volatile memory hash index scheme that addresses this issue.
Fingerprint technology proposed in recent years can reduce unnecessary scanning. It is an important data structure that can reduce the read overhead of the scanning process. A fingerprint is a one-byte hash value that is used to predict whether a key value key of a storage element may exist. The least significant byte of the hash value storing the element key is typically used as the fingerprint. When the storage element is scanned to be existed or not, the fingerprint of the key value key to be searched is calculated firstly, and then the search thread checks whether any fingerprint in the hash bucket is matched with the fingerprint of the search key value key or not. If the fingerprints are successfully matched, the search thread only accesses the key values corresponding to the fingerprints successfully matched, and skips all other key values which are failed in matching. And if the matching of the fingerprints in the hash bucket is not successful, the key value to be searched is absolutely not in the hash bucket.
Baitonghu et al propose Dash (a paper published in international conference VLDB2020), a dynamic and extensible NVM hash index structure that stores hash buckets in NVM. Fig. 1 is a hash table structure diagram of Dash, which uses a bucket array structure to construct a hash table, where the hash table includes N hash buckets, N is a positive integer, each hash bucket includes 8 hash units, each hash unit has a size of 33 bytes, one hash unit stores a key value pair < key, value > (32 bytes) and a fingerprint (1 byte) corresponding to the key value pair, key represents a key of a storage element, value represents a value of the storage element, and the fingerprint is a one-byte hash value obtained by key calculation. During the query process, the query program can determine whether a key-value pair exists in the hash-bucket by scanning a fingerprint that is much smaller than the actual key-value pair, avoiding unnecessary NVM reads by using fingerprinting for the key matching process of the hash-table. Although the fingerprint identification method can avoid scanning a long key value pair in the NVM so as to reduce the query access delay, because the fingerprint structure is stored in the NVM, a large amount of access to the NVM is still required to read the fingerprint in the query process, and the NVM read-write delay is 2-3 times that of the DRAM, which hinders the promotion of the query access delay.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a fingerprint identification method based on a hybrid DRAM-NVM memory, which solves the technical problem of high access delay caused by frequently detecting key values of hash bucket key values in the query process of the existing hash table on the premise of satisfying data recoverability during system crash.
In order to achieve the above object, the present invention provides a fingerprint identification method based on a hybrid DRAM-NVM memory, comprising the following steps:
the first step, construct and initialize the NVM table, store the NVM table in the NVM, the method is:
1.1, an NVM table is constructed by adopting a barrel array structure, the NVM table comprises N hash buckets, N is a positive integer, each hash bucket comprises 8 hash units, the size of each hash unit is 32 bytes, one hash unit stores a key value pair < key, value >, the key represents a key of a storage element, and the value represents a value of the storage element. Through the bucket address translation function, the key of the key value pair is used for calculating to obtain a hash bucket address (order is p) as the hash bucket address for indexing the NVM table, the hash bucket of the NVM table can be accurately positioned through the hash bucket address, and the calculation formula of the bucket address translation function is as follows:
p＝hash(key)％N
wherein p is a hash bucket address, hash () is a standard hash function in a c + + standard template library, N is the total number of hash buckets in the NVM table, and "%" is a modulo operation;
1.2M key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserting into NVM table, recording M key value pairs and hash unit index of hash bucket1，…，indexm，…，indexM(ii) a Wherein will be<keym,valuem>The method for inserting into the NVM table is as follows:
calculated by barrel address translation function<keym,valuem>Corresponding hash bucket address Pm，Pm＝hash(keym) % N, and<keym,valuem>the address sequentially inserted into the NVM table is PmRecords the hash unit index of the key-value pair inserted into the hash bucketm。
1.3 storing the NVM table in the NVM.
Secondly, constructing and initializing a fingerprint table, and storing the fingerprint table in a DRAM (dynamic random access memory), wherein the method comprises the following steps:
2.1 fingerprint extraction is carried out on the keys of the M key value pairs in the NVM table by utilizing a fingerprint extraction function, and M fingerprints f corresponding to the M key value pairs stored in the NVM table are obtained1，…，fm，…，fMWherein the mth key value pair in the NVM table is obtained<keym,valuem>F of the fingerprintmThe method comprises the following steps: key is calculated by taking hash function hash () of c + + standard template library as fingerprint extraction functionmThe lower eight bits (i.e. 1 byte) of the hash value is used as the fingerprint fm。
2.2 adopt bucket formula array structure to construct the fingerprint table, fingerprint table structure and NVM table one-to-one, the fingerprint table also contains N fingerprint buckets, and every bucket contains 8 fingerprint units, and the size of every fingerprint unit is 1 byte. And a fingerprint unit stores corresponding fingerprints of a key value pair in the NVM table, a fingerprint bucket address p is obtained by using the bucket address conversion function in the first step and key calculation of the key value pair and is used as a fingerprint bucket address for indexing the fingerprint table, and the fingerprint bucket of the fingerprint table can be accurately positioned through the fingerprint bucket address. For example: by bucket address translation function Pm＝hash(keym) Calculated as% N<keym,valuem>Corresponding fingerprint bucket address P in fingerprint tablemThrough PmFingerprint bucket of accurate location fingerprint table can find f in this fingerprint bucketm。
2.3M fingerprints f corresponding to M key value pairs in the NVM table1，…，fm，…，fMInserting into fingerprint table, inserting NVMThe mth key-value in the table corresponds to the fingerprint fmThe method of insertion into the fingerprint table is: finger print fmInsert to address PmIn the fingerprint bucket of the fingerprint table of (f)mFingerprint unit label index stored in the fingerprint bucketmFinger unit of (index)mObtained in the first step, which represents a one-to-one correspondence of the fingerprint table to the NVM table).
2.4 storing the fingerprint table in the DRAM;
thirdly, determining the address of the key value pair to be inquired in the fingerprint bucket, wherein the method comprises the following steps:
receiving the key of the key value pair to be inquired input by the user from the keyboard, wherein the key is a keyXCalculating a fingerprint bucket address by using keyx by adopting a bucket address conversion function, and making the fingerprint bucket address be pp;
fourthly, matching the fingerprint of the keyx with 8 fingerprint elements in the fingerprint bucket with the address of pp, wherein the method comprises the following steps:
4.1, fingerprint extraction is carried out on the keyx by utilizing a fingerprint conversion function, and a fingerprint corresponding to the keyx is obtained, wherein the order is ff;
4.2 let the variable of fingerprint unit i equal to 1;
4.3 matching the ith fingerprint unit of the fingerprint bucket with the address of pp in the fingerprint table by using ff, if the matching is successful, finding the fingerprint matched with the fingerprint ff in the ith fingerprint unit of the fingerprint bucket with the address of pp, and enabling the index of the matched fingerprint unit to be labeledpIf i is less than or equal to 8, rotating to 4.3 to continue matching; if i>8, the fingerprint bucket with the whole address of pp traversed is not matched successfully, which indicates that ff is not in the fingerprint table, and the key value pair corresponding to the key value keyx to be inquired is also in one-to-one correspondence with the NVM table<keyx,valuex>Also not in NVM tables, description<keyx,valuex>If the user is not in the NVM table, the query is failed, and the query failure information is sent to the user, turning to the seventh step;
the fifth step, key-value pairs corresponding to the fingerprints successfully matched in step 4.3 are mapped (since the fingerprint tables correspond to the NVM tables one-to-one, through the fourth step, the key-value pairs are located<keyx,valuex>Hash bucket with address pp in NVM table, according to fingerprint unit indexpThe key corresponding to the fingerprint successfully matched in the fingerprint table is judged to be the target element (the fingerprint only contains part of information of the key value pair, the successful fingerprint matching indicates that the key value pair is possible to be the target element, the key value pair is determined to be the key value pair to be inquired only by finally matching the key value, the key element is determined to be the target element if the key matching is successful), if the matching is successful, the sixth step is executed, if the matching is failed, i is equal to i +1, if i is equal to or less than 8, the step 4.3 is carried out to match the rest fingerprint elements in the fingerprint bucket with the address of pp; if i>8, indicating that the key value pair corresponding to the fingerprint successfully matched in the fingerprint table is not the target element, sending the 'query failure' information to the user, and turning to the seventh step;
and sixthly, obtaining a value in the key value pair corresponding to the keyx value, and sending the value to the inquiry user.
And step seven, finishing.
The invention can achieve the following technical effects:
1. the invention provides a fingerprint identification method based on a hybrid DRAM-NVM (dynamic random Access memory-non-volatile memory), which is characterized in that a fingerprint table capable of reducing NVM reading delay is stored on the DRAM, and the more excellent reading and writing performance of the DRAM is utilized to reduce Hash table access delay; the key value pair is stored in the NVM in a persistent mode, so that the function of quick recovery after system crash is realized; the method fully utilizes the advantages of high read-write performance of the DRAM in the hybrid memory and the durability of the NVM, and solves the technical problem of high access delay caused by frequently detecting the key value of the hash bucket in the query process of the conventional hash table on the premise of meeting the requirement of quick recovery of a system crash course.
2. The invention provides a fingerprint identification method based on a hybrid DRAM-NVM memory, which inserts the fingerprint of a key value pair into a fingerprint table in the DRAM, so that a fingerprint table is constructed in the DRAM with lower reading delay, when related elements are inquired, an inquiry thread does not need to complete a key value matching process by scanning a hash table in the NVM, and instead, linear detection is carried out on target elements by a fingerprint identification directory in the DRAM, a large amount of NVM reading is filtered, and the access delay is greatly reduced.
Description of the drawings:
FIG. 1 is a schematic diagram of a hash table structure of a Dash fingerprint identification method;
FIG. 2 is a schematic diagram of a NVM table structure constructed in the first step of the present invention;
FIG. 3 is a diagram illustrating a fingerprint table constructed in the second step of the present invention;
FIG. 4 is an overall flow chart of the present invention;
fig. 5 is a comparison diagram of the delay of Dash inquiry operation in the fingerprint identification method of the present invention and the background art.
Detailed Description
In order to further explain the technical scheme of the invention, the following description refers to the accompanying drawings. The invention, as shown in fig. 4, includes the following steps:
the first step, construct and initialize the NVM table, store the NVM table in the NVM, the method is:
1.1, an NVM table is constructed by using a barrel array structure, and as shown in fig. 2, the NVM table includes N hash buckets, where N is a positive integer, each hash bucket includes 8 hash units, the size of each hash unit is 32 bytes, one hash unit stores a key value pair < key, value >, where key represents a key of a storage element, and value represents a value of the storage element. Through the bucket address translation function, the key of the key value pair is used for calculating to obtain a hash bucket address (order is p) as the hash bucket address for indexing the NVM table, the hash bucket of the NVM table can be accurately positioned through the hash bucket address, and the calculation formula of the bucket address translation function is as follows:
p＝hash(key)％N
wherein p is a hash bucket address, hash () is a standard hash function in a c + + standard template library, N is the total number of hash buckets in the NVM table, and "%" is a modulo operation;
1.2M key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserting into NVM table, recording M key value pairs and hash unit index of hash bucket1，…，indexm，…，indexM(ii) a Wherein will be<keym,valuem>The method for inserting into the NVM table is as follows:
calculated by barrel address translation function<keym,valuem>Corresponding hash bucket address Pm，Pm＝hash(keym) % N, and<keym,valuem>the address sequentially inserted into the NVM table is PmRecords the hash unit index of the key-value pair inserted into the hash bucketm。
1.3 storing the NVM table in the NVM.
Secondly, constructing and initializing a fingerprint table, and storing the fingerprint table in a DRAM (dynamic random access memory), wherein the method comprises the following steps:
2.1 fingerprint extraction is carried out on the keys of the M key value pairs in the NVM table by utilizing a fingerprint extraction function, and M fingerprints f corresponding to the M key value pairs stored in the NVM table are obtained1，…，fm，…，fMWherein the mth key value pair in the NVM table is obtained<keym,valuem>F of the fingerprintmThe method comprises the following steps: key is calculated by taking hash function hash () of c + + standard template library as fingerprint extraction functionmThe lower eight bits (i.e. 1 byte) of the hash value is used as the fingerprint fm。
2.2 adopt bucket formula array structure to construct the fingerprint table, the fingerprint table structure is as shown in fig. 3, with NVM table one-to-one, the fingerprint table also contains N fingerprint buckets, every bucket contains 8 fingerprint units, the size of every fingerprint unit is 1 byte. And a fingerprint unit stores corresponding fingerprints of a key value pair in the NVM table, a fingerprint bucket address p is obtained by using the bucket address conversion function in the first step and key calculation of the key value pair and is used as a fingerprint bucket address for indexing the fingerprint table, and the fingerprint bucket of the fingerprint table can be accurately positioned through the fingerprint bucket address. For example: by bucket address translation function Pm＝hash(keym) Calculated as% N<keym,valuem>Corresponding fingerprint bucket address P in fingerprint tablemThrough PmFingerprint bucket of accurate location fingerprint table can find f in this fingerprint bucketm。
2.3M fingerprints f corresponding to M key value pairs in the NVM table1，…，fm，…，fMInserting the key value m in the NVM table into a fingerprint table, and corresponding to the key value m in the NVM table to obtain a fingerprint fmThe method of insertion into the fingerprint table is: finger print fmInsert to address PmIn the fingerprint bucket of the fingerprint table of (f)mFingerprint unit label index stored in the fingerprint bucketmFinger unit of (index)mObtained in the first step, which represents a one-to-one correspondence of the fingerprint table to the NVM table).
2.4 storing the fingerprint table in the DRAM;
thirdly, determining the address of the key value pair to be inquired in the fingerprint bucket, wherein the method comprises the following steps:
receiving the key of the key value pair to be inquired input by the user from the keyboard, wherein the key is a keyXCalculating a fingerprint bucket address by using keyx by adopting a bucket address conversion function, and making the fingerprint bucket address be pp;
fourthly, matching the fingerprint of the keyx with 8 fingerprint elements in the fingerprint bucket with the address of pp, wherein the method comprises the following steps:
4.1, fingerprint extraction is carried out on the keyx by utilizing a fingerprint conversion function, and a fingerprint corresponding to the keyx is obtained, wherein the order is ff;
4.2 let the variable of fingerprint unit i equal to 1;
4.3 matching the ith fingerprint unit of the fingerprint bucket with the address of pp in the fingerprint table by using ff, if the matching is successful, finding the fingerprint matched with the fingerprint ff in the ith fingerprint unit of the fingerprint bucket with the address of pp, and enabling the index of the matched fingerprint unit to be labeledpIf i is less than or equal to 8, rotating to 4.3 to continue matching; if i>8, the fingerprint bucket with the whole address of pp traversed is not matched successfully, which indicates that ff is not in the fingerprint table, and the key value pair corresponding to the key value keyx to be inquired is also in one-to-one correspondence with the NVM table<keyx,valuex>Also not in NVM tables, description<keyx,valuex>If the user is not in the NVM table, the query is failed, and the query failure information is sent to the user, turning to the seventh step;
fifthly, matching keyx with keys of key value pairs corresponding to the fingerprints successfully matched in the step 4.3, judging whether the key value pairs corresponding to the fingerprints successfully matched in the fingerprint table are target elements, if the key value pairs are successfully matched, executing the sixth step, if the key value pairs are not matched, making i equal to i +1, and if i is not greater than 8, turning to the step 4.3 to match the rest fingerprint elements in the fingerprint bucket with the address pp; if i is greater than 8, the key value pair corresponding to the fingerprint successfully matched in the fingerprint table is not the target element, the 'query failure' information is sent to the user, and the seventh step is carried out;
and sixthly, obtaining a value in the key value pair corresponding to the keyx value, and sending the value to the inquiry user.
And step seven, finishing.
Fig. 5 is a comparison diagram of the delay of Dash inquiry operation in the fingerprint identification method of the present invention and the background art. The query operation latency test was performed on a 32-core Langchao server with 32KB/1024KB/32MB L1/L2/L3 cache. The server is configured with 192GB DRAM (6 × 32GB/DIMM) and 1.5TB NVM (6 × 256GB/DIMM), and the NVM memory is configured in App Direct mode and mounted on the ext4-DAX file system. Software environments ubuntu Linux and PMDK 1.7 are used in the test, and all realized codes are optimized and compiled by adopting GCC 10.2.0. The ordinate of the comparison graph is the query average access delay (1000 query operations are repeated, and the average delay is taken), the vertical line graph represents the query average delay of Dash, the horizontal line graph represents the query average delay of the invention, the query average delay of Dash is 0.535us, the query average access delay of the invention is only 0.29us, and compared with Dash, the query average delay of the invention is improved by 84.5% -0.29/0.29%.
The fingerprint identification method based on the hybrid DRAM-NVM provided by the invention is described in detail above. The principles and embodiments of the present invention are explained herein, with the above description being included to assist in understanding the core concepts of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
It will be understood by those skilled in the art that the foregoing is merely a preferred embodiment of this invention, and is not intended to limit the invention, such that any modification, equivalent replacement, or improvement made within the spirit and scope of the invention should be construed broadly.
Claims (4)
1. A fingerprint identification method based on a hybrid DRAM-NVM adopts a barrel array structure to construct a hash table, and is characterized by comprising the following steps:
the first step, construct and initialize the NVM table, store the NVM table in the NVM, the method is:
1.1, constructing an NVM (non volatile memory) table by adopting a barrel array structure, wherein the NVM table comprises N hash buckets, N is a positive integer, each hash bucket comprises 8 hash units, the size of each hash unit is 32 bytes, one hash unit stores a key value pair < key, value >, the key represents a key of a storage element, and the value represents a value of the storage element; calculating by using key of key value pair to obtain hash bucket address p as hash bucket address of index NVM table through bucket address conversion function, where the calculation formula of the bucket address conversion function is:
p＝hash(key)％N
wherein, the hash () is a standard hash function in a c + + standard template library, and the "%" is a modulus operation;
1.2M key value pairs that the user needs to store<key1，value1>，…，<keym，valuem>，…，<keyM，valueM>Inserting into NVM table, recording M key value pairs and hash unit index of hash bucket1，...，indexm，...，indexM；
1.3 storing the NVM table in the NVM;
secondly, constructing and initializing a fingerprint table, and storing the fingerprint table in a DRAM (dynamic random access memory), wherein the method comprises the following steps:
2.1 fingerprint extraction is carried out on the keys of the M key value pairs in the NVM table by utilizing a fingerprint extraction function, and M fingerprints f corresponding to the M key value pairs stored in the NVM table are obtained1，...，fm，...，fM；
2.2, a fingerprint table is constructed by adopting a barrel array structure, the fingerprint table structure corresponds to the NVM table one by one, the fingerprint table also comprises N fingerprint barrels, each barrel comprises 8 fingerprint units, and the size of each fingerprint unit is 1 byte; a fingerprint unit stores corresponding fingerprints of a key value pair in the NVM table, and a fingerprint bucket address p is obtained by using the key of the key value pair to calculate by using the bucket address conversion function in the first step and is used as the fingerprint bucket address of the index fingerprint table;
2.3M fingerprints f corresponding to M key value pairs in the NVM table1，...，fm，...，fMInserting the fingerprint into a fingerprint table;
2.4 storing the fingerprint table in the DRAM;
thirdly, determining the address of the key value pair to be inquired in the fingerprint bucket, wherein the method comprises the following steps:
receiving the key of the key value pair to be inquired input by the user from the keyboard, wherein the key is a keyXCalculating a fingerprint bucket address by using keyx by adopting a bucket address conversion function, and making the fingerprint bucket address be pp;
fourthly, matching the fingerprint of the keyx with 8 fingerprint elements in the fingerprint bucket with the address of pp, wherein the method comprises the following steps:
4.1, fingerprint extraction is carried out on the keyx by utilizing a fingerprint conversion function, and a fingerprint corresponding to the keyx is obtained, wherein the order is ff;
4.2 let the variable of fingerprint unit i equal to 1;
4.3 matching the ith fingerprint unit of the fingerprint bucket with the address of pp in the fingerprint table by using ff, if the matching is successful, finding the fingerprint matched with the fingerprint ff in the ith fingerprint unit of the fingerprint bucket with the address of pp, and enabling the index of the matched fingerprint unit to be labeledpTurning to the fifth step if the number is i; otherwise, if i is equal to i +1, if i is less than or equal to 8, rotating to 4.3; if i is more than 8, the fingerprint buckets traversing the whole address pp are not matched successfully, ff is not in the fingerprint table, and ff is also shown<keyx，valuex>If the user is not in the NVM table, the query is failed, and the query failure information is sent to the user, turning to the seventh step;
fifthly, matching keyx with keys of key value pairs corresponding to the fingerprints successfully matched in the step 4.3, judging whether the key value pairs corresponding to the fingerprints successfully matched in the fingerprint table are target elements, if the matching is successful, executing the sixth step, if the matching is failed, making i equal to i +1, and if i is less than or equal to 8, turning to the step 4.3; if i is more than 8, the key value pair corresponding to the fingerprint successfully matched in the fingerprint table is not the target element, the 'query failure' information is sent to the user, and the seventh step is carried out;
sixthly, obtaining a value in a key value pair corresponding to the keyx value, and sending the value to a query user;
and step seven, finishing.
2. The method according to claim 1, wherein the step 1.2 is to perform the fingerprint identification based on the hybrid DRAM-NVM memory<keym，valuem>The method for inserting into the NVM table is as follows: calculated by barrel address translation function<keym，valuem>Corresponding hash bucket address Pm，Pm＝hash(keym) % N, will<keym，valuem>The address sequentially inserted into the NVM table is PmRecords the hash unit index of the key-value pair inserted into the hash bucketm。
3. The method according to claim 1, wherein the step 2.1 of obtaining the mth key value pair in the NVM table<keym，valuem>F of the fingerprintmThe method comprises the following steps: key is calculated by taking hash function hash () of c + + standard template library as fingerprint extraction functionmThe lower eight bits, i.e. 1 byte, of the hash value is used as the fingerprint fm。
4. The method according to claim 1, wherein step 2.3 is performed by associating the m-th key value in the NVM table with the corresponding fingerprint fmThe method for inserting into the fingerprint table is as follows: finger print fmInsert to address PmIn the fingerprint bucket of the fingerprint table of (f)mFingerprint unit label index stored in the fingerprint bucketmThe fingerprint cell.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110768156.4A CN113434092B (en) | 2021-07-07 | 2021-07-07 | Fingerprint identification method based on hybrid DRAM-NVM |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110768156.4A CN113434092B (en) | 2021-07-07 | 2021-07-07 | Fingerprint identification method based on hybrid DRAM-NVM |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113434092A true CN113434092A (en) | 2021-09-24 |
| CN113434092B CN113434092B (en) | 2022-04-01 |
Family
ID=77759526
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110768156.4A Active CN113434092B (en) | 2021-07-07 | 2021-07-07 | Fingerprint identification method based on hybrid DRAM-NVM |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113434092B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105786722A (en) * | 2014-12-25 | 2016-07-20 | 研祥智能科技股份有限公司 | NVM erasing and writing control method and system based on heterogeneous hybrid memory |
| CN107728937A (en) * | 2017-09-15 | 2018-02-23 | 上海交通大学 | A kind of key-value pair persistence methods and system using Nonvolatile memory medium |
| CN107862064A (en) * | 2017-11-16 | 2018-03-30 | 北京航空航天大学 | One high-performance based on NVM, expansible lightweight file system |
| CN111459846A (en) * | 2020-03-12 | 2020-07-28 | 华中科技大学 | Dynamic hash table operation method based on hybrid DRAM-NVM |
| KR20200092710A (en) * | 2019-01-25 | 2020-08-04 | 주식회사 리얼타임테크 | Hybride index appratus in database management system based heterogeneous storage |
-
2021
- 2021-07-07 CN CN202110768156.4A patent/CN113434092B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105786722A (en) * | 2014-12-25 | 2016-07-20 | 研祥智能科技股份有限公司 | NVM erasing and writing control method and system based on heterogeneous hybrid memory |
| CN107728937A (en) * | 2017-09-15 | 2018-02-23 | 上海交通大学 | A kind of key-value pair persistence methods and system using Nonvolatile memory medium |
| CN107862064A (en) * | 2017-11-16 | 2018-03-30 | 北京航空航天大学 | One high-performance based on NVM, expansible lightweight file system |
| KR20200092710A (en) * | 2019-01-25 | 2020-08-04 | 주식회사 리얼타임테크 | Hybride index appratus in database management system based heterogeneous storage |
| CN111459846A (en) * | 2020-03-12 | 2020-07-28 | 华中科技大学 | Dynamic hash table operation method based on hybrid DRAM-NVM |
Non-Patent Citations (3)
| Title |
|---|
| BAOTONG LU等: "Dash: Scalable Hashing on Persistent Memory", 《VLDB》 * |
| ISMAIL OUKID等: "FPTree: A Hybrid SCM-DRAM Persistent and Concurrent", 《ACM》 * |
| LING ZHAN 等: "RangeKV: An Efficient Key-Value Store Based on", 《IEEE ACCESS》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113434092B (en) | 2022-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11789860B2 (en) | Logical to physical mapping management using low-latency non-volatile memory | |
| US9990276B2 (en) | Read-write control method for memory, and corresponding memory and server | |
| US8341336B2 (en) | Region-based management method of non-volatile memory | |
| CN107153707B (en) | Hash table construction method and system for nonvolatile memory | |
| US20110202709A1 (en) | Optimizing storage of common patterns in flash memory | |
| US8661190B2 (en) | Flash memory device and data access method thereof | |
| US10977189B2 (en) | Reducing forward mapping table size using hashing | |
| CN110442529B (en) | Configurable memory system and method of configuring and using same | |
| TW201308078A (en) | Unaligned data coalescing | |
| WO2015096698A1 (en) | Data writing and reading methods for flash | |
| US20100011156A1 (en) | Memory device and management method of memory device | |
| CN101901169B (en) | Scanner and method | |
| CN107221351B (en) | Optimization processing method of error correcting code in solid-state disk system and application thereof | |
| CN113419675B (en) | Write operation method and read operation method for memory | |
| US11341036B2 (en) | Biased sampling methodology for wear leveling | |
| US20090319721A1 (en) | Flash memory apparatus and method for operating the same | |
| CN113076218B (en) | Method for rapidly processing data reading errors of NVM (non-volatile memory) chip and controller thereof | |
| CN115295045A (en) | Redundancy and majority voting in key value data storage systems using content addressable memories | |
| KR102321346B1 (en) | Data journaling method for large solid state drive device | |
| CN113434092B (en) | Fingerprint identification method based on hybrid DRAM-NVM | |
| CN111984651A (en) | Column type storage method, device and equipment based on persistent memory | |
| CN113434091A (en) | Cold and hot key value identification method based on hybrid DRAM-NVM | |
| US20170220634A1 (en) | Method and device to access auxiliary mapping data for a data structure | |
| CN112052190B (en) | Solid state disk hot data identification method based on bloom filter and secondary LRU table | |
| CN111190834B (en) | Data placement method for asymmetric read delay sensing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |