CN113434091A - Cold and hot key value identification method based on hybrid DRAM-NVM - Google Patents

Cold and hot key value identification method based on hybrid DRAM-NVM Download PDF

Info

Publication number
CN113434091A
CN113434091A CN202110767199.0A CN202110767199A CN113434091A CN 113434091 A CN113434091 A CN 113434091A CN 202110767199 A CN202110767199 A CN 202110767199A CN 113434091 A CN113434091 A CN 113434091A
Authority
CN
China
Prior art keywords
key
cache
hash
nvm
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110767199.0A
Other languages
Chinese (zh)
Inventor
朱俊豪
黄成龙
方粮
李成
易品筠
马德胜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202110767199.0A priority Critical patent/CN113434091A/en
Publication of CN113434091A publication Critical patent/CN113434091A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0611—Improving I/O performance in relation to response time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Abstract

The invention discloses a cold and hot key value identification method based on a hybrid DRAM-NVM (dynamic random Access memory-non-volatile memory), aiming at solving the problem of high data index access delay caused by the lack of the cold and hot key value identification method in the existing hash table of the hybrid memory on the premise of satisfying the condition that data can be recovered when a system crashes. The technical scheme is that an NVM table and a cache table are respectively constructed, the NVM table is stored in the NVM, and the cache table is stored in the DRAM; then determining the address pp of the key x of the key value pair to be inquired in the cache bucket and the address pt of the hash bucket, matching the key x with the cache element in the cache bucket with the address pp, and returning an inquiry result if the matching is successful; matching the keyx with the hash elements in the hash bucket with the address pt to obtain a matching result or 'query failure' information. The method solves the problem of high data index access delay caused by the lack of a cold and hot key value identification method in the existing hybrid memory hash table, filters out a large number of NVM reading, and greatly reduces the access delay.

Description

Cold and hot key value identification method based on hybrid DRAM-NVM
Technical Field
The invention belongs to the field of computer data storage, and particularly relates to a cold and hot key value identification method based on a hybrid DRAM-NVM.
Background
In recent years, several new Non-Volatile Memory (NVM) technologies have come into the spotlight, such as Phase Change Memory (PCM), memristor (ReRAM), spin torque magnetic Memory (STT-MRAM), and 3D-XPoint. Compared with the traditional DRAM memory technology, the NVM has the characteristics of power-down non-volatility, high storage density and low power consumption of a magnetic disk, and also has the characteristics of byte-based modification addressing and low read-write delay of the DRAM. Therefore, the NVM technology is used for making up the defects of insufficient memory capacity and high power consumption of the DRAM memory system. However, the current NVM technology still has the disadvantages of asymmetric read/write delay and limited wear times, so the NVM is more suitable for being connected to a memory bus of a computer system together with a DRAM to form a hybrid memory.
The key value pair is the simplest organization form of the database, wherein the key is the index number of the value, the value is the data to be stored in the database, and when a user inquires the data, the corresponding value can be inquired and obtained in the database only by providing the key to be inquired. For example in a telephone number database, the key: pony, value: 18753137777, < horse, 18753137777> is a key-value pair, and the user can obtain his phone number 18753137777 by the key "horse" to be queried. Key-value pairs are currently widely used in various databases, such as Redis, voldemert, Oracle BDB. The hash index is a data structure of a common query key value pair in a database, and is widely applied to various databases and file systems, and currently, with the rise of big data and cloud computing, the requirement for query delay of the memory hash index is higher and higher, which also prompts researchers to research the hash index structure of a novel nonvolatile memory.
Many studies of hash indexing based on persistent memory have been made, however, most of them are based on a system with only NVM as memory, which does not well exploit the advantages of the hybrid DRAM-NVM memory system. In an actual workload, a part of key value pairs in the hash table need to be frequently read, and the frequently read key value pairs are called hot key values; while key-value pairs that are rarely accessed are referred to as cold key-values. Reading hot key values from the NVM each time results in higher access latency and wastes NVM bandwidth. The natural idea is to build a cache table in DRAM to store key values, in which case the process of querying a key-value pair is: firstly, inquiring whether the key value pair exists in a cache table in the DRAM, and if so, reading the value of the key value pair and returning the value to an inquiring user; if the data table does not exist in the cache table, the data table in the NVM is searched. By establishing the cache table, reading of the hot key values is limited in the DRAM, and the data indexing efficiency can be effectively improved. However, because the capacity of the cache table is limited, how to identify the hot and cold key values and perform hot and cold replacement in time becomes a problem to be solved at present. At present, a cold and hot key value identification method aiming at a persistent hash table does not exist, so that a cache table cannot play a due role in the hash table, and the data index speed of the hash table is prevented from being improved.
Rewo (a paper published in International conference DATE2020) was proposed by Kaixin Huang et al. As shown in FIG. 1, there are two tables in Rewo, one in NVM space and one in DRAM space. The NVM table in the NVM is mainly used to ensure the persistent storage of key-value pairs in the NVM, and the cache table is a copy of the NVM table and serves for querying requests. The size ratio of the cache table to the NVM table is 1:2, and the cache table reserves a part of data of the NVM table, and the query request of the part of data can be satisfied in the DRAM space. Although Rewo realizes the fast query of key value pairs by setting a cache table in a DRAM, the method does not provide a method for identifying cold and hot key values, so that the cache table cannot identify cold and hot keys for hot data sets, and the utilization efficiency of the cache table is reduced.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a cold and hot key value identification method based on a hybrid DRAM-NVM, which solves the problem of high data index access delay caused by the lack of the cold and hot key value identification method in the existing hybrid memory hash table on the premise of satisfying the data recoverability in system crash.
In order to achieve the above object, the present invention comprises the steps of:
the first step, construct and initialize the NVM table, store the NVM table in the NVM, the method is:
1.1, an NVM table is constructed by adopting a barrel array structure, the NVM table comprises N hash buckets, N is a positive integer, each hash bucket comprises 8 hash units, the size of each hash unit is 32 bytes, one hash unit stores a key value pair < key, value >, the key represents a key of a storage element, and the value represents a value of the storage element. Through the bucket address translation function, the key of the key value pair is used for calculating to obtain a hash bucket address (order is p) as the hash bucket address for indexing the NVM table, the hash bucket of the NVM table can be accurately positioned through the hash bucket address, and the calculation formula of the bucket address translation function is as follows:
p＝hash(key)％N
wherein p is a hash bucket address, hash () is a standard hash function in a c + + standard template library, N is the total number of hash buckets in the NVM table, and "%" is a modulo operation;
1.2M key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserted into an NVM table, will<keym,valuem>The method for inserting into the NVM table is as follows:
calculated by barrel address translation function<keym,valuem>Corresponding hash bucket address Pm，Pm＝hash(keym) % N, and<keym,valuem>the address sequentially inserted into the NVM table is PmIn the hash bucket of (2). 1.3 storing the NVM table in the NVM.
Secondly, constructing and initializing a cache table, and storing the cache table in a DRAM (dynamic random access memory), wherein the method comprises the following steps:
2.1 adopt the bucket array structure to construct the cache table, the cache table also contains N cache buckets, each bucket contains 4 cache units, the size of each cache unit is 33 bytes (including 1 byte hot mark identifier for identifying the cold and hot degree of the key value pair in the cache unit and 32 byte hot key value), use the first step said bucket address conversion function, utilize the key of the key value pair to calculate and obtain the cache tableAnd the bucket storage address p is used as a cache bucket address of the index cache table, and the cache bucket of the cache table can be accurately positioned through the cache bucket address. For example: by bucket address translation function Pm＝hash(keym) Calculated as% N<keym,valuem>Corresponding cache bucket address P in cache tablemThrough PmCache bucket for accurately locating cache table where it can be found<keym,valuem>。
2.2M Key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserted into a cache table, will<keym,valuem>The method for inserting the cache table comprises the following steps:
calculated by barrel address translation function<keym,valuem>Corresponding cache bucket address Pm，Pm＝hash(keym) % N, and<keym,valuem>sequentially inserting the addresses of the cache table into PmThen, the hotmap value corresponding to the cache unit is initialized to 1.
2.3 store the cache table in the DRAM.
Thirdly, determining the addresses of the key value pair to be inquired in the cache bucket and the hash bucket, wherein the method comprises the following steps:
3.1 receiving the key of the key value pair to be inquired input by the user from the keyboard, and making the key as a keyXCalculating the address of a cache bucket by using a bucket address conversion function and using keyx, wherein the instruction is pp;
3.2 calculating the hash bucket address by using a bucket address conversion function and using keyx, wherein the order is pt;
fourthly, matching keyx with elements in 4 cache units in a cache bucket with address pp in the cache table, wherein the method comprises the following steps:
4.1 making the variable i of the cache unit equal to 1;
4.2, matching the ith cache unit of the cache bucket with the address of pp in the cache table by using keyx, if the matching is successful (namely, finding the key value pair matched with keyx in the ith cache unit of the cache bucket with the address of pp in the cache table), adding 1 to the hotmap value corresponding to the key value pair (indicating that the inquired times are increased once), and turning to the sixth step; otherwise, if i is equal to i +1, if i is less than or equal to 4, rotating to 4.2 for continuous matching; if i is greater than 4, the cache barrel with the traversal address pp is still unsuccessfully matched, the key value pair corresponding to the keyx is not in the cache table, and the fifth step is carried out;
fifthly, matching keyx with 8 hash elements in a hash bucket with pt address in the NVM table, wherein the method comprises the following steps:
5.1 let hash unit variable j equal to 1;
5.2 matching the jth hash unit of the hash bucket with the address pt in the NVM table by using keyx, if the matching is successful (i.e. finding the key-value pair matched with keyx in the hash bucket with the address pt), traversing the cache bucket with the address pp in the cache table, deleting the key-value pair in the cache unit with the minimum hot value in the cache bucket with the address pp, inserting the successfully-matched key-value pair into the cache unit, initializing the hot value of the cache unit to 1, and going to the sixth step; otherwise, j is made to be j +1, if j is less than or equal to 8, 5.2 is rotated to continue matching; if j is greater than 8, the hash bucket with the address pt of the traversal NVM table is still unsuccessfully matched, the key value pair corresponding to the keyx to be inquired is not in the NVM table, the inquiry is failed, the 'inquiry failure' information is sent to the user, and the seventh step is carried out;
and sixthly, obtaining a value in the key value pair corresponding to the keyx value, and sending the value to the inquiry user.
And step seven, finishing.
The invention can achieve the following technical effects:
1. the invention provides a cold and hot key value identification method based on a hybrid DRAM-NVM (dynamic random Access memory) -NVM, which is used for storing a cache table capable of reducing NVM reading delay on the DRAM, and reducing hash table access delay by utilizing the more excellent reading and writing performance of the DRAM; the key value pair is stored in the NVM in a persistent mode, so that the function of quick recovery after system crash or power failure is realized; the invention fully utilizes the advantages of high read-write performance of DRAM in the hybrid memory and durability of NVM, and solves the technical problem of high data index access delay caused by the lack of a cold-hot identification method in the existing hash table of the hybrid memory on the premise of meeting the requirement of quick recovery of system crash.
2. The invention provides a cold and hot key value identification method based on a hybrid DRAM-NVM (dynamic random access memory-non-volatile memory), which is characterized in that the cold and hot key value pairs are identified in real time by recording the access frequency of different key value pairs, and the hot key value data set is dynamically updated in the query process, so that a cache table suitable for the actual working load is constructed in the DRAM with lower read delay, the corresponding query process can be completed in the DRAM for the hot key value, a large amount of NVM reading is filtered, and the access delay is greatly reduced.
Description of the drawings:
FIG. 1 is a schematic diagram of a hash table structure of a Rewo hot and cold key value recognition method; FIG. 1(a) is a schematic diagram of a cache table structure; FIG. 1(b) is a schematic diagram of a NVM table structure.
FIG. 2 is a schematic diagram of a NVM table structure constructed in the first step of the present invention;
FIG. 3 is a structural diagram of a cache table constructed in the second step of the present invention;
FIG. 4 is an overall flow chart of the present invention;
fig. 5 is a diagram illustrating a comparison of the delay of the Rewo query operation in the cold and hot key value recognition method according to the present invention and the related art.
Detailed Description
In order to further explain the technical scheme of the invention, the following description refers to the accompanying drawings. The general flow chart of the invention is shown in fig. 4, and comprises the following steps:
the first step, construct and initialize the NVM table, store the NVM table in the NVM, the method is:
1.1, an NVM table is constructed by adopting a barrel array structure, the NVM table comprises N hash buckets, N is a positive integer, each hash bucket comprises 8 hash units, the size of each hash unit is 32 bytes, one hash unit stores a key value pair < key, value >, the key represents a key of a storage element, and the value represents a value of the storage element. Through the bucket address translation function, the key of the key value pair is used for calculating to obtain a hash bucket address (order is p) as the hash bucket address for indexing the NVM table, the hash bucket of the NVM table can be accurately positioned through the hash bucket address, and the calculation formula of the bucket address translation function is as follows:
p＝hash(key)％N
wherein p is a hash bucket address, hash () is a standard hash function in a c + + standard template library, N is the total number of hash buckets in the NVM table, and "%" is a modulo operation;
1.2M key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserted into an NVM table, will<keym,valuem>The method for inserting into the NVM table is as follows:
calculated by barrel address translation function<keym,valuem>Corresponding hash bucket address Pm，Pm＝hash(keym) % N, and<keym,valuem>the address sequentially inserted into the NVM table is PmIn the hash bucket of (2).
1.3 storing the NVM table in the NVM.
Secondly, constructing and initializing a cache table, and storing the cache table in a DRAM (dynamic random access memory), wherein the method comprises the following steps:
2.1 adopt the bucket formula array structure to construct the buffer memory table, the buffer memory table also contains N buffer memory buckets, each bucket contains 4 buffer memory units, the size of each buffer memory unit is 33 bytes (including 1 byte hot identifier and 32 byte hot key value used for identifying the hot and cold degree of the key value pair in the buffer memory unit), use the bucket address transfer function stated in the first step, utilize the key of the key value pair to calculate and obtain the buffer memory bucket address p, as the buffer memory bucket address of the index buffer memory table, can position the buffer memory bucket of the buffer memory table accurately through the buffer memory bucket address. For example: by bucket address translation function Pm＝hash(keym) Calculated as% N<keym,valuem>Corresponding cache bucket address P in cache tablemThrough PmCache bucket for accurately locating cache table where it can be found<keym,valuem>。
2.2M Key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserted into a cache table, will<keym,valuem>The method for inserting the cache table comprises the following steps:
calculated by barrel address translation function<keym,valuem>Corresponding cache bucket address Pm，Pm＝hash(keym) % N, and<keym,valuem>sequentially inserting the addresses of the cache table into PmThen, the hotmap value corresponding to the cache unit is initialized to 1.
2.3 store the cache table in the DRAM.
Thirdly, determining the addresses of the key value pair to be inquired in the cache bucket and the hash bucket, wherein the method comprises the following steps:
3.1 receiving the key of the key value pair to be inquired input by the user from the keyboard, and making the key as a keyXCalculating the address of a cache bucket by using a bucket address conversion function and using keyx, wherein the instruction is pp;
3.2 calculating the hash bucket address by using a bucket address conversion function and using keyx, wherein the order is pt;
fourthly, matching keyx with elements in 4 cache units in a cache bucket with address pp in the cache table, wherein the method comprises the following steps:
4.1 making the variable i of the cache unit equal to 1;
4.2, matching the ith cache unit of the cache bucket with the address of pp in the cache table by using keyx, if the matching is successful (namely, finding the key value pair matched with keyx in the ith cache unit of the cache bucket with the address of pp in the cache table), adding 1 to the hotmap value corresponding to the key value pair (indicating that the inquired times are increased once), and turning to the sixth step; otherwise, if i is equal to i +1, if i is less than or equal to 4, rotating to 4.2 for continuous matching; if i is greater than 4, the cache barrel with the traversal address pp is still unsuccessfully matched, the key value pair corresponding to the keyx is not in the cache table, and the fifth step is carried out;
fifthly, matching keyx with 8 hash elements in a hash bucket with pt address in the NVM table, wherein the method comprises the following steps:
5.1 let hash unit variable j equal to 1;
5.2 matching the jth hash unit of the hash bucket with the address pt in the NVM table by using keyx, if the matching is successful (i.e. finding the key-value pair matched with keyx in the hash bucket with the address pt), traversing the cache bucket with the address pp in the cache table, deleting the key-value pair in the cache unit with the minimum hot value in the cache bucket with the address pp, inserting the successfully-matched key-value pair into the cache unit, initializing the hot value of the cache unit to 1, and going to the sixth step; otherwise, j is made to be j +1, if j is less than or equal to 8, 5.2 is rotated to continue matching; if j is greater than 8, the hash bucket with the address pt of the traversal NVM table is still unsuccessfully matched, the key value pair corresponding to the keyx to be inquired is not in the NVM table, the inquiry is failed, the 'inquiry failure' information is sent to the user, and the seventh step is carried out;
and sixthly, obtaining a value in the key value pair corresponding to the keyx value, and sending the value to the inquiry user.
And step seven, finishing.
Fig. 5 is a diagram illustrating a comparison of the delay of the Rewo query operation in the cold and hot key value recognition method according to the present invention and the related art. The query operation latency test was performed on a 32-core Langchao server with 32KB/1024KB/32MB L1/L2/L3 cache. The server is configured with 192GB DRAM (6 × 32GB/DIMM) and 1.5TB NVM (6 × 256GB/DIMM), and the NVM memory is configured in App Direct mode and mounted on the ext4-DAX file system. Software environments ubuntu Linux and PMDK 1.7 are used in the test, and all realized codes are optimized and compiled by adopting GCC 10.2.0. The ordinate of the comparison graph is the query average access delay (1000 query operations are repeated, and the average delay is taken), the vertical line graph represents the query average delay of Rewo, the horizontal line graph represents the query average delay of the invention, the query average delay of Rewo is 0.649us, the query average access delay of the invention is 0.36us, and compared with Rewo, the query average delay of the invention is improved by 80.3% ((0.649-0.36)/0.36).
The method for identifying cold and hot key values based on the hybrid DRAM-NVM provided by the invention is described in detail above. The principles and embodiments of the present invention are explained herein, with the above description being included to assist in understanding the core concepts of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
It will be understood by those skilled in the art that the foregoing is merely a preferred embodiment of this invention, and is not intended to limit the invention, such that any modification, equivalent replacement, or improvement made within the spirit and scope of the invention should be construed broadly.

Claims (3)

1. A cold and hot key value identification method based on a hybrid DRAM-NVM adopts a barrel array structure to construct a hash table, and is characterized by comprising the following steps:
the first step, construct and initialize the NVM table, store the NVM table in the NVM, the method is:
1.1, constructing an NVM (non volatile memory) table by adopting a barrel array structure, wherein the NVM table comprises N hash buckets, N is a positive integer, each hash bucket comprises 8 hash units, the size of each hash unit is 32 bytes, one hash unit stores a key value pair < key, value >, the key represents a key of a storage element, and the value represents a value of the storage element; through a bucket address conversion function, a key of a key value pair is used for calculating to obtain a hash bucket address, the order is p, the hash bucket address is used as a hash bucket address for indexing the NVM table, the hash bucket of the NVM table can be accurately positioned through the hash bucket address, and the calculation formula of the bucket address conversion function is as follows:
P＝hash(key)％N
wherein p is a hash bucket address, hash () is a standard hash function in a c + + standard template library, N is the total number of hash buckets in the NVM table, and "%" is a modulo operation;
1.2M key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Insert into NVM table;
1.3 storing the NVM table in the NVM;
secondly, constructing and initializing a cache table, and storing the cache table in a DRAM (dynamic random access memory), wherein the method comprises the following steps:
2.1, constructing a cache table by adopting a barrel type array structure, wherein the cache table also comprises N cache barrels, each barrel comprises 4 cache units, the size of each cache unit is 33 bytes, and the cache table comprises 1-byte hotmap identifier for identifying the cold and hot degrees of key values in the cache units and 32-byte hot key values;
2.2M Key value pairs that the user needs to store<key1,value1>，…，<keym,valuem>，…，<keyM,valueM>Inserting the key value pair into a cache table, and initializing a hotmap value of a cache unit where the key value pair is positioned to be 1 after inserting the key value pair;
2.3 storing the cache table in the DRAM;
thirdly, determining the addresses of the key value pair to be inquired in the cache bucket and the hash bucket, wherein the method comprises the following steps:
3.1 receiving the key of the key value pair to be inquired input by the user from the keyboard, and making the key as a keyXCalculating the address of a cache bucket by using a bucket address conversion function and using keyx, wherein the instruction is pp;
3.2 calculating the hash bucket address by using a bucket address conversion function and using keyx, wherein the order is pt;
fourthly, matching keyx with elements in 4 cache units in a cache bucket with address pp in the cache table, wherein the method comprises the following steps:
4.1 making the variable i of the cache unit equal to 1;
4.2, matching the ith cache unit of the cache bucket with the address of pp in the cache table by using keyx, if the matching is successful, finding a key value pair matched with the keyx in the ith cache unit of the cache bucket with the address of pp in the cache table, adding 1 to the hotmap value corresponding to the key value pair to indicate that the inquired times are increased once, and turning to the sixth step; otherwise, if i is equal to i +1, if i is less than or equal to 4, rotating to 4.2 for continuous matching; if i is greater than 4, the cache barrel with the traversal address pp is still unsuccessfully matched, the key value pair corresponding to the keyx is not in the cache table, and the fifth step is carried out;
fifthly, matching keyx with 8 hash elements in a hash bucket with pt address in the NVM table, wherein the method comprises the following steps:
5.1 let hash unit variable j equal to 1;
5.2 matching the jth hash unit of the hash bucket with the address pt in the NVM table by using keyx, if the matching is successful, namely finding out the key value pair matched with the keyx in the hash bucket with the address pt, traversing the cache bucket with the address pp in the cache table, deleting the key value pair in the cache unit with the minimum hot value in the cache bucket with the address pp, inserting the successfully matched key value pair into the cache unit, initializing the hot value of the cache unit to 1, and going to the sixth step; otherwise, j is made to be j +1, if j is less than or equal to 8, 5.2 is rotated to continue matching; if j is greater than 8, the hash bucket with the address pt of the traversal NVM table is still unsuccessfully matched, the key value pair corresponding to the keyx to be inquired is not in the NVM table, the inquiry is failed, the 'inquiry failure' information is sent to the user, and the seventh step is carried out;
sixthly, obtaining a value in a key value pair corresponding to the keyx value, and sending the value to a query user;
and step seven, finishing.
2. The method according to claim 1, wherein 1.2 the step of identifying cold and hot key values is to identify the memory based on the hybrid DRAM-NVM<keym,valuem>The method of inserting into the NVM table is: calculated by barrel address translation function<keym,valuem>Corresponding hash bucket address Pm，Pm＝hash(keym) % N, will<keym,valuem>The address sequentially inserted into the NVM table is PmIn the hash bucket of (2).
3. The method according to claim 1, wherein 2.2 steps of the method for identifying cold and hot key values based on the hybrid DRAM-NVM memory<keym,valuem>The method for inserting the cache table comprises the following steps: calculated by barrel address translation function<keym,valuem>Corresponding cache bucket address Pm，Pm＝hash(keym) % N, will<keym,valuem>The addresses sequentially inserted into the cache table are PmIn the cache bucket.
CN202110767199.0A 2021-07-07 2021-07-07 Cold and hot key value identification method based on hybrid DRAM-NVM Pending CN113434091A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110767199.0A CN113434091A (en) 2021-07-07 2021-07-07 Cold and hot key value identification method based on hybrid DRAM-NVM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110767199.0A CN113434091A (en) 2021-07-07 2021-07-07 Cold and hot key value identification method based on hybrid DRAM-NVM

Publications (1)

Publication Number Publication Date
CN113434091A true CN113434091A (en) 2021-09-24

Family

ID=77759369

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110767199.0A Pending CN113434091A (en) 2021-07-07 2021-07-07 Cold and hot key value identification method based on hybrid DRAM-NVM

Country Status (1)

Country Link
CN (1) CN113434091A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023056946A1 (en) * 2021-10-09 2023-04-13 上海淇馥信息技术有限公司 Data caching method and apparatus, and electronic device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105893274A (en) * 2016-05-11 2016-08-24 华中科技大学 Device for building checkpoints for heterogeneous memory system
CN107451071A (en) * 2017-08-04 2017-12-08 郑州云海信息技术有限公司 A kind of caching replacement method and system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105893274A (en) * 2016-05-11 2016-08-24 华中科技大学 Device for building checkpoints for heterogeneous memory system
CN107451071A (en) * 2017-08-04 2017-12-08 郑州云海信息技术有限公司 A kind of caching replacement method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KAIXIN HUANG: "Revisiting Persistent Hash Table Design for Commercial Non-Volatile Memory", 《IEEE XPLORE》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023056946A1 (en) * 2021-10-09 2023-04-13 上海淇馥信息技术有限公司 Data caching method and apparatus, and electronic device

Similar Documents

Publication Publication Date Title
CN111475427B (en) Logical-to-physical mapping management using low latency nonvolatile memory
JP5445682B2 (en) Storage system
CN107153707B (en) Hash table construction method and system for nonvolatile memory
CN113196259B (en) Key value storage using logging in selective data storage formats
CN104794070A (en) Solid-state flash memory write cache system and method based on dynamic non-covering RAID technology
CN111033477A (en) Logical to physical mapping
CN106055679A (en) Multi-level cache sensitive indexing method
CN110018971B (en) cache replacement technique
CN109407985B (en) Data management method and related device
CN113419675B (en) Write operation method and read operation method for memory
US20130297877A1 (en) Managing buffer memory
US20090319721A1 (en) Flash memory apparatus and method for operating the same
WO2013075306A1 (en) Data access method and device
CN109165321B (en) Consistent hash table construction method and system based on nonvolatile memory
CN111309262B (en) Distributed storage cache reading and writing method
CN113434091A (en) Cold and hot key value identification method based on hybrid DRAM-NVM
CN110427347A (en) Method, apparatus, memory node and the storage medium of data de-duplication
CN112732725B (en) NVM (non volatile memory) hybrid memory-based adaptive prefix tree construction method, system and medium
CN111061652B (en) Nonvolatile memory management method and system based on MPI-IO middleware
CN105988720A (en) Data storage device and method
CN111338569A (en) Object storage back-end optimization method based on direct mapping
CN106168883A (en) A kind of efficient data tissue and access method
CN114003169B (en) Data compression method for SSD
CN113434092B (en) Fingerprint identification method based on hybrid DRAM-NVM
CN111143313B (en) Method for improving log block read-write performance of hybrid mapping algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210924

WD01 Invention patent application deemed withdrawn after publication