CN107479833A - A remote non-volatile memory access and management method for key-value storage - Google Patents

Abstract

The invention relates to a remote nonvolatile memory access and management method facing key value storage, which is characterized in that a nonvolatile memory cache region is set for a key value storage system and is registered on a network card capable of supporting remote direct memory access technology; this non-volatile memory cache area is divided into two parts: the memory blocks in the sparse area are mainly used for receiving data of a remote client, different memory blocks are allocated for remote data writing each time, the memory block in each sparse area only belongs to one key value pair, the memory block in the compact area is a main cache area of the key value pair, each memory block comprises a plurality of key value pairs, and a read request of the client is served; compressing key value pairs in a plurality of allocated memory blocks in the sparse area into memory blocks in one or more compact areas by a compressor at regular intervals by the server; when the number of available memory blocks of the compact area is insufficient, data is replaced to the data storage area. The invention can relieve the fixed point write abrasion problem of the remote nonvolatile memory and improve the remote read-write performance.

Description

A remote non-volatile memory access and management method for key-value storage

technical field

The invention relates to a remote non-volatile memory access and management method, in particular to a key-value storage-oriented remote non-volatile memory access and management method.

Background technique

In the big data environment, it is necessary to face the storage requirements of massive data. On the one hand, higher requirements are put forward for the scalability of the application system, while the NoSQL system based on the key-value storage architecture uses independent key-value pairs as the basic unit for data storage, which reduces the correlation between data and has a good Scalability; on the other hand, computer storage systems based on traditional memory and external memory architectures are faced with the challenge of massive data processing performance; new storage devices, such as PCM (phase-change random access memory), 3D Xpoint, non-volatile memory with external memory characteristics such as performance, high integration and memory-level access speed, considering the volatile characteristics, high energy consumption and limited capacity of the existing DRAM (Dynamic Random Access Memory), the non-volatile memory ( NVM (Non-Volatile Memory) has become a hot spot for computer designers and researchers to improve the performance of storage systems. Although the integration of NVM is higher than that of DRAM, the capacity of NVM in a single machine is still limited, and the maturity of the current NVM technology is not high. According to the report of Intel, in the future computer system, it will deploy 4 times the traditional memory. capacity of commercial NVM products (such as 3D Xpoint); it is true that the NVM resources of a single machine are far from meeting the storage capacity requirements of big data application systems, so horizontal expansion through the network has become a must. The traditional network based on the TCP/IP protocol will become the performance bottleneck of remote memory expansion. The main reason is the data processing method of the TCP/IP protocol. The application program prepares the data to be sent in the buffer area of the user space, and then copies the data through the system call In the kernel space, the header information required by each layer protocol needs to be added to the sent data in the kernel space, and finally sent to the remote machine through the network card, and the remote machine receives the data through the network card, according to the logic of sending to the reverse Parse the data packet, and finally copy the data to the buffer area of the user space of the remote application program; it is not difficult to find that using traditional network technology for data transmission requires multiple data copies. Data copying not only consumes CPU resources, but also It also reduces the utilization of network bandwidth. Compared with TCP/IP, RDMA (Remote Direct Memory Access, remote direct memory access technology) allows remote direct access to remote user memory space without the participation of the remote CPU. During data transmission, the remote CPU Other programs can be run in parallel. The current remote access memory is based on DRAM. Due to the memory access characteristics of NVM, remote access to NVM is reasonable, and NVM will play an important role in the future RDMA technology.

However, while NVM has the advantages of memory and external memory, NVM also has problems such as read-write imbalance and write wear. Because RDMA technology can make remote non-volatile memory directly accessible to clients, and RDMA The mapping between the accessed virtual address and physical address is fixed, which accelerates the write loss of NVM. Therefore, how to design the access mode and management mechanism of NVM has become a new challenge.

Contents of the invention

In order to solve the above-mentioned technical problems, the purpose of the present invention is to provide a remote non-volatile memory access and management method for key-value storage.

This method sets a non-volatile memory cache area for key-value storage and registers it on a network card that supports remote direct memory access technology; divides this non-volatile memory cache area into two parts: sparse area and compact area, sparse The memory blocks in the area are mainly used to receive data from remote clients, and different memory blocks are allocated for remote data writing each time. The memory blocks in each sparse area only belong to one key-value pair, and its function is to improve remote writing. performance and extended non-volatile memory life.

The memory block in the compact area is the main cache area for key-value pairs. Each memory block contains multiple key-value pairs to serve the client's read request; The key-value pairs in the memory block of the compact area are compressed into one or more memory blocks in the compact area; when the number of available memory blocks in the compact area is insufficient, the data will be replaced to the data storage area. Its role is to improve the efficiency of the cache.

The technical solution adopted by the present invention to solve its technical problems is:

An RDMA-based remote non-volatile memory access and management method, comprising:

In the first step, the server divides memory into three categories: data storage area, non-volatile memory cache area, and command buffer; among them, the memory in the data storage area and the non-volatile memory cache area is NVM, and the memory in the command buffer area is DRAM;

In the second step, the server registers the memory of the non-volatile memory buffer and command buffer to the network card that supports remote direct memory access technology, while the data storage area is selectively registered according to the parameters set by the user, and the non-volatile memory The volatile memory cache area is divided into a compact area and a sparse area. The size of the memory block in the sparse area is uniformly set as a multiple of page bytes, and the multiple is 1-n times, and n is a positive integer;

In the third step, the server receives the connection request from the client, and allocates a memory block in the sparse area and a memory block in the command buffer for each client;

In the fourth step, when the client transmits data, it writes the sparse memory block through RDMA (Remote Direct Memory Access, remote direct memory access technology), and at the same time sends the control information to the command buffer memory block, and the server returns the next write The sparse memory block address is used as a write confirmation (ACK) message; if a data transmission requires multiple sparse memory blocks, multiple sparse memory blocks are connected in series according to the control information to ensure data integrity;

Step 5, when the number of available memory blocks in the sparse area drops to a certain threshold (the threshold refers to the ratio of the number of available memory blocks to the total number of memory blocks, ranging from 0.1 to 0.6), the server will pass The memory compression mechanism compresses the data in multiple selected sparse memory blocks to the memory blocks in the compact area, and reclaims the sparse memory blocks containing garbage data;

Step 6, when the number of available memory blocks in the compact area drops to a certain threshold (the threshold refers to the ratio of the number of available memory blocks to the total number of memory blocks, ranging from 0.1 to 0.6), the server will pass The data migration mechanism copies the data in memory blocks in selected multiple compact areas to the data storage area.

Using the present invention can achieve the following beneficial effects: First, due to the limited on-chip cache on the RDMA network card, the non-volatile memory of the server is divided into a data storage area and a data buffer area, and the data buffer area is registered and the data storage area is selectively registered, reducing It reduces the miss overhead of the on-chip cache of the network card while ensuring the performance of data operations; secondly, the data cache is divided into a sparse area and a compact area. For the client, each time data is sent, the memory block in the sparse area is not Similarly, this means that the data writing of the client is not fixed-point, which can prolong the writing life of the non-volatile memory; moreover, a sparse memory block is allocated after each writing of the client as a confirmation message of the completion of the writing. While ensuring the reliability of data transmission, the performance of continuous data writing is improved; finally, the effective data occupying sparse memory blocks is compressed into compact memory blocks through the memory compression mechanism, which improves the efficiency of memory usage.

Description of drawings

The present invention will be further described below in conjunction with drawings and embodiments.

Figure 1 is a flow chart of remote non-volatile memory access and management for key-value storage;

Figure 2 is an organization diagram of sparse and compact memory blocks in the data cache.

detailed description

With reference to Fig. 1, the present invention provides a kind of key-value storage-oriented remote non-volatile memory access and management method, comprising:

In the first step, the server registers the communication memory area with the RDMA network card. The communication memory area includes non-volatile memory and volatile (traditional) memory. The non-volatile memory is used for caching key-value pairs, and the volatile memory receives user data transmission. command information;

In the second step, the server divides the non-volatile memory into two areas: sparse and compact. By default, these two areas are composed of multiple memory blocks of uniform size. The size of the memory block in the sparse area ranges from 4096 bytes to 16384 words. Between sections;

In the third step, the client initiates a connection establishment request to the server, and the server allocates a buffer memory block for receiving commands and a memory block in a sparse area for each client, and returns these information to the client;

In the fourth step, the client first writes the key-value pair into this sparse memory block through the RDMA remote write operation, and sends the control message to the command buffer;

In the fifth step, the server continues to allocate a memory block in a sparse area for the client by parsing the control message in the command buffer to store the data to be sent next time; if this is the last data block, then in the last sparse area Write its own identification information in the memory block to indicate the end of this data transmission, otherwise, write the identification of the memory block in the sparse area allocated for the next data transmission at the end of the previous sparse memory block;

Step 6, if the remaining length of the key-value pair data of the client exceeds the size of a bound memory block, continue to repeat step 4;

The seventh step, at intervals, the server counts the number of available memory blocks in the sparse area. If the data of the available memory blocks drops to a certain threshold, the server selects a certain number of memory blocks, and then compresses the number of them through the data compression mechanism. Data is copied to one or more memory blocks in the compact area;

The eighth step, at regular intervals, the server counts the number of available memory blocks in the compact area. If the available memory block data drops to a certain threshold, the server selects a certain number of memory blocks, and then transfers the available memory blocks through the data migration mechanism. Copy data to the data storage area to increase the number of available memory blocks;

Further as a preferred embodiment, the second step is specifically:

The memory of the registered cache area is shared by all clients. The default memory block in the compact area is the same size as the memory block in the sparse area. For the memory block in the sparse area, its main function is to cache more For key-value pairs, the client only has read permission. For the convenience of management, the size of all memory blocks in this area is uniform. However, in order to allow a single memory block to cache more data, the size of the memory block in the sparse area The maximum can be several times the size of the memory block in the compact area; since each client will bind a memory block in the compact area, if the size of the memory block in this area is set too large, in the case of more clients, memory resources will be reduced. It consumes more, so the memory block in the compact area is set to multiple levels according to the size, such as 4096 bytes, 8192 bytes, 16384 bytes, refer to Figure 2, establish a connection on the client side or transmit key-value pairs every time When finished, only the smallest memory block, which is 4096 bytes, is bound; the allocation of other memory blocks is dynamically adjusted according to the needs of the client.

Further as a preferred embodiment, the step three is specifically:

Due to the intermittent nature of the client's data sending request, if a long-term fixed memory block is bound for each connection, it will make other clients that need memory blocks lack resources, so set each connection within a certain period of time, if there is no The data transmission will reclaim the memory block in the bound compact area. There are two ways to recycle: the client actively notifies to give up the permission of this memory block; block of memory.

Claims (5)

1. a kind of long-range nonvolatile memory towards key assignments storage accesses and management method, non-volatile for key assignments storage setting one Internal memory cache region, and be registered on the network interface card that can support remote direct memory access technique；It is characterized in that, described non-volatile Internal memory cache region is divided into two parts：Rarefaction and compact area, the memory block in rarefaction are used to receive Terminal Server Client Data, and write per the different memory block of sub-distribution for teledata, the memory block of each rarefaction is pertaining only to a key Value pair, to improve the performance of remote write and extend the nonvolatile memory life-span；

The memory block in compact area is the main buffer area of key-value pair, and each memory block includes multiple key-value pairs, service client Read request；Service end is per at regular intervals, by compression mechanism by the key-value pair in the multiple allocated memory blocks in rarefaction It is compressed in the memory block in one or more compact areas；Improve the efficiency of caching；When compact area free memory block quantity not When sufficient, data are replaced to data storage area；

Specific steps include：

The first step, service end division in save as three classes：Data storage area, nonvolatile memory buffer area and command buffer；Its In, the internal memory of command buffer is DRAM；

Second step, the internal memory of nonvolatile memory buffer area and command buffer is registered to by service end supports remote direct memory to visit Ask on the network interface card of technology, and the registration for the parameter selectivity that data storage area is set according to user；By nonvolatile memory buffer area Compact area and rarefaction are divided into, the size of the memory block of rarefaction is uniformly set as the multiple of page byte, and the multiple is 1-n Times, n is positive integer；

3rd step, service end receives the connection request of client, and distributes a rarefaction memory block and one for each client The memory block of individual command buffer；

4th step, client enter sparse memory block, while send control information to order when transmitting data by RDMA Write In buffer memory block, service end returns to the sparse block address memory write next time as write acknowledgement message；An if data transfer Multiple sparse memory blocks are needed, then according to control information, multiple sparse memory blocks is cascaded and ensure the integrality of data；

5th step, when the quantity of the free memory block of sparse region drops to certain threshold values, service end will pass through internal memory pressure Contracting mechanism, the data compression in multiple sparse memory blocks will be selected into the memory block in compact area, and reclaim and include rubbish number According to sparse memory block；

6th step, when the quantity of the free memory block in compact area drops to certain threshold values, service end will pass through Data Migration Mechanism is by the data copy in the memory block for selecting multiple compact areas to data storage area.

2. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is that its effect of the memory block of the rarefaction is for caching more key-value pairs, and client only has the authority read, in order to Convenient management, it is unified in the size of all memory blocks of rarefaction, in order to allow single memory block to cache more data, The size maximum of memory block in rarefaction can be several times of compact area's internal memory block size；Because each client can be bound The memory block in one compact area, if the memory block setting in compact area is too big, in the case where client increases, the resource of internal memory Consume more, the memory block in compact area is set as several grades according to size.

3. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is, the 4th step,

If without the transmission of data, the memory block in the compact area of binding is reclaimed, reclaiming has two ways：Client proactive notification Abandon the authority of this memory block and service end forces recovery memory block and notifies client to send data needs Shen again next time Please memory block.

4. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is, the 4th step,

Client is it needs to be determined that the validity of the binding relationship of memory block before data is transmitted, if the relation of binding is solved Remove, then the memory block of the sparse memory field of first to file, be over if service end judges that a key-value pair is sent, can be this The transmission that a rarefaction memory block is used for key-value pair next time is bound in connection again.

5. a kind of long-range nonvolatile memory towards key assignments storage according to claim 1 accesses and management method, it is special Sign is that the threshold value in the 5th step, the 6th step refers to：The quantity of free memory block and the ratio of total internal memory number of blocks, Scope 0.1~0.6.