WO2020199760A1 - Procédé de stockage de données, mémoire et serveur - Google Patents

Procédé de stockage de données, mémoire et serveur Download PDF

Info

Publication number
WO2020199760A1
WO2020199760A1 PCT/CN2020/075170 CN2020075170W WO2020199760A1 WO 2020199760 A1 WO2020199760 A1 WO 2020199760A1 CN 2020075170 W CN2020075170 W CN 2020075170W WO 2020199760 A1 WO2020199760 A1 WO 2020199760A1
Authority
WO
WIPO (PCT)
Prior art keywords
value
storage
key
request
size
Prior art date
Application number
PCT/CN2020/075170
Other languages
English (en)
Chinese (zh)
Inventor
閤先军
尹文波
王伶俐
邱云辉
任予琛
陈华
余博伟
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020199760A1 publication Critical patent/WO2020199760A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0611—Improving I/O performance in relation to response time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device

Definitions

  • This application relates to the field of storage technology, and in particular to a data storage method, storage and server.
  • Key-value storage technology is a data access method that uses key pairs to store data. Each key corresponds to a value. Key-value storage technology is widely used in data centers. With the development of semiconductor technology, the cost of storage has dropped rapidly, and more and more high-performance storage has been applied to data centers, forming different types of key-value storage systems.
  • the key-value storage system has gradually developed from a system based on mechanical hard disks to a cache-type system that uses dynamic random access memory (DRAM) as a cache, and has developed to store all data including keys and values in DRAM A memory-based system as memory.
  • DRAM dynamic random access memory
  • the size of the value corresponding to the key is large, the data storage efficiency is low in the process of writing the value.
  • the value needs to be cached. Because the size of the value is large, a large cache space is required. In order to store the value completely in DRAM, the value needs to be stored The transmission process from the cache to the DRAM is started after the complete cache. The cache time and transmission time of this value are relatively long during the entire process, resulting in a large transmission delay and low data storage efficiency.
  • This application provides a data storage method, memory and server to solve the problem of low data storage efficiency of large-size data in the key-value storage system in the traditional technology.
  • the present application provides a data storage method.
  • the method includes: first, the key-value storage receives a key-value storage request, the key-value storage request includes a key and a first value, and the key and the first value are corresponding;
  • the key-value memory may divide the first storage space for storing the first value in the first memory, where the first memory may be a memory other than the key-value memory; if the first value is a large size Data; Exemplarily, the capacity of the data contained in the first value is greater than or equal to the first threshold, the key-value storage can perform data segmentation, divide the first value into multiple second values, and generate multiple secondary key-value stores Request, each secondary key value storage request can carry a key and one of the second values; the number of secondary key value storage requests is the same as the number of multiple second values obtained by dividing the first value.
  • the key-value storage can separately store the second value carried in each secondary key-value storage request according to the position order of the second value carried in each secondary key-value storage request in the first value.
  • the value is stored in the first storage space.
  • the capacity of the data contained in the first value can also be referred to as the size of the first value, which refers to the total size of the data contained in the first value, or the minimum storage space required to store the first value. the size of. For example, if the content contained in the first value is "0123456789", the capacity of the data contained in the first value is 10 bytes.
  • the key-value memory can be divided into multiple smaller second values, and the value of the smaller size can effectively shorten the time of data buffering and transmission, reduce the delay, and be more effective Improve the data storage efficiency of large-size data.
  • the key-value memory when the key-value memory divides the first value into multiple second values, it can be divided while caching; for example, the key-value memory can cache the first value (such as in the key-value memory).
  • the cached data In the process of caching the first value in the second memory), whenever the size of the cached data is less than or equal to the second threshold, the cached data is taken as the second value and carried in a secondary key value storage request until The first value cache is completed.
  • the cached data can be directly used as the second value and carried in a secondary key value storage request; in this case, each time The size of the second value carried in the level key value storage request is not necessarily the same.
  • the data division method may also be adopted after the cache is completed.
  • the key value storage may divide the first value into multiple second values, and the key value storage may divide the first value into N+1 second values, in which the order of division is Among the multiple second values arranged, the size of each of the first N second values can be equal to the second threshold, and the size of the N+1th second value can be less than or equal to the second threshold, and N is A positive integer greater than or equal to 1.
  • the key value storage may also set the second value to multiple second values.
  • the key value memory can set the second threshold value according to the size of the first value, which can segment the first value more flexibly.
  • a field may be set in the secondary key value storage request to indicate the relevant information of the second value carried, such as the size and offset of the second value.
  • a value offset may be included, and the value offset may indicate the position of the second value carried in the secondary key value storage request in the first value.
  • the key value storage can more conveniently determine the related information of the second value, which can make it convenient to determine that the second value carried in each secondary key value storage request is in the first value.
  • the key-value storage when the key-value storage stores the second value carried in each secondary key-value storage request to the first storage space, it needs to first determine the second value carried in each secondary key-value storage request.
  • the position order of the binary value in the first value; since the secondary key value storage request may include a value offset; the key value storage may determine the position order according to the value offset in each secondary key value storage request. For example, the position order is determined by comparing the magnitude of the value offset, and then the key value memory stores the second value in the first storage space according to the position order.
  • the key-value storage can determine the position sequence more conveniently by comparing the value offset in each secondary key-value storage request, improve data storage efficiency, and further ensure an efficient data storage process.
  • the second storage for the first value carried in one or more key-value storage requests can be configured in the first storage in advance. Space.
  • the key-value memory selects the first storage space from the second storage space, and the size of the first storage space may be equal to or greater than the size of the first value.
  • the key-value storage can pre-configure the second storage space before receiving the key-value storage request; it can also pre-configure the second storage space after receiving the key-value storage request and before storing the first value. In this case, the first 2.
  • the storage space can be equal to the size of the first value or greater than the size of the first value.
  • the key-value memory can select the first storage space from the pre-configured second storage space, and the first storage space can be determined more quickly, so that the first value can be stored in the first storage space faster, which can be realized Efficient data storage process.
  • the key-value memory allocates the first storage space for the first value, if there is not enough space in the second storage space to store the first value, that is, the amount of free space in the second storage space
  • the size is smaller than the size of the first value, and the key value storage may allocate the first storage space for the first value from the first storage.
  • the key value storage can select the first storage space from the first storage to store the first value, which can ensure that the first value can be stored in the first storage, and the efficiency of data storage can be guaranteed.
  • the key-value storage may further include a second storage, and after receiving the key-value storage request, the key-value storage may store the keys in the key-value storage request in the second storage; exemplary, the first 2.
  • a hash table can be stored in the storage, and the key value storage can update the hash table and add a hash table entry including the key; wherein the added hash table entry also includes the address of the first storage space.
  • the key value storage can update the hash table in time, and record the key and the corresponding first storage space address.
  • the key-value storage device determines that the storage of the first value has been completed, and stops the data storage operation ;
  • the secondary key value storage request includes the value size, which is used to indicate the size of the second value carried in the secondary key value storage request.
  • the key value storage device can accumulate the stored second value corresponding to the secondary key value storage request The value size, until the accumulated value is equal to the size of the first value, indicating that the first value has been stored.
  • the key value storage can determine that the first value has been stored in different ways, and can stop the data storage operation in time.
  • the present application provides a key-value memory, which has the function of the key-value memory in the first aspect and any one of the possible designs of the first aspect.
  • the function can be realized by hardware, or by hardware executing corresponding software.
  • the hardware or software includes one or more modules corresponding to the above-mentioned functions.
  • the structure of the device includes a receiving unit, a distributing unit, a dividing unit, and a storage unit. These units can perform the corresponding functions in the method example of the first aspect. For details, please refer to the detailed description in the method example. Do not repeat it.
  • the present application also provides a key value memory.
  • the structure of the key-value memory includes a processor and a memory, and the processor is configured to support the key-value memory to perform corresponding functions in the first aspect and any one of the possible design methods of the first aspect.
  • the memory is coupled with the processor, and it stores the program instructions and data necessary for the key-value memory.
  • the structure of the key-value memory also includes a transceiver for communicating with other devices.
  • this application provides a key-value storage server.
  • the key-value storage server includes a first storage and a key-value storage.
  • the first storage is a storage other than the key-value storage.
  • the key-value storage server is used to execute the first aspect. And the operation steps of the method provided in any one of the possible designs in the first aspect.
  • the present application also provides a computer-readable storage medium that stores instructions in the computer-readable storage medium, which when run on a computer, causes the computer to execute the methods of the above aspects.
  • the present application also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the methods of the above aspects.
  • the present application also provides a computer chip, which is connected to a memory, and the chip is used to read and execute a software program stored in the memory, and execute the methods of the above aspects.
  • Figure 1 is a schematic diagram of the architecture of a key-value storage system provided by this application.
  • FIG. 2 is a schematic structural diagram of a key-value storage server provided by this application.
  • FIG. 3 is a schematic diagram of a data storage method provided by this application.
  • FIG. 4 is a schematic diagram of a flow of data fragmentation and splicing provided by this application.
  • FIG. 5 is a schematic diagram of the format of a key-value storage request provided by this application.
  • FIG. 6 is a schematic diagram of another key-value storage request format provided by this application.
  • FIG. 7 is a schematic diagram of a first value segmentation provided by this application.
  • FIG. 8 is a schematic diagram of state switching of a state machine provided by this application.
  • FIG. 9 is a schematic structural diagram of a key value storage provided by this application.
  • FIG. 10 is a schematic structural diagram of another key value storage provided by this application.
  • This application provides a data storage method, memory and server to solve the problem of low data storage efficiency of large-size data in the key-value storage system in the traditional technology.
  • FIG. 1 a schematic diagram of a network architecture provided by an embodiment of this application, in which a plurality of application server key-value clients 101 and key-value storage servers (key-value servers) 102 are provided.
  • the application server key-value client 101 and the key-value storage server 102 interact through the network to send and receive information.
  • the embodiment of the present application does not limit the type of the network.
  • the transmission control protocol/Internet interconnection protocol transmission control protocol/ The wired transmission method of internet protocol (TCP/IP) can also be the wireless transmission method using wireless fidelity (Wifi).
  • the key-value storage server 102 may also be a key-value storage device composed of other hardware.
  • Each key-value storage server 102 can be provided with a memory 103, and the memory 103 can be used to store values.
  • the memory 103 can be a cache, flash memory (flash), or hard disk drive (HDD) in the key-value storage server. Or storage media such as solid-state drives (SSD).
  • the memory 103 may also be used to store a hash table for recording the correspondence between keys and values, and the storage location of the value, so that the hash table can be used in data processing to query the storage location of the specified value.
  • the hash table entries are the correspondence between the key and the address of the storage space where the value corresponding to the key is stored.
  • the storage of the value corresponding to the key can be retrieved through the hash table Address (the storage address is the address of the storage space in the memory 103).
  • the storage address of the value is determined by the hash table, and then the value is obtained from the memory 103; in the process of storing the value, the corresponding hash table entry needs to be added to the hash table to record the value The storage address.
  • the multiple key-value storage servers 102 shown in FIG. 1 can respectively receive key-value storage requests sent by different application server key-value clients 101 via the network in parallel. If the key-value storage request is a read request, the key-value storage server 102 may write the value carried in the key-value storage request into the memory 103. In addition, the key-value storage server 102 may also update the hash table and insert new hash table entries according to the key carried in the key-value storage request and the storage space allocated for the value in the memory 103.
  • the inserted hash table entry includes a key and a storage address; by hashing the key, the generated hash value is the index of the inserted hash table entry.
  • the key-value storage server 102 can read the corresponding value from the storage according to the key carried in the key-value storage request. Specifically, the key-value storage server 102 may perform a hash operation on the key carried in the key-value storage request to generate a hash value, query the corresponding hash table entry in the hash table through the hash value, and then compare the queried Whether the key in the hash table entry is consistent with the key carried in the key value storage request, if they are the same, read the value from the storage according to the storage address in the hash table entry and feed it back to the application server key value client 101.
  • multiple processing logics may be called in parallel to store different data in the same key-value storage server 102.
  • Each hash function corresponds to a set of processing logic, and the address of the storage space mapped by each processing logic corresponds to 4 buckets to form a set of buckets.
  • each key can correspond to 8 storage buckets for storing hash table entries; when updating the hash table entries, you can search the hash table entries stored in 8 storage buckets at the same time (keys pass a hash function
  • the hash table items stored in the mapped 4 buckets are a group of hash table items), find out the unoccupied buckets, and then use the polling mode to insert new hash table items; for each group of storage Buckets are inserted into the previous empty bucket first.
  • two groups can be inserted alternately.
  • a hash collision occurs, that is, when all 8 buckets corresponding to a key are occupied, a bucket is selected through preset rules, and some of the hash table entries are deleted to insert new hash table entries; for example, , Delete the earliest stored 10 hash table entries.
  • a polling mode can also be used to alternately delete the hash table entries in the two groups of buckets.
  • any application server key value client 101 can send a key to any key value storage server 102.
  • Value storage request may also communicate with the application server key-value client 101 through a key-value proxy (not shown in FIG. 1), and the key-value proxy is used to key the application server.
  • the key-value storage requests of the value client 101 are managed and scheduled, and distributed to different key-value storage servers. For example, the key-value agent can allocate key-value storage requests according to the current load of the key-value storage server.
  • the key-value proxy and the key-value storage server 102 can be deployed in one server, or can be deployed using a single server, or deployed in a cluster using multiple servers.
  • IP Internet protocol
  • the application server key-value client 101 can send a key-value storage request to the key-value proxy, and the key-value proxy is based on load balancing The algorithm distributes the key-value storage request to the key-value storage server 102.
  • variable-length key value storage method only involves the key hash operation, the hash table query, and the process of determining the hash table entry.
  • the embodiments of the present application are also applicable to variable-length key value storage methods.
  • additional fields are allowed in the key field.
  • variable-length keys are used to indicate different version numbers in scenarios such as data upgrades and snapshots. The value corresponding to the key will change to a certain extent.
  • the variable-length key is processed as follows: first perform a hash operation on the variable-length key, generate a hash value, query the hash table, and confirm Hash table entry.
  • the corresponding block address and the tree structure information stored in the block address are determined according to the hash table entry.
  • the parent node and the leaf node in the tree structure are determined according to the tree structure information, and the tree node may indicate the address of the storage space of the value corresponding to the variable-length key.
  • the tree structure may be a B-tree structure that identifies the size of the node, that is, in the tree structure, the value of the left leaf node is greater than the value of the parent node, and the value of the parent node is less than the value of the right leaf node.
  • the data storage methods provided in the embodiments of this application are all applicable. For ease of description, the following embodiments of this application only take fixed-length keys as an example for description.
  • the processor in the key-value storage server may be a central processing unit (CPU), an application specific integrated circuit (ASIC), and an on-site Field-programmable gate array (FPGA), artificial intelligence (AI) chip, system on chip (SoC) or complex programmable logic device (CPLD), of which,
  • the data required by the key-value storage server for example, the hash table and value, can be stored in the same memory or in different memories.
  • the hash table and value can be stored in the memory of the host.
  • the hash table can be stored in the memory of the host, and the value is stored in the flash memory.
  • the key-value storage server 201 is a key-value storage device constructed based on FPGA.
  • the key-value storage server 201 includes a key-value storage 202, The first storage 203; optionally, the key-value storage server 201 may also include a central processing unit 204.
  • the key value storage 202 includes a key value storage engine 205 and a second storage 206, and optionally, may also include a network offload engine (NOE) 207.
  • the second memory 206 may be a double data rate (DDR) memory, which may be specifically used to store a hash table.
  • DDR double data rate
  • the value is stored in the first memory 203.
  • the first memory 203 may be a memory in a host, and may be specifically implemented by a dynamic random access memory.
  • FPGA is a processing logic that uses hardware such as capacitors and resistors to implement specified functions.
  • the processing logic that implements specific functions in Figure 2 is named after the function name, for example, network offloading engine and key-value storage engine.
  • the key-value storage server 201 includes a central processing unit 204 and an FPGA-based key-value storage.
  • the same server including two or more types of processors it can also be called a heterogeneous server.
  • the network offloading engine 207 is used to receive the key-value storage request from the key-value client of the application server, process the key-value storage request based on the user datagram protocol (UDP), and send the processed key-value storage request to the key
  • the value storage engine 205 reduces the overhead incurred in the process of processing user datagram protocol messages by the CPU 204.
  • the CPU 204 can continue to complete the processing of the user datagram protocol.
  • the CPU 204 receives the key value storage request from the key value client of the application server, and based on the user data packet
  • the protocol processes the key-value storage request, and then sends the processed key-value storage request to the key-value storage engine 205.
  • the processing of the key-value storage request by the network offloading engine 207 in the key-value storage is taken as an example for description.
  • the specific processing process is to parse the message data, that is, to remove the header content of the protocol and obtain the payload data in the message.
  • the embodiment of the present application does not limit the process of the network offloading engine 207 or the CPU 204 processing the user data protocol message.
  • the key-value storage engine 205 is used to process the key-value storage request, for example, read the value from the first memory 203 according to the key in the key-value storage request or the value carried in the key-value storage request (corresponding to the value in the embodiment of this application).
  • the first key value of is written into the first memory 203.
  • the key-value storage engine 205 may add the value carried in the key-value storage request The first value is divided into multiple second values to generate multiple secondary key storage requests; where each secondary key value storage request carries a key and a second value; then, each secondary key value is stored separately The second value carried in the request is stored in the first memory 203.
  • the key-value memory 202 may also be connected to other processors, such as the CPU 204.
  • the key-value memory 202 may transfer the operation of data storage to the CPU 204.
  • the key-value storage 202 may directly forward the key-value storage request to the CUP 204; or after determining that the first value carried in the key-value storage request is greater than or equal to the first threshold, The key-value storage request is forwarded to CUP 204, and CUP 204 executes the data storage method provided in the embodiment of this application.
  • the structure of the key-value storage server shown in FIG. 2 is only an example, in which only one CPU and one key-value storage are shown. In fact, the embodiment of the present application does not limit the number of CPUs and key-value storages. One CPU or multiple key-value memories.
  • the key-value storage request can be forwarded between multiple key-value storages to cooperate with the data storage method provided in the embodiment of this application, and the data storage operation can also be selectively transferred to any CUP connected to the key-value storage device .
  • the embodiments of the present application do not limit the modules included in the key-value storage, the names of the modules, and the functions of each module. Any device that can implement the data storage method provided in the embodiments of the present application can be used as the key-value storage.
  • the method includes:
  • Step 301 The key-value storage receives a key-value storage request.
  • the key-value storage request carries a key and a first value, where the first value is a value corresponding to the key.
  • Step 302 The key value storage allocates a first storage space for the first value in the first storage.
  • a second storage space may be pre-configured in the first memory in the key-value storage for storing values.
  • the key-value storage allocates the storage space in the first storage for the first value, it may Priority is given to allocating storage space for the first value from the second storage space.
  • the size of the second storage space may be equal to or greater than the first value.
  • the second storage space may store the first value carried in one or more key-value storage requests.
  • the second storage space may be the storage space configured by the key-value storage from the first storage in advance after receiving the key-value storage request and before storing the first value carried in the key-value storage request,
  • the size of the second storage space may be equal to the first value or greater than the first value, so that the key value storage can flexibly allocate the first storage space for the first value from the second storage space.
  • the second storage space may be a storage space pre-configured by the key value storage from the first storage before the key value storage request is received. Since it is pre-configured before receiving the key-value storage request, the key-value storage can be pre-configured with a larger storage space as the second storage space, so that the key-value storage can store the first carried in multiple key-value storage key values. value.
  • the key-value memory can determine whether there is free space in the second storage space, and whether the free space can store the first value; if there is enough free space in the second storage space to store the first value , Select the free space from the second storage space as the first storage space.
  • the key value storage can be the first value from the first storage. Value allocation storage space.
  • the storage space allocated by the key-value storage for the first value from the first storage and the free space in the second storage space can be used to store the first value (such as storing one or Multiple second values). That is, the sum of the allocated storage space and the free space in the second storage space may be equal to the size of the first value.
  • the key value memory may also allocate a first storage space with a size equal to the size of the first value for the first value from the first storage.
  • the key value storage may also configure a third storage space from the first storage, and the third storage space is larger than The storage space required for the first value.
  • the third storage space may be a relatively large storage space such as 1G or 500M.
  • the third storage space can also be used to store the first value carried in the key-value storage request subsequently received by the key-value storage.
  • the key-value storage allocates the first value in the third storage space. The first storage space.
  • the key-value storage may not be configured with the second storage space in the first storage in advance, and when data storage is required, the first storage space is dynamically allocated in the first storage for the first value.
  • the key value storage may allocate the first storage space in the first storage according to the size of the first value.
  • the size of the first storage space is equal to the size of the storage space required for the first value as an example.
  • the embodiment of the present application does not limit the size of the first storage space.
  • the maximum threshold of the first value carried in the key-value storage request may be set. In other words, the maximum threshold is used to represent the maximum size of the value carried in the key-value storage request.
  • the key value storage allocates the first storage space in the first storage for the first value
  • the allocated first storage space may be equal to the maximum threshold.
  • the key-value memory stores the second value, it can store the second value in the allocated first storage space, and the storage location of the second value in the first storage space can be flexibly set, so that the way of storing the second value is more flexible .
  • the first storage space may be a continuous storage space or a discontinuous storage space, which is not limited in the embodiment of the present application.
  • Step 303 When the capacity of the data contained in the first value is greater than or equal to the first threshold, the key value store divides the first value into multiple second values, and generates multiple secondary key value storage requests, each secondary The key-value storage request carries the key and a second value.
  • the number of secondary key value storage requests is the same as the number of multiple second values obtained by dividing the first value.
  • the embodiment of the present application does not limit the sequence of step 302 and step 303, and can be performed simultaneously or sequentially.
  • the key value storage first caches the first value in the second storage. After that, the first value is transferred to the first memory, which will lead to a longer time for data storage.
  • the capacity of the data contained in the first value can also be referred to as the size of the first value, which refers to the total size of the data contained in the first value (size), or it can be said to refer to the storage space required to store the first value.
  • the size of the minimum storage space For example, if the content contained in the first value is "0123456789", the capacity of the data contained in the first value is 10 bytes.
  • the first threshold can be set here.
  • the first value is considered to be larger and belongs to large-size data; otherwise, the first value belongs to small-size data.
  • the specific value of the first threshold can be an empirical value, such as 1 mega (M), or it can be set according to specific scenarios and key-value storage performance. For example, in some key-value storage scenarios, the size of the value that needs to be stored in a period of time can be counted, and the first threshold can be determined according to the size of the value that needs to be stored in the statistics.
  • the storage efficiency of the key value memory is significantly slowed down, and 5M may be used as the first threshold.
  • the foregoing method of setting the threshold is only an example, and the embodiment of the present application does not limit the specific value and setting method of the threshold.
  • the embodiment of the present application proposes a method of data partitioning.
  • the first value is divided into multiple second values.
  • the second value is stored in the first memory in stages, and the key value memory is Complete the positive sequence, storage and merging process of multiple second values.
  • the embodiment of the present application does not limit the specific implementation of data segmentation, and any method that can finally divide the first value into multiple smaller second values is applicable to the embodiment of the present application.
  • FIG. 4 it is a schematic diagram of generating multiple secondary key value storage requests and merging after dividing the first value, which is divided into two different processes, the slicing process and the splicing process.
  • the key-value storage can divide the first value carried in the key-value storage request into multiple second values, and generate multiple secondary key-value storage requests.
  • Each secondary key-value storage request includes a message Header and a second value (the format of the secondary key value storage request will be explained later);
  • the splicing process is the process in which the key value storage reads the third value from the first storage, which can be seen as the inverse process of the sharding process.
  • the third threshold can be equal to the second threshold, or other values
  • the third threshold is 10M
  • the key value memory reads multiple fourth values from the first memory, and generates After multiple secondary key-value storage requests, multiple fourth values are combined into a third value, and the third value is carried in the key-value storage response that needs to be fed back to the key-value client of the application server.
  • FIG. 5 a schematic diagram of the format of a key-value storage request provided by an embodiment of the present application.
  • the key-value storage request includes two parts, one is a header and the other is a packet.
  • the message header carries some attribute information of keys and values.
  • the message header includes key identifier (k ID), type (type), key size (key size), value total size, value size, and value offset (value offset) ;
  • the data packet includes keys and values.
  • the key identifier is used to indicate the serial number of the key.
  • the type is used to indicate the operation type of the key-value storage request, for example, it can be insert (put), search (get), and delete (delete).
  • the insertion instruction key value storage request is a write request, and the value carried in the key value storage request (such as the first value in the embodiment of this application) needs to be stored in the first memory;
  • the search instruction key value storage request is read To fetch the request, the value corresponding to the key carried in the key-value storage request needs to be read from the first storage;
  • the deletion indicates that the value corresponding to the key carried in the key-value storage request needs to be deleted from the first storage.
  • the above three operation types are only examples, and the embodiments of the present application are not limited, and other operation types, such as update, may also exist.
  • the key size is used to indicate the size of the key carried in the key-value storage request.
  • the total value size is used to indicate the total size of the value carried in the key-value storage request (the corresponding key-value storage request is a write request, and the type is put), or the total size of the value obtained by the key-value storage request (the corresponding key-value storage request is Read request, the type is get), or the total size of the value deleted by the key value storage request (corresponding to the key storage request of type delete).
  • the value offset is used to indicate the offset of the value carried in the key value storage request (such as the second value in the embodiment of the present application) relative to the first position of the total value (corresponding to the first value in the embodiment of the present application).
  • the value offset may indicate the position of the second value in the first value. The larger the value offset, the lower the position of the second value in the first value.
  • the value size is used to indicate the size of the value (corresponding to the first value or the second value in the embodiment of the present application) carried in the key value storage request. Exemplarily, if the total length of a value is 2024, it is divided according to 1024. After the division, for the first value (such as the second value in the embodiment of this application), the value offset is 0 and the value size is 1024; The second value, the value offset is 1024, and the value size is 1000.
  • the format of a secondary key value storage request provided by the embodiment of the present application is similar to the key value storage request shown in FIG. 5, which can be referred to FIG. 5.
  • the secondary key value storage request can also be divided into two parts, one is the message header, and the other is the data packet.
  • the message header carries some attribute information of keys and values. Specifically, the message header includes key identifiers, types, key sizes, total value sizes, value sizes, and value offsets; data packets include keys and values.
  • the type of the key-value storage request is insert as an example, and correspondingly, the type in the secondary key-value storage request is also insert.
  • the total value size indicates the size of the value before data segmentation, that is, the size of the first value carried in the key value storage request.
  • the value size indicates the size of the value carried in the secondary key value storage request (corresponding to the second value in the embodiment of the present application).
  • the value offset indicates the offset position of the value carried in the secondary key value storage request (such as the second value in the embodiment of this application) in the value (corresponding to the first value in the embodiment of this application) in the key value storage request.
  • the value offset in the secondary key value storage request may be 0, indicating that the second value carried in the secondary key value storage request is divided from the starting position of the second value in the first value of.
  • the value offsets in the secondary key value storage requests that carry different second values are different, and the value offsets can reflect the position order of the second value in the first value.
  • the type may also indicate whether the key value storage request is a key value storage request before data segmentation or a secondary key value storage request formed after data segmentation.
  • an identification bit can be added to the type to indicate whether the key-value storage request is a key-value storage request before data segmentation or a secondary key-value storage request.
  • a field can be added to the key-value storage request as a slice, which is used to indicate whether the request is a key-value storage request before data segmentation or a secondary key. Value storage request.
  • the fragment has the first value (such as null), it indicates that the request is a key-value storage request before data segmentation, and when the fragment has the second value (such as 1), it indicates that the request is a secondary key-value storage request.
  • the key value memory can divide the first value into five second values, and the size of each second value is 1M. In this way, five secondary key value storage requests can be generated. Each secondary key value storage request carries a second value.
  • the value size is 1M, Indicates that the size of the second value carried is 1M, and the value offset can be set to 0, indicating that the second value is divided from the starting position of the first value; for the secondary key value storage that carries the second second value
  • the size of the value in the request is 1M
  • the size of the second value carried in the instruction is 1M
  • the value offset can be set to 1M, indicating that the second value is segmented from the 1M position of the first value.
  • the secondary key value storage request may set a field to indicate the order of the carried second value in the first value more intuitively.
  • a fragment field can be used.
  • 1 indicates that the key-value storage request is a secondary key-value storage request, and the second value carried in the secondary key-value storage request is the first value after the first value is split.
  • Binary value indicates that the key value storage request is a secondary key value storage request, and the second value carried in the secondary key value storage request is a second value whose position order is second after the first value is divided.
  • the above indication of using the fragment field is only an example, for example, other fields may be added, which is not limited in the embodiment of the present application.
  • the value in the data packet in the secondary key-value storage request is part of the value in the key-value storage request.
  • the value in the data packet in the secondary key-value storage request is the second value
  • the key-value storage The value in the data packet in the request is the first value.
  • the key identifier, type, key size, and total value of the message header in the secondary key-value storage request are the same as the content of the corresponding field in the key-value storage request.
  • the key and key value in the data packet in the secondary key-value storage request The keys in the storage request are the same.
  • Method 1 In the process of caching the first value, partition while caching.
  • the key value storage may first buffer the first value.
  • the caching process when the size of the cached data reaches the second threshold, the cached data is taken as a second value, and a corresponding secondary key value storage request is generated.
  • the value size field can be set to the size of the second value
  • the value offset is the position of the second value in the first value, indicating the offset of the second value relative to the first value of the first value.
  • the second value carried in the secondary key value storage request can be stored in the first storage space allocated for the first value, because the second value is only a part of the first value , The second value only takes up part of the storage space.
  • the key-value storage will continue to cache the remaining first value, when the size of the cached data reaches the second threshold (the previously cached data has been carried in the secondary key-value storage request, in this case, The size of the cached data starts from 0), the cached data is used as a second value; the corresponding secondary key request is generated, and the same operation as before is performed, including setting the secondary storage key request, and setting the secondary key
  • the second value carried in the value storage request is stored in the first storage space allocated for the first value; when the second value is stored, because a second value is already stored in the first storage space allocated for the first value. Value, the second value carried in the secondary key value storage request is stored in the storage space after the stored second value to achieve sequential storage.
  • the above process is looped until the first value is cached. It should be noted that when the first value is cached in the final stage, the first value may be cached, but the cached data does not reach the second threshold. In this case, the key-value storage may directly use the cached data as a second value, generate a corresponding secondary key request, and store the second value in the secondary storage key request into the first storage. In this case, it can be found that the sizes of the divided second values are not completely the same, and there may be a second value whose size is smaller than the second threshold.
  • step 303 and step 304 are performed alternately, and this method is only an example.
  • the operation of storing the second value can be executed after the first value is cached and all secondary key value storage requests (step 303) are generated, which is not limited in this embodiment of the application.
  • the key-value memory may adopt a state machine control mode.
  • Fig. 8 is a schematic diagram of a state machine provided by an embodiment of the application.
  • the finite state machine can be set in 6 states, namely, idle state (IDLE), waiting command (WAIT_CMD), and waiting descriptor (WAIT_DSC) , Wait for data (WAIT_DATA), generate descriptor (GEN_DSC), complete (FINISH) state, the state transition diagram is shown in Figure 8.
  • IDLE idle state
  • WAIT_CMD waiting command
  • WAIT_DSC waiting descriptor
  • WAIT_DATA Wait for data
  • G_DSC generate descriptor
  • complete complete
  • IDLE state When the first value is received, jump to WAIT_CMD state; otherwise, keep in IDLE state.
  • WAIT_CMD state Waiting to receive insert command. In this state, if the type of the key-value storage request is parsed by the key-value store, it will jump to the IDLE state, and the first value received is invalid and discarded; if the type of the key-value storage request is parsed, it will jump to WAIT_DSC State; otherwise, keep in WAIT_CMD state.
  • WAIT_DSC state waiting for receiving card to host (card to host, C2H) descriptor.
  • the C2H descriptor is used to indicate the size of the first value and the first storage space allocated by the key value storage for the first value. After the key value storage receives the key value storage request and allocates the first storage space for the first value, it may generate the C2H descriptor.
  • the key value memory determines the C2H descriptor, it jumps to the WAIT_DATA state; otherwise, it remains in the WAIT_DSC state.
  • WAIT_DATA state Waiting for the first value to be buffered to one frame (taking the second threshold as one frame as an example). When the first value buffer reaches one frame, the buffered data is used as the second value.
  • the first value is cached for one frame, and the first value has not been cached, that is, it is not the last frame, jump to the GEN_DSC state; when the last frame of the first value is cached, jump to the FINISH state; otherwise , Remain in the WAIT_DATA state; here, taking the size of the second value as one frame as an example, the embodiment of the present application does not limit the size of the second value.
  • GEN_DSC state Generate C2H fragment descriptor.
  • the key value memory can start a data transfer and transfer the second value to the first storage space.
  • the C2H fragment descriptor is used to indicate the size of the second value, the storage address of the second value, or the source address of the second value, etc., where the source address of the second value is the address of the key value memory cache second value, such as the key
  • the value memory caches the address of the second value in the second memory; it should be noted that caching the second value here means that when the key value memory caches the first value, the cached data is used as the second value.
  • the key value memory may store the second value carried in the corresponding secondary key value storage request in the first storage space according to the information indicated by the C2H fragment descriptor.
  • the storage address indication of the second value is a storage space for storing the second value.
  • the key value memory may determine the location of the storage space in the first storage space according to the size of the second value and the position of the second value in the first value, and determine the storage address of the second value.
  • the storage space for storing the second value is one frame from the starting position in the first storage space
  • the storage space can then determine the storage address of the second value; if the second value currently cached in the key-value memory is the third frame of the first value, the storage space used to store the second value is 3M in the first storage space
  • the storage space starting from the position and the size is one frame, and then the storage address of the second value is determined.
  • the corresponding secondary key value storage request is generated, and then it jumps to the WAIT_DATA state; otherwise, it remains in the GEN_DSC state.
  • FINISH state Generate the last piece of C2H fragment descriptor (that is, cache the last frame of the first value, and generate the last second value), marking the completion of the cache operation of the first value.
  • the first value When the first value is completely transmitted, it jumps to the IDLE state; otherwise, it remains in the FINISH state.
  • the embodiment of the present application is not a specific value of the second threshold, but may be an empirical value, or may be set according to a specific scenario, and the embodiment of the present application is not limited.
  • Method 2 After the first value is cached, segmentation is performed.
  • the key value memory can also divide the first value into multiple second values after fully buffering the first value.
  • the key-value storage may evenly divide the first value into a plurality of second values with the same size.
  • the second value is less than the second threshold.
  • the key-value memory may also randomly divide the first value, and the size of each second value after division is different.
  • each secondary key value storage request carries one of the second values
  • the message in each secondary key value storage request The value offset in the header indicates the position of the second value carried in the first value.
  • Step 304 The key-value storage stores the second value carried in each secondary key-value storage request to the first storage according to the order in which the second value carried in each secondary key-value storage request is located in the first value space.
  • the key value storage may store the second value carried in the secondary key value storage request in the order of the position of the second value in the first value, in the order from front to back in the order of position.
  • the position of the second value carried in the secondary key value storage request in the first value can be determined according to the value offset in the header of each secondary key value storage request, and each secondary key value can be The value offsets in the message header in the value storage request are sorted in order of magnitude, from small to large, and the second value carried in the corresponding secondary key value storage request is sequentially stored in the first storage space.
  • the key value storage when the key value storage stores the second value carried in the secondary key value storage request.
  • the storage position of the second value in the first storage space may be determined according to the position of the second value in the first value.
  • the second value is stored in the storage location in the first storage space, and the location of the second value in the first value can be determined by the value offset in the message header of the secondary key value storage request.
  • the value offset in the header of the secondary key value storage request is 0, it indicates that the second value carried in the secondary key value storage request is located at the beginning of the first value, and the second value is in the first value.
  • the storage location of a storage space is a storage space with a size of a second value starting from the starting point of the storage space.
  • the value offset in the message header of the secondary key value storage request is 1M, which indicates that the second value carried in the secondary key value storage request starts at the 1M position of the first value, and the second value is in the first storage space
  • the storage location starts from the location corresponding to the 1M data that is stored empty, and the size is the second value size storage space.
  • the key value storage After storing the second value in all secondary key value storage requests in the first storage space, the key value storage ends the data storage operation.
  • the key value memory judges whether the total size of the second value stored in the first storage space is consistent with the size of the first value, and determines whether the first value is stored through the judgment result; if the key value memory can query the first storage space For the stored second value, if the total size of the stored second value is consistent with the size of the first value, it is determined that the storage of the first value is completed.
  • the key value storage may accumulate the key value size in the header of each stored secondary key value storage request to generate an accumulated value until the accumulated value is equal to the total size of the key value. Determine whether the storage of the first value is complete, and end the data storage operation.
  • the embodiment of the application also provides a key-value memory for executing the method of the key-value memory in the above-mentioned method embodiment.
  • the device includes a receiving unit 901, a distributing unit 902, a dividing unit 903, and a storage unit 904:
  • the receiving unit 901 is configured to receive a key-value storage request.
  • the key-value storage request includes a key and a first value, where the key corresponds to the first value.
  • the allocating unit 902 is configured to allocate a first storage space for the first value in the first storage, and the first storage is a storage other than the key value storage.
  • the dividing unit 903 is configured to divide the first value into multiple second values when the total length of the first value is greater than or equal to the first threshold, and generate multiple secondary key value storage requests, each secondary key value
  • the storage request carries the key and a second value, and the number of secondary key value storage requests is the same as the number of multiple second values obtained by dividing the first value.
  • the storage unit 904 is configured to store the second value carried in each secondary key value storage request in the first storage according to the position order of the second value carried in each secondary key value storage request in the first value space.
  • the dividing unit 903 when it divides the first value into multiple second values, it may divide while buffering.
  • the dividing unit 903 in the process of caching the first value, the dividing unit 903 may use the cached data as the second value whenever the size of the cached data is equal to the second threshold, and carry the second value in a secondary key. In the value storage request, until the first value cache is completed.
  • the remaining data size of the last first value is less than the second threshold, you can directly use the cached data as the second value and carry it in a secondary key value storage request; in this case ,
  • the size of the second value carried in each secondary key value storage request may not be exactly the same.
  • the dividing unit 903 when the dividing unit 903 divides the first value into multiple second values, it may also use a method of data division after the buffering is completed; for example, the dividing unit 903 may perform data division after the first value is buffered.
  • the first value is divided into multiple second values; for example, the dividing unit 903 may divide the first value into N+1 second values, and among the multiple second values arranged in the order of division, the first N second values
  • the size of can be equal to the second threshold, the last second value, that is, the size of the N+1th second value can be less than or equal to the second threshold, and N is a positive integer greater than or equal to 1.
  • the dividing unit 903 may also set the second values to be multiple second values.
  • the secondary key value storage request may include some fields for indicating related information of the second value carried, such as the size and offset of the second value.
  • a value offset may be included, and the value offset may indicate the position of the second value carried in the secondary key value storage request in the first value.
  • the storage unit 904 stores the second value carried in each secondary key value storage request to the first value in the order of the second value carried in each secondary key value storage request.
  • the position order needs to be determined first, and then the second value is stored; for example, the storage unit 904 may determine each secondary key value storage request according to the value offset in each secondary key value storage request The position order of the second value carried in the first value in the first value; after that, the second value is stored in the first storage space according to the position order.
  • the second storage for the first value carried in one or more key value storage requests may be pre-configured in the first storage.
  • Storage space The storage unit 904 may allocate the first storage space for the first value from the second storage space.
  • the size of the first storage space may be equal to or greater than the first value.
  • the storage unit 904 may pre-configure the second storage space before the receiving unit 901 receives the key-value storage request; it may also pre-configure the second storage space after the receiving unit 901 receives the key-value storage request and before the storage unit 904 stores the first value.
  • the size of the second storage space can be equal to or greater than the size of the first value.
  • the storage unit 904 when the storage unit 904 allocates storage space for the first value in the first storage, if the size of the free space in the second storage space is less than the size of the first value, the storage unit 904 may not store the first value.
  • the first storage space is allocated for the first value from the first storage.
  • the storage unit 904 may also determine whether the first value has been stored; for example, when the total size of the second value stored in the first storage space is equal to the size of the first value, the storage unit 904 may determine The storage of the first value has been completed.
  • the storage unit 904 may accumulate the secondary key value corresponding to the stored second value Store the size of the requested value until the accumulated value is equal to the size of the first value, and it is determined that the first value has been stored.
  • the division of units in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the foregoing embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination.
  • the above-mentioned embodiments may be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more sets of available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium.
  • the semiconductor medium may be a solid state drive (SSD).
  • FIG. 10 is a schematic diagram of a key value storage 1000 according to an embodiment of the application.
  • the key value storage 1000 includes a processor 1001 and a memory 1002.
  • the key value storage 1000 may further include a communication interface 1003.
  • the number of the memory 1002 and the communication interface 1003 does not constitute a limitation to the embodiment of the present application, and can be configured arbitrarily according to business requirements during specific implementation.
  • the memory 1002 may be a volatile memory, such as a random access memory; the memory may also be a nonvolatile memory, such as a read-only memory, flash memory, hard disk drive (HDD) or solid-state drive (SSD) Or the memory 1002 is any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.
  • the memory 1002 may be a combination of the above-mentioned memories.
  • the memory 1002 may be the second memory in the foregoing method embodiment.
  • connection medium between the foregoing processor 1001 and the memory 1002 is not limited in the embodiment of the present application.
  • the processor 1001 may be a CPU, and the processor 1001 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate array (FPGA) or other programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, artificial intelligence chips, chips on a chip, etc.
  • the general-purpose processor may be a microprocessor or any conventional processor. It has a data transceiver function and can communicate with other devices.
  • an independent data transceiver module such as a communication interface 1003, can be used to send and receive data; when the processor 1001 communicates with other devices, it can Data transmission is performed through the communication interface 1003, such as receiving a key-value storage request from an application server key-value client.
  • the processor 1001 in FIG. 10 can call the computer-executed instructions stored in the memory 1002, so that the key-value memory can execute the key-value memory execution in any of the foregoing method embodiments. The steps of the method.
  • the processor 1001 of the key-value storage in FIG. 10 may implement the functions of the key-value storage engine 205 and the network offloading engine 207 in FIG. 2.
  • the functions/implementation processes of the receiving unit, the allocation unit, the dividing unit, and the storage unit in FIG. 9 can all be implemented by the processor 1001 in FIG. 10 calling the computer execution instructions stored in the memory 1002.
  • the function/implementation process of the allocation unit, the division unit, and the storage unit in FIG. 9 can be implemented by the processor 1001 in FIG. 10 calling the computer execution instructions stored in the memory 1002, and the function/implementation of the receiving unit in FIG. 9
  • the process can be implemented through the communication interface 1003 in FIG. 10.
  • the embodiment of the present application also provides a key-value storage server.
  • the structure of the key-value storage server can be seen in FIG. 2.
  • the key-value storage server may include a key-value storage and a first storage;
  • the memory is a memory other than the key value memory;
  • the key value memory is used to execute the method executed by the key value memory in the foregoing method embodiment, and the first memory is used to store the first value; optionally, it may also include a central processing unit.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé de stockage de données, une mémoire et un serveur. Le procédé spécifique comprend les étapes suivantes : une mémoire de valeur clé reçoit une demande de stockage de valeur clé, la demande de stockage de valeur clé comprenant une clé et une première valeur; lorsque la première valeur correspond à des données volumineuses, la mémoire de valeur clé segmente la première valeur en une pluralité de secondes valeurs, et génère une pluralité de requêtes de stockage de valeur clé secondaires, chaque demande de stockage de valeur clé secondaire portant la clé et l'une des secondes valeurs; puis, selon la séquence de position, dans la première valeur, de la seconde valeur transportée dans chaque demande de stockage de valeur clé secondaire, stocke dans un premier espace de stockage la seconde valeur transportée dans chaque demande de stockage de valeur clé secondaire, ce qui permet de résoudre le problème de faible efficacité de stockage de données de données volumineuses dans un système de stockage de valeur clé.
PCT/CN2020/075170 2019-03-29 2020-02-14 Procédé de stockage de données, mémoire et serveur WO2020199760A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910252054.X 2019-03-29
CN201910252054.XA CN110083307A (zh) 2019-03-29 2019-03-29 数据存储方法、存储器和服务器

Publications (1)

Publication Number Publication Date
WO2020199760A1 true WO2020199760A1 (fr) 2020-10-08

Family

ID=67413914

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/075170 WO2020199760A1 (fr) 2019-03-29 2020-02-14 Procédé de stockage de données, mémoire et serveur

Country Status (2)

Country Link
CN (1) CN110083307A (fr)
WO (1) WO2020199760A1 (fr)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110083307A (zh) * 2019-03-29 2019-08-02 华为技术有限公司 数据存储方法、存储器和服务器
CN111177025B (zh) * 2019-12-30 2022-05-17 深圳云天励飞技术股份有限公司 数据存储方法、装置及终端设备
CN114077490A (zh) * 2020-08-17 2022-02-22 富泰华工业（深圳）有限公司 数据处理方法、装置、电子设备及存储介质
CN112511450B (zh) * 2020-11-02 2022-05-31 杭州迪普信息技术有限公司 一种流量控制设备与方法
CN113609090B (zh) * 2021-08-06 2024-06-18 杭州网易云音乐科技有限公司 数据存储方法及装置、计算机可读存储介质、电子设备
CN113742555A (zh) * 2021-09-10 2021-12-03 北京锐安科技有限公司 热点侦测方法、装置、侦测服务器、热点侦测系统及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120284317A1 (en) * 2011-04-26 2012-11-08 Dalton Michael W Scalable Distributed Metadata File System using Key-Value Stores
CN102971732A (zh) * 2010-07-02 2013-03-13 华为技术有限公司 键/值存储器的集成分级查询处理的系统结构
CN106155586A (zh) * 2016-05-31 2016-11-23 华为技术有限公司 一种存储方法、服务器及存储控制器
CN110083307A (zh) * 2019-03-29 2019-08-02 华为技术有限公司 数据存储方法、存储器和服务器

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102971732A (zh) * 2010-07-02 2013-03-13 华为技术有限公司 键/值存储器的集成分级查询处理的系统结构
US20120284317A1 (en) * 2011-04-26 2012-11-08 Dalton Michael W Scalable Distributed Metadata File System using Key-Value Stores
CN106155586A (zh) * 2016-05-31 2016-11-23 华为技术有限公司 一种存储方法、服务器及存储控制器
CN110083307A (zh) * 2019-03-29 2019-08-02 华为技术有限公司 数据存储方法、存储器和服务器

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHU, XING: "Non-official translation: Redis Big Key Partition Scheme", 27 August 2017 (2017-08-27), Retrieved from the Internet <URL:https://blog.csdn.net/foreverling/article/details/77612060> DOI: 20200423145148Y *

Also Published As

Publication number Publication date
CN110083307A (zh) 2019-08-02

Similar Documents

Publication Publication Date Title
WO2020199760A1 (fr) Procédé de stockage de données, mémoire et serveur
US11467975B2 (en) Data processing method and NVMe storage device
CN110555001B (zh) 数据处理方法、装置、终端及介质
US20160132541A1 (en) Efficient implementations for mapreduce systems
CN110119304B (zh) 一种中断处理方法、装置及服务器
US9307024B2 (en) Efficient storage of small random changes to data on disk
EP4318251A1 (fr) Système et procédé d&#39;accès à des données et dispositif et carte réseau
EP3077914B1 (fr) Système et procédé de gestion et de soutien hôte virtuel adaptateur de bus (vhba) sur infiniband (ib) et pour soutenir efficace tampon usage avec une interface de mémoire externe unique
US11977548B2 (en) Allocating partitions for executing operations of a query
CN117312201B (zh) 一种数据传输方法、装置及加速器设备、主机和存储介质
WO2023125630A1 (fr) Procédé de gestion de données et appareil associé
CN116795736A (zh) 数据预读取方法、装置、电子设备和存储介质
CN106294191B (zh) 处理表的方法、访问表的方法和装置
WO2017177400A1 (fr) Procédé et système de traitement de données
US8898353B1 (en) System and method for supporting virtual host bus adaptor (VHBA) over infiniband (IB) using a single external memory interface
CN115904211A (zh) 一种存储系统、数据处理方法及相关设备
CN111125011B (zh) 一种文件处理方法、系统及相关设备
WO2019201091A1 (fr) Procédé et dispositif de traitement de données et support de stockage lisible par ordinateur
WO2024113090A1 (fr) Procédé, appareil et système d&#39;accès mémoire
WO2022222523A1 (fr) Procédé et appareil de gestion de journal
US20230105067A1 (en) Metadata Processing Method in Storage Device, and Related Device
WO2024055679A9 (fr) Procédé, appareil et système de stockage de données, puce et dispositif d&#39;accélération
US20160292096A1 (en) Packet processing system, method and device to optimize packet buffer space
US20150154142A1 (en) System and method for managing virtual host bus adaptor (vhba) over infiniband (ib) using a single external memory interface
CN116610598A (zh) 一种数据存储系统、数据存储方法、装置及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20783720

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20783720

Country of ref document: EP

Kind code of ref document: A1