CN111752480A - Data writing method, data reading method, related equipment and system - Google Patents

Data writing method, data reading method, related equipment and system Download PDF

Info

Publication number
CN111752480A
CN111752480A CN202010420635.2A CN202010420635A CN111752480A CN 111752480 A CN111752480 A CN 111752480A CN 202010420635 A CN202010420635 A CN 202010420635A CN 111752480 A CN111752480 A CN 111752480A
Authority
CN
China
Prior art keywords
data
read
storage node
written
remote storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010420635.2A
Other languages
Chinese (zh)
Inventor
张羽
郭海涛
俞柏峰
陈昊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202010420635.2A priority Critical patent/CN111752480A/en
Publication of CN111752480A publication Critical patent/CN111752480A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The embodiment of the invention discloses a data writing method, a data reading method, related equipment and a system. The data writing method comprises the following steps: receiving a write request aiming at a remote storage node and initiated by an application program calling a key value storage interface for writing data, wherein the write request carries a key value pair consisting of a key and a value, the key is used for indicating a destination address of the data to be written, and the value is used for indicating a source address of the data to be written and the size of the data to be written; generating a write command conforming to a high-speed network storage protocol according to the key value pair; the write command includes: the data writing device comprises a first field used for indicating a destination address of data to be written, a second field used for indicating a source address of the data to be written, and a third field used for indicating the size of the data to be written; and sending the write command to the remote storage node through the interface of the high-speed network storage protocol. The scheme can be compatible with the respective storage advantages of the key value storage and the high-speed network storage protocol.

Description

Data writing method, data reading method, related equipment and system
Technical Field
The present invention relates to the field of storage, and in particular, to a data writing method, a data reading method, and related devices and systems.
Background
With the rapid development of information technology, mass storage of data poses new requirements and challenges for storage system systems. To address this challenge, a memory management technique for Key-Value storage (i.e., KV storage for short) has been developed. KV storage can meet the requirements for efficient storage and access of mass data. In addition, the Non-Volatile Memory Express (NVMe) has been supported by many flash Memory, server and storage vendors as an efficient storage protocol, and its ecology is becoming mature. Meanwhile, a more efficient network storage protocol is called along with the evolution of the flash memory technology, namely NVMe Over Fabric. The NVMe Over Fabric replaces iSCSI (e.g., iSER under Linux or SMB3 under Windows) used by the former front-end, so that the end user can directly realize fast and low-delay access to the remote NVMe device through the NVMe Over Fabric protocol.
Based on the advantage that the NVMe Over Fabric supports remote storage and the advantage that the KV storage management technology supports high-efficiency mass data storage management, the compatibility of the KV interface and the NVMe Over Fabric plays an important role in the future storage field, and the method has good and wide market prospects.
Disclosure of Invention
Embodiments of the present invention provide a data writing method, a data reading method, a related device, and a system, which can implement that a key value storage interface is carried on a high-speed network storage protocol for network storage, and are compatible with respective storage advantages of the key value storage and the high-speed network storage protocol.
In a first aspect, a data writing method is provided, which is applied to a host side, and includes:
receiving a write request aiming at a far-end storage node, wherein the write request carries a key value pair consisting of a key and a value, generating a write command conforming to a high-speed network storage protocol according to the key value pair, finally sending the write command to the far-end storage node through an interface of the high-speed network storage protocol, triggering the far-end storage node to respond to the write command, and writing the data to be written stored at the source address into a physical address mapped by the destination address.
Specifically, the write request is a request initiated by an application program by calling a key value storage interface for writing data, such as KV _ write (); the input parameters of the key value storage interface for writing data comprise the key value pair, wherein the key is used for indicating a destination address of data to be written, and the value is used for indicating a source address of the data to be written and the size of the data to be written.
Specifically, the write command may include: the data writing device comprises a first field used for indicating the destination address of the data to be written, a second field used for indicating the source address of the data to be written, and a third field used for indicating the size of the data to be written.
It can be understood that, since the key-value pair includes values of the first field, the second field and the third field in the write command, the host is able to fill the corresponding field of the write command according to the key-value pair carried by the write request.
By implementing the data writing method described in the first aspect, it is possible to implement that the key value storage interface for writing data is carried on the high-speed network storage protocol for network storage, so that an application program can implement data writing operation on the remote storage node through the key value storage interface, and respective storage advantages of the key value storage interface and the high-speed network storage protocol are compatible.
In a second aspect, a data writing method is provided, which is applied to a remote storage node side, and includes:
receiving a write command which is sent by a host and accords with a high-speed network storage protocol through an interface of the high-speed network storage protocol, storing a destination address of the data to be written and metadata of the data to be written in a target database in a target cache region in a one-to-one correspondence manner, analyzing a physical address mapped by the destination address, and then responding to the write command and writing the data to be written in the physical address mapped by the destination address.
Specifically, the write command may include: the data writing device comprises a first field used for indicating a destination address of data to be written, a second field used for indicating a source address of the data to be written, a third field used for indicating the size of the data to be written and a fourth field used for indicating metadata of the data to be written. The target database provides metadata access interfaces, such as add, delete, change, check, etc., to the host.
The method for writing data described in the second aspect can also be implemented to bear a key value storage interface for writing data on the high-speed network storage protocol for network storage, so that an application program can implement data writing operation on the remote storage node through the key value storage interface, and the respective storage advantages of the key value storage interface and the high-speed network storage protocol are compatible.
Moreover, it can be understood that the target database provided by the invention can realize that: the host accesses the metadata of the data to be written directly from the memory of the remote storage node (namely, the target database), and does not need to search the metadata of the data to be written on the storage hard disk block by block, thereby realizing the high-efficiency access and management of the metadata of the data written in the storage hard disk.
In one implementation, after the remote storage node writes the data to be written into the physical address mapped by the destination address, the remote storage node may return a write response to a host through an interface of the high-speed network storage protocol.
Correspondingly, the host can receive the write response returned by the remote storage node through the interface of the high-speed network storage protocol.
In a specific implementation, after the host receives the write Response, the host may convert the data format of the write Response into a key value storage interface for receiving the write Response, for example, KV _ Get _ Response (), which can be identified, so that the application program identifies the write Response by calling the key value storage interface.
In one implementation, after receiving the write command sent by the host, the remote storage node may send a get request for the data to be written to the host, where the get request may include a source address of the data to be written.
Correspondingly, the host receives an acquisition request for the data to be written, which is sent by the remote storage node.
Then, the host may send the data to be written stored at the source address to a remote storage node through an interface of the high-speed network storage protocol in response to the obtaining request.
Correspondingly, the remote storage node can receive the data to be written sent by the host through the interface of the high-speed network storage protocol.
In a third aspect, a data reading method is provided, which is applied to a host side, and includes:
receiving a read request aiming at a remote storage node, wherein the read request carries a key value pair consisting of a key and a value, generating a read command conforming to a high-speed network storage protocol according to the key value pair, finally sending the read command to the remote storage node through an interface of the high-speed network storage protocol, triggering the remote storage node to respond to the read command, and reading the data to be read from a physical address mapped by the source address.
Specifically, the read request is a request initiated by an application program by calling a key-value storage interface for reading data, for example, KV _ read (), where input parameters of the key-value storage interface for reading data include the key-value pair; the key is used for indicating a source address of data to be read, and the value is used for indicating a destination address of the data to be read and the size of the data to be read;
specifically, the read command may include: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read, and a third field used for indicating the size of the data to be read.
It can be understood that, since the key-value pair includes the values of the first field, the second field and the third field in the read command, the host is able to fill the corresponding field of the read command according to the key-value pair carried by the read request.
By implementing the data reading method described in the first aspect, it is possible to implement that a key value storage interface for reading data is carried on the high-speed network storage protocol for network storage, so that an application program can implement data reading operation on the remote storage node through the key value storage interface, and respective storage advantages of the key value storage interface and the high-speed network storage protocol are compatible.
In a fourth aspect, a data reading method is provided, which is applied to a remote storage node side, and includes:
and receiving a read command which is sent by a host and accords with the high-speed network storage protocol through an interface of the high-speed network storage protocol, analyzing a physical address mapped by the source address, and then responding to the read command to read the data to be read from the physical address mapped by the source address.
Specifically, the read command may include: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read, and a third field used for indicating the size of the data to be read.
The same beneficial effects as those of the implementation of the data reading method described in the third aspect are obtained, and by implementing the data reading method described in the fourth aspect, the key value storage interface for reading data can be similarly carried on the high-speed network storage protocol for network storage, so that an application program can implement data reading operation on the remote storage node through the key value storage interface, and respective storage advantages of the key value storage interface and the high-speed network storage protocol are compatible.
In one implementation, after the remote storage node reads the data to be read from the physical address mapped by the source address, the remote storage node may return a read response to the host through the interface of the high-speed network storage protocol.
Correspondingly, the host can receive the read response returned by the remote storage node through the interface of the high-speed network storage protocol.
In a specific implementation, after receiving the read Response, the host may convert the data format of the read Response into a key value storage interface used for receiving the read Response, for example, KV _ Get _ Response (), which can identify the read Response format, so that the application program identifies the read Response by calling the key value storage interface.
In one implementation, after receiving the read command sent by the host, the remote storage node may further send a send request for the data to be read to the host, where the send request includes a destination address of the data to be read.
Correspondingly, the host receives the sending request sent by the remote storage node and returns a response to the sending request to the remote storage node.
And then, the remote storage node can receive the response returned by the host and send the data to be read to the host through an interface of a high-speed network storage protocol.
Correspondingly, the host can receive the data to be read sent by the remote storage node through the interface of the high-speed network storage protocol, and store the data to be read to the destination address.
In a fifth aspect, there is provided a data writing device comprising means for performing the method of the first aspect.
In a sixth aspect, there is provided a storage node comprising means for performing the method of the second aspect.
In a seventh aspect, there is provided a data reading apparatus comprising means for performing the method of the third aspect.
In an eighth aspect, there is provided a storage node comprising means for performing the method of the fourth aspect.
In a ninth aspect, there is provided a data writing apparatus for executing the data writing method described in the first aspect. The data writing device station may include: a memory and a processor, a transmitter, and a receiver coupled with the memory, wherein: the transmitter is configured to transmit a mobile communication signal to a remote storage node, the receiver is configured to receive the mobile communication signal transmitted by the remote storage node, the memory is configured to store implementation codes of the data writing method described in the first aspect, and the processor is configured to execute the program codes stored in the memory, that is, execute the data writing method described in the first aspect.
In a tenth aspect, there is provided a data reading apparatus for performing the data reading method described in the third aspect. The data reading device station may include: a memory and a processor, a transmitter, and a receiver coupled with the memory, wherein: the transmitter is configured to transmit a mobile communication signal to a remote storage node, the receiver is configured to receive the mobile communication signal transmitted by the remote storage node, the memory is configured to store implementation codes of the data reading method described in the third aspect, and the processor is configured to execute the program codes stored in the memory, that is, execute the data reading method described in the third aspect.
In an eleventh aspect, there is provided a storage system comprising: data write device and storage node, wherein: the data writing device is the data writing device according to the fifth aspect, and the storage node is the storage node according to the sixth aspect.
In some possible implementations, the data writing device may also be the storage node described in the above ninth aspect.
In a twelfth aspect, there is provided a storage system comprising: data reading device and storage node, wherein: the data reading device is the data reading device according to the seventh aspect, and the storage node is the storage node according to the eighth aspect.
In some possible implementations, the data reading device may also be the data reading device described in the above tenth aspect.
By implementing the embodiment of the invention, a host can generate a read-write command conforming to a high-speed network storage protocol according to a key value pair carried by a read-write request aiming at a remote storage node sent by an application program, wherein the read-write request is a request initiated by the application program by calling a key value storage interface; and the host can send the read-write command conforming to the high-speed network storage protocol to the remote storage node through the transmission interface conforming to the high-speed network storage protocol so as to trigger the remote storage node to respond to the read-write command conforming to the high-speed network storage protocol and complete remote read-write operation. According to the scheme, the key value storage interface can be borne on the high-speed network storage protocol, so that an application program can perform read-write operation on the remote storage node through the key value storage interface, and the respective storage advantages of the key value storage and the high-speed network storage protocol are compatible.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below.
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present invention;
FIG. 2 is a flow chart of a data writing method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of NVM commands according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the mapping of logical addresses to physical addresses according to an embodiment of the present invention;
FIG. 5 is a diagram of a message flow to which an embodiment of the present invention relates;
FIG. 6 is a flow chart illustrating a data reading method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a data writing device according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a storage node according to an embodiment of the present invention;
FIG. 9 is a schematic structural diagram of another data writing device provided by an embodiment of the present invention;
fig. 10 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of a storage node according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of another data reading apparatus according to an embodiment of the present invention.
Detailed Description
The terminology used in the description of the embodiments of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
For better understanding of the embodiment of the present invention, an application scenario of the embodiment of the present invention is described below by taking the NVMe Over Fabric protocol as an example.
Referring to fig. 1, fig. 1 is a schematic diagram of an application scenario disclosed in the embodiment of the present invention. In the application scenario shown in fig. 1, a Host (Host) and a remote storage node form a network storage system, and both the Host and the remote storage node are integrated with NVMe Over Fabric (hereinafter, referred to as NOF) interfaces, and communicate with each other through the NOF interfaces. Wherein the host is operable to generate a standard NVM command and send the NVM command to a remote storage node via an NOF interface integrated on the host; the remote storage node may be configured to receive, through an NOF interface, an NVM command sent by the host, and respond to the NVM command to perform read/write operations on a storage device corresponding to the remote storage node, for example, a plurality of Solid State Drives (SSDs) shown in the drawings.
Specifically, as shown in fig. 1, the interaction between the host and the remote storage node may mainly include: a control plane and a data plane. Wherein the control plane may be primarily responsible for: the host sending an NVM standard command to the remote storage node, and the remote storage node returning a Response (Response) to the NVM standard command to the host; the data plane may be primarily responsible for: and the host sends data to be written to the remote storage node, and the remote storage node sends data to be read to the host.
It should be noted that the host according to the embodiment of the present invention may include: a computing device compatible with the NOF interface, capable of accessing the remote storage node, such as a server, a storage management device, and the like.
It should be noted that the remote storage node according to the embodiment of the present invention may refer to a storage node integrated with the high-speed network storage protocol interface. Here, the remote storage node may refer to a storage node in a network storage system located remotely from the host, or may refer to a storage node located in a distributed storage system (e.g., cloud storage).
It should be noted that the high-speed network storage protocol according to this embodiment refers to a storage protocol for network storage and with a storage bandwidth reaching Gbps level (e.g., 1Gbps or higher), such as NVMe Over Fabric protocol. The embodiment of the present invention does not limit the specific form of the high-speed network storage protocol, and a high-speed storage protocol for network storage that may appear in the future also belongs to an implementation object of the scheme of the present invention.
It should be noted that the network storage system shown in fig. 1 is only a simple example provided by the embodiment of the present invention, and the specific networking form of the network storage system may be determined according to the practical application scenario and requirements, and is not limited herein.
The embodiment of the invention provides a data writing method, a data reading method, related equipment and a system, wherein a host can generate a reading and writing command conforming to a high-speed network storage protocol according to a key-value pair (key-value) carried by a reading and writing request aiming at a remote storage node sent by an application program, wherein the reading and writing request is a request initiated by the application program by calling a key-value storage interface (hereinafter referred to as KV interface); and the host can send the read-write command conforming to the high-speed network storage protocol to the remote storage node through the transmission interface conforming to the high-speed network storage protocol so as to trigger the remote storage node to respond to the read-write command conforming to the high-speed network storage protocol and complete remote read-write operation. The scheme of the invention can realize that the KV interface is carried on the high-speed network storage protocol for network storage, so that an application program can carry out read-write operation on the remote storage node through the key value storage interface, and the respective storage advantages of the KV storage and the high-speed network storage protocol (such as NVMe Over Fabric) are compatible. The following are detailed below.
Referring to fig. 2, fig. 2 is a schematic flowchart of a data writing method according to an embodiment of the present invention. As shown in fig. 2, the method includes:
s101, a host receives a write request aiming at a remote storage node, wherein the write request carries a key value pair consisting of a key and a value, the key is used for indicating a destination address of data to be written, and the value is used for indicating a source address of the data to be written and the size of the data to be written. Specifically, the write request may be a request initiated by an application by calling a KV interface, such as KV _ write (), for writing data. In practical applications, the write request may also be a request initiated by a remote application program calling the KV interface for writing data, which is not limited herein.
Here, the source address of the data to be written refers to an address on the host for storing the data to be written before writing, and the source address is usually located in a memory space of the host; the destination address of the data to be written refers to a logical address used for writing the data to be written on a remote storage node.
S103, the host can generate a write command conforming to the high-speed network storage protocol according to the key value pair contained in the write request. Specifically, the write command conforming to the high-speed network storage protocol may include: the data writing device comprises a first field used for indicating the destination address of the data to be written, a second field used for indicating the source address of the data to be written, and a third field used for indicating the size of the data to be written.
S105, after the write command is generated, the host can send the write command to a remote storage node through the interface of the high-speed network storage protocol. The write command is used for triggering a remote storage node to write the data to be written stored at the source address into the physical address mapped by the destination address. Correspondingly, the remote storage node can receive the write command sent by the host through the interface of the high-speed network storage protocol.
S107, after receiving the write command sent by the host, the remote storage node may analyze a physical address mapped by the destination address.
And S109, the remote storage node responds to the write command and writes the data to be written into the physical address mapped by the destination address.
The data writing method provided by the embodiment of the invention is described below by taking a high-speed network storage protocol of NOF as an example.
FIG. 3 illustrates an NVM command according to an embodiment of the present invention, which mainly includes: a Name Space Identifier (NSID), a Starting logical block address (staring LBA), a Number of logical blocks (Number of logical blocks), and a hash table (Scatter Gather List, SGL). Wherein the NSID and Starting LBA in combination may indicate a logical address involved by the command on the remote storage node, such as a destination address of the write data or a source address of the read data; the number of logical blocks may indicate the length of data involved in the command, such as the length of data to be written, or the length of data to be read; the SGL is a cache area in a memory address space for storing data, such as a source address for storing the data to be written or a destination address for storing the data to be read.
In this embodiment of the present invention, the application program may call a KV interface for writing data, for example, KV _ write (), and initiate the write request, so as to request the host to write the data to be written to the remote storage node. The key-value pair (key-value) carried by the write request may be as shown in table 1:
Key NSID Starting LBA
Value SGL Length
TABLE 1
Wherein the key (key) may include: the NSID and the Starting LBA are used for indicating the destination address of the data to be written; the value (value) may include: and the Length is used for indicating the size of the data to be written, and is equivalent to the number of logic blocks in the NVM command.
It should be noted that if the write request is a write (e.g., a full disk write) to the entire domain name space of the remote storage node, the key (key) may also include only the NSID.
It can be understood from the NVM command shown in fig. 3 and the key-value pair shown in table 1 that, since the key-value pair includes the values of the first field, the second field, and the third field in the write command, the host can fill the corresponding fields of the NVM write command according to the key-value pair carried by the write request. Here, other fields included in the NVM write command may be filled by the host according to NVMe protocol specification, specifically refer to NVM Express delivery 1.2, which is not described herein again.
It should be noted that table 1 is only key-value pairs designed for NVM write commands, and for write commands of other types of high-speed network storage protocols, the key-value pairs carried by the write request according to the embodiments of the present invention may not be limited by table 1 as long as the key satisfying the key-value pair is used to indicate the destination address of the data to be written, and the value of the key-value pair is used to indicate the source address of the data to be written and the size of the data to be written.
In the embodiment of the present invention, after the remote storage node receives the write command conforming to the high-speed network storage protocol sent by the host, the remote storage node may analyze a physical address mapped by the destination address represented by the first field.
Specifically, as shown in fig. 4, the storage device corresponding to the remote storage node may be integrated with a plurality of storage hard disks (e.g., SSD), and the plurality of storage hard disks map a segment of consecutive logical block addresses together. As shown in fig. 4, one piece of data stored on the storage device corresponds to a logical Block Address (PBA), and the PBA can be actually mapped to a plurality of discrete Physical Block Addresses (PBAs). For example, the Data1 corresponds to the LBA-1, and the LBA-1 maps PBA-11, PBA-12, etc. In a specific implementation, the remote storage node may analyze, through a Flash Translation Layer (FTL), a physical address mapped by the destination address represented by the first field, and finally write the data to be written into the physical address.
Optionally, the data writing method provided in the embodiment of the present invention may further include S111 in fig. 2, that is: after the data to be written is written into the physical address mapped by the destination address, the remote storage node can return a write response to the host through the interface of the high-speed network storage protocol. Accordingly, the host can receive the write response returned by the remote storage node through the interface of the high-speed network storage protocol.
The following describes in detail a message processing procedure of write data according to an embodiment of the present invention, taking a high-speed network storage protocol, which is NOF, as an example, with reference to a message flow diagram shown in fig. 5.
As shown in fig. 5, the message processing procedure corresponding to S101 to S109 may include: 1. an application program (APP) initiates a write request for a remote storage node by calling a KV interface for writing data, such as KV _ write (), and transmits a key-value pair key-value to a host through KV _ write (); 2. the host generates a standard NVM write Command, namely NVM _ Command in the figure, according to the key value pair transmitted by the application program; 3. and the host sends the standard NVM write command to a remote storage node through the NOF interface.
As shown in fig. 5, the message processing procedure corresponding to S111 may specifically include: 4. the host receives a write Response returned by the remote storage node through the NOF interface, namely NVM _ Response in the graph; 5. the host converts the data format of the write Response into a KV interface for receiving the write Response, such as KV _ Get _ Response (), and the recognizable write Response format is the data format of KV _ Response in the graph; 6. and the application program (APP) receives the write Response returned by the remote storage node through a KV dropping interface, such as KV _ Get _ Response ().
The above details describe the data writing method provided by the embodiment of the present invention in detail from the control layer.
In terms of data, the data writing method provided in the embodiment of the present invention may further include:
after receiving the write command sent by the host, the remote storage node may send an acquisition request for the data to be written to the host, where the acquisition request may include a source address of the data to be written; correspondingly, the host receives an acquisition request for the data to be written, which is sent by the remote storage node. Then, the host can respond to the acquisition request and send the data to be written stored at the source address to a remote storage node through an interface of the high-speed network storage protocol; correspondingly, the remote storage node can receive the data to be written sent by the host through the interface of the high-speed network storage protocol.
Further, in this embodiment of the present invention, the Value (Value) in the key Value pair carried by the write request may further include: metadata (Metadata) of the data to be written, the Metadata being a kind of data for describing attributes (e.g. service type, creation time, etc.) of the data to be written. Correspondingly, the write command generated by the host computer and conforming to the high-speed network storage protocol further comprises: a fourth field for indicating metadata of the data to be written.
That is, the write command received by the remote storage node further includes: a fourth field for indicating metadata of the data to be written. In a specific implementation, the remote storage node may store the destination address of the data to be written and the metadata of the data to be written in a target database in a target cache region in a one-to-one correspondence manner, where the target cache region is located in a memory address space of the remote storage node, and may provide a metadata access interface for the host. For example, the target database may be as shown in table 2:
NSID Starting LBA Metadata
01 LBA-11 metadata1
01 LBA-12 metadata2
02 LBA-21 metadata3
… … …
TABLE 2
The combination of "NSID" and "Starting LBA" is used to indicate the destination address of the data to be written, and "Metadata" is used to indicate the Metadata of the data to be written. It should be noted that table 2 is only one implementation manner of the embodiment of the present invention, and may also be different in practical application, and should not be limited.
Specifically, when the destination address of the data to be written and the metadata of the data to be written are stored in the target database in the target cache area in a one-to-one correspondence manner, the remote storage node may: and judging whether the destination address exists in the target database or not, if so, replacing the metadata corresponding to the destination address in the target database with the metadata of the data to be written, otherwise, establishing a mapping relation in the target database for correspondingly storing the destination address of the data to be written and the metadata of the data to be written.
In a specific implementation, the providing, by the target database, the data access interface to the host may include: and the interfaces of adding, deleting, modifying, searching and the like are used for managing the metadata in the target database.
It can be understood that, since the target cache region is located in the memory address space of the remote storage node, rather than the physical storage space on the storage hard disk corresponding to the storage node, the target database provided by the present invention can implement: the host accesses the metadata of the data to be written directly from the memory of the remote storage node (namely, the target database), and does not need to search the metadata of the data to be written on the storage hard disk block by block, thereby realizing the high-efficiency access and management of the metadata of the data written in the storage hard disk.
After receiving a write request (the write request carries a key value pair, the key is used for indicating a destination address of data to be written, and the value is used for indicating a source address of the data to be written and the size of the data to be written) for a remote storage node sent by an application program, a host can generate a write command conforming to a high-speed network storage protocol according to the key value pair, send the write command to the remote storage node through an interface of the high-speed network storage protocol, and trigger the remote storage node to write the data to be written into a physical address mapped by the destination address. The data writing method can realize that the KV interface used for writing data is borne on the high-speed network storage protocol used for network storage, so that an application program can realize data writing operation on the remote storage node through the KV interface used for writing data, and respective storage advantages of KV storage and the high-speed network storage protocol are compatible.
Referring to fig. 6, fig. 6 is a schematic flowchart of a data reading method according to an embodiment of the present invention. As shown in fig. 6, the method includes:
s201, a host receives a read request aiming at a far-end storage node, wherein the read request carries a key value pair consisting of a key and a value, the key is used for indicating a source address of data to be read, and the value is used for indicating a destination address of the data to be read and the size of the data to be read. Specifically, the read request may be a request initiated by an application program (APP) on the host by calling a KV interface, such as KV _ read (), for reading data. In practical applications, the read request may also be a request initiated by a remote application calling the KV interface for reading data, which is not limited herein.
Here, the source address of the data to be read refers to a logical address on a remote storage node for storing the data to be read before reading; the destination address of the data to be read refers to an address on the host for storing the read data to be read, and the destination address is usually located in a memory space of the host.
S203, the host can generate a read command conforming to the high-speed network storage protocol according to the key value pair contained in the read request. Specifically, the read command may include: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read, and a third field used for indicating the size of the data to be read.
S205, after generating the read command, the host may send the read command to the remote storage node through the interface of the high-speed network storage protocol. Correspondingly, the remote storage node can receive the read command sent by the host through the interface of the high-speed network storage protocol.
S207, after receiving the read command sent by the host, the remote storage node may analyze a physical address mapped by the source address.
Specifically, the method for analyzing the physical address mapped by the source address may refer to relevant contents in the embodiment of the method in fig. 2, and details are not repeated here.
S209, the remote storage node responds to the read command, and reads the data to be read from the physical address mapped by the source address.
The data to be read related to the embodiment of the present invention may be written into the storage device corresponding to the remote storage node by the data writing method provided in the embodiment of the method in fig. 2. It should be noted that, in practical applications, the data to be read may also be written to the storage device through other ways, which is not limited herein.
The data reading method provided by the embodiment of the invention is described below by taking a high-speed network storage protocol of NOF as an example. Fig. 3 shows an NVM command, and the specific format can refer to relevant contents in the embodiment of the method in fig. 2, which is not described herein again.
In the embodiment of the present invention, the application program may call a KV interface for reading data, for example, KV _ read (), and initiate the read request to request the host to read the data to be read from the remote storage node. The key-value pair (key-value) carried by the read request may be as shown in table 3:
Key NSID Starting LBA
Value SGL Length
TABLE 3
Wherein the key (key) may include: the NSID and the Starting LBA are used for indicating the source address of the data to be read; the value (value) may include: and the Length is used for indicating the size of the data to be read, and is equivalent to the number of logic blocks in the NVM command.
It should be noted that if the read request is a write (e.g., a full disk read) to the entire domain name space of the remote storage node, the key (key) may also include only the NSID.
It can be understood from the NVM read-write command shown in fig. 3 and the key-value pair shown in table 3 that the host can fill the corresponding field of the NVM read command according to the key-value pair carried by the read request. Here, other fields included in the NVM read command may be filled by the host according to NVMe protocol specification, specifically refer to NVM Express delivery 1.2, which is not described herein again.
It should be noted that table 3 is only key-value pairs designed for NVM read commands, and for read commands of other types of high-speed network storage protocols, the key-value pairs carried by the read request according to the embodiments of the present invention may not be limited by table 3 as long as the key satisfying the key-value pair is used to indicate the source address of the data to be read, and the value of the key-value pair is used to indicate the destination address of the data to be read and the size of the data to be read.
Optionally, the data reading method provided in the embodiment of the present invention may further include S211 in fig. 6, that is: after the data to be read is read from the physical address mapped by the source address, the remote storage node may return a read response to the host through the interface of the high-speed network storage protocol. Accordingly, the host can receive the read response returned by the remote storage node through the interface of the high-speed network storage protocol.
The following describes in detail a message processing procedure for reading data according to an embodiment of the present invention, taking a high-speed network storage protocol, which is NOF, as an example, in conjunction with a message flow diagram shown in fig. 5.
As shown in fig. 5, the message processing procedures corresponding to the foregoing S201 to S209 may include: 1. an application program (APP) initiates a read request for a remote storage node by calling a KV interface for reading data, such as KV _ read (), and transmits a key-value pair key-value to a host through KV _ read (); 2. the host generates a standard NVM read Command, namely NVM _ Command in the figure, according to the key value pair transmitted by the application program; 3. and the host sends the standard NVM read command to a remote storage node through the NOF interface.
As shown in fig. 5, the message processing procedure corresponding to the foregoing S211 may specifically include: 4. the host receives a read Response, namely NVM _ Response in the graph, returned by the remote storage node through the NOF interface; 5. the host converts the data format of the read Response into a KV interface, such as KV _ Get _ Response (), and the read Response format which can be identified, namely the data format of KV _ Response in the graph; 6. and the application program (APP) receives the read Response returned by the remote storage node through a KV dropping interface, such as KV _ Get _ Response ().
The above description explains the data reading method provided in the embodiment of the present invention in detail from the control layer.
In terms of data, the data reading method provided in the embodiment of the present invention may further include:
after receiving the read command sent by the host, the remote storage node may send a sending request for the data to be read to the host, where the sending request includes a destination address of the data to be read; correspondingly, the host receives the sending request sent by the remote storage node and returns a response aiming at the sending request to the remote storage node. Then, the remote storage node may receive the response returned by the host, and send the data to be read to the host through an interface of a high-speed network storage protocol; correspondingly, the host can receive the data to be read sent by the remote storage node through the interface of the high-speed network storage protocol, and store the data to be read to the destination address.
After receiving a read request (the read request carries a key value pair, the key is used for indicating a source address of data to be read, and the value is used for indicating a destination address of the data to be read and the size of the data to be read) for a remote storage node sent by an application program, a host can generate a read command conforming to a high-speed network storage protocol according to the key value pair, send the read command to the remote storage node through an interface of the high-speed network storage protocol, and trigger the remote storage node to read the data to be read from a physical address mapped by the source address. The data reading method can realize that the KV interface for reading data is loaded on the high-speed network storage protocol for network storage, so that an application program can read data from a remote storage node through the KV interface for reading data, and respective storage advantages of the KV storage and the high-speed network storage protocol are compatible.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a data writing device according to an embodiment of the present invention. As shown in fig. 7, the data writing apparatus 10 may include: a first receiving unit 101, a write command generating unit 103, and a write command transmitting unit 105.
Wherein:
a first receiving unit 101, configured to receive a write request for a remote storage node, where the write request carries a key-value pair composed of a key and a value, where the key is used to indicate a destination address of data to be written, and the value is used to indicate a source address of the data to be written and a size of the data to be written;
a write command generating unit 103, configured to generate a write command conforming to a high-speed network storage protocol according to the key value pair; the write command includes: a first field for indicating a destination address of the data to be written, a second field for indicating a source address of the data to be written, and a third field for indicating a size of the data to be written;
a write command sending unit 105, configured to send the write command to the remote storage node through the interface of the high-speed network storage protocol, and trigger the remote storage node to respond to the write command, and write the data to be written stored at the source address into the physical address mapped by the destination address.
As shown in fig. 7, the data writing apparatus 10 may further include: a second receiving unit 107 and a data transmitting unit 109, wherein:
a second receiving unit 107, configured to receive an obtaining request for the data to be written sent by the remote storage node after the write command sending unit 105 sends the write command to the remote storage node through the interface of the high-speed network storage protocol, where the obtaining request includes a source address of the data to be written;
a data sending unit 109, configured to send, in response to the obtaining request, the data to be written stored at the source address to the remote storage node through an interface of the high speed network storage protocol.
In this embodiment of the present invention, the values in the key value pair may further include: the metadata of the data to be written, the write command may further include: a fourth field for indicating metadata of the data to be written.
It is understood that the specific implementation of the functional unit included in the data writing device 10 may refer to the function of the host described in the embodiment of the method in fig. 2, and is not described herein again.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a storage node according to an embodiment of the present invention. The storage node 20 corresponds to the data writing device 10 shown in fig. 7, and is configured to receive a write command conforming to a high-speed network storage protocol sent by the data writing device 10 and perform a write operation. As shown in fig. 8, the storage node 20 may include: a write command receiving unit 201, a holding unit 203, an analyzing unit 205, and a writing unit 207. Wherein:
a write command receiving unit 201, configured to receive a write command that conforms to a high-speed network storage protocol and is sent by a host through an interface of the high-speed network storage protocol; the write command includes: the data writing method comprises the steps of writing data to be written into a storage device, wherein the storage device comprises a first field used for indicating a destination address of the data to be written, a second field used for indicating a source address of the data to be written, a third field used for indicating the size of the data to be written and a fourth field used for indicating metadata of the data to be written;
a storing unit 203, configured to store the destination address of the data to be written and the metadata of the data to be written in a target database in a target cache area in a one-to-one correspondence manner, where the target database provides a metadata access interface for the host
An analyzing unit 205, configured to analyze a physical address to which the destination address is mapped;
a writing unit 207, configured to respond to the write command, write the data to be written into the physical address mapped by the destination address.
As shown in fig. 8, the storage node 20 may further include: a transmitting unit 211 and a data receiving unit 209, wherein:
a sending unit 211, configured to send, after the write command receiving unit 201 receives, through an interface of a high-speed network storage protocol, a write command that conforms to the high-speed network storage protocol and is sent by a host, an obtaining request for the data to be written to the host, where the obtaining request includes a source address of the data to be written;
a data receiving unit 209, configured to receive, through an interface of the high speed network storage protocol, the data to be written, which is sent by the host in response to the obtaining request.
In a specific implementation, the saving unit may be specifically configured to: and judging whether the destination address exists in the target database or not, if so, replacing the metadata corresponding to the destination address in the target database with the metadata of the data to be written, otherwise, establishing a mapping relation in the target database for correspondingly storing the destination address of the data to be written and the metadata of the data to be written.
It is understood that specific implementation of the functional units included in the storage node 20 may refer to the functions of the remote storage node in the embodiment of the method in fig. 2, and details are not described here.
To facilitate the implementation of embodiments of the present invention, the present invention provides another data writing apparatus. Referring to fig. 9, the data writing apparatus 90 may include: a transmitter 903, a receiver 904, a memory 902, and a processor 901 coupled to the memory 902 (the number of processors 901 in the data writing device 90 may be one or more, and one processor is taken as an example in fig. 9). In some embodiments of the invention, the transmitter 903, the receiver 904, the memory 902 and the processor 901 may be connected by a bus or other means, wherein the connection by the bus is exemplified in fig. 9.
Wherein the transmitter 903 is used for transmitting data to a remote storage node, and the receiver 904 is used for receiving data to the remote storage node. In a specific implementation, the transmitter 903 and the receiver 904 may be integrated as: and a transceiver such as a wireless transceiver module and a wired transceiver module. The Memory 902 is used for storing a program code, and in a specific implementation, the Memory 902 may be a Read Only Memory (ROM) and may be used for storing an implementation code of the data writing method described in the foregoing embodiment of the method in fig. 2. A processor 901, e.g. a CPU, is used to call the program code stored in the memory 902 to perform the following steps:
receiving a write request aiming at a remote storage node, wherein the write request carries a key value pair consisting of a key and a value, the key is used for indicating a destination address of data to be written, and the value is used for indicating a source address of the data to be written and the size of the data to be written; the write request is initiated by an application program calling a key value storage interface for writing data, and input parameters of the key value storage interface for writing data comprise the key value pair;
generating a write command conforming to a high-speed network storage protocol according to the key value pair; the write command includes: a first field for indicating a destination address of the data to be written, a second field for indicating a source address of the data to be written, and a third field for indicating a size of the data to be written;
and sending the write command to the remote storage node by using a transmitter 903 through an interface of the high-speed network storage protocol, triggering the remote storage node to respond to the write command, and writing the data to be written stored at the source address into a physical address mapped by the destination address.
In this embodiment of the present invention, after sending the write command to the remote storage node by using the transmitter 903, the processor 901 may further:
receiving, by a receiver 904, an acquisition request for the data to be written, which is sent by the remote storage node, where the acquisition request includes a source address of the data to be written;
and responding to the acquisition request, and sending the data to be written stored at the source address to the remote storage node by using a transmitter 903 through an interface of the high-speed network storage protocol.
Further, the values in the key-value pair may further include: the metadata of the data to be written, the write command further includes: a fourth field for indicating metadata of the data to be written.
It is understood that the steps executed by the processor 901 can also refer to the content related to the host in the method embodiment of fig. 2, which is not described herein again.
Referring to fig. 10, fig. 10 is a schematic structural diagram of a data reading device according to an embodiment of the present invention. As shown in fig. 10, the data reading device 30 may include: a first receiving unit 301, a read command generating unit 303, and a read command transmitting unit 305.
Wherein:
a first receiving unit 301, configured to receive a read request for a remote storage node, where the read request carries a key-value pair consisting of a key and a value, where the key is used to indicate a source address of data to be read, and the value is used to indicate a destination address of the data to be read and a size of the data to be read;
a read command generating unit 303, configured to generate a read command conforming to a high-speed network storage protocol according to the key value pair; the read command includes: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read and a third field used for indicating the size of the data to be read;
a read command sending unit 305, configured to send the read command to a remote storage node through an interface of the high-speed network storage protocol, trigger the remote storage node to respond to the read command, and read the data to be read from the physical address mapped by the source address.
As shown in fig. 10, the data reading device 30 may further include: a second receiving unit 307, a response unit 309 and a data receiving unit 311, wherein:
a second receiving unit 307, configured to receive a sending request for the data to be read sent by a remote storage node after the read command sending unit sends the read command to the remote storage node through the interface of the high-speed network storage protocol, where the sending request includes a destination address of the data to be read;
a response unit 309, configured to, in response to the sending request, return a response to the remote storage node in response to the sending request;
and a data receiving unit 311, configured to receive the data to be read sent by the remote storage node through an interface of the high-speed network storage protocol, and store the data to be read to the destination address.
It is understood that the specific implementation of the functional unit included in the data reading device 30 may refer to the function of the host described in the embodiment of the method in fig. 6, and is not described herein again.
Referring to fig. 11, fig. 11 is a schematic structural diagram of a storage node according to an embodiment of the present invention. The storage node 40 corresponds to the data reading device 30 shown in fig. 9, and is configured to receive a read command conforming to the high-speed network storage protocol sent by the data reading device 30 and perform a read operation. As shown in fig. 11, the storage node 40 may include: a read command receiving unit 401, an analyzing unit 403, and a reading unit 405. Wherein:
a read command receiving unit 401, configured to receive, through an interface of a high-speed network storage protocol, a read command that conforms to the high-speed network storage protocol and is sent by a host; the read command includes: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read and a third field used for indicating the size of the data to be read;
an analyzing unit 403, configured to analyze a physical address mapped by the source address;
a reading unit 405, configured to respond to the read command, and read the data to be read from the physical address mapped by the source address.
As shown in fig. 11, the storage node 40 may further include: a transmitting unit 407, an acknowledgement receiving unit 409, and a data transmitting unit 411, wherein:
a sending unit 407, configured to send, to the host, a sending request for the data to be read after the reading unit reads the data to be read from the physical address mapped by the source address, where the sending request includes a destination address of the data to be read;
a response receiving unit 409, configured to receive a response to the sending request returned by the host;
and the data sending unit 411 may be configured to send the data to be read to the host through an interface of a high-speed network storage protocol.
It can be understood that specific implementation of the functional unit included in the storage node 40 may refer to the function of the remote storage node in the embodiment of the method in fig. 6, and details are not described here.
To facilitate the implementation of embodiments of the present invention, another data reading device is provided. Referring to fig. 12, the data reading apparatus 100 may include: a transmitter 1003, a receiver 1004, a memory 1002, and a processor 1001 coupled to the memory 1002 (the number of the processors 1001 in the data reading apparatus 100 may be one or more, and one processor is taken as an example in fig. 12). In some embodiments of the present invention, the transmitter 1003, the receiver 1004, the memory 1002 and the processor 1001 may be connected by a bus or other means, wherein the connection by the bus is exemplified in fig. 12.
The transmitter 1003 is configured to transmit data to a remote storage node, and the receiver 1004 is configured to receive data from the remote storage node. In a specific implementation, the transmitter 1003 and the receiver 1004 may be integrated as: and a transceiver such as a wireless transceiver module and a wired transceiver module. The Memory 1002 is used for storing a program code, and in a specific implementation, the Memory 1002 may be a Read Only Memory (ROM) and may be used for storing an implementation code of the data reading method described in the foregoing embodiment of the method in fig. 6. The processor 1001, e.g. a CPU, is adapted to call the program code stored in the memory 1002 to perform the following steps:
receiving a read request aiming at a remote storage node, wherein the read request carries a key value pair consisting of a key and a value, the key is used for indicating a source address of data to be read, and the value is used for indicating a destination address of the data to be read and the size of the data to be read; the read request is initiated by an application program calling a key value storage interface for reading data, and input parameters of the key value storage interface for reading data comprise the key value pair;
generating a read command conforming to a high-speed network storage protocol according to the key value pair; the read command includes: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read and a third field used for indicating the size of the data to be read;
and sending the read command to a remote storage node through an interface of the high-speed network storage protocol by using a transmitter 1003, triggering the remote storage node to respond to the read command, and reading the data to be read from the physical address mapped by the source address.
In this embodiment of the present invention, after reading the data to be read from the physical address mapped by the source address, the processor 1001 may further:
sending a sending request aiming at the data to be read to the host by using a transmitter 1003, wherein the sending request comprises a destination address of the data to be read;
the receiver 1004 receives the response returned by the host to the sending request, and sends the data to be read to the host through the interface of the high-speed network storage protocol by using the transmitter 1003.
It is understood that the steps executed by the processor 1001 may refer to the content related to the host in the embodiment of fig. 6, and are not described herein again.
In addition, an embodiment of the present invention provides a storage system (as shown in fig. 1), where the storage system includes: a host and a remote storage node. Wherein: the host may be the data writing device 10 depicted by fig. 7 and the remote storage node may be the storage node 20 depicted by fig. 8.
It should be noted that the host may be the host mentioned in the embodiment of the method in fig. 2, and the remote storage node may be the remote storage node mentioned in the embodiment of the method in fig. 2, which is not described herein again.
An embodiment of the present invention further provides a storage system (as shown in fig. 1), where the storage system includes: a host and a remote storage node. Wherein: the host may be the data reading device 30 depicted by fig. 10 and the remote storage node may be the storage node 40 depicted by fig. 11.
It should be noted that the host may be the host mentioned in the embodiment of the method in fig. 6, and the remote storage node may be the remote storage node mentioned in the embodiment of the method in fig. 6, which is not described herein again.
In summary, with the embodiment of the present invention, a host may generate a read-write command conforming to a high-speed network storage protocol according to a key value pair carried by a read-write request for a remote storage node sent by an application program, where the read-write request is a request initiated by the application program by calling a key value storage interface; and the host can send the read-write command conforming to the high-speed network storage protocol to the remote storage node through the transmission interface conforming to the high-speed network storage protocol so as to trigger the remote storage node to respond to the read-write command conforming to the high-speed network storage protocol and complete remote read-write operation. According to the scheme, the key value storage interface can be borne on the high-speed network storage protocol, so that an application program can perform read-write operation on the remote storage node through the key value storage interface, and the respective storage advantages of the key value storage and the high-speed network storage protocol are compatible.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

Claims (20)

1. A method of data processing, the method comprising:
the host acquires a key value pair carried by a read-write request aiming at a remote storage node;
the host generates a read-write command conforming to a high-speed network storage protocol according to the key value pair carried by the read-write request aiming at the remote storage node;
the host sends the read-write command conforming to the high-speed network storage protocol to the remote storage node through the transmission interface conforming to the high-speed network storage protocol so as to trigger the remote storage node to respond to the read-write command conforming to the high-speed network storage protocol to complete the operation of the read-write request aiming at the remote storage node.
2. The method of claim 1, wherein the high-speed network storage protocol comprises NVMe overlay fabric protocol.
3. The method of claim 1, wherein the read/write request to the remote storage node carries a key-value pair consisting of a key and a value.
4. The method of claim 1, wherein the read and write requests comprise requests initiated by the application by invoking a key-value store interface.
5. The method according to any one of claims 1 to 4, wherein when the read-write request for the remote storage node is a write request, a key carried in the read-write request for the remote storage node is used to indicate a destination address of data to be written, and the value includes a source address used to indicate the data to be written, where the destination address of the data to be written is used to indicate a logical address used for the data to be written in the remote storage node, and the source address of the data to be written is used to indicate an address used for storing the data to be written in the host.
6. The method of claim 5, wherein the value further comprises a size indicating the data to be written.
7. The method of claim 5, wherein the host generates the read/write command conforming to the high-speed network storage protocol, and the read/write command comprises: the data writing device comprises a first field used for indicating the destination address of the data to be written and a second field used for indicating the source address of the data to be written.
8. The method of claim 7, wherein the host generates the read and write commands conforming to the high speed network storage protocol further comprises: a third field for indicating a size of the data to be written.
9. The method of claim 7, wherein the value of the read/write request for the remote storage node further comprises: if the metadata of the data to be written is metadata, the read-write command for the remote storage node further includes: a fourth field for indicating metadata of the data to be written.
10. The method of claim 5, wherein after sending the write command to the remote storage node over the interface of the high speed network storage protocol, the method further comprises:
receiving an acquisition request aiming at the data to be written, which is sent by the remote storage node, wherein the acquisition request comprises a source address of the data to be written;
and responding to the acquisition request, and sending the data to be written stored at the source address to the remote storage node through an interface of the high-speed network storage protocol.
11. The method according to any one of claims 1 to 4, wherein when the read/write request for the remote storage node is a read request, the read request carries a key-value pair consisting of a key and a value, wherein the key is used for indicating a source address of data to be read, and the value is used for indicating a destination address of the data to be read and a size of the data to be read; the read request is a request initiated by an application by calling a key-value store interface for reading data.
12. The method of claim 11, wherein generating the read command compliant with the high speed network storage protocol comprises: the device comprises a first field used for indicating the source address of the data to be read and a second field used for indicating the destination address of the data to be read.
13. The method of claim 11, wherein generating the read command compliant with the high speed network storage protocol further comprises: a third field to indicate a size of the data to be read.
14. The method of claim 11, after sending the read command to a remote storage node through the interface of the high speed network storage protocol, further comprising:
receiving a sending request aiming at the data to be read, which is sent by the remote storage node, wherein the sending request comprises a destination address of the data to be read;
responding to the sending request, and returning a response to the remote storage node aiming at the sending request;
and receiving the data to be read sent by the remote storage node through an interface of the high-speed network storage protocol, and storing the data to be read to the destination address.
15. A method of data processing, the method comprising:
a remote storage node receives a read-write command which is sent by a host and accords with a high-speed network storage protocol through an interface of the high-speed network storage protocol;
the remote storage node analyzes the address of the remote storage node to be operated by the read-write command conforming to the high-speed network storage protocol according to the read-write command conforming to the high-speed network storage protocol;
and the remote storage node finishes the operation of the read-write command conforming to the high-speed network storage protocol.
16. The method of claim 15, wherein when the read/write command conforming to the high speed network storage protocol is a write command, the write command comprises: the data writing device comprises a first field used for indicating a destination address of data to be written, a second field used for indicating a source address of the data to be written, a third field used for indicating the size of the data to be written and a fourth field used for indicating metadata of the data to be written.
17. The method of claim 16, wherein the analyzing, by the remote storage node, the address of the remote storage node to be operated by the read/write command conforming to the high-speed network storage protocol according to the read/write command conforming to the high-speed network storage protocol comprises:
the remote storage node stores the destination address of the data to be written and the metadata of the data to be written in a target database in a target cache region in a one-to-one correspondence manner, and the target database provides a metadata access interface for the host;
the remote storage node analyzes a physical address mapped by the destination address;
and the remote storage node responds to the write command and writes the data to be written into the physical address mapped by the destination address.
18. The method according to claim 15, wherein when the read/write command conforming to the high speed network storage protocol is a read command, the read request carries a key-value pair consisting of a key and a value, wherein the key is used for indicating a source address of data to be read, and the value is used for indicating a destination address of the data to be read and a size of the data to be read; the read request is a request initiated by an application by calling a key-value store interface for reading data.
19. The method of claim 18, wherein the analyzing, by the remote storage node, the address of the remote storage node to be operated by the read/write command conforming to the high-speed network storage protocol according to the read/write command conforming to the high-speed network storage protocol comprises:
generating a read command conforming to a high-speed network storage protocol according to the key value pair; the read command includes: the data reading device comprises a first field used for indicating a source address of the data to be read, a second field used for indicating a destination address of the data to be read and a third field used for indicating the size of the data to be read;
and sending the read command to a remote storage node through an interface of the high-speed network storage protocol, triggering the remote storage node to respond to the read command, and reading the data to be read from the physical address mapped by the source address.
20. A storage system comprising a host computer adapted to implement the operational steps of the method of any one of the preceding method claims 1 to 14 and a remote storage node adapted to implement the operational steps of the method of any one of the preceding method claims 15 to 19.
CN202010420635.2A 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system Pending CN111752480A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010420635.2A CN111752480A (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610176106.6A CN107229415B (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system
CN202010420635.2A CN111752480A (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201610176106.6A Division CN107229415B (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system

Publications (1)

Publication Number Publication Date
CN111752480A true CN111752480A (en) 2020-10-09

Family

ID=59931730

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202010420635.2A Pending CN111752480A (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system
CN201610176106.6A Active CN107229415B (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN201610176106.6A Active CN107229415B (en) 2016-03-24 2016-03-24 Data writing method, data reading method, related equipment and system

Country Status (1)

Country Link
CN (2) CN111752480A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115292420A (en) * 2022-10-10 2022-11-04 天津南大通用数据技术股份有限公司 Method and device for rapidly loading data in distributed database
WO2024037193A1 (en) * 2022-08-17 2024-02-22 北京超弦存储器研究院 Network storage processing device, storage server, and data storage and reading method

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108153682B (en) * 2017-12-12 2020-10-27 西安交通大学 Method for mapping addresses of flash translation layer by utilizing internal parallelism of flash memory
CN112615917B (en) 2017-12-26 2024-04-12 华为技术有限公司 Storage device management method in storage system and storage system
CN110110003A (en) * 2018-01-26 2019-08-09 广州中国科学院计算机网络信息中心 The data storage control method and device of M2M platform
WO2020185679A1 (en) * 2019-03-11 2020-09-17 Replixio Ltd. System and method for optimizing write requests of a write queue
CN110674106B (en) * 2019-08-30 2022-03-15 中国人民财产保险股份有限公司 Storage method, device and equipment of policy data and computer readable storage medium
CN111090869B (en) * 2019-12-16 2022-04-05 海光信息技术股份有限公司 Data encryption method, processor and computer equipment
CN113037772B (en) * 2021-03-30 2023-05-02 苏州科达科技股份有限公司 Data processing method, system, equipment and storage medium
CN112988681A (en) * 2021-04-07 2021-06-18 拉卡拉支付股份有限公司 File processing method, file processing device, electronic equipment, storage medium and program product
CN117675759A (en) * 2022-09-06 2024-03-08 成都华为技术有限公司 Data processing method and device
CN115766889A (en) * 2022-09-28 2023-03-07 成都赛力斯科技有限公司 Data frame structure and data communication method
CN117234427B (en) * 2023-11-09 2024-02-09 苏州元脑智能科技有限公司 Data reading and writing method, device, equipment, system, storage medium and storage system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130042060A1 (en) * 2011-08-08 2013-02-14 Takao Marukame Memory system including key-value store
WO2014089828A1 (en) * 2012-12-14 2014-06-19 华为技术有限公司 Method for accessing storage device and storage device
CN103973810A (en) * 2014-05-22 2014-08-06 华为技术有限公司 Data processing method and device based on IP disk
CN104007938A (en) * 2014-05-29 2014-08-27 华为技术有限公司 Key value generating method and device in storage network
CN104238963A (en) * 2014-09-30 2014-12-24 华为技术有限公司 Data storage method, device and system
US20150370700A1 (en) * 2014-06-23 2015-12-24 Google Inc. Managing storage devices

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8151323B2 (en) * 2006-04-12 2012-04-03 Citrix Systems, Inc. Systems and methods for providing levels of access and action control via an SSL VPN appliance
US9367346B2 (en) * 2013-02-20 2016-06-14 Nec Corporation Accelerating distributed transactions on key-value stores through dynamic lock localization

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130042060A1 (en) * 2011-08-08 2013-02-14 Takao Marukame Memory system including key-value store
WO2014089828A1 (en) * 2012-12-14 2014-06-19 华为技术有限公司 Method for accessing storage device and storage device
CN103973810A (en) * 2014-05-22 2014-08-06 华为技术有限公司 Data processing method and device based on IP disk
CN104007938A (en) * 2014-05-29 2014-08-27 华为技术有限公司 Key value generating method and device in storage network
US20150370700A1 (en) * 2014-06-23 2015-12-24 Google Inc. Managing storage devices
CN104238963A (en) * 2014-09-30 2014-12-24 华为技术有限公司 Data storage method, device and system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024037193A1 (en) * 2022-08-17 2024-02-22 北京超弦存储器研究院 Network storage processing device, storage server, and data storage and reading method
CN115292420A (en) * 2022-10-10 2022-11-04 天津南大通用数据技术股份有限公司 Method and device for rapidly loading data in distributed database

Also Published As

Publication number Publication date
CN107229415B (en) 2020-06-02
CN107229415A (en) 2017-10-03

Similar Documents

Publication Publication Date Title
CN107229415B (en) Data writing method, data reading method, related equipment and system
US11467975B2 (en) Data processing method and NVMe storage device
KR102104288B1 (en) Methods for accessing stored data and associated controllers, devices, hosts and systems
AU2013388031B2 (en) Data processing system and data processing method
CN107092439B (en) Data storage method and equipment
KR101378270B1 (en) Data prefetch in sas expanders
US10365858B2 (en) Thin provisioning in a storage device
CN109791522B (en) Data migration method and system and intelligent network card
CN108701004A (en) A kind of system of data processing, method and corresponding intrument
CN110837479B (en) Data processing method, related equipment and computer storage medium
CN103530066B (en) A kind of date storage method, Apparatus and system
US20220222016A1 (en) Method for accessing solid state disk and storage device
EP4216515A1 (en) Method and apparatus for transmitting data processing request
CN113032293A (en) Cache manager and control component
KR20200135715A (en) Method, apparatus, device and medium for processing data
CN109478171A (en) Improve the handling capacity in OPENFABRICS environment
CN115934002B (en) Solid state disk access method, solid state disk, storage system and cloud server
KR102623061B1 (en) Apparatus for performing iterator operation in database
CN113031849A (en) Direct memory access unit and control unit
US20230048813A1 (en) Method of storing data and method of reading data
KR102275706B1 (en) Method of operating data storage device and method of operating data processing system having same
CN116955228A (en) Accelerator for processing write command
CN114610678A (en) File access method, storage node and network card
CN110737396A (en) Method, apparatus and computer program product for data replication
CN103914525A (en) FAT32-formatted data reading method and system for use after USB equipment redirection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination