CN116909484A - Data processing method, device, equipment and computer readable storage medium - Google Patents
Data processing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN116909484A CN116909484A CN202310967591.9A CN202310967591A CN116909484A CN 116909484 A CN116909484 A CN 116909484A CN 202310967591 A CN202310967591 A CN 202310967591A CN 116909484 A CN116909484 A CN 116909484A
- Authority
- CN
- China
- Prior art keywords
- data
- queue
- nvme
- request
- dpu
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 33
- 230000000977 initiatory effect Effects 0.000 claims description 36
- 238000004590 computer program Methods 0.000 claims description 14
- 239000003999 initiator Substances 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 230000001133 acceleration Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 239000004744 fabric Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002023 wood Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/10—Packet switching elements characterised by the switching fabric construction
- H04L49/111—Switch interfaces, e.g. port details
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a data processing method, apparatus, device, and computer-readable storage medium. According to the method and the device, the command management queue, the data processing queue and the Target data oF the data processing request are directly stored in the internal storage space oF the DPU, so that the NVMe-oF Target is completely unloaded, and memory resources oF a host at the Target side are not occupied.
Description
Technical Field
The disclosure relates to the technical field of cloud storage, and in particular relates to a data processing method, a device, equipment and a computer readable storage medium.
Background
In the age of today's data bursts, previous data transfer protocols have failed to meet the transmission of large amounts of data. NVMe over Fabrics (NVMe-ofr) is a relatively new protocol specification aimed at using NVMe to connect hosts to storage through a network structure, supporting the resolution oF data center computation and storage. The advent oF NVMe-orf not only solves the performance bottleneck problem oF the above protocol, it also allows organization oF storage that implements lateral expansion for highly distributed, highly available applications.
The data processing unit (Data Process Unit, DPU) is an evolution device oF the intelligent network card, and generally has the capabilities oF unloading network, storage, management, etc., and can be used for unloading acceleration oF data processing oF the NVMe-oh request receiving end (Target). In the prior art, the DPU is generally utilized to temporarily store data into the memory of the host at the request receiving end, so as to reduce the resource consumption of the central processing unit (Central Processing Unit, CPU) in the host. However, the configuration management and data processing oF NVMe still need to use host memory, so this method does not achieve complete offloading oF NVMe-oh Target, and there is still a certain consumption oF resources oF the host.
Disclosure of Invention
In order to solve the above technical problems, the present disclosure provides a data processing method, apparatus, device and computer readable storage medium to achieve complete offloading oF NVMe-oh Target.
In a first aspect, an embodiment of the present disclosure provides a data processing method, applied to a DPU, including:
configuration information of NVMe equipment is obtained through a command management queue, the command management queue is stored in a first storage space of the DPU, and the NVMe equipment is in communication connection with the DPU;
creating a data processing queue in a second storage space of the DPU according to the configuration information of the NVMe equipment, wherein the data processing queue comprises a data submitting queue and a data completing queue;
responding to a data exchange request of a request initiating end, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU;
updating the data submitting queue, and submitting the data exchange request to the NVMe equipment;
and responding to the update of the NVMe equipment to the data completion queue, and forwarding the target data according to the data exchange request.
In some embodiments, before the root obtains the configuration information of the NVMe device through the command management queue, the method further comprises:
according to a configuration instruction of a user, acquiring a physical address of an NVMe equipment register;
and initializing the NVMe equipment according to the physical address of the NVMe equipment register.
In some embodiments, initializing the NVMe device according to the physical address of the NVMe device register includes:
and writing the physical address of the first storage space into a preset register of the NVMe equipment.
In some embodiments, the management queue includes a command commit queue and a command completion queue;
the creating a data processing queue in the second storage space of the DPU according to the configuration information of the NVMe device includes:
submitting a configuration acquisition command to the NVMe equipment through the command management queue to acquire configuration information of the NVMe equipment;
submitting a function setting command to the NVMe equipment through the command management queue, and determining the preset number of the data processing queues;
and submitting a queue creation command to the NVMe equipment through the command management queue, and creating a preset number of data processing queues in a second storage space of the DPU.
In some embodiments, when the data exchange request is a write data request, the responding to the data exchange request of the request initiator, obtaining the target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU, includes:
initiating a read data request to the request initiating terminal through an internal network card;
acquiring target data in the request initiating terminal;
the target data is stored in a third memory space in the DPU.
In some embodiments, the forwarding the target data according to the data exchange request in response to the update of the data completion queue by the NVMe device includes:
writing the target data in the third storage space into NVMe equipment;
and sending request completion information corresponding to the data exchange request to the request initiating terminal.
In some embodiments, when the data exchange request is a read data request, the forwarding the target data according to the data exchange request in response to the update of the data completion queue by the NVMe device includes:
and sending target data in the third storage space and request completion information corresponding to the data exchange request to the request initiating terminal.
In a second aspect, embodiments of the present disclosure provide a data processing apparatus, including:
the first acquisition module is used for acquiring configuration information of the NVMe equipment through a command management queue, wherein the command management queue is stored in a first storage space of the DPU, and the NVMe equipment is in communication connection with the DPU;
the creation module is used for creating a data processing queue in a second storage space of the DPU according to the configuration information of the NVMe equipment, wherein the data processing queue comprises a data submitting queue and a data completing queue;
the second acquisition module is used for responding to a data exchange request of a request initiating terminal, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU;
the submitting module is used for updating the data submitting queue and submitting the data exchange request to the NVMe equipment;
and the forwarding module is used for responding to the update of the NVMe equipment to the data completion queue and forwarding the target data according to the data exchange request.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method according to the first aspect.
In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium having stored thereon a computer program for execution by a processor to implement the method of the first aspect.
In a fifth aspect, the disclosed embodiments also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement a data processing method as described above.
According to the data processing method, the device, the equipment and the computer readable storage medium, the complete unloading oF the NVMe-oF Target is realized by directly storing the command management queue, the data processing queue and the Target data oF the data processing request into the internal storage space oF the DPU, and the memory resource oF a host at the Target side is not occupied.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic diagram of a DPU acceleration method;
FIG. 2 is a schematic diagram of another DPU acceleration method;
FIG. 3 is a flowchart of a data processing method according to an embodiment of the present disclosure;
fig. 4 is a schematic view of an application scenario provided in an embodiment of the present disclosure;
FIG. 5 is a flow chart of a data processing method according to another embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.
The Non-volatile memory host controller interface specification ((Non-Volatile Memory express, NVMe) is a storage protocol based on high-speed serial computer expansion bus standard (Peripheral Component Interconnect express, PCIe) devices the goal is to have higher performance than small computer system interfaces (Small Computer System Interface, SCSI), which are commonly used in computers to connect high-performance Solid State Drives (SSDs).
NVMe over Fabrics (NVMe-ofs) is a relatively new protocol specification, encapsulating NVMe protocols in various fabrics protocols, such as transmission control protocol (Transmission Control Protocol, TCP), remote direct address access (Remote Direct Memory Access, RDMA), fibre Channel (Fibre Channel). NVMe-orf enables NVe protocol transport over TCP or RDMA networks. NVMe over TCP, NVMe over rdma all belong to NVMe over Fabrics.
There are currently two general implementations oF hardware acceleration oF NVMe-oh request receiver (target) by DPU. As shown in fig. 1, the storage function oF the NVMe device is implemented by the DPU providing PCIe Root Complex capability and accelerating hardware offloading at the NVMe-oftarget end. Alternatively, as shown in fig. 2, the DPU intelligent network card is used as a PCIe End Point (the End Point is located at the End of the PCIe bus system topology structure, and is generally used as an initiator or terminator of a bus operation, and generally represents a serial or I/O device), and uses the RDMA capability of the DPU to temporarily store data in the memory of the target side host, and then reads and writes the memory through the PCIe interface, thereby reducing the participation of the CPU.
However, in the first solution, the target after acceleration does not occupy the CPU and memory resources of the host at the target side any more, but the NVMe disk needs to be connected to the DPU, and the bits that the DPU can provide are limited and not easy to expand, and for the existing storage system, the PCIe topology structure of the storage system needs to be changed, so that the implementation is not friendly enough.
In the second scheme, queues required for data and data processing still need to be built in the memory of the host at the target side, a certain consumption is still required for the resources of the host at the target side, and the acceleration products on the market at present can only support a single transmission protocol, such as RDMA or TCP, and have low expansibility.
In view of the foregoing, embodiments of the present disclosure provide a data processing method, which is described below with reference to specific embodiments.
Fig. 3 is a flowchart of a data processing method according to an embodiment of the present disclosure. The method can be applied to an application scenario shown in fig. 4, where the application scenario includes a Target side HOST (HOST) 41, an NVMe device (NVMe-SSD) 42, and a DPU43, the Target side HOST 41 is communicatively connected to the NVMe device 42, and the Target side HOST 41 is communicatively connected to the DPU 43. Wherein the RC device is used to connect the CPU/memory subsystem and the I/O device, and the EP device represents a serial or I/O device.
The Target side host 41 includes a memory 411, configured to store related information of the NVMe device 42 and the DPU 43; the DPU43 includes a network card (Network Interface Card, NIC) 431 and a double-rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR) 432.
It can be appreciated that the data processing method provided by the embodiment of the present disclosure may also be applied in other scenarios.
The following describes the data processing method shown in fig. 3 in conjunction with the application scenario shown in fig. 4, where the method is applied to a DPU, and includes the following specific steps:
s301, configuration information of NVMe equipment is obtained through a command management queue, the command management queue is stored in a first storage space of the DPU, and the NVMe equipment is in communication connection with the DPU.
Specifically, the Identify command is submitted to the NVMe device through the command management queue to obtain configuration information of the NVMe device, such as configuration of the NVMe controller and information of the NVMe nacespace.
S302, creating a data processing queue in a second storage space of the DPU according to the configuration information of the NVMe equipment, wherein the NVMe equipment is in communication connection with the DPU, and the data processing queue comprises a data submitting queue and a data completing queue.
Typically, NVMe devices can be divided into three parts, the HOST-side drive (NVMe-end and linux, windows have integrated the corresponding drives), NVMe controller and storage medium. The NVMe controller is essentially a direct memory access (Direct Memory Access, DMA) that is responsible for data handling (instruction data, user data, etc.) and multiple queues that are responsible for exploiting the parallel capabilities of the flash memory. NVMe queue () consists of a commit queue (Submission Queues, SQ) and a completion queue (Completion Queues, CQ), HOST commits commands through SQ, and NVMe controller commits completion commands through CQ.
There is a storage medium (flash space) in the NVMe device, if the flash space is divided into several independent logical spaces, the address range of each space logical block is 0 to N-1 (N is the logical space size), and each of the thus divided logical spaces is called a naspace.
After the DPU obtains the configuration information of the NVMe device from the NVMe device, the DPU may obtain the NVMe queue information and the storage information required in the NVMe device, and allocate a second storage space inside the DPU for storing the NVMe queue. Specifically, in this step, the NVMe queue includes a data processing queue (IO queue) that includes a data commit queue (IO SQ) for submitting a request to the NVMe device and a data completion queue (IO CQ) for characterizing a processing state of the NVMe device for the corresponding request.
In some embodiments, the DPU allocates a second memory space in the internal DDR for storing the NVMe queue.
S303, responding to a data exchange request of a request initiating end, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU.
The DPU communicates with the request Initiator through an internal network card, and receives a data exchange request from the request Initiator. The data exchange request can be a read data request, namely, the request initiating terminal applies for reading target data of the request receiving terminal; or a data writing request, namely, the request initiating terminal applies for writing target data into the request receiving terminal.
According to the data exchange request of the request initiating end, the DPU acquires target data corresponding to the data exchange request, and allocates a third storage space internally for storing the target data.
In some embodiments, the DPU allocates a third memory space in the internal DDR for storing the target data.
S304, updating the data submitting queue and submitting the data exchange request to the NVMe equipment.
The DPU submits the data exchange request to the NVMe device through the data submission queue created in the steps.
Specifically, the DPU updates the doorbell register of the SQ queue using PCIe P2P DMA to submit the data exchange request to the NVMe controller.
And S305, responding to the update of the NVMe equipment to the data completion queue, and forwarding the target data according to the data exchange request.
The DPU traverses the data completion queue, and after the NVMe device updates the data completion queue, the DPU can know that the data exchange request submitted to the NVMe device in the steps is completed, has the condition of data exchange, and further forwards the target data in the third storage space according to the data exchange request.
In some embodiments, after the DPU completes forwarding the target data in the third storage space according to the data exchange request, an execution result of the data exchange request is fed back to the request initiator.
The method comprises the steps that configuration information of NVMe equipment is obtained through a command management queue, the command management queue is stored in a first storage space of the DPU, and the NVMe equipment is in communication connection with the DPU; creating a data processing queue in a second storage space of the DPU according to the configuration information of the NVMe equipment, wherein the data processing queue comprises a data submitting queue and a data completing queue; responding to a data exchange request of a request initiating end, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU; updating the data submitting queue, and submitting the data exchange request to the NVMe equipment; and in response to the updating oF the data completion queue by the NVMe equipment, forwarding the Target data according to the data exchange request, and directly storing the command management queue, the data processing queue and the Target data oF the data processing request into an internal storage space oF the DPU by the DPU, so that the NVMe-oF Target is completely unloaded without occupying memory resources oF a host at the Target side.
On the basis of the above embodiment, before the configuration information of the NVMe device is obtained through the command management queue, the method further includes: according to a configuration instruction of a user, acquiring a physical address of an NVMe equipment register; and initializing the NVMe equipment according to the physical address of the NVMe equipment register.
Wherein initializing the NVMe device according to the physical address of the NVMe device register includes: and writing the physical address of the third storage space into a preset register of the NVMe equipment.
Wherein the management queue includes a command submit queue and a command complete queue, and the creating a data processing queue in the second storage space of the DPU according to the configuration information of the NVMe device includes: submitting a configuration acquisition command to the NVMe equipment through the command management queue to acquire configuration information of the NVMe equipment; submitting a function setting command to the NVMe equipment through the command management queue, and determining the preset number of the data processing queues; and submitting a queue creation command to the NVMe equipment through the command management queue, and creating a preset number of data processing queues in a first storage space of the DPU.
Fig. 5 is a flowchart of a data processing method according to another embodiment of the present disclosure, and the method is explained below with reference to the flowchart shown in fig. 5.
As shown in fig. 5, the method comprises the following steps:
s501, according to a configuration instruction of a user, acquiring a physical address of an NVMe device register.
When the Target side DPU starts acceleration, a user configures the PCI address of the NVMe equipment in the DPU through the host, and the DPU acquires the physical address of the NVMe equipment register according to the PCI address of the NVMe equipment and stores the physical address in the DPU equipment register.
S502, a first storage space is allocated in the DPU, wherein the first storage space is used for storing a command management queue, and the management queue comprises a command submitting queue and a command completing queue.
The DPU initializes the NVMe device according to the physical address of the NVMe device register stored in the DPU device register.
In some embodiments, the DPU uses PCIe P2P DMA to DMA between the physical address of the internal DDR and the physical address of the NVMe device register.
Specifically, the RDY of the CSTS register of the waiting NVMe is set to "0", and then a first memory space is allocated inside the DPU to store a command management queue (NVMe admin queue).
The command management queues include a command commit queue (Admin SQ), and a command completion queue (Admin CQ).
S503, writing the physical address of the first storage space into a preset register of the NVMe device.
The DPU writes the physical address of the first storage space into a preset register of the NVMe device. Specifically, the physical address of the first storage space is written into Admin Queue Attributes (AQA) and Admin Submission Queue Base Address (ASQ) registers of the NVMe device.
Further, the DPU sets the EN position of the CC register of NVMe to be 1, and when the RDY of the CSTS register of NVMe is changed to 1, the initialization of the NVMe equipment is completed. At this point, the DPU and NVMe device may already be in direct communication.
S504, submitting a configuration acquisition command to the NVMe equipment through the command management queue to acquire the configuration information of the NVMe equipment.
The DPU submits an admin command to the NVMe controller by updating a doorbell register of a command management queue in a PCIe P2PDMA mode according to related instructions written in advance in the DDR.
Specifically, the DPU submits a configuration acquisition command (identity) to the NVMe device through the command management queue to acquire configuration information of the NVMe device, including information of the NVMe controller and the naspace.
S505, submitting a function setting command to the NVMe equipment through the command management queue, and determining the preset number of the data processing queues.
The DPU submits a function setup command (Set Features) to the NVMe device via the command management queue for determining a number of data processing queues, including a number of data commit queues and a number of data completion queues.
In the embodiment of the present disclosure, the number of data processing queues is a preset number.
S506, submitting a queue creation command to the NVMe equipment through the command management queue, and creating a preset number of data processing queues in the second storage space of the DPU.
The DPU allocates a second storage space from the internal DDR for storing a preset number of data processing queues, submits a queue creation command (Create IO SQ/CQ) to the NVMe device through the command management queue, and sets a base address of the queue as a physical address of the second storage space.
S507, responding to a data exchange request of a request initiating terminal, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU.
According to the user's configuration, the DPU initializes the Target service and starts internal network card listening (RDMA protocol or TCP protocol can be used according to the user's configuration), waiting for the request initiator to initiate a connection request. If the connection request exists, the connection is established, and a waiting request initiating terminal initiates a standard NVMeoF data exchange request.
When the request initiating terminal initiates a standard NVMe over Fabrics NVMeoF data exchange request, the request is sent to the target side through the network card oF the request initiating terminal, and the DPU receives and analyzes the request.
S508, updating the data submitting queue and submitting the data exchange request to the NVMe equipment.
S509, responding to the update of the NVMe equipment to the data completion queue, and forwarding the target data according to the data exchange request.
Specifically, the implementation processes and principles of S507 to S509 and S103 to S105 are identical, and will not be described here again.
According to the embodiment of the disclosure, the initialization of the NVMe equipment is realized by using the PCIe P2P mode, and the read-write operation is directly initiated, so that the CPU of the Target side host computer is prevented from participating in the initialization of the NVMe equipment, and the resources of the host computer are further saved.
Meanwhile, in the embodiment of the disclosure, the DPU integrates network cards (or IP cores) of various network protocols, and can shield network protocol differences for upper layers, select any network protocol for data processing according to user configuration, and the DPU is not limited to a specific network protocol, so that the effect of flexible expansion is achieved.
On the basis of the foregoing embodiment, when the data exchange request is a write data request, the obtaining, in response to the data exchange request of the request initiator, target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU includes: initiating a read data request to the request initiating terminal through an internal network card; acquiring target data in the request initiating terminal; the target data is stored in a third memory space in the DPU.
Specifically, the responding to the update of the data completion queue by the NVMe device, forwarding the target data according to the data exchange request, includes: writing the target data in the third storage space into NVMe equipment; and sending request completion information corresponding to the data exchange request to the request initiating terminal.
When the data exchange request is a data writing request, the DPU initiates a read request to a request initiating terminal through an internal network card, acquires target data in the request initiating terminal, and stores the target data in a third storage space in the DPU. And updating a doorbell register of a data submitting queue by using a PCIe P2P DMA mode, submitting a data writing request to an NVMe controller, traversing a data completion queue to know that the request is completed, and sending request completion information corresponding to an execution result to a request initiating terminal by the DPU through an internal network card.
On the basis of the foregoing embodiment, when the data exchange request is a read data request, the responding to the update of the data completion queue by the NVMe device, forwarding the target data according to the data exchange request includes: and sending the target data in the third storage space and the request completion information corresponding to the data exchange request to a request initiating terminal.
When the data exchange request is a read data request, the DPU allocates a third storage space in the DDR for temporarily storing target data, updates a doorbell register of a data submitting queue in a PCIe P2P DMA mode, submits the read data request to an NVMe controller, traverses a data completing queue, and feeds back request completion information corresponding to an execution result in the third storage space to the request completion information through a network card when the data exchange request is the read data request.
In some embodiments, the request initiator initiates a data exchange request (write data request) for writing a block with a size of 4k, the Target side DPU receives the data exchange request through the internal network card, parses the data exchange request, knows that the data exchange request is a write data request, obtains a Target address of Target data and a size of wood packet data, initiates a read request to the request initiator, and moves the Target data to a third storage space of the DDR in the Target side DPU. After the data is moved, the DPU submits a data writing request to the NVMe controller in a PCIe P2P DMA mode, and the 4k data is directly written into the NVMe device to complete the persistence of the data. And then, the Target side DPU sends request completion information corresponding to the data exchange request to the request initiating terminal, and informs the request initiating terminal that the data exchange request is processed.
In the data processing method provided by the embodiment of the disclosure, the Target side does not need the host to allocate any memory, so that the occupation of memory resources in the host is reduced. The Target side host does not need NVMe drive to read and write NVMe equipment, the whole data processing process does not need CPU participation in the host, delay fluctuation caused by operating system scheduling is avoided, and the fluctuation of the whole data processing delay is reduced. Meanwhile, the original PCIe topology structure at the Target side is not changed, only the DPU is needed to be inserted, and the PCIe topology structure is flexible and extensible and convenient to use. According to various network protocols integrated in the DPU card, network protocols such as RDMA, TCP, FC and the like can be flexibly supported.
Fig. 6 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure. The data processing apparatus may be a DPU as described in the above embodiments, or the data processing apparatus may be a part or component in the DPU. The data processing apparatus provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the data processing method, as shown in fig. 6, the data processing apparatus 60 includes: a first acquisition module 61, a creation module 62, a second acquisition module 63, a submitting module 64, a forwarding module 65; the first obtaining module 61 is configured to obtain configuration information of an NVMe device through a command management queue, where the command management queue is stored in a first storage space of the DPU, and the NVMe device is in communication connection with the DPU; the creating module 62 is configured to create a data processing queue in the second storage space of the DPU according to the configuration information of the NVMe device, where the data processing queue includes a data commit queue and a data completion queue; the second obtaining module 63 is configured to obtain, in response to a data exchange request from a request initiator, target data corresponding to the data exchange request and store the target data in a third storage space in the DPU; the submitting module 64 is configured to update the data submitting queue and submit the data exchange request to the NVMe device; the forwarding module 65 is configured to forward the target data according to the data exchange request in response to the update of the data completion queue by the NVMe device.
Optionally, the data processing apparatus 60 further includes a third acquiring module 66 and an initializing module 67; the third obtaining module 66 is configured to obtain a physical address of the NVMe device register according to a configuration instruction of a user; the initialization module 67 is configured to initialize the NVMe device according to the physical address of the NVMe device register;
optionally, the initialization module 67 is specifically configured to write the physical address of the first storage space into a preset register of the NVMe device.
Optionally, the management queue includes a command submitting queue and a command completing queue; the creation module 62 includes an acquisition unit 621, a determination unit 622, a creation unit 623; the obtaining unit 621 is configured to submit a configuration obtaining command to the NVMe device through the command management queue, and obtain configuration information of the NVMe device; the determining unit 622 is configured to submit a function setting command to the NVMe device through the command management queue, and determine a preset number of the data processing queues; the creating unit 623 is configured to submit a queue creating command to the NVMe device through the command management queue, and create a preset number of data processing queues in the second storage space of the DPU.
Optionally, when the data exchange request is a data writing request, the second obtaining module 63 is configured to initiate a data reading request to the request initiating end through an internal network card; acquiring target data in the request initiating terminal; the target data is stored in a third memory space in the DPU.
Optionally, the forwarding module 65 is configured to write the target data in the third storage space into an NVMe device; and sending request completion information corresponding to the data exchange request to the request initiating terminal.
Optionally, when the data exchange request is a read data request, the forwarding module 65 is configured to send target data in the third storage space and request completion information corresponding to the data exchange request to the request initiator.
The data processing apparatus of the embodiment shown in fig. 6 may be used to implement the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effects are similar, and are not described herein again.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure. The electronic device may be a device to be upgraded as described in the above embodiments. The electronic device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the data processing method, as shown in fig. 7, where the electronic device 70 includes: memory 71, processor 72, computer programs and communication interface 73; wherein the computer program is stored in the memory 71 and configured to be executed by the processor 72 for performing the data processing method as described above.
In addition, the embodiment of the present disclosure also provides a computer-readable storage medium having stored thereon a computer program that is executed by a processor to implement the data processing method described in the above embodiment.
Furthermore, the disclosed embodiments also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement a data processing method as described above.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data processing method, for use in a DPU, the method comprising:
configuration information of NVMe equipment is obtained through a command management queue, the command management queue is stored in a first storage space of the DPU, and the NVMe equipment is in communication connection with the DPU;
creating a data processing queue in a second storage space of the DPU according to the configuration information of the NVMe equipment, wherein the data processing queue comprises a data submitting queue and a data completing queue;
responding to a data exchange request of a request initiating end, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU;
updating the data submitting queue, and submitting the data exchange request to the NVMe equipment;
and responding to the update of the NVMe equipment to the data completion queue, and forwarding the target data according to the data exchange request.
2. The method of claim 1, wherein before the root obtains the configuration information of the NVMe device via the command management queue, the method further comprises:
according to a configuration instruction of a user, acquiring a physical address of an NVMe equipment register;
and initializing the NVMe equipment according to the physical address of the NVMe equipment register.
3. The method of claim 2, wherein initializing the NVMe device according to the physical address of the NVMe device register comprises:
and writing the physical address of the first storage space into a preset register of the NVMe equipment.
4. The method of claim 1, wherein the management queue comprises a command commit queue and a command completion queue;
the creating a data processing queue in the second storage space of the DPU according to the configuration information of the NVMe device includes:
submitting a configuration acquisition command to the NVMe equipment through the command management queue to acquire configuration information of the NVMe equipment;
submitting a function setting command to the NVMe equipment through the command management queue, and determining the preset number of the data processing queues;
and submitting a queue creation command to the NVMe equipment through the command management queue, and creating a preset number of data processing queues in a second storage space of the DPU.
5. The method according to claim 1, wherein when the data exchange request is a write data request, the responding to the data exchange request of the request initiator, obtaining the target data corresponding to the data exchange request and storing the target data in the third storage space in the DPU, includes:
initiating a read data request to the request initiating terminal through an internal network card;
acquiring target data in the request initiating terminal;
the target data is stored in a third memory space in the DPU.
6. The method of claim 5, wherein forwarding the target data according to the data exchange request in response to the update of the data completion queue by the NVMe device comprises:
writing the target data in the third storage space into NVMe equipment;
and sending request completion information corresponding to the data exchange request to the request initiating terminal.
7. The method of claim 1, wherein when the data exchange request is a read data request, the forwarding the target data according to the data exchange request in response to the NVMe device updating the data completion queue comprises:
and sending target data in the third storage space and request completion information corresponding to the data exchange request to the request initiating terminal.
8. A data processing apparatus, comprising:
the first acquisition module is used for acquiring configuration information of the NVMe equipment through a command management queue, wherein the command management queue is stored in a first storage space of the DPU, and the NVMe equipment is in communication connection with the DPU;
the creation module is used for creating a data processing queue in a second storage space of the DPU according to the configuration information of the NVMe equipment, wherein the data processing queue comprises a data submitting queue and a data completing queue;
the second acquisition module is used for responding to a data exchange request of a request initiating terminal, acquiring target data corresponding to the data exchange request and storing the target data in a third storage space in the DPU;
the submitting module is used for updating the data submitting queue and submitting the data exchange request to the NVMe equipment;
and the forwarding module is used for responding to the update of the NVMe equipment to the data completion queue and forwarding the target data according to the data exchange request.
9. An electronic device, comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any of claims 1-7.
10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310967591.9A CN116909484B (en) | 2023-08-02 | 2023-08-02 | Data processing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310967591.9A CN116909484B (en) | 2023-08-02 | 2023-08-02 | Data processing method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116909484A true CN116909484A (en) | 2023-10-20 |
| CN116909484B CN116909484B (en) | 2024-08-02 |
Family
ID=88353100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310967591.9A Active CN116909484B (en) | 2023-08-02 | 2023-08-02 | Data processing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116909484B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117744118A (en) * | 2023-12-21 | 2024-03-22 | 北京星驰致远科技有限公司 | High-speed encryption storage device and method based on FPGA |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190042151A1 (en) * | 2017-08-02 | 2019-02-07 | Samsung Electronics Co., Ltd. | Hybrid framework of nvme-based storage system in cloud computing environment |
| CN110770710A (en) * | 2017-05-03 | 2020-02-07 | 艾德蒂克通信公司 | Apparatus and method for controlling data acceleration |
| US20200050402A1 (en) * | 2018-08-08 | 2020-02-13 | Marvell World Trade Ltd. | Managed Switching Between One or More Hosts and Solid State Drives (SSDs) Based on the NVMe Protocol to Provide Host Storage Services |
| CN111427808A (en) * | 2019-01-10 | 2020-07-17 | 三星电子株式会社 | System and method for managing communication between a storage device and a host unit |
| US10891156B1 (en) * | 2017-04-26 | 2021-01-12 | EMC IP Holding Company LLC | Intelligent data coordination for accelerated computing in cloud environment |
| CN112256601A (en) * | 2020-10-19 | 2021-01-22 | 凌云光技术股份有限公司 | Data access control method, embedded storage system and embedded equipment |
| US20220253238A1 (en) * | 2019-10-28 | 2022-08-11 | Huawei Technologies Co., Ltd. | Method and apparatus for accessing solid state disk |
| US20220350545A1 (en) * | 2021-04-29 | 2022-11-03 | EMC IP Holding Company LLC | Method and systems for storing data in a storage pool using memory semantics with applications utilizing object semantics |
| CN115543474A (en) * | 2022-08-30 | 2022-12-30 | 中科驭数（北京）科技有限公司 | NVMe-oF semi-unloading processing method and device and hardware acceleration equipment |
| CN115826872A (en) * | 2022-12-27 | 2023-03-21 | 上海交通大学 | Full hardware offload NVMe storage acceleration system |
-
2023
- 2023-08-02 CN CN202310967591.9A patent/CN116909484B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10891156B1 (en) * | 2017-04-26 | 2021-01-12 | EMC IP Holding Company LLC | Intelligent data coordination for accelerated computing in cloud environment |
| CN110770710A (en) * | 2017-05-03 | 2020-02-07 | 艾德蒂克通信公司 | Apparatus and method for controlling data acceleration |
| US20190042151A1 (en) * | 2017-08-02 | 2019-02-07 | Samsung Electronics Co., Ltd. | Hybrid framework of nvme-based storage system in cloud computing environment |
| US20200050402A1 (en) * | 2018-08-08 | 2020-02-13 | Marvell World Trade Ltd. | Managed Switching Between One or More Hosts and Solid State Drives (SSDs) Based on the NVMe Protocol to Provide Host Storage Services |
| CN111427808A (en) * | 2019-01-10 | 2020-07-17 | 三星电子株式会社 | System and method for managing communication between a storage device and a host unit |
| US20220253238A1 (en) * | 2019-10-28 | 2022-08-11 | Huawei Technologies Co., Ltd. | Method and apparatus for accessing solid state disk |
| CN112256601A (en) * | 2020-10-19 | 2021-01-22 | 凌云光技术股份有限公司 | Data access control method, embedded storage system and embedded equipment |
| US20220350545A1 (en) * | 2021-04-29 | 2022-11-03 | EMC IP Holding Company LLC | Method and systems for storing data in a storage pool using memory semantics with applications utilizing object semantics |
| CN115543474A (en) * | 2022-08-30 | 2022-12-30 | 中科驭数（北京）科技有限公司 | NVMe-oF semi-unloading processing method and device and hardware acceleration equipment |
| CN115826872A (en) * | 2022-12-27 | 2023-03-21 | 上海交通大学 | Full hardware offload NVMe storage acceleration system |
Non-Patent Citations (1)
| Title |
|---|
| 朱佳平等: "NVMeoF网络存储协议及硬件卸载技术研究", 第二十二届计算机工程与工艺年会暨第八届微处理器技术论坛, 16 August 2018 (2018-08-16), pages 15 - 24 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117744118A (en) * | 2023-12-21 | 2024-03-22 | 北京星驰致远科技有限公司 | High-speed encryption storage device and method based on FPGA |
| CN117744118B (en) * | 2023-12-21 | 2024-05-28 | 北京星驰致远科技有限公司 | High-speed encryption storage device and method based on FPGA |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116909484B (en) | 2024-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10908841B2 (en) | Increasing throughput of non-volatile memory express over fabric (NVMEoF) via peripheral component interconnect express (PCIe) interface | |
| CN107145459B (en) | System and method for remote shared access of cascade plate and SSD | |
| CN114138702B (en) | Computing system, PCI device manager and initialization method thereof | |
| US7340551B2 (en) | Bridge permitting access by multiple hosts to a single ported storage drive | |
| US8892964B2 (en) | Methods and apparatus for managing asynchronous dependent I/O for a virtual fibre channel target | |
| CN116909484B (en) | Data processing method, device, equipment and computer readable storage medium | |
| US20220222016A1 (en) | Method for accessing solid state disk and storage device | |
| US10564898B2 (en) | System and method for storage device management | |
| US20190196989A1 (en) | Method, Apparatus, and System for Accessing Memory Device | |
| CN112214157A (en) | Executing device and method for host computer output and input command and computer readable storage medium | |
| US11029847B2 (en) | Method and system for shared direct access storage | |
| CN114706531A (en) | Data processing method, device, chip, equipment and medium | |
| CN113672172A (en) | Data interaction method and receiving card applied to LED display control system | |
| US20150326684A1 (en) | System and method of accessing and controlling a co-processor and/or input/output device via remote direct memory access | |
| CN116719764A (en) | Data synchronization method, system and related device | |
| EP4425334A1 (en) | Storage control method, storage controller, storage chip, network card, and readable medium | |
| US10289550B1 (en) | Method and system for dynamic write-back cache sizing in solid state memory storage | |
| CN112732176B (en) | SSD (solid State disk) access method and device based on FPGA (field programmable Gate array), storage system and storage medium | |
| CN113031863B (en) | SSD command correlation management method, SSD command correlation management device, computer equipment and storage medium | |
| US11294832B2 (en) | Systems and methods for queuing device management configuration requests | |
| US20050177648A1 (en) | Data processing system | |
| CN106325377B (en) | The data processing method of Principle of External Device Extension card and I/O peripheral equipment | |
| CN116601616A (en) | Data processing device, method and related equipment | |
| US11507441B2 (en) | Asymmetric fulfillment of remote procedure calls by multi-core systems | |
| CN117880364B (en) | Data transmission method, system and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |