CN110874336B - Distributed block storage low-delay control method and system based on Shenwei platform - Google Patents

Distributed block storage low-delay control method and system based on Shenwei platform Download PDF

Info

Publication number
CN110874336B
CN110874336B CN201911001077.XA CN201911001077A CN110874336B CN 110874336 B CN110874336 B CN 110874336B CN 201911001077 A CN201911001077 A CN 201911001077A CN 110874336 B CN110874336 B CN 110874336B
Authority
CN
China
Prior art keywords
data packet
solid state
state disk
disk device
uio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911001077.XA
Other languages
Chinese (zh)
Other versions
CN110874336A (en
Inventor
许浒
周威
戴永涌
王刚
李紫阳
陈露
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Yuanhen Cloud Technology Co ltd
Wuxi Jiangnan Computing Technology Institute
Original Assignee
Suzhou Yuanhen Cloud Technology Co ltd
Wuxi Jiangnan Computing Technology Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Yuanhen Cloud Technology Co ltd, Wuxi Jiangnan Computing Technology Institute filed Critical Suzhou Yuanhen Cloud Technology Co ltd
Priority to CN201911001077.XA priority Critical patent/CN110874336B/en
Publication of CN110874336A publication Critical patent/CN110874336A/en
Application granted granted Critical
Publication of CN110874336B publication Critical patent/CN110874336B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1004—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/06—Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication
    • G06F12/0615—Address space extension
    • G06F12/063—Address space extension for I/O modules, e.g. memory mapped I/O
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/16—Handling requests for interconnection or transfer for access to memory bus
    • G06F13/1605—Handling requests for interconnection or transfer for access to memory bus based on arbitration
    • G06F13/161—Handling requests for interconnection or transfer for access to memory bus based on arbitration with latency improvement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/22—Handling requests for interconnection or transfer for access to input/output bus using successive scanning, e.g. polling
    • G06F13/225—Handling requests for interconnection or transfer for access to input/output bus using successive scanning, e.g. polling with priority control

Abstract

The invention discloses a distributed block storage low-delay control method and a distributed block storage low-delay control system based on a Shenwei platform, wherein the method comprises the following steps: the data packet is promoted through the UIO technology, specifically, the data packet of the NVMe solid state disk device is promoted to a user state, the data packet of the NVMe solid state disk device is read in the user state, and the data packet is mapped to a process space of the user state; calling a polling mode driver to process the sending and receiving of the data packets in a polling mode, wherein the NVMe solid state disk equipment corresponds to the polling mode driver one by one; and storing the user mode and sending the data packet to a storage core of the Shenwei platform for analysis and processing. The data drive of the NVMe solid state disk device is promoted to a user state, and the data packet sending and receiving are polled in a polling mode, so that the extra loss and consumption of the power application processor are reduced, and the IO efficiency of a power application platform of the system is improved.

Description

Distributed block storage low-delay control method and system based on Shenwei platform
Technical Field
The invention relates to the field of data storage, in particular to a distributed block storage low-delay control method and system based on a Shenwei platform.
Background
As server systems based on domestic processors are applied to various cloud computing environments, the application capability of the domestic processor technology in the general technology market is more and more important. At present, the domestic processor has a certain foundation on the technical support capability overall, but has a little gap with the international mainstream similar chip technology.
Compared with the perfect ecological environment of the X86 processor, the domestic processor has obvious defects in application support, and particularly, how to build a distributed storage system on a domestic chip through technical optimization can provide stable and low IO delay storage system service support for computing virtualization, and is an application ecological key means for rapidly promoting a domestic processor platform to form virtuous circle.
The Shenwei processor was sourced from DEC Alpha 21164, the development of which was supported by national "nuclear high base" special funds. Under the support of the important special item of the national 'nuclear high-base', an autonomous instruction set is adopted, the unit which is particularly responsible for research and development is a computer institute in the south of the Yangtze river, and the optical supercomputers of Shenwei Taihu lake, which take the processor series with completely autonomous intellectual property rights as the calculation basis, have been the fastest supercomputers in cicada union in the world for many years.
In the traditional IO model, an application program sleeps after submitting a read-write request, once IO is completed, the interrupt awakens the application program, and frequent interrupt and awakening bring a large amount of processor loss overhead. However, in the era of solid-state devices, lower-latency persistency devices are continuously introduced, and the interrupt overhead becomes a non-negligible part of the total IO time, and the problem only becomes more and more serious on the lower-latency devices.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art. Therefore, an object of the present invention is to provide a distributed block storage low-latency control method based on an Shenwei platform, which can reduce extra consumption and consumption of a CPU to improve overall IO efficiency.
To this end, a second object of the present invention is to provide a distributed block storage low latency control system based on the Shenwei platform.
The technical scheme adopted by the invention is as follows:
in a first aspect, the invention provides a distributed block storage low-latency control method based on an Shenwei platform, which comprises the following steps:
the data packet is promoted through the UIO technology, specifically, the data packet of the NVMe solid state disk device is promoted to a user state, the data packet of the NVMe solid state disk device is read in the user state, and the data packet is mapped to a process space of the user state;
calling a polling mode driver to process the sending and receiving of the data packet in a polling mode, wherein the NVMe solid state disk equipment corresponds to the polling mode driver one by one;
and the user state stores and sends the data packet to a storage core of the Shenwei platform for analysis and processing.
Further, the packet lifting specifically comprises:
intercepting the interruption of the NVMe solid state disk equipment through an UIO technology, and resetting the interruption behavior of the NVMe solid state disk equipment;
reading a data packet in the NVMe solid state disk device in a user mode through a DPDK technology;
and carrying out Shenwei processor kernel state registration through UIO technology, and mapping registration information to the process space of the user state.
Further, intercepting the interruption of the NVMe solid state disk device through the UIO technology, and resetting the interruption behavior of the NVMe solid state disk device specifically includes:
the user state waits for the interruption of one NVMe solid state disk device and blocks the read-write operation of the NVMe solid state disk device, and when the NVMe solid state disk device is interrupted, the read-write operation of the NVMe solid state disk device returns immediately;
registering a uniform interrupt handling function for the UIO device, the interrupt handling function being associated with a UIO listener data structure to reset interrupt behavior to bypass the Shenwei processor kernel state and promote data drivers to a user state.
Further, performing state registration of the processor kernel of the miwei through the UIO technology, and mapping the registration information to the process space of the user state specifically includes:
filling a UIO information structure body of the UIO equipment, wherein the registration information of the UIO information structure body comprises: information name, memory size, physical address, offset address;
calling a register UIO device function to register the UIO device in the Shenwei processor kernel state; and reading the registration information of the UIO information structure body and mapping the registration information to the process space of the user mode.
Further, invoking a polling mode driver to process the sending and receiving of the data packet in a polling manner specifically includes:
calling and executing the polling mode drive through a CORE binding technology;
providing an application programming interface according to the polling mode driver, and configuring the NVMe solid state disk device and a transmission queue of data packets of the NVMe solid state disk device;
and driving and controlling the processor core to poll one or more ports through the application programming interface according to the polling mode and sending a data packet to the Shenwei platform processing core for processing.
Further, the specific steps of storing the user mode and sending the data packet to a storage core of the shenwei platform for analysis and processing are as follows:
reading a data packet of the NVMe solid state disk;
setting labels with different priorities according to the storage characteristics of the data packets, and pre-reading and pre-caching the data packets to the storage core of the shenwei platform according to the priorities of the labels in sequence;
and analyzing the data packet according to the own protocol specification, and carrying out request identity check and/or CRC check and/or compression and/or decryption processing on the data packet.
Further, the storage characteristics include: the number of times the packet is accessed, updated, and the time it is read in a fixed time.
Further, the above description applies to the Shenwei processor in general.
In a second aspect, the present invention provides a distributed block storage low latency control system based on a Shenwei platform, including:
the data packet hoisting module is used for hoisting the data packet transmission of the NVEe solid state disk device to a user state through UIO technology, and directly reading the data packet of the NVEe solid state disk device and mapping the data packet to a process space of the user state;
the polling module is used for calling a polling mode driver to process the sending and receiving of the data packet in the NVEe solid state disk device in a polling mode;
the storage module is used for reading the data packet of the NVEe solid state disk device and storing the data packet to a storage core of the Shenwei platform;
and the analysis processing module is used for analyzing and processing the data packet in the storage core of the Shenwei platform.
The invention has the beneficial effects that:
according to the invention, the data driver of the NVMe solid state disk device is promoted to the user state by bypassing the kernel state of the Shenwei processor through the UIO technology and is mapped to the process space of the user state, then the polling mode driver is adopted to poll the sending and receiving of the data packet, and finally the data packet is stored, analyzed and processed in the user state.
Drawings
FIG. 1 is a block diagram of an embodiment of a distributed storage low latency control system based on an Shenwei platform;
FIG. 2 is a block diagram of a packet lifting module in an embodiment of a distributed storage low latency control system based on a Shenwei platform;
FIG. 3 is a block diagram of a polling module, a storage module, and a parsing module in an embodiment of a distributed storage low latency control system based on an Shenwei platform;
FIG. 4 is a flow chart of an embodiment of a distributed storage low latency control method based on the Shenwei platform.
Reference numerals: 10. a packet lifting module; 11. an interception resetting unit; 111. an interceptor unit; 112. resetting the subunit; 12. a control right transfer unit; 13. a registration unit; 131. a filler subunit; 132. a registration subunit; 133. a mapping subunit; 20. a polling module; 21. a binding unit; 22. an application programming interface configuration unit; 23. a polling unit; 30. a storage module; 31. a reading unit; 32. a label setting unit; 33. a storage unit; 40. an analysis processing module; 41. an analysis unit; 42. and a processing unit.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
Referring to fig. 1, an embodiment of the present invention discloses a distributed storage low-latency control system based on a power application platform, and in order to meet a storage accessibility requirement realized based on a domestic power application processor platform, an end-to-end IO full-path performance service quality assurance method is researched to realize a stable and low-IO-latency distributed storage system based on a power application platform, and IO processing performance optimization is realized. The distributed storage low-delay control system based on the Shenwei platform comprises: the data package upgrading method comprises a data package upgrading module 10, a polling module 20, a storage module 30 and an analysis processing module 40, wherein the data package upgrading module 10 is used for upgrading data package transmission of the NVEe solid state disk device to a user state through a UIO technology, and the UIO technology is implemented based on a chenille platform. The data packet lifting module 10 directly reads a data packet of the NVEe solid state disk device and maps the data packet to a process space of a user state, the polling module 20 is used for calling a polling mode driver to process sending and receiving of the data packet in the NVEe solid state disk device in a polling mode, the NVEe solid state disk device corresponds to the polling mode driver one by one, and the storage module 30 is used for reading the data packet of the NVEe solid state disk device and storing the data packet to a storage core of a shenwei platform. The parsing processing module 40 is configured to parse and process the data packets in the storage core of the shenwei platform, and dequeue the parsed and processed data packets to the storage space of the NVEe solid state disk device in a first-in first-out manner.
The UIO technique is an I/O technique that runs in user space. The general driving device in the Linux system runs in a kernel space, and only needs to be called by an application program in a user space, while the UIO technology runs a part of the driving device in the kernel space, and realizes most functions of the driving device in the user space. The drive of the NVEe solid state disk device can be operated by calling the application program through the user mode, so that the kernel mode drive of the Shenwei processor is saved, and the drive efficiency of the NVEe solid state disk device is improved.
The data packet transmission in the NVEe solid state disk device bypasses a kernel state of a Shenwei processor and is directly moved up to a user state, the polling module 20 polls the receiving and sending of the data packet between the NVEe solid state disk device and the user state, the polling replaces interruption to perform packet receiving and sending processing in the user state, the storage module 30 directly reads the data packet in the NVEe solid state disk device in the user state and stores the data packet in a Shenwei platform storage kernel, and the analysis processing module 40 receives the processing packet to perform analysis and processing. In conclusion, the extra loss and consumption of the CPU can be reduced, so that the overall IO efficiency is improved, and the IO delay is reduced.
The driving of the NVEe solid state disk device has two main tasks, wherein the first task is to store the memory of the device and the second task is to process the interrupt generated by the device. For the first task, the UIO technology realizes that mapping can process physical memory, logic memory and virtual memory, and the device driving performed by the UIO technology does not need to consider the fussy details. The second task, the interrupt response to the device, must be done in kernel space, so there is a portion of code in kernel space to answer interrupts and disable interrupts, leaving the rest of the work entirely for user space processing. In the embodiment, the interrupt to the device is reset interrupt on one hand and is replaced by polling on the other hand, so that the efficiency of the NVEe solid state disk device driver is improved.
Referring to fig. 1 and 2, the data packet hoisting module 10 includes an interception resetting unit 11, a control right transferring unit 12, and a registering unit 13, where the interception resetting unit 11 is configured to intercept an interrupt of the NVMe solid state disk device through the UIO technology and reset an interrupt behavior of the NVMe solid state disk device so as to bypass the application processor core state to promote the data driver of the NVMe solid state disk device to the user state operation. The control right transfer unit 12 is configured to control the kernel mode and the user mode of the nwer processor to perform control right transfer by using a DPDK technique, so that only control right transfer is performed without copying the memory when the kernel mode and the user mode of the nwer processor interact with each other, and the user mode directly reads and writes the memory buffer of the NVMe solid state disk device to directly read a data packet. The registration unit 13 completes registration in the kernel mode of the Shenwei processor through the UIO technology and maps the registration information to the process space of the user mode, and data interaction is facilitated to be driven and processed in the user mode through registration in the kernel mode of the Shenwei processor. The interruption of the NVMe solid state disk device is intercepted by the interception resetting unit 11, the interruption is reset to bypass the data processing flow of the kernel state of the Shenwei processor, then the control right transfer unit 12 is used for transferring the control right instead of copying the data when the kernel state of the Shenwei processor interacts with the user state so as to be convenient for directly reading and writing the memory buffer area of the NVMe solid state disk device in the user state, then the registration unit 13 is used for registering in the kernel state of the Shenwei processor to realize that the user state directly drives and processes the data packet sent by the NVMe solid state disk device, thus a frame for directly sending and receiving the data in the user state is constructed, and the efficient reading is realized by bypassing the kernel state of the Shenwei processor and directly driving the data packet through the user state.
The DPDK is a data plane development kit, is developed by multiple companies such as 6WIND, Intel and the like, is mainly operated based on a Linux system, is used for a function library and a drive set for rapidly processing data packets, can greatly improve the data processing performance and throughput, and improves the working efficiency of data plane application programs.
The interception resetting unit 11 includes an interception subunit 111 and a resetting subunit 112, the UIO technology is executed by the UIO device, in this embodiment, the interception subunit 111 is configured to intercept an interruption of the NVMe solid state disk device and block a read-write operation of the UIO device, the interception subunit 111 waits for the interruption of the NVEe solid state disk device in the user state, and by blocking the read-write operation of the UIO device, when the device generates an interruption, the read-write operation immediately returns, that is, the UIO device stops reading and writing and returns to the previous operation. In other embodiments, the interception subunit 111 is configured to wait for an interrupt to occur and intercept using a selection function, which has a timeout parameter that can be used to implement a wait for a limited time for the interrupt. The interruption is intercepted by the interception subunit 111 to realize that the NVMe solid state disk device can skip the state of the inner core of the application processor to process data in the user state.
The reset subunit 112 is configured to register a uniform interrupt processing function for the UIO device, where the interrupt processing function is associated with a UIO monitoring data structure, and increment an interrupt event counter of the UIO device by calling an interrupt processing function provided by the UIO device and calling an UIO event notification function at the interrupt processing function, so as to notify a read-write program of the NVMe solid-state disk device to notify that the read-write operation of the UIO device is "data readable". Since the interrupt handling functions of each UIO device are individually registered, the UIO listener data structure has mainly two attributes: a device pointer for easy access to save the pointer of the UIO device, and an event number for tracking the interrupt event counter of the UIO device. The UIO snoop data structure is used to track interrupt event counter values for each UIO device. The listening structure associated with the UIO device is destroyed while the user-mode data-driven operations are performed. When the UIO equipment is registered, a general interrupt processing function is registered for the UIO equipment, in the function, an interrupt processing function of the UIO equipment is called, when an interrupt occurs, an UIO event notification function is called, the UIO event notification function is used for increasing one to an interrupt event counter of the NVMe solid state disk equipment and notifying the NVMe solid state disk equipment that data can be read. The reset subunit 112 resets the interrupt and performs the interrupt processing in the user mode, so as to not affect the normal reading and writing of the data and improve the reading and writing efficiency of the data.
The UIO device includes a UIO information structure body, and the registration unit 13 includes: a padding subunit 131, a registration subunit 132, and a mapping subunit 133, where the padding subunit 131 is configured to pad registration information of the UIO information structure, where the pad registration information includes an information name, a memory size, a physical address, and an offset address, and the information name is a UIO device name. The registration subunit 132 is configured to call an UIO device registration function after the completion of the filling to register the UIO information structure into the processor kernel state, the registration is completed when the registration information is stored at the address of the folder, the folder has the registration information of the UIO information structure, and the mapping subunit 133 is configured to directly map the registration information of the UIO information structure to the process space of the user state, so that the user state directly operates the memory space of the NVMe solid state disk device to implement sending and receiving of the data packet.
Referring to fig. 3, the polling module 20 includes: the binding unit 21 is used for driving a corresponding NVMe solid state disk device by a polling mode, the polling mode driving is executed by a CORE binding technology of a Winser platform, and the polling mode driving is a UIO technology which is not dispatched but continuously waits for work and depends on a kernel. The application programming configuration unit is used for providing an application programming interface according to the polling mode driver, configuring the application programming interface on the NVMe solid state disk device and a transmission queue of a data packet of the NVMe solid state disk device, wherein the transmission queue is a data packet sending and receiving queue, and the polling unit 23 is used for calling the polling mode driver in a user mode, controlling the processor core to poll one or more ports of the NVMe solid state disk device through the application programming interface according to the polling mode driver, and sending the data packet to the processor core for processing. The binding unit 21 uses the Shenwei processor core binding technology to correspond to each polling mode driver, one logic CPU binds one NVMe solid state disk device, the polling mode driver can continuously have 100% utilization rate when running in the logic CPU, and the polling unit 23 continuously polls datagram sending and receiving of the NVMe solid state disk device according to the polling mode driver so as to replace the past interrupt through polling and save the delay caused by the interrupt.
The storage module 30 includes: the NVMe solid-state disk device comprises a reading unit 31, a tag setting unit 32 and a storage unit 33, wherein the reading unit 31 is used for directly reading a data packet sent by the NVMe solid-state disk device in a user mode, the tag setting unit 32 is used for setting tags with different priorities according to storage characteristics of the data packet, and the storage characteristics include: the number of times the packet is accessed, the number of updates, and the time it is accessed within a fixed time. The storage unit 33 pre-reads and pre-buffers the data packets to the storage core of the shenwei platform according to the priority order of the tags. In this embodiment, different priorities are set for the reading times from small to large and the access heat from strong to weak, and the packet storage unit 33 for the priorities is stored in advance, which is beneficial to rapidly storing the frequently used packets with multiple accesses.
The parsing processing module 40 includes a parsing unit 41 and a processing unit 42, where the parsing unit 41 receives the data packet stored in the storage unit 33 and parses the data packet, and the processing unit 42 performs request identity check, CRC check, compression, and decryption on the parsed data packet to complete data processing.
In summary, a data transmission architecture of the NVMe solid state disk device and the user mode is established by the data packet lifting module 10, the polling module 20 polls the data packet sent and received by the NVMe solid state disk device to the user mode, so that the polling mode replaces the interrupt mode, the storage module 30 directly reads the data packet from the memory space of the NVMe solid state disk device and stores the data packet in the user mode, and the analysis processing module 40 analyzes and processes the stored data packet, thereby implementing data processing in the user mode bypassing the kernel mode of the waiwei processor, reducing delay caused by interrupt, and thereby improving efficiency of device port driving.
Referring to fig. 4, embodiment two: the embodiment of the invention discloses a distributed block storage low-delay control method based on a Shenwei platform, which comprises the following steps of:
before driving processing of the NVMe solid state disk device, whether the NVMe solid state disk device is the NVMe solid state disk device needs to be judged by identifying a classifier of the NVMe solid state disk device;
the classifier is defined by the specification of the NVMe solid state disk device, whether the NVMe solid state disk device is determined by identifying the classifier of the NVMe solid state disk device, so that the next operation is executed, and if the NVMe solid state disk device is not identified through classification, the next operation is stopped.
The data packet is promoted through the UIO technology, specifically, the data packet of the NVMe solid state disk device is promoted to a user state, the data packet of the NVMe solid state disk device is read in the user state, and the data packet is mapped to a process space of the user state;
generally, in a user state, the interruption of an NVMe solid state disk device needs to be waited, then the read-write operation of the NVMe solid state disk device is blocked, when the NVMe solid state disk device is interrupted, the read-write operation returns immediately, wherein the return is the stop of the read-write operation of the UIO device and the return of the previous operation to achieve the interruption. The interrupt interception can bypass the kernel mode of the Shenwei processor to process data in the user mode. In other embodiments, interception of the interrupt may wait for the interrupt to occur by selecting a function. The selection function has a timeout parameter for implementing a limited time waiting for the interrupt, so as to intercept the interrupt after waiting for the interrupt to arrive within a set time, thereby eliminating the interrupt. Since the interrupt needs to be processed in the kernel state of the Shenwei processor, the kernel state processing of the Shenwei processor is saved by interrupt interception. Therefore, the port driving efficiency of the NVMe solid state disk device is improved.
The data packet promotion specifically comprises the following steps:
intercepting the interruption of the NVMe solid state disk equipment through the UIO technology, resetting the interruption behavior of the NVMe solid state disk equipment, and specifically:
the user state waits for the interruption of one NVMe solid state disk device and blocks the read-write operation of the NVMe solid state disk device, and when the NVMe solid state disk device is interrupted, the read-write operation of the NVMe solid state disk device returns immediately;
a unified interrupt handling function is registered for the UIO device, the interrupt handling function being associated with the UIO listener data structure to reset interrupt behavior to bypass the nwer processor kernel state to promote data driven to the user state.
When UIO equipment is registered, a general terminal processing function is registered for the UIO equipment, in the function, an interrupt processing function provided by the UIO equipment is called, an event notification function is called to increase the interrupt event count of the UIO equipment by one, and the read-write operation of the UIO equipment is notified to be 'data readable', wherein the interrupt processing function of each UIO equipment is independently registered. And associating each registered UIO device with an UIO monitoring data structure, wherein the UIO monitoring data structure has two attributes, the two attributes are a device pointer and an event number, the device pointer is a pointer for storing the UIO device, memory space access of the NVMe solid state disk device is facilitated, and the event number is an interrupt event counter for tracking the UIO device. The data processing flow of the Shenwei kernel is bypassed by intercepting the interrupt and resetting the interrupt processing function.
The data package in the NVMe solid state disk device is read in the user mode through a DPDK technology, a DPDK technology architecture is realized on a Shenwei platform, and the DPDK technology realizes a kernel bypass technology, so that data are output from the NVMe solid state disk device, then are polled through a polling mode drive, then pass through a DPDK base library, and finally are data storage processing services, so that the user mode directly reads and writes a memory buffer area of the NVMe solid state disk device, the data storage processing from the NVMe solid state disk device is completed through basically zero-copying of the data of the NVMe solid state disk device, and the efficient reading of the data is realized.
And carrying out Shenwei processor kernel state registration through UIO technology, and mapping registration information to a process space of a user state.
Performing Shenwei processor kernel state registration through UIO technology, and mapping registration information to a process space of a user state specifically as follows:
filling a UIO information structure body of the UIO equipment, wherein the registration information of the UIO information structure body comprises: information name, memory size, physical address, offset address;
calling a function of registering UIO equipment to register UIO information equipment into a core state of the Shenwei processor; and reading the registration information of the UIO information structure body and mapping the registration information to a process space of a user state.
The UIO application programming interface provided by a kernel of the Shenwei platform is called for registration, firstly, structure body information of the UIO equipment needs to be filled, after the structure body information is filled, a function of the UIO equipment is called for registration to register the UIO information structure body to the kernel state of the Shenwei processor, wherein the function of the UIO equipment is a registration function stored in the UIO equipment, and the UIO equipment is registered to the kernel state of the Shenwei processor by calling the registration function of the UIO equipment. After registration, the structure information just filled is available under the storage file of the UIO device, the structure information of the UIO device can be directly read and mapped to the process space of the user state, and the structure information of the UIO device is mapped to the process space of the user state, so that the memory space of the NVMe solid state disk device can be operated by the user state.
Calling a polling mode drive to process the transmission and the reception of the data packet in a polling mode, wherein the specific mode is as follows:
calling and executing polling mode driving through a CORE binding technology;
providing an application programming interface according to the polling mode driver, and configuring NVMe solid state disk equipment and a transmission queue of a data packet of the NVMe solid state disk equipment;
and driving the control processor core to poll one or more ports through the application programming interface according to the polling mode and sending the data packet to the Shenwei platform processing core for processing. The NVMe solid-state disk devices correspond to the polling mode drives one by one.
The NVMe solid-state disk device corresponds to a polling mode drive, the CORE running under the polling mode drive is in a state of 100% of a user state CPU, and an application programming interface is configured on the NVMe solid-state disk device and can directly access data packets of the NVMe solid-state disk device to be sent and received at the same time, so that completely uninterrupted rapid data processing is realized. While polling for the transmission and reception of data packets through the application programming interface in order to prevent packet transmission confusion.
And storing the user mode and sending the data packet to a storage core of the Shenwei platform for analysis and processing.
Reading a data packet of the NVMe solid state disk;
setting labels with different priorities according to the storage characteristics of the data packets, pre-reading and pre-caching the data packets to a storage core of a shenwei platform according to the priorities of the labels in sequence, wherein the storage characteristics comprise: the number of times the data packet is accessed, updated and read within a fixed time; and judging the data packets processed by the current user mode for multiple times through the accessed times, the updated times and the read time of the data packets within the fixed time, and preferentially storing the data packets with the priority so as to preferentially store the data packets accessed, updated and read for multiple times, so that the user mode processing is more in line with the port drive requirement of the NVMe solid state disk.
And analyzing the data packet according to the own protocol specification, and carrying out request identity check and/or CRC check and/or compression and/or decryption processing on the data packet.
The storage processing program is arranged in the storage core of the Shenwei platform so as to analyze the stored data packet according to the free protocol specification and carry out operations such as request identity verification and/or CRC verification and/or compression and/or decryption and the like on the data packet. Meanwhile, the analyzed data is constructed into an annular first-in first-out and lock-free queue in a zero-copy mode to provide a storage processing program for processing, and the data is released to a memory area of the NVMe solid state disk device after being dequeued and copied once.
In summary, by constructing a receiving and sending framework of a user-mode NVMe solid-state disk device data packet, then bypassing a kernel mode of the power application processor to directly read the data packet of the NVMe solid-state disk device in the user mode, and polling is performed on the sending and receiving of the data packet of the NVMe solid-state disk device through a polling mode driver, the extra consumption and consumption of a CPU are reduced, and the IO efficiency of the power application platform is improved.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (6)

1. A distributed block storage low-delay control method based on a Shenwei platform is characterized by comprising the following steps:
the data packet is promoted through the UIO technology, specifically, the data packet of the NVMe solid state disk device is promoted to a user state, the data packet of the NVMe solid state disk device is read in the user state, and the data packet is mapped to a process space of the user state;
calling a polling mode driver to process the sending and receiving of the data packet in a polling mode, wherein the NVMe solid state disk equipment corresponds to the polling mode driver one by one;
the user state stores and sends the data packet to a storage core of the Shenwei platform for analysis and processing;
the step of calling the polling mode driver to process the transmission and the reception of the data packet in a polling mode specifically comprises the following steps:
calling and executing the polling mode drive through a CORE binding technology;
providing an application programming interface according to the polling mode driver, and configuring the NVMe solid state disk device and a transmission queue of data packets of the NVMe solid state disk device;
driving and controlling a processor core to poll one or more ports through the application programming interface according to the polling mode and sending a data packet to the Shenwei platform processing core for processing;
the user state storing and sending the data packet to a storage core of the Shenwei platform for analysis and processing specifically comprises the following steps:
reading a data packet of the NVMe solid state disk;
setting labels with different priorities according to the storage characteristics of the data packets, and pre-reading and pre-caching the data packets to the storage core of the shenwei platform according to the priorities of the labels in sequence;
and analyzing the data packet according to the own protocol specification, and carrying out request identity check and/or CRC check and/or compression and/or decryption processing on the data packet.
2. The distributed block storage low-latency control method based on the Shenwei platform as claimed in claim 1, wherein the packet promotion is specifically:
intercepting the interruption of the NVMe solid state disk equipment through an UIO technology, and resetting the interruption behavior of the NVMe solid state disk equipment;
reading a data packet in the NVMe solid state disk device in a user mode through a DPDK technology;
and carrying out Shenwei processor kernel state registration through UIO technology, and mapping registration information to the process space of the user state.
3. The distributed block storage low-latency control method based on the claims 2, wherein the intercepting of the NVMe solid state disk device by the UIO technology and the resetting of the interruption behavior of the NVMe solid state disk device are specifically:
the user state waits for the interruption of one NVMe solid state disk device and blocks the read-write operation of the NVMe solid state disk device, and when the NVMe solid state disk device is interrupted, the read-write operation of the NVMe solid state disk device returns immediately;
registering a uniform interrupt handling function for the UIO device, the interrupt handling function associated with a UIO listener data structure to reset interrupt behavior to bypass the Shenwei processor kernel state and promote data drivers to a user state.
4. The distributed block storage low-latency control method based on the unwatched platform as claimed in claim 3, wherein the performing of unwatched processor kernel-state registration by UIO technology and mapping of registration information to the process space of the user state specifically include:
filling a UIO information structure body of UIO equipment, wherein the registration information of the UIO information structure body comprises: information name, memory size, physical address, offset address;
calling a function of registering UIO equipment to register UIO equipment into the Shenwei processor kernel state; and reading the registration information of the UIO information structure body and mapping the registration information to the process space of the user mode.
5. The Shenwei platform-based distributed block storage low-latency control method according to claim 4, wherein the storage characteristics comprise: the number of times the packet is accessed, updated, and the time it is read in a fixed time.
6. A distributed block storage low latency control system based on a explain-mean platform, comprising:
the data packet hoisting module is used for hoisting the data packet transmission of the NVEe solid state disk device to a user state through UIO technology, and directly reading the data packet of the NVEe solid state disk device and mapping the data packet to a process space of the user state;
the polling module is used for calling a polling mode driver to process the sending and receiving of the data packet in the NVEe solid state disk device in a polling mode;
the storage module is used for reading the data packet of the NVEe solid state disk device and storing the data packet to a storage core of the Shenwei platform;
the analysis processing module is used for analyzing and processing the data packet in the storage core of the Shenwei platform;
the polling module comprises:
the binding unit is used for driving a corresponding NVMe solid state disk device by a polling mode, and the polling mode driving is executed by a CORE binding technology of a Shenwei platform;
the application programming configuration unit is used for providing an application programming interface according to the polling mode drive and configuring the application programming interface on the NVMe solid state disk device and a transmission queue of a data packet of the NVMe solid state disk device;
the polling unit is used for calling a polling mode driver in a user mode, controlling the processor core to poll one or more NVMe solid state disk equipment ports through the application programming interface according to the polling mode driver and sending the data packet to the processor core for processing;
the memory module includes:
the reading unit is used for directly reading a data packet sent by the NVMe solid state disk device in a user mode;
the label setting unit is used for setting labels with different priorities according to the storage characteristics of the data packets;
the storage unit reads in advance and caches the data packets to a storage core of the Shenwei platform in advance according to the priority order of the tags;
the analysis processing module comprises:
the analysis unit receives the data packet stored in the storage unit and analyzes the data packet;
and the processing unit performs request identity verification, CRC verification, compression and decryption processing on the analyzed data packet to complete data processing.
CN201911001077.XA 2019-10-21 2019-10-21 Distributed block storage low-delay control method and system based on Shenwei platform Active CN110874336B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911001077.XA CN110874336B (en) 2019-10-21 2019-10-21 Distributed block storage low-delay control method and system based on Shenwei platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911001077.XA CN110874336B (en) 2019-10-21 2019-10-21 Distributed block storage low-delay control method and system based on Shenwei platform

Publications (2)

Publication Number Publication Date
CN110874336A CN110874336A (en) 2020-03-10
CN110874336B true CN110874336B (en) 2021-09-24

Family

ID=69717837

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911001077.XA Active CN110874336B (en) 2019-10-21 2019-10-21 Distributed block storage low-delay control method and system based on Shenwei platform

Country Status (1)

Country Link
CN (1) CN110874336B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111488219B (en) * 2020-04-07 2022-08-02 中国科学院自动化研究所 Ethernet data flow recording method for high-speed data acquisition system
CN111796912A (en) * 2020-07-09 2020-10-20 山东省计算中心（国家超级计算济南中心） Virtualization performance optimization method and system for storage input/output device of Shenwei platform
CN111897492B (en) * 2020-07-15 2023-02-28 杭州海康威视系统技术有限公司 Data processing method and device based on block device driver and electronic device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11290572B2 (en) * 2016-11-07 2022-03-29 Constructive Labs System and method for facilitating sharing of virtual three-dimensional space
CN108964959B (en) * 2017-05-27 2022-02-25 阿里巴巴集团控股有限公司 Network card direct connection system for virtualization platform and data packet supervision method
CN107908365A (en) * 2017-11-14 2018-04-13 郑州云海信息技术有限公司 The method, apparatus and equipment of User space memory system data interaction
US10332235B1 (en) * 2018-05-01 2019-06-25 At&T Intellectual Property I, L.P. Direct memory access for graphics processing unit packet processing
CN109803016A (en) * 2019-01-24 2019-05-24 山东超越数控电子股份有限公司 A kind of method and apparatus using virtual switch forwarding data packet

Also Published As

Publication number Publication date
CN110874336A (en) 2020-03-10

Similar Documents

Publication Publication Date Title
CN110874336B (en) Distributed block storage low-delay control method and system based on Shenwei platform
WO2017114283A1 (en) Method and apparatus for processing read/write request in physical host
JP5180373B2 (en) Lazy processing of interrupt message end in virtual environment
US9383997B2 (en) Apparatus, system, and method for persistent user-level thread
US10540294B2 (en) Secure zero-copy packet forwarding
CN107967225B (en) Data transmission method and device, computer readable storage medium and terminal equipment
CN114095251B (en) SSLVPN implementation method based on DPDK and VPP
CN111133425A (en) Hardware-based virtualization of input/output (I/O) memory management units
KR20170095607A (en) Computing device, data transfer method between coprocessor and non-volatile memory, and program including the same
US11740812B2 (en) Data storage device idle time processing
CN110865868A (en) Low-delay control method, device and equipment
CN115033188B (en) Storage hardware acceleration module system based on ZNS solid state disk
US11435958B2 (en) Shared memory mechanism to support fast transport of SQ/CQ pair communication between SSD device driver in virtualization environment and physical SSD
CN102662910A (en) Network interaction system based on embedded system and network interaction method
US20120311180A1 (en) Facilitating routing by selectively aggregating contiguous data units
CN109857545A (en) A kind of data transmission method and device
US8055817B2 (en) Efficient handling of queued-direct I/O requests and completions
US8495261B2 (en) Redispatching suspended tasks after completion of I/O operations absent I/O interrupts
CN114397999A (en) Communication method, device and equipment based on nonvolatile memory interface-remote processing message transmission
CN116257471A (en) Service processing method and device
US10771554B2 (en) Cloud scaling with non-blocking non-spinning cross-domain event synchronization and data communication
US9288163B2 (en) Low-latency packet receive method for networking devices
US8793329B2 (en) Facilitating communication between isolated memory spaces of a communications environment
CN110837482B (en) Distributed block storage low-delay control method, system and equipment
US20180167340A1 (en) Technologies for multi-core wireless network data transmission

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant