WO2024037172A1 - 一种dpu的存储空间扩展方法、装置及系统 - Google Patents

一种dpu的存储空间扩展方法、装置及系统 Download PDF

Info

Publication number
WO2024037172A1
WO2024037172A1 PCT/CN2023/101068 CN2023101068W WO2024037172A1 WO 2024037172 A1 WO2024037172 A1 WO 2024037172A1 CN 2023101068 W CN2023101068 W CN 2023101068W WO 2024037172 A1 WO2024037172 A1 WO 2024037172A1
Authority
WO
WIPO (PCT)
Prior art keywords
host
storage space
dpu
data
memory
Prior art date
Application number
PCT/CN2023/101068
Other languages
English (en)
French (fr)
Inventor
覃国
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2024037172A1 publication Critical patent/WO2024037172A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/40—Bus structure
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]

Definitions

  • the present application relates to the field of communication technology, and in particular, to a DPU storage space expansion method, device and system.
  • a data processing unit is a dedicated processor that is connected to the host and can assist the host in performing some data processing tasks. Due to the superior performance of DPU in data processing, DPU can support the offloading of basic functions such as virtualization, storage, network, and security. In other words, basic functions such as virtualization, storage, network, and security that the host needs to have can be offloaded to the DPU, and the DPU can implement these functions to achieve accelerated processing.
  • the DPU In order to facilitate the data processing of the DPU, the DPU is configured with a dedicated memory to provide storage space for the DPU.
  • the storage space of the memory will not be too large, which will limit the data processing capabilities of the DPU. For this reason You can borrow the host memory to expand the storage space of the DPU memory.
  • the traditional method is to reserve a fixed-size storage space for the DPU in the host's memory and map the storage space to the DPU's storage space.
  • the storage space reserved in this section is pre-allocated to the DPU and can only be used by the DPU. The host cannot use this section of storage space.
  • the DPU does not require large-capacity storage space, it is easy to waste storage space.
  • This application provides a DPU storage space expansion method, device and system to accelerate format conversion and reduce CPU consumption.
  • embodiments of the present application provide a method for expanding the storage space of a DPU.
  • the method is used to expand the storage space of the DPU.
  • the DPU is connected to the host through a system bus.
  • the DPU can be located outside the host as an external device of the host; the DPU can also be deployed inside the host, for example, deployed on the motherboard or backplane of the host, and the DPU is connected to the host through a PCIe bus or other types of buses. motherboard or backplane.
  • the DPU needs to expand its own storage space, or when its own storage space is insufficient, it initiates a request message to the host.
  • the request message is used to apply for the host storage space; after the host obtains the request message, it stores the request message in the host's memory based on the request message. Configure a host storage space for the DPU, and send a response message carrying the address of the host storage space to the DPU. After obtaining the response message, the DPU maps the host storage space to the storage space used by the DPU, and reads and writes data to the host storage space.
  • the DPU can apply for storage space from the host according to its own needs, so that the storage space configured by the host for the DPU is no longer fixed and avoids waste of storage space.
  • the DPU can also release the host storage space.
  • the DPU sends a notification message to the host.
  • the notification message is used to notify the release of the host storage space.
  • the host reclaims the host storage space and uses the storage space.
  • the DPU can promptly notify the host to release the host storage space to ensure that the host can continue to use the host storage space and improve the resource utilization of the host storage space.
  • the request message sent by the DPU also indicates the size of the host storage space.
  • the size of the host storage space is expressed as the number of pages.
  • the DPU can determine the size of the host storage space to be applied for according to its own needs, and initiate a request message to the host, making the application for host storage space more flexible and avoiding the waste of host storage space.
  • the DPU has its own memory.
  • the DPU is connected to the memory through a system bus.
  • the memory includes a host page cache. After the DPU applies for the host storage space from the host, it uses DMA to transfer the host storage space to the host. The data is written to the host page cache or the host page cache is written back to the host storage space through DMA.
  • the DPU when the DPU needs to write data to the host storage space, the DPU writes the data in the host page cache and uses DMA to The data in the host page cache is written to the host storage space.
  • the DPU when the DPU needs to read data in the host storage space, the DPU writes the data in the host storage space into the host page cache through DMA, and reads the data from the host page cache.
  • the DPU can use its own memory and DMA to read and write data to the host's storage space without going through the host (such as the processor in the host), so that the data to the host's storage space can be read and written.
  • the way to write data is simpler and faster.
  • the DPU when the DPU releases the host storage space, if the data in the host page cache includes data in the host storage space, the DPU marks the data in the host storage space stored in the host page cache. is invalid data, or the data in the host storage space is deleted.
  • the data in the host storage space in the host page cache is marked as invalid data in a timely manner, or the data in the host storage space in the host page cache is deleted to reduce the host page cache. of occupation.
  • the interaction between the DPU and the host can be implemented based on the virtualization input and output (virtual I/O) protocol, or can also be implemented based on other protocols.
  • the DPU and the host interact based on the mature virtual I/O protocol, making the relationship between the DPU and the host simpler.
  • the DPU when the DPU maps the storage space of the host to the storage space of the DPU, it maps the address of the host storage space to the logical address of the storage space of the DPU.
  • the address of the host storage space may be the physical address of the host storage space. Establish a mapping relationship between the address of the host storage space and the logical address of the DPU's storage space.
  • the DPU maps the address of the host storage space to the logical address of the DPU's storage space.
  • the upper-layer software of the DPU only senses the logical address of the DPU's storage space and does not need to clarify the DPU's storage space.
  • the storage space indicated by the logical address of the storage space is located in the host's memory or in the DPU's own memory.
  • the DPU uses the data in the host storage space to perform some data processing operations. For example, the DPU obtains the data stored in the host storage space, encapsulates the data, and sends the encapsulated data to a device outside the DPU.
  • This processing method is suitable for scenarios where the DPU supports network functions.
  • the DPU encrypts data and sends the encrypted data to a device outside the DPU.
  • This processing method is suitable for scenarios where the DPU has security functions.
  • the DPU stores data in a local storage device of the host, or stores the data in a remote storage device connected to the DPU through a network. This method is suitable for scenarios where the DPU has a storage function.
  • the local storage device of the host is the storage device used by the host for persistent storage, such as a hard disk, etc.
  • the remote storage device is a storage device deployed remotely for persistent storage, such as a storage device deployed at the remote end. devices in the system.
  • the DPU calls to apply for the storage space of the host, helping the host to undertake part of the data processing operations and realizing some basic functions.
  • the DPU stores some data in the host storage space after applying for the host storage space.
  • the DPU can decapsulate the data packets from the device outside the DPU and convert the decoded data into the host storage space. The data obtained after encapsulation is written to the host storage space.
  • This processing method is suitable for scenarios where the DPU supports network functions.
  • the DPU can decrypt data from a device or host outside the DPU, and write the decrypted data into the host storage space. This processing method is suitable for scenarios where the DPU has security functions.
  • the DPU accesses a storage device external to the DPU, reads data from the host's local storage device or the host's remote storage device, and stores the read data in the host's storage space.
  • This method is suitable for scenarios where the DPU has storage functions. middle.
  • the DPU calls to apply for the storage space, helping the host to undertake part of the data processing operations and realizing some basic functions.
  • embodiments of the present application also provide a data processing device, which has the function of implementing the behavior in the method example of the above-mentioned aspect.
  • the beneficial effects can be found in the description of the first aspect and will not be described again here.
  • the functions described can be implemented by hardware, or can be implemented by hardware executing corresponding software.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the structure of the data processing device includes a transmission module, a mapping module, and optionally a writing module, a reading module, a first processing module, and a second processing module. These modules can perform the corresponding functions in the above method examples of the first aspect. For details, please refer to the detailed description in the method examples, which will not be described again here.
  • inventions of the present application also provide a data processing device that has the function of implementing the behavior in the method example of the first aspect.
  • the beneficial effects can be found in the description of the first aspect and will not be described again here.
  • the structure of the device includes a DPU and a power supply circuit.
  • the power supply circuit is used to supply power to the DPU.
  • it may also include a memory and a communication interface.
  • the DPU is connected to the host through a system bus, and the DPU is configured to support the data processing device in executing corresponding functions in the method of the first aspect.
  • the memory is connected to the DPU through a system bus, and it stores the necessary computer program instructions and data (such as data in the storage space) of the communication device.
  • the memory can be one of RAM, ROM, DRAM, flash memory media, hard disk and other media, or Various combinations.
  • This application does not limit the connection method between the DPU and the host.
  • the DPU is connected to the backplane or motherboard of the host through a PCIe bus or other types of buses.
  • the structure of the data processing device also includes a communication interface for communicating with other devices or hosts, such as sending request messages, receiving response messages, sending notification messages, etc.
  • inventions of the present application also provide a computing system.
  • the computing system includes a data processing device and a host.
  • the data processing device includes a DPU.
  • the DPU is connected to the host through a system bus.
  • the DPU is connected to the host through a PCIe bus.
  • the DPU has the function of realizing the behavior in the method example of the first aspect. The beneficial effects can be found in the description of the first aspect and will not be described again here.
  • the data processing device may be part of the host and be deployed inside the host.
  • the DPU is connected to the backplane or motherboard of the host through the PCIe bus.
  • the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium stores instructions that, when run on a computer, cause the computer to execute the above-mentioned first aspect and various possibilities of the first aspect. The method described in the embodiment.
  • the present application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the method described in the above-mentioned first aspect and each possible implementation of the first aspect.
  • the present application also provides a computer chip, the chip is connected to a memory, the chip is used to read and execute the software program stored in the memory, and execute the above first aspect and each possibility of the first aspect. The method described in the embodiment.
  • Figure 1 is an architectural schematic diagram of a computing system provided by this application.
  • Figure 2 is a schematic diagram of a storage space expansion method provided by this application.
  • Figure 3 is a schematic diagram of sending a request message provided by this application.
  • Figure 4 is a schematic diagram of sending a response message provided by this application.
  • Figure 5 is a schematic structural diagram of a data processing device provided by this application.
  • the system 10 includes a host 20 and a data processing device 30 .
  • the host 20 is a common computer device.
  • the host 20 includes but is not limited to a personal computer, a server, a mobile phone, a tablet computer or a smart car.
  • the host 20 can communicate with other devices outside the system and receive data sent by other devices to the host 20 to request the host 20 to process the data.
  • both the host 20 and the data processing device 30 have data processing functions.
  • the host 20 can process the data by itself.
  • the host 20 can also transfer the data to the data processing device 30, and the data processing device 30 can process the data. This data is processed. After the host 20 or the data processing device 30 completes the data processing, the data processing results can be fed back to other devices.
  • the data processing device 30 is connected to the host 20.
  • the data processing device 30 can be used as an external device of the host 20, or can be deployed inside the host 20.
  • the data processing device 30 is located on the mainboard or backplane of the host 20.
  • the data processing device 30 can be used as an external device of the host 20.
  • the device 30 (such as the DPU 301 in the data processing device 30) exchanges data with the host 20 (such as the processor 201 in the host 20) through a bus.
  • the bus 300 can be a peripheral component interconnect express (PCIe) bus, It can also be a bus of Compute Express Link (CXL), Universal Serial Bus (USB) protocol or other protocols.
  • PCIe peripheral component interconnect express
  • CXL Compute Express Link
  • USB Universal Serial Bus
  • the data processing device 30 can be used as a module with a data processing function attached to the host 20 and assume part of the functions of the host 20 . That is to say, part of the functions of the host 20 are offloaded to the data processing device 30, and the data processing device 30 processes data instead of the host 20 and performs part of the tasks to reduce the pressure on the host 20, especially the processing in the host 20.
  • the pressure on the processor 201 is released, and the computing power of the processor 201 is released.
  • the embodiment of the present application does not limit the functions that the data processing device 30 can realize in place of the host 20 .
  • the data processing device 30 can be installed on the host 20 as a network card of the host 20 .
  • the data processing device 30 serves as a network card and can process data packets based on network protocols, such as encapsulating and transmitting data.
  • the data processing device 30 can also be used to support data security and can encrypt and decrypt data.
  • the data processing device 30 can also serve as a storage entry, through data
  • the data processing device 30 implements distributed storage of data and remote access of data.
  • the data processing device 30 can access some local storage devices of the host 20 (such as hard disks and other devices used for persistent storage). You can also access remote storage devices (such as devices in a remote storage system) through the network.
  • the data processing device 30 and the host 20 each have a memory 302 internally.
  • the memory 302 in the data processing device 30 can provide storage space to support some data processing operations of the data processing device 30 .
  • the memory 202 in the host 20 can provide storage space to support some data processing operations of the host 20 .
  • the data processing device 30 is allowed to "borrow" the storage space in the host 20 according to its own needs (the storage space in the host 20 can also be referred to as the storage space of the host 20 for short), that is, the data processing device 30 can use
  • the storage space of the memory 202 in the host 20 is used to support its own data processing operations.
  • the data processing device 30 applies for host storage space from the host 20 according to its own needs.
  • the host storage space is the storage space in the memory 202.
  • the host 20 can send the address of the host storage space to the data processing device 30, so that the data processing device 30
  • the host storage space can be mapped to the data processing device 30's own storage space, and the host storage space can be used.
  • the data processing device 30 When the data processing device 30 no longer needs the storage space in the memory 202, the data processing device 30 releases the host storage space and notifies the host 20 to release the host storage space. The host 20 reclaims the host storage space upon notification from the data processing device 30 .
  • the data processing device 30 can apply for the storage space of the host 20 according to its own needs, or release the storage space of the host. That is to say, the host 20 no longer needs to reserve a fixed size of storage space for the data processing device 30 , and only needs to allocate storage space according to the needs of the data processing device 30 when the data processing device 30 applies for host storage space.
  • the data processing device 30 can access the storage space in the memory 202 of the host 20 on demand, thereby avoiding excessive occupation of the memory 202 and reducing the waste of storage space.
  • the host 20 includes an I/O interface 203, a processor 201, and a memory 202.
  • the I/O interface 203 is used to communicate with devices located outside the host 20 .
  • the external device can send data to the host 20 through the I/O interface 203.
  • the host 20 After the host 20 processes the input data, the host 20 then sends the output result of processing the data to the external device through the I/O interface 203.
  • the processor 201 is the computing core and control core of the host 20. It can be a central processing unit (CPU) or other specific integrated circuits.
  • the processor 201 can also be other general-purpose processors, digital signal processing (DSP), application specific integrated circuit (ASIC), field programmable gate array (field programmable gate array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • Programmable logic devices discrete gate or transistor logic devices, discrete hardware components, etc.
  • the memory 202 is usually used to store various running computer program instructions in the operating system of the host 20, data that needs to be processed, and data processing results. In order to improve the access speed of the processor 201, the memory 202 needs to have the advantage of fast access speed.
  • the memory 202 usually uses dynamic random access memory (DRAM) as the memory 202.
  • DRAM dynamic random access memory
  • the memory 202 can also be other random access memories, such as static random access memory (Static random access memory, SRAM), etc.
  • the memory 202 may also be a read only memory (ROM).
  • read-only memory can be programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc.
  • the memory 202 can also be a flash memory medium (FLASH), a hard disk drive (hard disk drive, HDD) or a solid state drive (solid state disk, SSD), etc.
  • the processor 201 is connected to the memory 202 through a double data rate (DDR) bus or other types of buses.
  • the memory 202 is understood as the memory (internal memory) of the host 20, and the memory is also called main memory (main memory).
  • the processor 201 executes some methods that the host 20 needs to execute in the embodiment shown in FIG. 2 below by calling computer program instructions in the memory 202 .
  • the data processing device 30 includes a data processing unit (DPU) 301 and a memory 302.
  • the DPU 301 and the memory 302 are connected through a system bus.
  • the system bus can be a line based on PCIe, or it can be CXL, USB protocol or other protocols. bus.
  • the DPU 301 is the main computing unit of the data processing device 30 and the core unit of the data processing device 30 .
  • the DPU 301 assumes the main functions of the data processing device 30 .
  • some functions of the host 20 can be offloaded to the DPU 301 , and the DPU 301 processes data and performs tasks assigned by the host 20 to the data processing device 30 .
  • the DPU 301 executes some methods that the DPU 301 needs to execute in the embodiment shown in FIG. 3 below by calling computer program instructions in the memory 302 .
  • DPU 301 can also apply for host storage space or release host storage space according to its own needs.
  • the memory 302 can support the data processing operations of the DPU 301 and provide data storage space for the DPU 301, similar to the memory 202.
  • the memory 302 is used to store computer program instructions that need to be called by the DPU 301, data that needs to be processed, data processing results, etc.
  • the type of memory 302 is similar to the type of memory 202. For details, please refer to the foregoing content and will not be described again here.
  • the memory 302 is understood to be the internal memory of the data processing device 30 .
  • data is organized in page granularity. That is to say, the physical storage space of the memory 302 in the memory 302 is divided into pages.
  • the DPU 301 also accesses the memory 302 with a page granularity, and the processor 201 also accesses the memory 202 with a page granularity.
  • the data processing device 30 can be directly inserted into the card slot on the motherboard of the host 20 , and exchanges data with the processor 201 through the PCIe bus 204 .
  • the PCIe bus 204 in Figure 1 can be replaced by a bus using the compute express link (CXL), universal serial bus (USB) protocol or other protocols to allow the data processing device 30 to perform data transmission.
  • CXL compute express link
  • USB universal serial bus
  • the physical storage space of the memory 302 is mapped into a virtual storage space.
  • the address of the physical storage space is called the physical address
  • the address of the virtual storage space is called the logical address.
  • the DPU 301 is provided with a management unit for the memory 302 .
  • the management unit can implement management of the memory 302, which management includes but is not limited to: management of the mapping relationship between physical addresses and logical addresses, applying for storage space (storage space with page granularity) from the host 20, The software allocates storage space (storage space with page granularity) and reads and writes data.
  • the management unit can realize the mapping between physical storage space and virtual storage space, establish the mapping relationship between physical address and logical address, and delete the established mapping relationship between physical address and logical address.
  • the software program (also called software) on the DPU 301 only needs to sense the logical address, and the conversion between the logical address and the physical address is implemented by the management unit.
  • the management unit will provide the logical address of the storage space allocated to the software program to the software of DPU301.
  • the software program running on DPU301 needs to write data to the storage space
  • the software program running on DPU301 provides the logical address and the data to be written to the management unit.
  • the management unit is based on the relationship between the physical address and the logical address. Mapping relationship, convert the logical address to a physical address, and write the data to the physical address.
  • the software program running on DPU301 when the software program running on DPU301 needs to read data from the storage space, the software program running on DPU301 provides a logical address to the management unit, and the management unit converts the logical address to the logical address based on the mapping relationship between the physical address and the logical address. Convert to a physical address, read data from the physical address, and feed the data back to the software program.
  • the software program on the DPU301 will require a certain amount of storage space to perform data processing operations during its operation.
  • the process of data processing operations is accompanied by reading or writing data in the storage space of the host. For example, reading data that needs to be processed, or writing processed data.
  • the software program on DPU301 applies for storage space from the management unit according to its own needs.
  • the management unit determines the free physical storage space in the memory 302, allocates the free physical storage space to the software running on the DPU 301, maps the physical address of the free physical storage space to a logical address, and saves the relationship between the physical address and the logical address. mapping relationship and provide that logical address to the software running on the DPU301.
  • the management unit can apply for host storage space from the host 20, and the host 20 allocates the storage space in the memory 202 to the management unit, and sends the address of the storage space in the memory 202 (the address can be the physical address of the storage space) to the management unit. .
  • the management unit After obtaining the physical address of the storage space in the memory 202, the management unit maps the physical address to a logical address, saves the mapping relationship between the physical address and the logical address, and provides the logical address to the software program running on the DPU 301.
  • the management unit writes data in the storage space applied by the software on the DPU301, or reads data from the storage space applied by the software program on the DPU301.
  • the software running on DPU301 When the software program running on DPU301 needs to write data to the storage space, the software running on DPU301 initiates a request to the management unit.
  • Write instruction which carries the logical address and the data to be written.
  • the management unit Based on the mapping relationship between the physical address and the logical address, the management unit converts the logical address into a physical address and writes the data to the physical address.
  • the software running on DPU301 When the software program running on DPU301 needs to read data from the storage space, the software running on DPU301 initiates a read instruction to the management unit.
  • the read instruction carries a logical address
  • the management unit is based on the mapping between the physical address and the logical address. relationship, convert the logical address into a physical address, read data from the physical address, and feed the data back to the software.
  • the management unit can be understood as a software program running on the DPU301, and can also be understood as a part of the hardware modules on the DPU301.
  • the embodiments of this application do not limit the specific form of the management unit.
  • a storage space application method provided by the embodiment of the present application will be described below with reference to Figure 2.
  • the embodiment shown in Figure 2 includes two parts. One part is for the DPU 301 to apply for storage space from the host 20. For details, see steps 201 to 204. The other part is that the DPU 301 requests the host 20 to release the storage space. For details, please refer to steps 205 to 206.
  • Step 201 DPU 301 initiates a request message to the host 20.
  • the request message is used to apply for host storage space from the host 20.
  • the request message also indicates the size of the host storage space to be applied for.
  • the size of the page in the memory 302 is the same as the size of the page in the memory 202, and the size of the storage space can be expressed by the number of pages.
  • One or more software programs can be run on the DPU301.
  • the operation of the software program requires storage space.
  • the software program needs to write data or read data in the storage space.
  • the DPU301 can perform steps 201.
  • the software program can apply for storage space to the management unit and inform the management unit of the size of the storage space that needs to be applied for.
  • the management unit determines that there is no free space in the memory 302. In the case of storage space, a request message is initiated to the host 20.
  • DPU301 when a software program running on DPU301 needs to expand the storage space, the software program applies to the management unit for storage space and informs the management unit of the size of the required expanded storage space.
  • the management unit When it is determined that there is no free storage space in the memory 302, a request message may be initiated to the host 20.
  • the embodiment of this application does not limit the interaction method between the DPU 301 and the host 20 .
  • the DPU 301 and the host 20 can interact based on existing protocols.
  • the DPU 301 and the host 20 can interact based on the virtual input/output (virt virtual I/O) protocol.
  • the virtual I/O protocol defines a general IO virtualization model.
  • virtual I/O is an abstraction for a common set of simulated devices in a paravirtualized hypervisor.
  • the devices connected to the host 20 are unified to achieve unified management, maintenance, expansion, etc. In this way, these devices implement functions according to the virtual I/O protocol definition and avoid updating the hypervisor of host 20.
  • DPU301 deploys a virtual I/O backend driver
  • the host 20 deploys a virtual I/O frontend driver.
  • the interaction between the DPU 301 and the host 20 can be realized through the virtual I/O queue jointly maintained by the virtual input and output back-end driver and the virtual input and output front-end driver.
  • the virtual I/O queue includes a request queue and a reply queue.
  • the request queue is used to carry the request instructions added by the virtual input and output front-end driver.
  • the reply queue is used to carry the reply instructions added to the request by the virtual input and output backend.
  • the host 20 When the host 20 needs to initiate a request instruction to the DPU 301, it adds the request instruction to the request queue of the virtual I/O queue through the virtual input and output front-end driver. In this way, the virtual input and output back-end driver deployed in the DPU 301 obtains the request instruction from the virtual I/O back-end driver.
  • the request instruction is obtained from the I/O queue, the DPU 301 processes the request instruction, and places the processing result as a reply instruction in the reply queue of the virtual I/O queue.
  • the virtual input and output front-end driver can send the virtual input to the virtual input through the virtual I/O queue.
  • the output front-end driver returns a reply command.
  • the DPU 301 and the host 20 need to configure the virtual input and output front-end driver deployed by the host 20 in advance in the virtual I/O Add a request instruction to the queue (the request instruction may not indicate any actual content), so that when the DPU 301 subsequently needs to apply for storage space in the memory 202 of the host 20 , the virtual input and output back-end driver deployed in the DPU 301 will be used to apply for storage.
  • the space application message is added to the virtual I/O queue as a reply instruction.
  • the management unit determines that there is no free storage space in the memory 302, the management unit initiates the request message to the virtual input and output backend driver, and the virtual input and output backend driver adds the request message to the virtual input and output backend driver as a reply instruction. in the I/O queue.
  • the above method of the DPU 301 sending the request message to the host 20 is only an example.
  • the DPU 301 and the host 20 can also interact based on other protocols.
  • a new protocol can be redefined between the DPU 301 and the host 20 and the interaction can be carried out based on the new protocol.
  • the interaction between the DPU 301 and the host 20 is essentially the interaction between the DPU 301 and the processor 201 in the host 20.
  • this application refers to the interaction between the DPU 301 and the processor 201 in the host 20.
  • the interaction is collectively referred to as the interaction between the DPU 301 and the host 20 .
  • Step 202 After receiving the request message, the host 20 applies for host storage space for the DPU 301 from the memory 202 of the host 20. If the request message carries the number of pages that need to be applied for, the host 20 applies for the corresponding number of pages from the memory 202 .
  • the virtual input and output front-end driver applies for storage space from the memory 202.
  • the memory 202 will think that the host storage space is used by the virtual input and output front-end driver, and it is not clear that the host storage space is used by the DPU 301.
  • the memory 202 configures a storage space for the virtual input and output front-end driver.
  • the size of the host storage space is consistent with the size of the storage space indicated in the request message, and feeds back the physical address (for example, the physical address of the page) of the host storage space.
  • the virtual input and output front-end driver is consistent with the size of the storage space indicated in the request message, and feeds back the physical address (for example, the physical address of the page) of the host storage space.
  • Step 203 The host 20 sends a response message to the DPU 301.
  • the response message carries the address of the host's storage space.
  • the response message may carry the address of the page in the memory 202 .
  • the virtual input and output front-end driver obtains the address of the host storage space from the memory 202, it adds the address of the host storage space to the virtual as a request instruction. in the I/O queue.
  • the virtual input and output front-end driver can also initiate another request instruction (the request instruction may not indicate any actual content) to add to the virtual I/O. O queue.
  • Step 204 After receiving the response message, DPU301 obtains the address of the host storage space, maps the host storage space to the storage space of DPU301, and uses the host storage space. For example, the DPU 301 maps the address of the host storage space into a logical address, and feeds back the logical address to the software program in the DPU 301 that applies for the storage space.
  • the management unit in the DPU 301 After obtaining the response message, the management unit in the DPU 301 obtains the physical address of the host storage space from the response message, maps the physical address to a logical address, and saves the mapping relationship between the physical address and the logical address.
  • the memory 302 inside the memory 302, the memory 302 includes a host page cache.
  • the host page cache is used to store data in the storage space of the host 20.
  • the host storage space is the storage of the memory 202 that the DPU 301 applies to the host 20. space.
  • the host page cache can be used to store the data in the pages of the memory 202 that the DPU 301 applies to the host 20 .
  • the memory 302 accesses the host storage space through direct memory access (DMA), such as reading data from the host storage space and storing the read data in the host page cache. Another example is writing the data saved in the host page cache to the host storage space.
  • DMA direct memory access
  • the software program running on the DPU301 After obtaining the logical address, the software program running on the DPU301 reads and writes data for the logical address.
  • the data processing device 30 can assume some functions of the host computer 20 .
  • the DPU 301 can call the host storage space to perform some data processing operations.
  • DPU301 can obtain the data stored in the host storage space and process the data.
  • DPU301 can encapsulate the acquired data based on the network protocol and send the encapsulated data, for example, sending the encapsulated data to a device external to the DPU301; the DPU301 can also encrypt the data and save the encrypted data in the The encrypted data can be stored in the host storage space or sent to a device external to the DPU.
  • DPU301 can also access external storage devices used to persistently store data. For example, DPU301 can access the local storage device of the host or a remote storage device connected to the DPU301 through a network, and store the data stored in the host's storage space in The local storage device of the host or the remote storage device connected to the DPU301 through the network.
  • the DPU 301 When the DPU 301 needs to obtain the data stored in the host storage space, the DPU 301 instructs the memory 302 to store the data in the host storage space in the host page cache through DMA.
  • the software program that needs to use the data initiates a read instruction to the management unit, and the read instruction carries the logical address of the host storage space.
  • the management unit determines the physical address based on the logical address, and the management unit instructs the memory 302 to read data from the host page cache based on the physical address. If the data currently stored in the host page cache is data in the host storage space, the memory 302 directly reads the data from the host page cache.
  • the memory 302 passes through according to the address of the host storage space. DMA reads this data from the host memory space, Write this data to the host page cache. The memory 302 will read the data from the host page cache and feed it back to the management unit, which will feed the data back to the software program.
  • DPU301 can store the processed data in the host storage space. For example, DPU301 decrypts the acquired data based on the network protocol, and stores the decrypted data in the host storage space; for another example, DPU301 decrypts the acquired encrypted data, and stores the decrypted data in the host storage space. in the host storage space. For another example, DPU301 accesses an external storage device, such as a local storage device of the host or a remote storage device connected to the DPU301 through a network, reads data from the external storage device, and stores the read data on the host. in storage space.
  • an external storage device such as a local storage device of the host or a remote storage device connected to the DPU301 through a network, reads data from the external storage device, and stores the read data on the host. in storage space.
  • the DPU 301 When the DPU 301 needs to store data in the host storage space, the DPU 301 instructs the memory 302 to store the data in the host page cache of the memory 302. Then the memory 302 stores the data in the host page cache in the host storage space through DMA.
  • the software program that needs to store the data initiates a write instruction to the management unit, and the write instruction indicates the logical address and the data that needs to be written.
  • the management unit determines the physical address based on the logical address. If the physical address is the address of the storage space of the host 20 , the management unit instructs the memory 302 to write the data into the host page cache. If the data in the current host page cache is not full, that is, there is free storage space in the host page cache, the memory 302 directly writes the data into the host page cache.
  • the memory 302 If the data in the current host page cache is full, the memory 302 writes the data in the host page cache back to the storage space of the host 20 through DMA. After the data is written back, memory 302 writes the data into the host page cache.
  • Step 205 DPU 301 initiates a notification message to the host 20.
  • the notification message is used to notify the host 20 to release the storage space.
  • the notification message also carries the address of the storage space.
  • the size of the page in the memory 302 is the same as the size of the page in the memory 202, and the size of the storage space can be expressed by the number of pages.
  • the physical address of the page that needs to be released is used as the physical address of the storage space.
  • the software program on the DPU301 can apply to release the storage space that the software program has previously applied for.
  • the storage space applied for release by the software program on the DPU301 may be all the storage space previously applied for by the software program, or it may be part of the storage space.
  • the software program on DPU301 can apply to the management unit to release storage space and inform it of the logical address of the storage space that needs to be released.
  • the management unit After obtaining the logical address of the host storage space, the management unit deletes the mapping relationship between the logical address and the physical address of the host storage space. If the data of the host storage space is still cached in the host page cache of the memory 302, the management unit marks the data of the host storage space in the host page cache as invalid data.
  • the management unit initiates the notification message to the virtual input and output backend driver, and the virtual input and output backend driver adds the notification message to the virtual I/O queue as a reply command.
  • Step 206 After receiving the notification message, the host 20 releases the host storage space.
  • the virtual input and output front-end driver deployed in the host 20 obtains the notification message from the virtual I/O queue
  • the virtual input and output front-end driver instructs the memory 202 to release storage space.
  • the memory 202 releases the host storage space according to the virtual input and output front-end driver instruction.
  • the embodiment of the present application also provides a data processing device, which is used to execute the method executed by the DPU 301 in the above method embodiment as shown in Figure 5.
  • the relevant features can be Please refer to the above method embodiments, which will not be described again here.
  • the data processing device can be understood as a software form of the management unit mentioned in the previous description.
  • the data processing device 500 includes a transmission module 501 and a mapping module 502 .
  • the transmission module 501 is used to initiate a request message to the host, the request message is used to apply for the host storage space; and obtain a response message fed back by the host, the response message carries the address of the host storage space, and the host storage space is located in the memory of the host.
  • the mapping module 502 is used to map the host storage space to the storage space used by the DPU.
  • the DPU and the host are connected through a system bus.
  • mapping module 502 when the mapping module 502 maps the physical address of the host storage space to the storage space of the DPU, the mapping module 502 can map the address of the host storage space to the logical address of the DPU's storage space, establishing The mapping relationship between the address of the host storage space and the logical address of the DPU storage space.
  • the mapping module 502 deletes the mapping relationship between the physical address and the virtual address; the transmission module 501 sends a notification message to the host, and the notification message is used to notify the release host storage.
  • the request message also indicates the size of the host storage space.
  • the DPU is connected to the memory through a system bus, and the memory includes a host page cache.
  • the data processing device 500 also includes a writing module 503 that can write data in the host storage space. The writing module 503 writes to the When the host storage space writes data, the data is written in the host page cache, and the data in the host page cache is written to the host storage space through DMA.
  • the data processing device 500 includes a reading module 504.
  • the reading module 504 can read data from the host storage space. When reading the data, the reading module 504 transfers the host storage space to the host storage space through DMA. Data is written to the host page cache and data is read from the host page cache.
  • the mapping module 502 when releasing the host storage space, marks the data in the host storage space stored in the host page cache as invalid data.
  • the transmission module 501 interacts with the host based on the virtual I/O protocol.
  • the data processing device 500 further includes a first processing module 505, which is capable of performing some data processing operations on the data in the host storage space.
  • the first processing module 505 obtains the data stored in the host storage space and performs some or all of the following:
  • the data processing device 500 further includes a second processing module 506, and the second processing module 506 can store the processed data in the storage space.
  • the second processing module 506 decapsulates the data packet from the external device and writes the decapsulated data into the host storage space.
  • the second processing module 506 decrypts the data from the external device and writes the decrypted data into the host storage space.
  • the second processing module 506 reads data from a local storage device of the host or a remote storage device of the host, and stores the read data in the host storage space.
  • the data processing device 500 may correspond to performing the device described in the embodiment of the present application, and the above and other operations and/or functions of each module in the device 500 are respectively to implement the corresponding functions of each device in Figure 2 The process, for the sake of brevity, will not be repeated here.
  • each functional module in the embodiment of the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules.
  • the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination.
  • the above-described embodiments may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer program instructions.
  • the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer program instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer program instructions may be transmitted from a website, computer, server, or
  • the data center transmits data to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or a data center that contains one or more sets of available media.
  • the usable media may be magnetic media (eg, floppy disk, hard disk, tape), optical media (eg, DVD), or semiconductor media.
  • the semiconductor medium may be a solid state drive (SSD).
  • embodiments of the present application may be provided as a device, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions
  • the device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.
  • These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device.
  • Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Memory System Of A Hierarchy Structure (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种DPU（301）的存储空间扩展方法、装置（500）及系统（10），DPU（301）在需要扩展自身的存储空间，或者自身存储空间不足的情况下，向与其通过总线（204）连接的主机（20）发起请求消息，请求消息用于申请主机存储空间；主机（20）根据请求消息在主机（20）的存储器（202）中为DPU（301）配置存储空间，向DPU（301）发送携带有存储空间的地址的响应消息；DPU（301）将主机存储空间映射为DPU（301）的存储空间使用。DPU（301）能够根据自身需求向主机（20）申请主机存储空间，使得主机（20）为DPU（301）配置的存储空间不再是固定不变的，避免存储空间的浪费。

Description

一种DPU的存储空间扩展方法、装置及系统
相关申请的交叉引用
本申请要求在2022年08月19日提交中国专利局、申请号为202210998035.3、申请名称为“一种DPU的存储空间扩展方法、装置及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信技术领域，尤其涉及一种DPU的存储空间扩展方法、装置及系统。
背景技术
数据处理单元(data process unit，DPU)是一种专用处理器，该数据处理单元连接到主机上，能够协助主机执行一些数据处理类的任务。由于DPU在数据处理方面优越的性能，DPU能够支持虚拟化、存储、网络、安全等基础功能的卸载。也就是说，主机需要具备的虚拟化、存储、网络、安全等基础功能可以卸载到DPU中，由DPU来实现这些功能，达到加速处理的效果。
为了便于DPU进行数据处理，DPU配置有专用的存储器，为该DPU提供存储空间，但从成本以及硬件的约束，该存储器的存储空间不会太大，这将限制DPU的数据处理能力，为此可以借用主机内存对DPU的存储器的存储空间进行扩展。传统方式是在主机的内存中为该DPU预留一段固定大小的存储空间，将该存储空间映射到DPU的存储空间中。该段预留的存储空间是预先分配给DPU，只能由该DPU使用，主机并不能使用该段存储空间。在该DPU不需要大容量的存储空间时，容易造成存储空间的浪费。
发明内容
本申请提供一种DPU的存储空间扩展方法、装置及系统，用以加快实现格式转换，减少对CPU的消耗。
第一方面，本申请实施例提供了一种DPU的存储空间扩展方法，该方法用于扩展DPU的存储空间，DPU与主机通过系统总线连接。该DPU可以位于主机外部，作为该主机的外接设备；该DPU也可以部署在主机内部，例如，部署在该主机的主板或者背板上，DPU通过PCIe总线或其他类型的总线连接在该主机的主板或者背板上。DPU在需要扩展自身的存储空间，或者自身存储空间不足的情况下，向主机发起请求消息，请求消息用于申请主机存储空间；主机在获取该请求消息后，根据该请求消息在该主机的存储器中为该DPU配置主机存储空间，向DPU发送携带有该主机存储空间的地址的响应消息。DPU在获取该响应消息后，将主机存储空间映射为DPU的存储空间使用，对该主机存储空间进行数据读取以及写入。
通过上述方法，DPU能够根据自身需求向主机申请存储空间，使得主机为该DPU配置的存储空间不再是固定不变的，避免存储空间的浪费。
在一种可能的实施方式中，DPU在向主机申请了该存储空间后，当无需使用该存储空间时，还可以释放该主机存储空间，DPU在释放该主机存储空间时，向主机发送通知消息，通知消息用于通知释放主机存储空间，主机在接收到该通知消息后，回收该主机存储空间，使用该存储空间。
通过上述方法，DPU能够及时的告知主机释放该主机存储空间，以保证主机可以继续使用该主机存储空间，以提高主机存储空间的资源利用率。
在一种可能的实施方式中，DPU所发送的请求消息还指示了主机存储空间的大小。该主机存储空间的大小表示为页的数量。
通过上述方法，DPU能够按照自身需求决定需申请的主机存储空间的大小，并向主机发起请求消息，使得主机存储空间的申请更加灵活，避免了主机存储空间的浪费。
在一种可能的实施方式中，DPU具备自己的存储器，该DPU与该存储器通过系统总线连接，存储器包括主机页面缓存，DPU在向主机申请了主机存储空间后，通过DMA的方式将主机存储空间的数据写入到主机页面缓存或通过DMA的方式将主机页面缓存写回至主机存储空间。
例如，DPU在需要向主机存储空间写入数据时，DPU在主机页面缓存中写入数据，通过DMA将 主机页面缓存中的数据写入到主机存储空间中。
又例如，DPU在需要读取主机存储空间中的数据时，DPU通过DMA将主机存储空间的数据写入到主机页面缓存中，从主机页面缓存读取数据。
通过上述方法，DPU能够借助自身的存储器以及DMA实现对该主机存储空间的数据读取以及数据写入，无需通过主机(如主机中的处理器)，使得对该主机存储空间的数据读取以及数据写入的方式更加简单、快捷。
在一种可能的实施方式中，DPU在释放该主机存储空间时，若该主机页面缓存中的数据包括该主机存储空间中的数据，将主机页面缓存中存储的该主机存储空间中的数据标注为无效数据，或删除该主机存储空间中的数据。
通过上述方法，在DPU释放主机存储空间时，及时将主机页面缓存中该主机存储空间中的数据标注为无效数据，或删除主机页面缓存中该主机存储空间中的数据，减少对该主机页面缓存的占用。
在一种可能的实施方式中，DPU与主机(如主机中的处理器)的交互可是基于虚拟化输入输出(virtual I/O)协议实现，也可以基于其他协议实现。
通过上述方法，DPU与主机基于已成熟的virtual I/O协议进行交互，使得DPU与主机之间更加简单。
在一种可能的实施方式中，DPU在将主机的存储空间映射为所述DPU的存储空间使用时，将主机存储空间的地址映射为所述DPU的存储空间的逻辑地址。主机存储空间的地址可以为该主机存储空间的物理地址。建立主机存储空间的地址与DPU的存储空间的逻辑地址的映射关系。
通过上述方法，DPU将主机存储空间的地址映射为DPU的存储空间的逻辑地址，面向该DPU的上层软件，该DPU的上层软件仅感知DPU的存储空间的逻辑地址，并不需要明确该DPU的存储空间的逻辑地址所指示的存储空间位于主机的存储器中，还是DPU自身的存储器中。
在一种可能的实施方式中，DPU在申请到该主机存储空间后，利用主机存储空间中的数据执行一些数据处理操作。例如，DPU在获取主机存储空间中存储的数据，对数据进行封装，并向该DPU外部的设备发送封装后的数据，这种处理方式适用于DPU具备支持网络功能的场景中。又例如，DPU对数据进行加密，向该DPU外部的设备发送加密后的数据，这种处理方式适用于DPU具备安全功能的场景中。又例如，DPU将数据存储在主机的本地存储设备中，或将该数据存储与所述DPU通过网络连接的远端存储设备中，这种方式适用于DPU具备存储功能的场景中。主机的本地存储设备为该主机用于进行持久化存储的存储设备，如硬盘等，该远端存储设备为部署远端的、用于实现持久化存储的存储设备，如部署在远端的存储系统中的设备。
通过上述方法，DPU调用申请到该主机存储空间，帮助主机承担一部分数据处理操作，能够实现一些基础功能。
在一种可能的实施方式中，DPU在申请到该主机存储空间后，在该主机存储空间中存储一些数据，例如，DPU能够在对来自该DPU外部的设备的数据包进行解封装，将解封装后获得的数据写入到主机存储空间中，这种处理方式适用于DPU具备支持网络功能的场景中。又例如，DPU能够对来自该DPU外部的设备或者主机的数据进行解密，将解密后获得的数据写入到主机存储空间中，这种处理方式适用于DPU具备安全功能的场景中。又例如，DPU访问DPU外部的存储设备，读取主机的本地存储设备或者主机远端存储设备中的数据，将读取的数据存储主机存储空间中，这种方式适用于DPU具备存储功能的场景中。
通过上述方法，DPU调用申请到该存储空间，帮助主机承担一部分数据处理操作，能够实现一些基础功能。
第二方面，本申请实施例还提供了一种数据处理装置，该数据处理装置具有实现上述第以方面的方法实例中行为的功能，有益效果可以参见第一方面的描述此处不再赘述。所述功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。所述硬件或软件包括一个或多个与上述功能相对应的模块。在一个可能的设计中，所述数据处理装置的结构中包括传输模块、映射模块，可选的，还包括写入模块、读取模块、第一处理模块、第二处理模块。这些模块可以执行上述第一方面方法示例中的相应功能，具体参见方法示例中的详细描述，此处不做赘述。
第三方面，本申请实施例还提供了一种数据处理装置，该数据处理装置具有实现上述第一方面的方法实例中行为的功能，有益效果可以参见第一方面的描述此处不再赘述。所述装置的结构中包括DPU 和供电电路，供电电路用于为所述DPU供电，可选的，还可以包括存储器和通信接口。所述DPU与主机通过系统总线连接，DPU被配置为支持所述数据处理装置执行上述第一方面方法中相应的功能。存储器与DPU通过系统总线连接，其保存所述通信装置必要的计算机程序指令和数据(如存储空间中的数据)，存储器可以为RAM、ROM、DRAM、闪存介质、硬盘等介质中的一种或多种的组合。本申请并不限定该DPU与主机的连接方式，例如DPU通过PCIe总线或其他类型的总线连接在所述主机的背板或主板上。
数据处理装置的结构中还包括通信接口，用于与其他设备或主机进行通信，如可以发送请求消息、接收响应消息、发送通知消息等。
第四方面，本申请实施例还提供了一种计算系统，该计算系统包括数据处理装置和主机，数据处理装置包括DPU，DPU与主机通过系统总线连接，例如，所述DPU通过PCIe总线连接在所述主机的背板或主板上。DPU具有实现上述第一方面的方法实例中行为的功能，有益效果可以参见第一方面的描述此处不再赘述。
在具体实现中，数据处理装置可以作为主机的一部分，部署在该主机的内部。例如，DPU通过PCIe总线连接在所述主机的背板或主板上。
第五方面，本申请还提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面以及第一方面的各个可能的实施方式中所述的方法。
第六方面，本申请还提供一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面以及第一方面的各个可能的实施方式中所述的方法。
第七方面，本申请还提供一种计算机芯片，所述芯片与存储器相连，所述芯片用于读取并执行所述存储器中存储的软件程序，执行上述第一方面以及第一方面的各个可能的实施方式中所述的方法。
附图说明
图1为本申请提供的一种计算系统的架构示意图；
图2为本申请提供的一种存储空间扩展方法示意图；
图3为本申请提供的一种发送请求消息的示意图；
图4为本申请提供的一种发送响应消息的示意图；
图5为本申请提供的一种数据处理装置的结构示意图。
具体实施方式
如图1所述为本申请实施例提供的一种系统结构示意图，该系统10包括主机20、数据处理装置30。主机20是一个常见的计算机设备。主机20包括但不限于个人电脑、服务器、手机、平板电脑或者智能车等。
主机20能够与该系统之外的其他设备进行通信，接收其他设备向该主机20发送的数据，以请求主机20处理该数据。在本申请实施例中，主机20以及数据处理装置30均具备数据处理功能，主机20可以自行对该数据进行处理，主机20也可以将该数据交由数据处理装置30，由数据处理装置30对该数据进行处理。主机20或数据处理装置30对该数据处理完成后，可以将数据处理结果反馈给其他设备。
数据处理装置30与主机20连接，该数据处理装置30可以作为该主机20的外接设备，也可以部署在该主机20内部，如数据处理装置30位于在主机20的主板或背板上，数据处理装置30(如数据处理装置30中的DPU301)通过总线与主机20(如主机20中的处理器201)交换数据，总线300可以为快捷外围部件互连标准(peripheral component interconnect express，PCIe)总线，也可以为计算快速互联(compute express link，CXL)、通用串行总线(universal serial bus，USB)协议或其他协议的总线。
该数据处理装置30可以作为主机20附带的具有数据处理功能的模块，承担该主机20的部分功能。也就是说，主机20的部分功能卸载到该数据处理装置30上，由该数据处理装置30代替主机20处理数据，执行部分的任务，以减轻主机20的压力，尤其是，对主机20中处理器201的压力，释放该处理器201的算力。本申请实施例并不限定该数据处理装置30代替主机20所能实现的功能。
举例来说，数据处理装置30能够作为主机20的网卡安插在主机20上。该数据处理装置30作为网卡，能够基于网络协议完成对数据包的处理，如实现数据的封装、传输等。该数据处理装置30还能够用于支持数据安全，能够对数据包括进行加解密。该数据处理装置30还能够作为存储的入口，通过数 据处理装置30实现数据的分布式存储以及数据的远程访问，如数据处理装置30可以访问该主机20的一些本地存储设备(如硬盘等用于持久化存储的设备)。还可以通过网络，访问远端的存储设备(如远端存储系统中的设备)。
数据处理装置30和主机20内部各自具备存储器302，数据处理装置30中的存储器302能够提供存储空间，以支持数据处理装置30的一些数据处理操作。类似的，主机20中的存储器202能够提供存储空间，以支持主机20的一些数据处理操作。
在本申请实施例中，允许数据处理装置30能够根据自身需求“借用”主机20中的存储空间(主机20中的存储空间也可以简称为主机20存储空间)，也即数据处理装置30能够使用主机20中的存储器202的存储空间，以支持自身的数据处理操作。数据处理装置30、根据自身需求向主机20申请主机存储空间，该主机存储空间为存储器202中的存储空间，主机20可以将该主机存储空间的地址发送给数据处理装置30，以便数据处理装置30能够将该主机存储空间映射为该数据处理装置30自己的存储空间，使用该主机存储空间。
当数据处理装置30不再需要存储器202中的存储空间时，数据处理装置30释放该主机存储空间，向主机20通知释放该主机存储空间。主机20在数据处理装置30的通知下回收该主机存储空间。
数据处理装置30能够按照自身需求申请主机20存储空间，或释放主机存储空间。也就是说，主机20不再需要为数据处理装置30预留固定大小的存储空间，仅需在数据处理装置30申请主机存储空间时，按照数据处理装置30的需求分配存储空间。数据处理装置30能够对主机20的存储器202中的存储空间进行按需取用，避免针对存储器202的过量占用，减少存储空间的浪费。
下面对主机20以及数据处理装置30的内部结构进行说明。
主机20包括I/O接口203、处理器201、存储器202。I/O接口203用于与位于主机20外部的设备通信。例如，外部设备可以通过I/O接口203向主机20发送数据，主机20对输入的数据进行处理之后，再通过I/O接口203向该外部设备发送对该数据处理后的输出结果。
处理器201是主机20的运算核心和控制核心，它可以是中央处理器(central processing unit，CPU)，也可以是其他特定的集成电路。处理器201还可以是其他通用处理器、数字信号处理器(digital signal processing,DSP)、专用集成电路(application specific integrated circuit,ASIC)、现场可编程门阵列(field programmable gate array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。
存储器202通常用来存放主机20操作系统中各种正在运行的计算机程序指令、需要处理的数据以及将数据处理结果等。为了提高处理器201的访问速度，存储器202需要具备访问速度快的优点。存储器202通常采用动态随机存取存储器(dynamic random access memory,DRAM)作为存储器202。除了DRAM之外，存储器202还可以是其他随机存取存储器，例如静态随机存取存储器(Static random access memory，SRAM)等。另外，存储器202也可以是只读存储器(read only memory,ROM)。而对于只读存储器，举例来说，可以是可编程只读存储器(programmable read only memory,PROM)、可抹除可编程只读存储器(erasable programmable read only memory,EPROM)等。存储器202还可以为闪存介质(FLASH)、硬盘驱动器(hard disk drive，HDD)或固态驱动器(solid state disk，SSD)等。
本实施例并不限定存储器202的数量。处理器201通过双倍速率(double data rate,DDR)总线或者其他类型的总线和存储器202相连。存储器202理解为主机20的内存(internal memory)，内存又称为主存(main memory)。处理器201通过调用该存储器202中的计算机程序指令执行如下文中图2所示的实施例中主机20需执行的一些方法。
数据处理装置30包括数据处理单元(data process unit，DPU)301以及存储器302，DPU301与存储器302通过系统总线连接，该系统总线可以为基于PCIe的线路，也可以为CXL、USB协议或其他协议的总线。
DPU301为数据处理装置30的主要运算单元，是数据处理装置30的核心单元，DPU301承担了数据处理装置30的主要功能。例如主机20的部分功能可以卸载到DPU301上，由DPU301处理数据，执行主机20交由数据处理装置30的任务。DPU301通过调用该存储器302中的计算机程序指令执行如下文中图3所示的实施例中DPU301需执行的一些方法。
在本申请实施例中，DPU301还可以按照自身需求申请主机存储空间，或释放主机存储空间。
存储器302能够支持DPU301的数据处理操作，为DPU301提供数据存储空间，与存储器202类似， 存储器302用于存储DPU301需要调用的计算机程序指令，需要处理的数据以及数据处理结果等。存储器302的类型与存储器202的类型类似，具体可以参见前述内容，此处不再赘述。存储器302理解为数据处理装置30的内存。
存储器302和存储器202中，数据是以页(page)为粒度组织的。也就是说，在存储器302中存储器302的物理存储空间以页为粒度进行划分。DPU301也是以页为粒度对存储器302进行访问的，处理器201也是以页为粒度对存储器202进行访问的。
如图1所示，数据处理装置30可以直接插在主机20的主板上的卡槽中，通过PCIe总线204与处理器201交换数据。需注意的是，图1中的PCIe总线204能够被替换成计算快速互联(compute express link，CXL)、通用串行总线(universal serial bus，USB)协议或其他协议的总线使得数据处理装置30进行数据传输。
在数据处理装置30中，为了便于对存储器302进行管理，将存储器302的物理存储空间映射为虚拟存储空间。物理存储空间的地址称为物理地址，虚拟存储空间的地址称为逻辑地址。DPU301部署有用于对存储器302的管理单元。管理单元能够实现对存储器302的管理，该管理包括但不限于：物理地址与逻辑地址之间的映射关系的管理、向主机20申请存储空间(以页为粒度的存储空间)、为DPU301上的软件分配存储空间(以页为粒度的存储空间)、数据读写。
下面对管理单元的管理操作进行说明：
(1)、物理地址与逻辑地址之间的映射关系的管理。
管理单元能够实现物理存储空间与虚拟存储空间的映射，建立物理地址与逻辑地址之间的映射关系，删除已建立的物理地址与逻辑地址之间的映射关系。
由于管理单元的存在，DPU301上的软件程序(也可以称为软件)仅需要感知逻辑地址，逻辑地址与到物理地址之间的转换交由管理单元实现。例如，DPU301上运行的软件程序需要申请存储空间时，管理单元将为该软件程序所分配的存储空间的逻辑地址提供给DPU301的软件。又例如，当DPU301上运行的软件程序需要向存储空间写入数据时，DPU301上运行的软件程序进行向管理单元提供逻辑地址以及需要写入的数据，管理单元基于物理地址与逻辑地址之间的映射关系，将该逻辑地址转换为物理地址，将该数据写到该物理地址上。又例如，当DPU301上运行的软件程序需要从存储空间读取数据时，DPU301上运行的软件程序向管理单元提供逻辑地址，管理单元基于物理地址与逻辑地址之间的映射关系，将该逻辑地址转换为物理地址，从该物理地址上去读取数据，将该数据反馈给该软件程序。
(2)、为DPU301上的软件分配存储空间。
DPU301上的软件程序在运行过程会需要一定的存储空间以进行数据处理操作，数据处理操作的过程伴随着在该主机存储空间中的数据读取或写入。例如，读取需要处理的数据、或写入处理后的数据。DPU301上的软件程序根据自身需要向管理单元申请存储空间。
需要说明的是，DPU301上的软件程序在申请存储空间时，是以页为粒度申请的，也即是说，DPU301所需要申请的存储空间的大小为页的整数倍。
管理单元确定存储器302中的空闲的物理存储空间，为DPU301上运行的软件分配空闲的物理存储空间，并将该空闲的物理存储空间的物理地址映射为逻辑地址，保存该物理地址与逻辑地址的映射关系，并将该逻辑地址提供为DPU301上运行的软件。
(3)、向主机20申请存储空间。
当DPU301上的软件程序向管理单元申请存储空间，若存储器302中的空闲的物理存储空间不足以支撑该DPU301上的软件程序所申请的存储空间，也即DPU301上的软件程序所申请的存储空间的大小小于存储器302中的空闲的物理存储空间的大小。管理单元可以向主机20申请主机存储空间，主机20将存储器202中的存储空间分配给管理单元，将存储器202中的存储空间的地址(该地址可以为该存储空间的物理地址)发送给管理单元。
管理单元在获取存储器202中的存储空间的物理地址后，将物理地址映射为逻辑地址，并保存物理地址与逻辑地址的映射关系，将逻辑地址提供给DPU301上运行的软件程序。
(4)、数据读写。
管理单元在DPU301上的软件程序的请求下在DPU301上的软件所申请的存储空间中写入数据，或者从DPU301上的软件程序所申请的存储空间中读取数据。
当DPU301上运行的软件程序需要向存储空间写入数据时，DPU301上运行的软件向管理单元发起 写入指令，该写入指令中携带逻辑地址以及需要写入的数据。管理单元基于物理地址与逻辑地址之间的映射关系，将该逻辑地址转换为物理地址，将该数据写到该物理地址上。
当DPU301上运行的软件程序需要从存储空间读取数据时，DPU301上运行的软件向管理单元发起读取指令，该读取指令中携带逻辑地址，管理单元基于物理地址与逻辑地址之间的映射关系，将该逻辑地址转换为物理地址，从该物理地址上读取数据，将该数据反馈给该软件。
管理单元理解为该DPU301上运行的一种软件程序，也可以理解为DPU301上的一部分硬件模块。本申请实施例并不限定管理单元的具体形态。
下面结合图2对本申请实施例提供的一种存储空间申请方法进行说明，图2所示的实施例包括两部分，一部分为DPU301向主机20申请存储空间，具体参见步骤201～步骤204。另一部分为DPU301向主机20请求释放存储空间，具体参见步骤205～步骤206。
步骤201：DPU301向主机20发起请求消息，该请求消息用于向主机20申请主机存储空间，该请求消息还指示所需申请的主机存储空间的大小。
当存储器302以及存储器202中数据存储的最小单位均为页，存储器302中页的大小与存储器202中页的大小相同，存储空间的大小可以用页的数量表示。
DPU301上可以运行一种或多种软件程序，软件程序的运行需要存储空间，软件程序需要在存储空间中写入数据或读取数据，在确定存储器302无足够的存储空间时，DPU301可以执行步骤201。
例如，在DPU301内部，当DPU301上的一个软件程序开始运行时，该软件程序可以向管理单元申请存储空间，并告知管理单元所需申请的存储空间的大小，管理单元在确定存储器302中无空闲的存储空间的情况下，向主机20发起请求消息。
又例如，在DPU301内部，当DPU301上已运行的软件程序在运行过程中，需要扩展存储空间，该软件程序向管理单元申请存储空间，并告知管理单元所需扩展的存储空间的大小，管理单元在确定存储器302中无空闲的存储空间的情况下，可以向主机20发起请求消息。
在本申请实施例并不限定DPU301与主机20的交互方式。DPU301与主机20之间可以基于已有的协议进行交互。
举例来说，DPU301与主机20之间可以基于虚拟化输入输出(virtual input/output，virt virtual I/O)协议进行交互的。virtual I/O协议定义了一种通用的IO虚拟化模型。virtual I/O是对半虚拟化超级监督者(hypervisor)中的一组通用模拟设备的抽象。通过这个IO虚拟化模型，将主机20连接的装置(如本申请实施例的数据处理装置30)统一起来，以便实现统一的管理、维护、以及扩展等。这样，这些装置根据virtual I/O协议定义实现所具备的功能，避免对主机20的hypervisor进行更新。
如图3所示，基于virtual I/O协议，DPU301部署虚拟输入输出后端驱动(virtual I/O backend driver)，主机20中部署虚拟输入输出前端驱动(virtual I/O frontend driver)。DPU301与主机20之间的交互可以通过虚拟输入输出后端驱动以及虚拟输入输出前端驱动共同维护的虚拟I/O队列实现。虚拟I/O队列包括请求队列以及回复队列，请求队列用于承载虚拟输入输出前端驱动添加的请求指令，回复队列用于承载虚拟输入输出后端添加针对请求的回复指令。
主机20在需要向DPU301发起请求指令时，通过虚拟输入输出前端驱动，将该请求指令添加到虚拟I/O队列的请求队列中，这样，部署在DPU301中的虚拟输入输出后端驱动从该虚拟I/O队列中获取该请求指令，DPU301处理该请求指令，并将处理结果作为回复指令放置在该虚拟I/O队列的回复队列中。
基于virtual I/O协议，只有在虚拟输入输出前端驱动通过虚拟I/O队列向虚拟输入输出后端驱动下发一个请求指令时，虚拟输入输出后驱动才可以通过虚拟I/O队列向虚拟输入输出前端驱动返回一个回复指令。
在本申请实施例中DPU301与主机20为了能够继续借助图3所示的这种交互方式实现请求消息的发送，在这种情况下，主机20部署的虚拟输入输出前端驱动预先在虚拟I/O队列中添加一个请求指令(该请求指令可以不指示任何实际性的内容)，以便后续DPU301需要申请主机20的存储器202的存储空间时，部署在DPU301中虚拟输入输出后端驱动将用于申请存储空间的申请消息作为回复指令添加到虚拟I/O队列中。
也即是说，在管理单元确定存储器302中无空闲的存储空间的情况下，管理单元向虚拟输入输出后端驱动发起该请求消息，虚拟输入输出后端驱动将请求消息作为回复指令添加到虚拟I/O队列中。
上述DPU301向主机20发送请求消息的方式仅是举例，在实际应用中，DPU301与主机20之间也可以基于其他协议进行交互。或者，DPU301与主机20之间可以重新定义新的协议，基于该新的协议进行交互。
应需理解的是，DPU301与主机20之间的交互实质上为DPU301与主机20中的处理器201之间的交互，本申请为了方便表述，将DPU301与主机20中的处理器201之间的交互统一称为DPU301与主机20的交互。
步骤202：主机20在接收到该请求消息之后，从该主机20的存储器202为该DPU301申请主机存储空间。若请求消息中携带了需要申请的页的数量，主机20从存储器202中申请相应数量的页。
以主机20与DPU301基于virtual I/O协议为例，主机20内部署的虚拟输入输出前端驱动从虚拟I/O队列中获取该请求消息后，虚拟输入输出前端驱动向存储器202申请存储空间。这种情况下，存储器202会认为该主机存储空间是虚拟输入输出前端驱动使用的，并不明确该主机存储空间是由DPU301所使用的。存储器202为该虚拟输入输出前端驱动配置存储空间，该主机存储空间的大小与请求消息中所指示的存储空间的大小一致，并将该主机存储空间的物理地址(例如，页的物理地址)反馈给虚拟输入输出前端驱动。
步骤203：主机20向DPU301发送响应消息，该响应消息携带有该主机存储空间的地址。如该响应消息中可以携带存储器202中页的地址。
以主机20与DPU301基于virtual I/O协议为例，如图4所示，虚拟输入输出前端驱动从存储器202获取该主机存储空间的地址后，将该主机存储空间的地址作为请求指令添加到虚拟I/O队列中。
另外，为了便于DPU301后续能够再次向主机20申请存储器202中的存储空间，虚拟输入输出前端驱动还可以再次发起另一个请求指令(该请求指令可以不指示任何实际性的内容)添加到虚拟I/O队列中。
步骤204：DPU301在接收到该响应消息，获取该主机存储空间的地址，将该主机存储空间映射为DPU301的存储空间，使用该主机存储空间。例如，DPU301将该主机存储空间的地址映射为逻辑地址，将该逻辑地址反馈给DPU301中申请存储空间的软件程序。
DPU301中的管理单元在获取该响应消息后，从该响应消息中获取该主机存储空间的物理地址，将物理地址映射为逻辑地址，保存该物理地址与逻辑地址之间的映射关系。
在本申请实施例中，存储器302内部，存储器302包括主机页面缓存，该主机页面缓存用于存储主机20的存储空间中的数据，该主机存储空间是DPU301向主机20申请的、存储器202的存储空间。在存储器202以及存储器302均以页为粒度存储数据的情况下，该主机页面缓存可用于以存储DPU301向主机20申请的、存储器202的页中的数据。存储器302通过直接存储器访问(direct memory access，DMA)访问主机存储空间，如从主机存储空间中读取数据，将读取的数据存储在主机页面缓存中。又如，将主机页面缓存中保存的数据写入到主机存储空间中。
DPU301上运行的软件程序在获取该逻辑地址后，针对该逻辑地址进行数据读写。
前述说明中提及，数据处理装置30能够承担主机20的部分功能。数据处理装置30在承担该部分功能时，DPU301能够调用该主机存储空间执行一些数据处理操作。
DPU301能够获取该主机存储空间中存储的数据，对该数据进行处理。DPU301可以基于网络协议对获取的数据进行封装，并发送封装后的数据，例如将该封装后的数据发送给该DPU301外部的设备；DPU301也可以对数据进行加密，将加密后的数据保存在该主机存储空间中、或将该加密后的数据发送给该DPU外部的设备。DPU301还可以访问外部的、用于持久化存储数据的存储设备，例如，DPU301可以访问主机的本地存储设备或与DPU301通过网络连接的远端存储设备，将该主机存储空间中存储的数据存储在主机的本地存储设备、或与DPU301通过网络连接的远端存储设备。
DPU301需要获取该主机存储空间中存储的数据时，DPU301指示存储器302通过DMA的方式将主机存储空间中的数据存储在主机页面缓存中。在DPU301内部，需要使用该数据的软件程序向管理单元发起读取指令，该读取指令中携带了该主机存储空间的逻辑地址。管理单元根据该逻辑地址确定物理地址，管理单元指示存储器302根据该物理地址从主机页面缓存读取数据。若当前主机页面缓存中存储的数据为该主机存储空间中的数据，存储器302直接从主机页面缓存中读取该数据。若当前主机页面缓存中存储的数据并非该主机存储空间中的数据(例如，该数据在写入之后已通过DMA写入到主机20的存储器202中)，存储器302根据该主机存储空间的地址通过DMA从主机存储空间中读取该数据， 将该数据写入到主机页面缓存。存储器302将从主机页面缓存中读取该数据反馈给管理单元，该管理单元将该数据反馈给软件程序。
DPU301能够将处理后的数据存储在该主机存储空间中。例如，DPU301基于网络协议对获取的数据进行解封装，并解密后获得的数据存储在该主机存储空间中；又例如，DPU301对获取的已经加密的数据进行解密，将解密后的数据保存在该主机存储空间中。又例如，DPU301访问外部的存储设备，如访问主机的本地存储设备、或与该DPU301通过网络连接的远端存储设备，从外部的存储设备中读取数据，将读取的数据存储在该主机存储空间中。
DPU301需要在该主机存储空间中存储数据时，DPU301指示存储器302将数据存储在存储器302的主机页面缓存中，之后存储器302通过DMA的方式将主机页面缓存中的数据存储在该主机存储空间中。在DPU301内部，需要存储该数据的软件程序向管理单元发起写入指令，该写入指令指示了逻辑地址以及需要写入的数据。管理单元在接收到写入指令后，根据该逻辑地址确定物理地址，若该物理地址为主机20的存储空间的地址，管理单元指示存储器302将该数据写入到主机页面缓存中。若当前主机页面缓存中的数据未满，也即主机页面缓存存在空闲的存储空间，存储器302直接将该数据写入到主机页面缓存中。
若当前主机页面缓存中的数据已满，存储器302通过DMA将主机页面缓存中的数据写回到主机20的存储空间中。在数据写回后，存储器302将该数据写入到主机页面缓存中。
步骤205：DPU301向主机20发起通知消息，该通知消息用于通知主机20释放存储空间，该通知消息还携带存储空间的地址。当存储器302以及存储器202中数据存储的最小单位均为页，存储器302中页的大小与存储器202中页的大小相同，存储空间的大小可以用页的数量表示。需要释放的页的物理地址作为存储空间的物理地址。
当DPU301上的一个软件程序在运行过程中或准确停止运行时，DPU301上的软件程序可以申请释放该软件程序之前所申请的存储空间。这里DPU301上的软件程序所申请释放的存储空间可以是该软件程序之前申请的所有存储空间，也可以为部分存储空间。
DPU301上的软件程序可以向管理单元申请释放存储空间，并告知需要释放的存储空间的逻辑地址。
管理单元在获取该主机存储空间的逻辑地址后，删除该主机存储空间的逻辑地址与物理地址的映射关系。若该主机存储空间的数据仍缓存在存储器302的主机页面缓存中，管理单元将该主机页面缓存中该主机存储空间的数据标注为无效数据。
以主机20与DPU301基于virtual I/O协议为例，管理单元向虚拟输入输出后端驱动发起该通知消息，虚拟输入输出后端驱动将通知消息作为回复指令添加到虚拟I/O队列中。
步骤206：主机20接收该通知消息后，释放该主机存储空间。
以主机20与DPU301基于virtual I/O协议为例，主机20内部署的虚拟输入输出前端驱动从虚拟I/O队列中获取该通知消息后，虚拟输入输出前端驱动指示该存储器202释放存储空间。存储器202根据该虚拟输入输出前端驱动指示释放该主机存储空间。
基于与方法实施例同一发明构思，本申请实施例还提供了一种数据处理装置，该数据处理装置用于执行上述如图5所示的方法实施例中所述DPU301执行的方法，相关特征可参见上述方法实施例，此处不再赘述。该数据处理装置可理解为前述说明提及的管理单元的一种软件形态。如图5所示，数据处理装置500包括传输模块501以及映射模块502。
传输模块501，用于向主机发起请求消息，请求消息用于申请主机存储空间；获取主机反馈的响应消息，响应消息携带主机存储空间的地址，主机存储空间位于主机的存储器中。
映射模块502，用于将主机存储空间映射为DPU的存储空间使用，DPU与主机通过系统总线连接。
在一种可能的实施方式中，映射模块502在将主机存储空间的物理地址映射为DPU的存储空间使用时，映射模块502可以将主机存储空间的地址映射为DPU的存储空间的逻辑地址，建立主机存储空间的地址与DPU的存储空间的逻辑地址的映射关系。
在一种可能的实施方式中，数据处理装置500在释放该主机存储空间时，映射模块502删除物理地址与虚拟地址的映射关系；传输模块501向主机发送通知消息，通知消息用于通知释放主机存储空间。
在一种可能的实施方式中，请求消息还指示了主机存储空间的大小。
在一种可能的实施方式中，DPU与存储器通过系统总线连接，存储器包括主机页面缓存，数据处理装置500还包括写入模块503，写入模块503能够在主机存储空间中写入数据。写入模块503在向该 主机存储空间写入数据时，在主机页面缓存中写入数据，通过DMA将主机页面缓存中的数据写入到主机存储空间中。
在一种可能的实施方式中，数据处理装置500包括读取模块504，读取模块504能够从主机存储空间中读取数据，读取模块504在读取数据时，通过DMA将主机存储空间的数据写入到主机页面缓存中，从主机页面缓存读取数据。
在一种可能的实施方式中，在释放该主机存储空间时，映射模块502将主机页面缓存中存储的该主机存储空间中的数据标注为无效数据。
在一种可能的实施方式中，传输模块501与主机之间基于virtual I/O协议交互。
在一种可能的实施方式中，数据处理装置500还包括第一处理模块505，第一处理模块505能够对主机存储空间中的数据进行一些数据处理操作。例如，第一处理模块505获取主机存储空间中存储的数据，执行下列的部分或全部：
获取存储空间中存储的数据，执行下列的部分或全部：
对数据进行封装，并向DPU外部的设备发送封装后的数据。
对数据进行加密，向DPU外部的设备发送加密后的数据。
将数据存储在主机的本地存储设备中，或者与DPU通过网络连接的远端存储设备中。
在一种可能的实施方式中，数据处理装置500还包括第二处理模块506，第二处理模块506可以将经过处理后的数据存储在存储空间中。例如，第二处理模块506对来自外部设备的数据包进行解封装，将解封装后获得的数据写入到主机存储空间中。又例如，第二处理模块506对来自外部设备的数据进行解密，将解密后获得的数据写入到主机存储空间中。又例如，第二处理模块506读取主机的本地存储设备或者主机远端存储设备中的数据，将读取的数据存储主机存储空间中。
根据本申请实施例的数据处理装置500可对应于执行本申请实施例中描述的装置，并且装置500中的各个模块的上述和其它操作和/或功能分别为了实现图2中的各个装置的相应流程，为了简洁，在此不再赘述。
需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。在本申请的实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
上述实施例，可以全部或部分地通过软件、硬件、固件或其他任意组合来实现。当使用软件实现时，上述实施例可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机程序指令。在计算机上加载或执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以为通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机程序指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机程序指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集合的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质。半导体介质可以是固态硬盘(solid state drive，SSD)。
本领域内的技术人员应明白，本申请的实施例可提供为装置、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请的装置、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (24)

  1. 一种DPU的存储空间扩展方法，其特征在于，所述方法用于扩展数据处理单元DPU的存储空间，所述DPU与主机通过系统总线连接，包括：
    向所述主机发起请求消息，所述请求消息用于申请主机存储空间；
    获取所述主机反馈的响应消息，所述响应消息携带所述主机存储空间的地址，所述主机存储空间位于所述主机的存储器中；
    将所述主机存储空间映射为所述DPU的存储空间使用。
  2. 如权利要求1所述的方法，其特征在于，所述方法还包括：
    向所述主机发送通知消息，所述通知消息用于通知释放所述主机存储空间。
  3. 如权利要求1或2所述的方法，其特征在于，所述请求消息还指示了所述主机存储空间的大小。
  4. 如权利要求1-3任一项所述的方法，其特征在于，所述DPU与存储器通过系统总线连接，所述存储器包括主机页面缓存，所述获取所述主机反馈的响应消息之后，还包括：
    在所述主机页面缓存中写入数据，通过直接存储器访问DMA将所述主机页面缓存中的数据写入到所述主机存储空间中。
  5. 如权利要求1-3任一项所述的方法，其特征在于，所述DPU与存储器通过系统总线连接，所述存储器包括主机页面缓存，所述获取所述主机反馈的响应消息之后，还包括：
    通过直接存储器访问DMA将所述主机存储空间的数据写入到所述主机页面缓存中，从所述主机页面缓存读取所述数据。
  6. 如权利要求4或5所述的方法，其特征在于，所述方法还包括：
    释放所述主机存储空间，将所述主机页面缓存存储的所述主机存储空间中的数据标注为无效数据。
  7. 如权利要求1-6任一项所述的方法，其特征在于，所述DPU与所述主机之间基于虚拟化输入输出virtual I/O协议交互。
  8. 如权利要求1～7任一项所述的方法，其特征在于，所述将所述主机的存储空间映射为所述DPU的存储空间使用具体包括：
    将所述主机存储空间的地址映射为所述DPU的存储空间的逻辑地址。
  9. 如权利要求1-8任一项所述的方法，其特征在于，所述方法还包括：
    获取所述存储空间中存储的数据，执行下列的部分或全部：
    对所述数据进行封装，并向所述DPU外部的设备发送封装后的数据；
    对所述数据进行加密，向所述DPU外部的设备发送加密后的数据；
    将所述数据存储在所述主机的本地存储设备中，或者与所述DPU通过网络连接的远端存储设备中。
  10. 如权利要求1-9任一项所述的方法，其特征在于，所述方法还包括：
    执行下列的部分或全部：
    对来自所述DPU外部的设备的数据包进行解封装，将解封装后获得的数据写入到所述主机存储空间中；
    对来自所述DPU外部的设备的数据进行解密，将解密后获得的数据写入到所述主机存储空间中；
    读取所述主机的本地存储设备或者所述主机远端存储设备中的数据，将读取的所述数据存储所述主机存储空间中。
  11. 一种数据处理装置，其特征在于，所述数据处理装置包括：
    所述传输模块，用于向主机发起请求消息，所述请求消息用于申请主机存储空间；获取所述主机反馈的响应消息，所述响应消息携带所述主机存储空间的地址，所述主机存储空间位于所述主机的存储器中；
    所述映射模块，用于将所述主机存储空间映射为所述DPU的存储空间使用，所述DPU与所述主机通过系统总线连接。
  12. 如权利要求11所述的装置，其特征在于，
    所述映射模块，还用于：释放所述主机存储空间；
    所述传输模块，还用于：向所述主机发送通知消息，所述通知消息用于通知释放所述主机存储空间。
  13. 如权利要求11或12所述的装置，其特征在于，所述请求消息还指示了所述主机存储空间的大小。
  14. 如权利要求11-13任一项所述的装置，其特征在于，所述DPU与存储器通过系统总线连接，所述存储器包括主机页面缓存，所述数据处理装置还包括写入模块，所述写入模块，用于：
    在所述主机页面缓存中写入数据，通过直接存储器访问DMA将所述主机页面缓存中的数据写入到所述主机存储空间中。
  15. 如权利要求11-13任一项所述的装置，其特征在于，所述DPU与存储器通过系统总线连接，所述存储器包括主机页面缓存，所述数据处理装置包括读取模块，所述读取模块，用于：
    通过直接存储器访问DMA将所述主机存储空间的数据写入到所述主机页面缓存中，从所述主机页面缓存读取所述数据。
  16. 如权利要求14或15所述的装置，其特征在于，所述映射模块，还用于：
    释放所述主机存储空间，将所述主机页面缓存存储的所述主机存储空间中的数据标注为无效数据。
  17. 如权利要求11-16任一项所述的装置，其特征在于，所述传输模块与所述主机之间基于虚拟化输入输出virtual I/O协议交互。
  18. 如权利要求11-17任一项所述的装置，其特征在于，所述映射模块在将所述主机的存储空间映射为所述DPU的存储空间使用，用于：
    将所述主机存储空间的地址映射为所述DPU的存储空间的逻辑地址。
  19. 如权利要求11-18任一项所述的装置，其特征在于，所述数据处理装置还包括第一处理模块，所述第一处理模块，用于获取所述存储空间中存储的数据，执行下列的部分或全部：
    获取所述存储空间中存储的数据，执行下列的部分或全部：
    对所述数据进行封装，并向所述DPU外部的设备发送封装后的数据；
    对所述数据进行加密，向所述DPU外部的设备发送加密后的数据；
    将所述数据存储在所述主机的本地存储设备中，或者与所述DPU通过网络连接的远端存储设备中。
  20. 如权利要求11-18任一项所述的装置，其特征在于，所述数据处理装置还包括第二处理模块，所述第二处理模块，用于对来自所述DPU外部的设备的数据包进行解封装，将解封装后获得的数据写入到所述主机存储空间中；对来自所述DPU外部的设备的数据进行解密，将解密后获得的数据写入到所述主机存储空间中；读取所述主机的本地存储设备或者所述主机远端存储设备中的数据，将读取的所述数据存储所述主机存储空间中。
  21. 一种数据处理装置，其特征在于，所述数据处理装置包括数据处理单元DPU以及供电电路，所述供电电路用于为所述DPU供电，所述DPU与主机连接，用于执行如权利要求1～10任一项所述的方法。
  22. 如权利要求21所述的装置，其特征在于，所述数据处理装置还包括存储器，所述存储器与所述DPU通过系统总线连接，所述存储器包括下列的部分或全部：
    随机存取存储器RAM、只读存储器ROM、动态随机存取存储器DRAM、闪存介质、硬盘。
  23. 如权利要求21所述的装置，其特征在于，所述DPU通过快捷外围部件互连标准PCIe总线连接在所述主机的背板或主板上。
  24. 一种计算系统，其特征在于，所述计算系统包括主机、以及DPU，所述DPU与主机通过系统总线连接，所述DPU用于执行如权利要求1～10任一项所述的方法。
PCT/CN2023/101068 2022-08-19 2023-06-19 一种dpu的存储空间扩展方法、装置及系统 WO2024037172A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210998035.3 2022-08-19
CN202210998035.3A CN117631958A (zh) 2022-08-19 2022-08-19 一种dpu的存储空间扩展方法、装置及系统

Publications (1)

Publication Number Publication Date
WO2024037172A1 true WO2024037172A1 (zh) 2024-02-22

Family

ID=89940592

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/101068 WO2024037172A1 (zh) 2022-08-19 2023-06-19 一种dpu的存储空间扩展方法、装置及系统

Country Status (2)

Country Link
CN (1) CN117631958A (zh)
WO (1) WO2024037172A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103677674A (zh) * 2013-12-27 2014-03-26 华为技术有限公司 一种数据处理方法及装置
CN103793330A (zh) * 2012-10-31 2014-05-14 国际商业机器公司 在虚拟机环境中进行数据交换的方法和装置
US20150319237A1 (en) * 2014-05-02 2015-11-05 Cavium, Inc. Systems and methods for enabling access to extensible storage devices over a network as local storage via nvme controller
CN105094997A (zh) * 2015-09-10 2015-11-25 重庆邮电大学 一种云计算主机节点间物理内存共享方法和系统
CN106959893A (zh) * 2017-03-31 2017-07-18 联想(北京)有限公司 加速器、用于加速器的内存管理方法以及数据处理系统
CN107077426A (zh) * 2016-12-05 2017-08-18 华为技术有限公司 NVMe over Fabric架构中数据读写命令的控制方法、设备和系统
CN114089926A (zh) * 2022-01-20 2022-02-25 阿里云计算有限公司 分布式存储空间的管理方法、计算设备及存储介质
CN114490433A (zh) * 2022-01-20 2022-05-13 哲库科技(上海)有限公司 存储空间的管理方法、数据处理芯片、设备和存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103793330A (zh) * 2012-10-31 2014-05-14 国际商业机器公司 在虚拟机环境中进行数据交换的方法和装置
CN103677674A (zh) * 2013-12-27 2014-03-26 华为技术有限公司 一种数据处理方法及装置
US20150319237A1 (en) * 2014-05-02 2015-11-05 Cavium, Inc. Systems and methods for enabling access to extensible storage devices over a network as local storage via nvme controller
CN105094997A (zh) * 2015-09-10 2015-11-25 重庆邮电大学 一种云计算主机节点间物理内存共享方法和系统
CN107077426A (zh) * 2016-12-05 2017-08-18 华为技术有限公司 NVMe over Fabric架构中数据读写命令的控制方法、设备和系统
CN106959893A (zh) * 2017-03-31 2017-07-18 联想(北京)有限公司 加速器、用于加速器的内存管理方法以及数据处理系统
CN114089926A (zh) * 2022-01-20 2022-02-25 阿里云计算有限公司 分布式存储空间的管理方法、计算设备及存储介质
CN114490433A (zh) * 2022-01-20 2022-05-13 哲库科技(上海)有限公司 存储空间的管理方法、数据处理芯片、设备和存储介质

Also Published As

Publication number Publication date
CN117631958A (zh) 2024-03-01

Similar Documents

Publication Publication Date Title
JP6796304B2 (ja) 最終レベルキャッシュシステム及び対応する方法
US11929927B2 (en) Network interface for data transport in heterogeneous computing environments
CN110647480B (zh) 数据处理方法、远程直接访存网卡和设备
US10324873B2 (en) Hardware accelerated communications over a chip-to-chip interface
US20150261720A1 (en) Accessing remote storage devices using a local bus protocol
WO2013170730A1 (zh) Dma传输方法及系统
CN113742269B (zh) 用于epa设备的数据传输方法、处理设备和介质
US10951741B2 (en) Computer device and method for reading or writing data by computer device
WO2019047834A1 (zh) 用于传输数据处理请求的方法和装置
CN114662136B (zh) 一种基于pcie通道的多算法ip核的高速加解密系统及方法
US20130346668A1 (en) Virtual Memory Module
WO2019141157A1 (zh) 一种核间数据传输的装置和方法
WO2023115852A1 (zh) 基于pcie的通信方法、装置、计算机设备和可读存储介质
CN113032293A (zh) 缓存管理器及控制部件
WO2014202003A1 (zh) 数据存储系统的数据传输方法、装置及系统
WO2023103704A1 (zh) 数据处理方法、存储介质和处理器
CN116185553A (zh) 数据迁移方法、装置及电子设备
WO2014100954A1 (zh) 数据控制方法及系统
CN114925012A (zh) 一种以太网帧的下发方法、上传方法及相关装置
WO2014180446A1 (zh) 一种网络设备写表方法及装置
WO2023186143A1 (zh) 一种数据处理方法、主机及相关设备
WO2024037172A1 (zh) 一种dpu的存储空间扩展方法、装置及系统
US9244824B2 (en) Memory sub-system and computing system including the same
CN110765065A (zh) 片上系统
WO2022126398A1 (zh) 消息通知方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23854077

Country of ref document: EP

Kind code of ref document: A1