CN112699069A - High-speed data transmission method and system based on PCIe interface - Google Patents
High-speed data transmission method and system based on PCIe interface Download PDFInfo
- Publication number
- CN112699069A CN112699069A CN202011576315.2A CN202011576315A CN112699069A CN 112699069 A CN112699069 A CN 112699069A CN 202011576315 A CN202011576315 A CN 202011576315A CN 112699069 A CN112699069 A CN 112699069A
- Authority
- CN
- China
- Prior art keywords
- pcie interface
- host
- pcie
- hardware equipment
- kernel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
- G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/382—Information transfer, e.g. on bus using universal interface adapter
- G06F13/385—Information transfer, e.g. on bus using universal interface adapter for adaptation of a particular data processing system to different peripheral devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
Abstract
The embodiment of the invention provides a high-speed data transmission method and a system based on a PCIe interface, wherein the method comprises the following steps: mapping all DMA function operations of a driver of the PCIe interface hardware equipment from a kernel mode to a user mode; when the host reads the PCIe interface hardware equipment, the program of the host application layer is directly connected with the PCIe interface hardware equipment through the address to read data or write data; and compiling the application layer of the host through a giant page technology to monopolize the multi-core CPU. The system comprises a host, a PCIe bus and hardware equipment of a PCIe interface; the host has a PCIe bus interface, the hardware equipment of the PCIe interface is accessed into the host through the PCIe bus, and the hardware equipment of the PCIe interface has a DMA function and can finish the read-write operation of data.
Description
Technical Field
The invention relates to the technical field of computer high-speed data communication, in particular to a high-speed data transmission method and a high-speed data transmission system based on a PCIe interface.
Background
With the rapid development of computer technology, new technologies such as cloud computing, internet of things, 5G, block chaining and the like appear, the number of devices is huge, and the high-speed transmission and processing of the acquired mass data are the main parts occupying the computing resources of a computer CPU.
The PCIe bus is used as a high-speed data transmission channel, and data can be directly carried to a memory address without passing through a CPU (central processing unit) by a DMA (direct memory access) technology. However, each time a packet of data is transmitted, the linux-based PCIe general driver DMA generates an interrupt, and the transmission of mass data frequently triggers the interrupt, so that the CPU is frequently interrupted, and enters the driver layer to process the interrupt request. This results in an inability to adequately deliver performance for PCIe bus transfers.
From the perspective of application programs, the current CPUs are all multi-core, programs can be switched among the multi-core, mass data needs to be distributed to different threads, dirty pages on different CPU cores need to be reloaded, page tables need to be searched when the pages are reloaded, a large amount of time is consumed, and the data processing speed is affected.
Disclosure of Invention
The embodiment of the invention provides a high-speed data transmission method and system based on a PCIe (peripheral component interconnect express) interface, which aim to solve the problems of frequent interruption of data transmission and low data processing speed in the prior art.
In order to solve the technical problem, the embodiment of the invention discloses the following technical scheme:
the invention provides a high-speed data transmission method based on a PCIe interface in a first aspect, which comprises the following steps:
mapping all DMA function operations of a driver of the PCIe interface hardware equipment from a kernel mode to a user mode;
when the host reads the PCIe interface hardware equipment, the program of the host application layer is directly connected with the PCIe interface hardware equipment through the address to read data or write data;
and compiling the application layer of the host through a giant page technology to monopolize the multi-core CPU.
With reference to the first aspect, in a first possible implementation manner of the first aspect, mapping all DMA function operations of a driver of a PCIe interface hardware device from a kernel mode to a user mode specifically includes:
converting a register physical address for operating the DMA function into an IO address;
converting a kernel virtual address;
the kernel virtual address is mapped to the user layer through a specific interface of the driver.
With reference to the first aspect, in a second possible implementation manner of the first aspect, when the host reads the PCIe interface hardware device, directly connecting, by an address, the program of the host application layer to the PCIe interface hardware device to read data or write data specifically includes: the application program of the user layer directly operates the register, and the DMA transmission function is realized in a polling register mode without transferring through the memory in the kernel.
With reference to the first aspect, in a third possible implementation manner of the first aspect, the compiling an application layer of a host by using a macro-page technology, where the monopolizing a multi-core CPU specifically includes:
configuring a CONFIG _ HUGETLB _ PAGE option and a CONFIG _ HUGETLFS option, and compiling the huge PAGE file into a kernel;
the application program binds the program on a CPU core to run through a CPU core binding technology provided by a linux inner core.
The second aspect of the present invention provides a high-speed data transmission system based on PCIe interface, including: hardware equipment of a host, a PCIe bus and a PCIe interface;
the host has a PCIe bus interface, the hardware equipment of the PCIe interface is accessed into the host through the PCIe bus, and the hardware equipment of the PCIe interface has a DMA function and can finish the read-write operation of data.
With reference to the second aspect, in a first possible implementation manner of the second aspect, a mapping module is disposed in the hardware device of the PCIe interface, and is configured to map the DMA function operation from a kernel state to a user state.
The system according to the second aspect of the present invention is capable of implementing the methods according to the first aspect and the respective implementation manners of the first aspect, and achieves the same effects.
The technical scheme shows that the driver does not adopt the traditional interrupt architecture mode, but reconstructs the mode of the driver, and adopts the application layer to register polling mode architecture, so that the expense of informing the user layer by the kernel in an interrupt mode can be reduced, the data of DMA operation is not transferred through the memory in the kernel, and the data transmission can not trigger interrupt frequently.
The invention adopts the giant page technology at the application layer, reduces the frequency of page table look-up and greatly reduces the expenditure of accessing the memory, and the application layer ensures that the phenomenon of CPU core switching can not occur when the application program runs in the multi-core CPU by the mode of monopolizing the CPU, thereby avoiding the problem of readdressing the memory and accelerating the data processing speed.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a flow chart of a PCIe interface-based high-speed data transmission method;
fig. 2 is a schematic structural diagram of a PCIe interface-based high-speed data transmission system according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, a method for high-speed data transmission based on PCIe interface includes:
step S1, mapping all DMA function operations of the driver of the PCIe interface hardware equipment from a kernel mode to a user mode;
step S2, when the host reads the PCIe interface hardware device, the program of the host application layer is directly connected with the PCIe interface hardware device through the address to read or write data;
and step S3, compiling the application layer of the host through the giant page technology and monopolizing the multi-core CPU.
The steps S1-S3 have no precedence.
In step S1, mapping all DMA function operations of the driver of the PCIe interface hardware device from the kernel mode to the user mode specifically includes:
s11, converting the physical address of the register operating the DMA function into an IO address;
s12, converting the kernel virtual address;
s13, mapping the kernel virtual address to the user layer through the specific interface of the driver.
The application program of the user layer can directly operate the register, and the DMA transmission function is realized in a polling register mode, so that the overhead of informing the application layer by the kernel in an interrupt mode can be reduced, and other related functions matched with the DMA are all mapped to the user mode. The data of the DMA operation is not transferred through the memory in the kernel.
In step S2, when the host reads the PCIe interface hardware device, the directly connecting the program of the host application layer to the PCIe interface hardware device through the address to read data or write data specifically includes: the application program of the user layer directly operates the register, and the DMA transmission function is realized in a polling register mode without transferring through the memory in the kernel.
In step S3, the application layer of the host is compiled by the macro page technique, and the exclusive multi-core CPU specifically includes:
s31, configuring CONFIG _ HUGETLB _ PAGE and CONFIG _ HUGETLLBFS options, and compiling the giant PAGE file into the kernel. When the application program uses a large paging unit, the number of TLB Miss and page fault interrupt can be greatly reduced; the frequency of page table lookup is reduced, and the expenditure for accessing the memory is greatly reduced.
S32, the application program binds the program on a CPU core to run through the CPU core binding technology provided by the linux kernel, so that the problem of memory readdressing is avoided, and the data processing speed is increased.
As shown in fig. 2, a PCIe interface-based high speed data transmission system includes: hardware equipment of a host, a PCIe bus and a PCIe interface;
the host has a PCIe bus interface, the hardware equipment of the PCIe interface is accessed into the host through the PCIe bus, and the hardware equipment of the PCIe interface has a DMA function and can finish the read-write operation of data.
And a mapping module is arranged in the hardware equipment of the PCIe interface and is used for mapping the DMA function operation from a kernel mode to a user mode.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. A high-speed data transmission method based on PCIe interface is characterized in that the method comprises the following steps:
mapping all DMA function operations of a driver of the PCIe interface hardware equipment from a kernel mode to a user mode;
when the host reads the PCIe interface hardware equipment, the program of the host application layer is directly connected with the PCIe interface hardware equipment through the address to read data or write data;
and compiling the application layer of the host through a giant page technology to monopolize the multi-core CPU.
2. The method of claim 1, wherein mapping all DMA function operations of a driver of the PCIe interface hardware device from the kernel mode to the user mode comprises:
converting a register physical address for operating the DMA function into an IO address;
converting a kernel virtual address;
the kernel virtual address is mapped to the user layer through a specific interface of the driver.
3. The method as claimed in claim 1, wherein when the host reads the PCIe interface hardware device, the directly connecting the program of the host application layer to the PCIe interface hardware device through the address to read or write data specifically includes: the application program of the user layer directly operates the register, and the DMA transmission function is realized in a polling register mode without transferring through the memory in the kernel.
4. The method of claim 1, wherein compiling the application layer of the host by the megapage technique, monopolizing the multi-core CPU specifically comprises:
configuring a CONFIG _ HUGETLB _ PAGE option and a CONFIG _ HUGETLFS option, and compiling the huge PAGE file into a kernel;
the application program binds the program on a CPU core to run through a CPU core binding technology provided by a linux inner core.
5. A PCIe interface based high speed data transfer system using the method of claims 1 to 4, the system comprising: hardware equipment of a host, a PCIe bus and a PCIe interface;
the host has a PCIe bus interface, the hardware equipment of the PCIe interface is accessed into the host through the PCIe bus, and the hardware equipment of the PCIe interface has a DMA function and can finish the read-write operation of data.
6. The system of claim 5, wherein a mapping module is provided within the hardware device of the PCIe interface for mapping DMA function operations from kernel mode to user mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011576315.2A CN112699069A (en) | 2020-12-28 | 2020-12-28 | High-speed data transmission method and system based on PCIe interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011576315.2A CN112699069A (en) | 2020-12-28 | 2020-12-28 | High-speed data transmission method and system based on PCIe interface |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112699069A true CN112699069A (en) | 2021-04-23 |
Family
ID=75511789
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011576315.2A Pending CN112699069A (en) | 2020-12-28 | 2020-12-28 | High-speed data transmission method and system based on PCIe interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699069A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107526620A (en) * | 2017-09-28 | 2017-12-29 | 东软集团股份有限公司 | A kind of User space input-output equipment collocation method and device |
| CN108897703A (en) * | 2018-05-30 | 2018-11-27 | 郑州云海信息技术有限公司 | A kind of high speed data transmission system and method based on PCIE |
| CN109710548A (en) * | 2018-12-21 | 2019-05-03 | 荆门博谦信息科技有限公司 | A kind of DMA control data transmission method, system and equipment |
| CN110175107A (en) * | 2019-05-13 | 2019-08-27 | 华中科技大学 | A kind of test method and test macro of FPGA Cloud Server performance |
| CN111124506A (en) * | 2019-11-12 | 2020-05-08 | 中国电子科技集团公司第三十研究所 | Operation card driving implementation method based on application layer |
-
2020
- 2020-12-28 CN CN202011576315.2A patent/CN112699069A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107526620A (en) * | 2017-09-28 | 2017-12-29 | 东软集团股份有限公司 | A kind of User space input-output equipment collocation method and device |
| CN108897703A (en) * | 2018-05-30 | 2018-11-27 | 郑州云海信息技术有限公司 | A kind of high speed data transmission system and method based on PCIE |
| CN109710548A (en) * | 2018-12-21 | 2019-05-03 | 荆门博谦信息科技有限公司 | A kind of DMA control data transmission method, system and equipment |
| CN110175107A (en) * | 2019-05-13 | 2019-08-27 | 华中科技大学 | A kind of test method and test macro of FPGA Cloud Server performance |
| CN111124506A (en) * | 2019-11-12 | 2020-05-08 | 中国电子科技集团公司第三十研究所 | Operation card driving implementation method based on application layer |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7533198B2 (en) | Memory controller and method for handling DMA operations during a page copy | |
| EP1896965B1 (en) | Dma descriptor queue read and cache write pointer arrangement | |
| CN115039077A (en) | Maintaining storage namespace identifiers for real-time virtualized execution environment migration | |
| US20060236063A1 (en) | RDMA enabled I/O adapter performing efficient memory management | |
| US6128674A (en) | Method of minimizing host CPU utilization in driving an adapter by residing in system memory a command/status block a soft interrupt block and a status block queue | |
| US11829309B2 (en) | Data forwarding chip and server | |
| EP2423826A2 (en) | Computer system, i/o device control method, and i/o drawer | |
| US20220327080A1 (en) | PCIe DEVICE AND OPERATING METHOD THEREOF | |
| CN114860329A (en) | Dynamic consistency biasing configuration engine and method | |
| US11928070B2 (en) | PCIe device | |
| US11741039B2 (en) | Peripheral component interconnect express device and method of operating the same | |
| US20180039518A1 (en) | Arbitrating access to a resource that is shared by multiple processors | |
| KR20180041037A (en) | Method for shared distributed memory management in multi-core solid state driver | |
| US11314673B2 (en) | Configurable multi-function PCIe endpoint controller in an SoC | |
| Shim et al. | Design and implementation of initial OpenSHMEM on PCIe NTB based cloud computing | |
| WO2020177567A1 (en) | Method, apparatus, and system for migrating data | |
| WO2012140669A2 (en) | Low pin count controller | |
| US20230153153A1 (en) | Task processing method and apparatus | |
| CN112699069A (en) | High-speed data transmission method and system based on PCIe interface | |
| CN114144768A (en) | Adaptive pipeline selection for accelerating memory copy operations | |
| CN116601616A (en) | Data processing device, method and related equipment | |
| US20130246670A1 (en) | Information processing system | |
| CN116069451B (en) | Virtualization method, device, equipment, medium, accelerator and system | |
| EP4339776A1 (en) | Task scheduling method, system, and hardware task scheduler | |
| WO2023226885A1 (en) | Memory access method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210423 |
|
| WD01 | Invention patent application deemed withdrawn after publication |