CN114253883A

CN114253883A - Access method and system for endpoint device and endpoint device

Info

Publication number: CN114253883A
Application number: CN202111475014.5A
Authority: CN
Inventors: 尹淇; 毛磊冰
Original assignee: Fiberhome Telecommunication Technologies Co Ltd
Current assignee: Fiberhome Telecommunication Technologies Co Ltd
Priority date: 2021-12-03
Filing date: 2021-12-03
Publication date: 2022-03-29
Anticipated expiration: 2041-12-03

Abstract

The application relates to an access method and system of an endpoint device and the endpoint device, wherein a DMA controller is preset in the endpoint device and used for writing data into a host CPU according to control information stored in a register of the preset DMA controller; the method comprises the following steps: the CPU of the host writes the control information of the data to be read into a DMA controller register arranged in the endpoint equipment; and the DMA controller writes the data to be read of the end device into the CPU of the host according to the control information. The method can solve the problem that the CPU of the host is suspended when the PCIe equipment is pulled out in the related technology.

Description

Access method and system for endpoint device and endpoint device

Technical Field

The invention relates to the technical field of data communication, in particular to an endpoint device access method, a system and an endpoint device.

Background

With the gradual evolution of the PCIe standard, the PCIe protocol has become the mainstream physical layer communication method for high-speed data exchange between peripheral devices and a CPU. The device has hot plug (hot plug) capability, and is incorporated in the evolution process of PCIe standard, and a series of requirements are provided for the hardware design of the PCIe peripheral, for example, under the condition of requiring external hardware intervention, hardware interruption is given to ensure that software terminates accessing PCIe peripheral resources in advance, and then hardware unplugging action is completed. However, this approach is still not safe enough, and in extreme cases, it may cause CPU instruction level hang, corresponding to Linux system, which represents system hang, and corresponding to Windows, we may observe a blue screen. The PCIe protocol implementation mechanism and the CPU implementation mechanism determine that the standard PCIe hot plug can not completely avoid the phenomenon.

There are two behaviors for standard PCIe device communication, post and non-post respectively. The post is unidirectional interaction, for example, the host sends data to the slave, and the slave sends data to the host, which are considered as post; the non-post indicates that the interaction between the host and the slave is bidirectional and the corresponding operation must be completed, in general, the read operation of the CPU to the PCIe peripheral is the non-post, the instruction corresponding to the CPU is an atomic instruction, and the instruction cannot be split, and not all CPUs design a timeout exception response mechanism for the atomic operation. If the non-post operation is executed when the peripheral is pulled out, the CPU instruction cannot be continuously executed to cause the system to be suspended; even if the CPU designs a timeout response mechanism, there may be a hang due to failure to obtain valid data.

Disclosure of Invention

The embodiment of the invention provides PCIe (peripheral component interface express) equipment, a PCIe equipment access method and a PCIe equipment access system, which are used for solving the problem that a host CPU (central processing unit) is suspended when the PCIe equipment is pulled out in the related art.

On one hand, the embodiment of the invention provides an end device access method, which is characterized in that a DMA controller is preset in the end device, and the DMA controller is used for writing data into a host CPU according to control information stored in a preset DMA controller register; the method comprises the following steps:

the CPU of the host writes the control information of the data to be read into a DMA controller register arranged in the endpoint equipment;

and the DMA controller writes the data to be read of the end device into the CPU of the host according to the control information.

In some embodiments, the writing, by the cpu of the host, the control information of the data to be read into the DMA controller register provided in the endpoint device includes:

applying for a memory segment matched with a corresponding DMA controller register at a root complex side, wherein the memory segment is used for storing the control information written by the CPU of the host in the corresponding DMA controller register;

writing the control information into a DMA controller register corresponding to a memory segment at the root complex side;

the control information comprises a source address of the data to be read, a destination address of the data to be read, the length of the data to be read and transmission control information, wherein the transmission control information comprises transmission enabling and transmission closing and is used for controlling the DMA controller to start or stop writing the data into the cpu.

In some embodiments, when there is a DMA transfer requirement for multiple pieces of data, a DMA controller in an endpoint device initializes multiple pieces of DMA controller registers;

the application of the memory segment matched with the corresponding DMA controller register at the root complex side comprises the following steps:

applying for a plurality of data spaces corresponding to the multiple sections of DMA controller registers in the endpoint device at a root complex side, wherein each data space is used for storing a data section corresponding to DMA transmission;

and enabling the value of the last member in each data space to point to the address corresponding to the next DMA transmission control register at the root complex side.

In some embodiments, the control information comprises transmission status information comprising transmission not initiated and transmission completed;

the DMA controller writes the data to be read of the end device into the cpu of the host according to the control information, and the method comprises the following steps:

the DMA controller writes the data to be read into a destination address corresponding to the root complex side according to the control information;

and after the transmission is finished, the DMA controller modifies the transmission state information in the DMA controller register in the endpoint device into transmission finished information and reports the transmission finished information to the CPU of the host.

In some embodiments, the method comprises:

the host CPU judges whether the end equipment to be accessed works abnormally according to the transmission state information acquired by the root complete side from the DMA controller of the end equipment to be accessed, and sends a prompt that the end equipment works abnormally when judging that the end equipment works abnormally.

In a second aspect, an embodiment of the present invention provides an endpoint device, where a DMA controller is preset in the endpoint device, the DMA controller is configured to write data to a host CPU according to control information stored in a preset DMA controller register, and the control information is written to the DMA controller register by a host CPU according to data reading requirements.

In a third aspect, an embodiment of the present invention further provides an endpoint device access system, which is characterized in that the system includes:

the host CPU is used for writing control information of data to be read into a DMA controller register arranged in the end device when the end device is accessed;

and the DMA controller is used for writing data to be read into the host CPU according to control information stored in a preset DMA controller register.

In some embodiments, the host cpu is configured to:

In some embodiments, the endpoint device is further configured to:

when the DMA transmission requirement of a plurality of data segments exists, initializing a plurality of DMA controller registers and storing control information required by the transmission of the corresponding data segments in each DMA controller register;

the host cpu is further configured to:

the DMA controller is to:

writing the data to be read into a destination address corresponding to the root complex side according to the control information;

after the transmission is finished, the DMA controller modifies the transmission state information in the DMA controller register in the endpoint equipment into transmission finished information and reports the transmission finished information to the CPU of the host;

the host cpu is further configured to:

and judging whether the end equipment to be accessed works abnormally or not according to the transmission state information acquired by the root complex side from the DMA controller of the end equipment to be accessed, and sending a prompt of the end equipment working abnormally when the work is judged to be abnormal.

The embodiment of the invention has the beneficial effects that:

because of the DMA controller preset in the end device, the host CPU does not access the slave machine in a non-post type bidirectional interaction mode, but writes the control information of the data to be read into the DMA controller in a post type unidirectional interaction mode, so as to realize the access to the end device. The hot plug of the end point device can be realized without the intervention of external hardware (such as a pluggable plate). When the CPU is not pulled up due to the fact that the CPU is hung due to the non-post mode interaction, the problem of the external endpoint device occurs under the timely and extreme condition, the CPU is not hung, the problems of halt, blue screen and the like are avoided, hardware design is simplified, and meanwhile system reliability is improved. In addition, under a large data throughput scene, the host CPU finishes read operation through a DMA controller of the endpoint device, and because PCIe protocol design is friendly to large data space devices, actual data acquisition efficiency is higher than that of direct reading for multiple times according to 32-bit data, and efficiency is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart of an endpoint device access method according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of an endpoint device access system according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.

As shown in fig. 1, an embodiment of the present invention provides an access method for an endpoint device, where a DMA controller is preset in the endpoint device, and the DMA controller is configured to write data to a host CPU according to control information stored in a preset DMA controller register; the method comprises the following steps:

s100, writing control information of data to be read into a DMA controller register arranged in the endpoint device by a host CPU;

and S200, the controller writes the data to be read of the end device into the host CPU according to the control information.

It should be noted that the DMA controller is a DMA (Direct Memory Access) controller; the Endpoint device includes DMA ctrl (DMA controller), DMA mem (DMA controller register for holding configuration DATA of the DMA controller), DATA mem (memory location of EP subsystem for storing DATA of all EP sides).

It can be understood that, before step S100, when the host cpu finds an end device, the specific information (including data length, address, etc.) of the end configuration space and the data space is obtained by using a standard end device enumeration method in the kernel.

According to the embodiment of the invention, because the DMA controller is preset in the endpoint equipment, the CPU of the host does not access the slave machine in a non-post type bidirectional interaction mode, but writes the control information of the data to be read into the DMA controller in a post type unidirectional interaction mode, so as to access the endpoint equipment. The hot plug of the end point device can be realized without the intervention of external hardware (such as a pluggable plate). When the CPU is not pulled up due to the fact that the external end point device is hung up due to the non-post mode interaction, the external PCIe device is not hung up in time under extreme conditions, the CPU is not hung up, the problems of crash, blue screen and the like are avoided, hardware design is simplified, and system reliability is improved. In addition, under a large data throughput scene, the host CPU finishes read operation through a DMA controller of the endpoint device, and because PCIe protocol design is friendly to large data space devices, actual data acquisition efficiency is higher than that of direct reading for multiple times according to 32-bit data, and efficiency is improved.

In some embodiments, step S100 comprises:

s110, applying for a memory segment matched with a corresponding DMA controller register at a root complex side, wherein the memory segment is used for storing the control information written by the CPU of the host in the corresponding DMA controller register;

s120, writing the control information into a DMA controller register corresponding to the memory segment at the root complex side;

the control information of the data to be read comprises a source address of the data to be read, a destination address of the data to be read, the length of the data to be read and transmission control information; the transfer control information includes transfer enable and transfer close, which are used to control the DMA controller to start or stop transferring data.

It should be noted that, a root complex (abbreviated as RC or RC) is a hub of the host cpu and the slave device, and may support a function of routing messages between different domains according to specific selection, where the host cpu further includes a kernel core, a storage unit mem, and the like.

It can be understood that, in the content used for storing by the DMA controller register, the source address of the data to be read refers to the source address of the data to be read at the end device side, the destination address of the data to be read refers to the destination address where the data to be read is finally stored at the root complex side, and the length of the data to be read refers to the length of the data that needs to be read in one DMA read operation.

When the RC side needs to read the data at the endpoint side, the source address, the destination address and the data length of the data to be read can be written into the DMA controller register, and when the transfer enable (which may be an enable bit of the transfer control information) is written, the DMA controller starts the corresponding data transfer to the host side.

It is understood that, in step S110, applying for the memory segment space matched with the DMA controller register in the slave end device on the root complete side is convenient for buffering and managing data. The buffer setting of data depends on the actual data space size and performance requirements of the EP, and is not strictly limited. The management space that operates in conjunction with the DMA controller may include the following pseudo code:

struct{

ARCH_POINTER*host_cfg_addr；

ARCH_POINTER*srcaddr；

ARCH_POINTER*distaddr；

int datalen；

int transfer_status；

int transfer_ctrl；

void*next；

}EP_DMACTL；

wherein, ARCH _ POINTED is the instruction bit width change used by the CPU, if the CPU is 64 bits, the address is 64 bits, and if the CPU is 32 bits, the address is 32 bits; since the PCIe protocol is currently accessed with 32-bit data bit width, datalen uses int type; transfer _ status indicates a transfer status bit, which is filled in by the DMA controller at the EP side after the DMA operation is completed.

In some embodiments, when there is a DMA transfer of multiple pieces of data, the DMA controller in endpoint initializes multiple pieces of DMA controller registers;

step S110 includes the steps of:

s111, applying for a plurality of data spaces corresponding to the multiple DMA controller registers in the endpoint complex side, wherein each data space is used for storing a data segment corresponding to DMA transmission;

s112, on the root complex side, enabling the value of the last member in each data space to point to the address corresponding to the next DMA transmission control register;

in the embodiment, the condition of multi-segment DMA transfer is considered, and the host CPU applies for a plurality of data spaces on the root complete side and is used for receiving data required by the multi-segment DMA transfer. For the condition of multi-segment DMA transmission, when the RC side writes the control information into the endpoint, the multi-time writing can be simplified into 1-time writing, and the data reading efficiency is improved.

The ability to allow multiple DMA control segments to be initialized, thus enabling multiple segmented DMA operations, may be specifically expressed in terms of the void used in the pseudo-code segment structure described above.

In some embodiments, step S200 includes the steps of:

s210, the DMA controller writes the data to be read into a destination address corresponding to the root complex side according to the control information;

s220, the DMA controller modifies the transmission state information in the DMA controller register in the end device into transmission completion and reports the transmission completion to the CPU of the host computer after the transmission is completed.

It should be noted that the control information includes transfer status information, and the transfer status information is used to reflect the status of data transfer by the DMA controller, including that the transfer is not started and the transfer is completed. In actual operation, the status bits of the transmission status information can be represented by 0 and 1 respectively (0 represents that the transmission is not started, and 1 represents that the transmission is completed), and the DMA controller can modify the transmission status according to the actual transmission condition.

Meanwhile, it can be understood that the control information further includes a host configuration address, which indicates an address for storing data in a corresponding DMA controller register on the root complete side; in step S220, the DMA controller first modifies the transmission status information in the DMA controller register located at the endpoint, calculates the transmission status bit offset of the root complex side according to the DMA controller register structure and the host configuration address, executes an independent DMA transmission operation, and writes the transmission status information indicating that the transmission is completed into the corresponding address of the root complex side.

In a specific implementation process, specific instructions (pseudo code) include:

Write(*ep_dmactl_src_addr,ep_data_addr,addrlen)；

Write(*ep_dmactl_dist_addr,cpu_data_addr,addrlen)；

Write(*ep_dmactl_rdlen_addr,datalen,addrlen)；

Write(*ep_dmactl_transfer,START,sizeof(START))；

in actual operation, the structure configuration data is written once using memset as:

struct EP_DMA_CTL*ep_dma_ctl；

ep_dma_ctl->host_cfg_addr＝host_cfg_addr；

ep_dma_ctl->srcaddr＝ep_data_addr；

ep_dma_ctl->distaddr＝cpu_data_addr；

ep_dma_ctl->datalen＝datalen

ep_dma_ctl->transfer_ctrl＝START；

ep_dma_ctl->transfer_status＝TRANSFERING；

memcpy(ep_dma_ctl,ep_dmactl_addr,(sizeof(struct EP_DMA_CTL)))；

wherein, host _ cfg _ addr points to the management space (the memory segment address matched with the corresponding DMA controller register applied at the root complex side) applied in the cpu subsystem memory and cooperating with the DMA controller;

after the operation of the DMA controller in the Endpoint device is finished, according to the configuration data in ep _ dmact, the host _ cfg _ addr is used as a base address, the offset of transfer _ status is superposed, and the transmission state is set to be DONE.

The atomic operation of read is changed to a multiple instruction operation and is all a write operation, with no read operation.

Based on data processing efficiency requirements, DMA data may be allowed to be transmitted in multiple fragments.

In some embodiments, the method further comprises:

and S300, the CPU of the host judges whether the end equipment to be accessed works abnormally according to the transmission state information acquired by the root complete side from the DMA controller of the end equipment to be accessed, and sends a prompt of the end equipment working abnormally when judging that the end equipment works abnormally.

It will be appreciated that an endpoint device operation exception includes multiple failures to initiate a DMA operation, a hot-unplug action or a failure, etc. When the end point has the abnormal working conditions, the CPU of the host can identify and judge by comparing the transmission state with the time required for transmitting normal data. After the upper layer software obtains the end equipment work abnormity prompt sent by the host CPU, the end equipment work abnormity condition can be further diagnosed or unloaded according to the requirement.

As shown in fig. 2, an embodiment of the present invention provides an endpoint device (ep subsystem), in which a DMA controller (DMA ctrl) is preset, and the DMA controller is configured to write data to a host CPU according to control information stored in a preset DMA controller register, where the control information is written to the DMA controller register by a host CPU according to data reading requirements.

Wherein, rc (rc) refers to a root complex (a hub of the host cpu and the slave device, and may support a function of routing messages between different domains according to specific selection) on the cpu side of the host, and the DMA controller is a DMA (Direct Memory Access) controller; the host cpu further comprises a kernel core and a storage unit mem; the endpoint (ep subsystem) of the PCIe device further includes DMA ctrl (DMA controller), DMA mem (DMA controller register for holding configuration DATA of the DMA controller), and DATA mem (memory location of the ep subsystem for storing DATA on all ep subsystems).

It should be noted that the endpoint device has a plurality of ways to preset the DMA controller, including ways to implement through a logic device or configure the embedded processor as endpoint, which can implement the ep subsystem in this embodiment. The end point device with the DMA controller is constructed by using a Programmable logic device, and can be implemented based on Field-Programmable Gate arrays (FPGA) of manufacturers Altera and Xilinx. In this embodiment, the PCIe controller (which may be configured as either a root complex or an endpoint) may be set as an endpoint device instead of the root complex, and the DMA controller of the CPU is used, so that a dedicated DMA control driver is developed according to the requirement of the embodiment.

It can be understood that, whichever of the above-described manners of constructing an endpoint device in which a DMA controller is preset, the characteristics include: a certain address space (CFG MEM as shown in fig. 1 for holding configuration data of the DMA controller) and a completely exclusive DMA controller (for performing DMA read actions based on data written in the CFG MEM).

By adopting the endpoint device of the embodiment, the cpu of the host accesses the slave in a non-post type bidirectional interactive manner, but writes the control information of the data to be read into the DMA controller in a post type unidirectional interactive manner, so as to access the endpoint device. The hot plug of the end point device can be realized without the intervention of external hardware (such as a pluggable plate). When the CPU is not pulled up due to the fact that the external endpoint equipment is hung up due to the non-post mode interaction, the problem of the external endpoint equipment occurs under the timely and extreme condition, the CPU is not hung up, the problems of halt, blue screen and the like are avoided, hardware design is simplified, and system reliability is improved. In addition, under a large data throughput scene, the host CPU finishes read operation through a DMA controller of the endpoint device, and because PCIe protocol design is friendly to large data space devices, actual data acquisition efficiency is higher than that of direct reading for multiple times according to 32-bit data, and efficiency is improved.

As shown in fig. 2, an embodiment of the present invention provides a PCIe device access system, which includes:

the host CPU is used for writing control information of data to be read into a DMA controller register arranged in the end equipment when the end equipment is accessed;

Wherein, rc (rc) refers to a root complex (a hub of the host cpu and the slave device, and may support a function of routing messages between different domains according to specific selection) on the cpu side of the host, and the DMA controller is a DMA (Direct Memory Access) controller; the host cpu further comprises a kernel core and a storage unit mem; the endpoint (ep subsystem) of the PCIe device further includes DMA ctrl (DMA controller), DMA mem (DMA controller register for holding configuration DATA of the DMA controller), and DATA mem (memory location of ep subsystem for storing DATA of all ep subsystems).

In some embodiments, the host cpu is configured to:

writing the control information into a DMA controller register corresponding to the memory segment at the root complex side;

In some embodiments, the endpoint device is further configured to:

when the DMA transmission requirement of a plurality of data segments exists, initializing a plurality of DMA controller registers and storing control information required by the transmission of the corresponding data segments in each DMA controller register.

The host cpu is also configured to:

In some embodiments, the control information includes transmission status information, the transmission status information including transmission not initiated and transmission completed;

the DMA controller is to:

The host cpu is also configured to:

according to the transmission state information acquired by the root complex side from the DMA controller of the end device to be accessed, whether the end device to be accessed works abnormally is judged, and in some embodiments, a prompt that the end device works abnormally is sent when the work is judged to be abnormal, the host cpu is configured to:

and judging whether the end point of the PCIe device to be accessed works abnormally or not according to the transmission state information acquired by the root complex side from the DMA controller of the end point of the PCIe device to be accessed, and sending a prompt of the end point of the PCIe device working abnormally when judging that the end point works abnormally.

One of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable storage media, which may include computer readable storage media (or non-transitory media) and communication media (or transitory media).

The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An access method of an endpoint device is characterized in that a DMA controller is preset in the endpoint device, and the DMA controller is used for writing data into a host CPU according to control information stored in a preset DMA controller register; the method comprises the following steps:

2. The endpoint device access method of claim 1,

the method comprises the following steps that a CPU (central processing unit) of a host writes control information of data to be read into a DMA (direct memory access) controller register arranged in an endpoint device, and comprises the following steps:

3. The endpoint device access method of claim 2,

when the DMA transmission requirement of a plurality of data sections exists, a DMA controller in the endpoint equipment initializes a plurality of DMA controller registers;

4. The endpoint device access method of claim 2,

the control information comprises transmission state information, and the transmission state information comprises transmission non-starting and transmission completion;

5. The endpoint device access method of claim 4, wherein the method comprises:

6. An end point device is characterized in that a DMA controller is preset in the end point device, the DMA controller is used for writing data into a host CPU according to control information stored in a preset DMA controller register, and the control information is written into the DMA controller register by a host CPU according to the requirement of reading the data.

7. An endpoint device access system, comprising:

8. The endpoint device access system of claim 7,

the host cpu is used for:

9. The endpoint device access system of claim 8,

the endpoint device is further configured to:

the host cpu is further configured to:

10. The endpoint device access system of claim 9,

the DMA controller is to:

the host cpu is further configured to: