CN117725012A - Sharing method, device, equipment and storage medium of PCIe equipment - Google Patents
Sharing method, device, equipment and storage medium of PCIe equipment Download PDFInfo
- Publication number
- CN117725012A CN117725012A CN202311773957.5A CN202311773957A CN117725012A CN 117725012 A CN117725012 A CN 117725012A CN 202311773957 A CN202311773957 A CN 202311773957A CN 117725012 A CN117725012 A CN 117725012A
- Authority
- CN
- China
- Prior art keywords
- pcie device
- pcie
- shadow
- configuration space
- shared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000006870 function Effects 0.000 claims description 63
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004891 communication Methods 0.000 description 11
- 230000002093 peripheral effect Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 239000004744 fabric Substances 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Landscapes
- Information Transfer Systems (AREA)
Abstract
The application discloses a sharing method, a device, equipment and a storage medium of PCIe equipment, which relate to the technical field of server clusters, wherein the method is applied to the server clusters, the server clusters comprise at least one server host, the server host is connected with the shared PCIe equipment of a data center through a non-transparent bridge, and the method comprises the following steps: acquiring the content of a first configuration space of the shared PCIe device; creating a shadow PCIe device according to the content of the first configuration space; establishing a data channel through the non-transparent bridge by the second address of the base address register of the shadow PCIe device and the first address of the base address register of the shared PCIe device; and controlling the shared PCIe device through the data channel. The method can reduce the use quantity of PCIe devices and reduce the cost.
Description
Technical Field
The present disclosure relates to the field of server clusters, and in particular, to a method, an apparatus, a device, and a storage medium for sharing PCIe devices.
Background
PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard. With the rapid development of PCIe technology and the continuous growth of large data transmission demands for computer peripherals in recent years, device manufacturers have become mainstream to produce peripheral devices at high speed using PCIe standards, for example, PCIe devices include an image processing unit (GPU) and a Data Processing Unit (DPU), and the like.
At present, each device in the server cluster has a requirement for using PCIe devices, so that the corresponding PCIe devices are required to be configured for each device in the server cluster, a large number of PCIe devices are required, and the cost is high.
Disclosure of Invention
The application provides a sharing method, device, equipment and storage medium of PCIe equipment, which can reduce the use quantity of the PCIe equipment and reduce the cost.
In order to achieve the above purpose, the present application adopts the following technical scheme:
in a first aspect, the present application provides a method for sharing PCIe devices, where the method is applied to a server cluster, where the server cluster includes at least one server host, where the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, and the method includes:
acquiring the content of a first configuration space of the shared PCIe device;
creating a shadow PCIe device according to the content of the first configuration space;
establishing a data channel through the non-transparent bridge by the second address of the base address register of the shadow PCIe device and the first address of the base address register of the shared PCIe device;
and controlling the shared PCIe device through the data channel.
In some possible implementations, the content of the second configuration space of the shadow PCIe device is the same as the content of the first configuration space of the shared PCIe device.
In some possible implementations, after the creating the shadow PCIe device according to the content of the first configuration space, the method further includes:
creating an interface function of the shadow PCIe device, the interface function being used to access content of the second configuration space;
and adding the interface function to an interface function set of an interconnection bus of the external equipment.
In some possible implementations, the method further includes:
retrieving a device number and a function number of a target device on the external device interconnection bus;
and if the equipment number is a preset equipment number and the function number is a preset function number, calling the interface function.
In some possible implementations, the method further includes:
performing write operation on a second configuration space of the shadow PCIe device through a write access function of the shadow PCIe device;
if the address corresponding to the writing operation is a preset address and the writing value is a preset value, or the address corresponding to the writing operation is a second address, ending the writing operation.
In a second aspect, the present application provides a sharing apparatus of PCIe devices, applied to a server cluster, where the server cluster includes at least one server host, where the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, and the apparatus includes:
the acquisition module is used for acquiring the content of the first configuration space of the shared PCIe device;
the creating module is used for creating a shadow PCIe device according to the content of the first configuration space;
a configuration module, configured to establish a data channel through the non-transparent bridge with a second address of a base address register of the shadow PCIe device and a first address of a base address register of the shared PCIe device;
and the control module is used for controlling the shared PCIe equipment through the data channel.
In some possible implementations, the content of the second configuration space of the shadow PCIe device is the same as the content of the first configuration space of the shared PCIe device.
In some possible implementations, the apparatus further includes an add module;
the creating module is further configured to create an interface function of the shadow PCIe device, where the interface function is used to access content of the second configuration space;
the adding module is configured to add the interface function to an interface function set of an interconnection bus of the external device.
In some possible implementations, the apparatus further includes: a retrieval module and a calling module;
the retrieval module is used for retrieving the device number and the function number of the target device on the external device interconnection bus;
and the calling module is used for calling the interface function if the equipment number is a preset equipment number and the function number is a preset function number.
In some possible implementations, the control module is further configured to perform a write operation on the second configuration space of the shadow PCIe device through a write access function of the shadow PCIe device; if the address corresponding to the writing operation is a preset address and the writing value is a preset value, or the address corresponding to the writing operation is a second address, ending the writing operation.
In a third aspect, the present application provides a computing device comprising a memory and a processor;
wherein one or more computer programs are stored in the memory, the one or more computer programs comprising instructions; the instructions, when executed by the processor, cause the computing device to perform the method of any of the first aspects.
In a fourth aspect, the present application provides a computer readable storage medium for storing a computer program for performing the method of any one of the first aspects.
According to the technical scheme, the application has at least the following beneficial effects:
the application provides a sharing method of PCIe devices, which is applied to a server cluster, wherein the server cluster comprises at least one server host, and the server host is connected with the shared PCIe devices of a data center through a non-transparent bridge, and the method comprises the following steps: and acquiring the content of a first configuration space of the shared PCIe device, creating a shadow PCIe device according to the content of the first configuration space, then establishing a data channel between a second address of a base address register of the shadow PCIe device and a first address of a base address register of the shared PCIe device through a non-transparent bridge, and finally controlling the shared PCIe device through the established data channel, thereby realizing the sharing of the PCIe device. Based on the method, the shared PCIe devices can be shared by the multiple server hosts through the respective shadow PCIe devices, so that the actual use quantity of the PCIe devices is reduced, and the cost is lowered.
It should be appreciated that the description of technical features, aspects, benefits or similar language in this application does not imply that all of the features and advantages may be realized with any single embodiment. Conversely, it should be understood that the description of features or advantages is intended to include, in at least one embodiment, the particular features, aspects, or advantages. Therefore, the description of technical features, technical solutions or advantageous effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and advantageous effects described in the present embodiment may also be combined in any appropriate manner. Those of skill in the art will appreciate that an embodiment may be implemented without one or more particular features, aspects, or benefits of a particular embodiment. In other embodiments, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments.
Drawings
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present application;
FIG. 2 is a flowchart of a method for sharing PCIe devices according to an embodiment of the present application;
FIG. 3 is a flow chart of a write operation provided by an embodiment of the present application;
FIG. 4 is a flow chart of a read operation provided in an embodiment of the present application;
FIG. 5 is a flowchart of a function call procedure provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of a sharing device of PCIe equipment according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a computing device according to an embodiment of the present application.
Detailed Description
The terms "first," "second," and "third," and the like, in the description and in the drawings, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
For clarity and conciseness in the description of the following embodiments, a brief description of the related art will be given first:
PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard for connecting external devices (referred to as PCIe devices) in a computer system, e.g., PCIe devices may include: graphics cards, network adapters, memory controllers, and the like. PCIe devices have several key features and advantages: high bandwidth: PCIe provides a scalable high bandwidth capable of meeting the data transmission speed requirements of modern devices. PCIe uses a high speed serial connection to achieve faster data transfer rates. Scalability: PCIe has multiple lanes, allowing multiple devices to transmit data simultaneously, and may add more lanes as needed, improving bus bandwidth. Low delay: compared with the traditional parallel bus, the serial connection mode of PCIe reduces communication delay and is beneficial to realizing faster data transmission and higher system performance. Multifunction: PCIe standards support different device types and functions, including Graphics Processing Units (GPUs), network Interface Cards (NICs), memory controllers, etc., enabling them to meet a variety of computing and communication requirements.
At present, each device in the server cluster has a requirement for using PCIe devices, so that the corresponding PCIe devices are required to be configured for each device in the server cluster, a large number of PCIe devices are required, and the cost is high.
In view of this, an embodiment of the present application provides a method for sharing PCIe devices, where the method may be performed by a server host in a server cluster, where the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, and specifically includes:
the server host acquires the content of a first configuration space of the shared PCIe device, then creates a shadow PCIe device according to the content of the first configuration space, establishes a data channel through a non-transparent bridge between a second address of a base address register of the shadow PCIe device and a first address of a base address register of the shared PCIe device, and then controls the shared PCIe device through the data channel, thereby realizing the sharing of the PCIe device.
Based on the method, the shared PCIe devices can be shared by the multiple server hosts through the respective shadow PCIe devices, so that the actual use quantity of the PCIe devices is reduced, and the cost is lowered.
In order to make the technical solution of the present application clearer and easier to understand, an application scenario provided by the embodiment of the present application is first described, as shown in fig. 1, which is a schematic diagram of an application scenario provided by the embodiment of the present application.
In this application scenario, the server cluster 100 includes a plurality of server hosts, such as a first server host 101 and a second server host 102, and the data center 300 is connected to the server hosts in the server cluster 100 through a Non-transparent bridge (Non-Transparent Bridge, NTB) 200. Typically, a computer system supports only one PCIe address domain, i.e., different computer systems have separate PCIe address domains. Direct data communication cannot be performed between different PCIe address domains, and after address conversion is performed through the NTB 200, a data channel is established between the different PCIe address domains, so that data communication is realized.
In this embodiment of the present application, each server host (taking the first server host 101 as an example) in the server cluster 100 may create a shadow PCIe device locally, where the shadow PCIe device is not a real PCIe device, but is a virtualized PCIe device, and it is required that the content of the second configuration space of the shadow PCIe device is consistent with the content of the configuration space of the shared PCIe device 301, after the creation of the shadow PCIe device is completed by the first server host 101, data communication may be established between the second address of the base address register of the shadow PCIe device and the first address of the base address register of the shared PCIe device 301 through the NTB 200, so that the first server host 101 controls the remote shared PCIe device 301 through the shadow PCIe device. After the multiple server hosts perform similar processing, the multiple server hosts can control the remote shared PCIe devices 301, so that the real PCIe devices do not need to be locally plugged in, and the sharing of the PCIe devices is realized. Therefore, in the application scene, the use quantity of real PCIe devices is reduced, and the cost is reduced.
In order to make the technical solution of the present application clearer and easier to understand, the method for sharing PCIe devices provided in the embodiments of the present application is described below with reference to the accompanying drawings, as shown in fig. 2, where the drawing is a flowchart of a method for sharing PCIe devices provided in the embodiments of the present application, the method may be applied to a server cluster, where the server cluster includes at least one server host, and the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, and the method includes:
s201, acquiring the content of a first configuration space of the shared PCIe device.
The shared PCIe device refers to a real PCIe device, where the shared PCIe device is plugged onto a host of the data center, and is used for sharing by a server host in the server cluster. The contents of the first configuration space include basic information of the shared PCIe device, such as connection capabilities, versions, and characteristics. Since the shared PCIe device is a truly existing device, that is, the content of the first configuration space of the shared PCIe device is known content, the content of the first configuration space of the shared PCIe device may be obtained.
In some embodiments, prior to this step, the server host may also look up the NTB fabric pointer, and in the event that the pointer is not empty, then perform this step. For example, the lookup may be by a unique identification of the NTB.
S202, creating a shadow PCIe device according to the content of the first configuration space.
After obtaining the contents of the first configuration space of the shared PCIe device, the shadow PCIe device may be created based on the contents of the first configuration space.
Specifically, the content of the second configuration space of the shadow PCIe device is set to the content of the first configuration space of the shared PCIe device, that is, the content of the second configuration space of the shadow PCIe device is the same as the content of the first configuration space of the shared PCIe device.
In preparing the second configuration space of the shadow PCIe device, a space of 4kb may be allocated in memory, and then the contents of the first configuration space of the shared PCIe device may be copied to that space.
It should be noted that, for some special shared PCIe devices, special processing is also required to facilitate multiple server hosts in the server cluster to share and use the shared PCIe device at the same time. For example, in the case where the shared PCIe DEVICE supporting Single Root I/O Virtualization is a Virtual Function (VF), and the values of the pci_vector_id (0 x 00) and the pci_device_id (0 x 02) registers of the first configuration space are both 0xFFFF, it is necessary to modify the pci_vector_id (0 x 00) and the pci_device_id (0 x 02) to be real VENDOR IDs and DEVICE IDs (DEVICE IDs) of the VF in the second configuration space of the shadow PCIe DEVICE.
In this embodiment, while creating the shadow PCIe device, the fabric pointer of the peripheral component interconnect BUS (PCI BUS) may be searched based on the fabric pointer of the NTB, and then the corresponding interface function set may be searched based on the fabric pointer of the peripheral component interconnect BUS.
After creating the shadow PCIe device, an interface function of the shadow PCIe device may also be created, the interface function being used to access the content of the second configuration space, and then the interface function being added to the set of interface functions of the external device interconnect bus. The interface function may be hung on the external device interconnection bus, for example, by means of a hook. In this way, during the process of enumerating the shadow PCIe device on the external device interconnection bus, the PCI subsystem can correctly access to the configuration space of the shadow PICe device, and the enumeration process can be controlled through the interface function, so that enumeration is determined to be successful, and then the initial structure body of the shadow PCIe device is obtained.
The enumeration process is described below.
The writing operation can be performed on the second configuration space of the shadow PCIe device through the write access function of the shadow PCIe device, if the address corresponding to the writing operation is a preset address and the writing value is a preset value, or the address corresponding to the writing operation is a second address, the writing operation is ended, so that the PCI subsystem considers that the writing is successful, and the preset address is not actually written to the preset value, or the writing operation is not actually performed only on the two addresses, thereby reducing the influence on the native system only slightly.
For ease of understanding, the following description is given by way of example with reference to the accompanying drawings.
As shown in fig. 3, the present application provides a flowchart of a write operation, where a process of the write operation includes:
s301, judging whether the offset address is legal.
In some examples, the determination of whether the offset address is legitimate may be based on a configuration space address range defined by the PCIe protocol specification, if the offset address is within the configuration space address range defined by the PCIe protocol specification, the offset address is deemed legitimate, otherwise the offset address is not legitimate.
If the offset address is legal, S302 is performed, and if the offset address is not legal, S307 is performed.
S302, judging whether the offset address is in the range of the second address.
Wherein the second address may range from 0x10 to 0x 24. If the offset address is in the range of [0x 10-0 x24], then S307 is performed; if the offset address is not in the range of 0x10 to 0x24, S303 is performed.
S303, judging whether the offset address is a preset address.
Wherein, the preset address may be 0x06, if the offset address is 0x06, S304 is performed, and if the offset address is not 0x06, S305 is performed.
S304, judging whether the written value is a preset value.
Wherein the preset value is 0xFF, and if the write value is 0xFF, S307 is performed; if the write value is not 0xFF, S305 is performed.
S305, acquiring a head address of a second configuration space of the shadow PCIe device.
S306, writing the writing value at the address after summing the head address and the offset address.
S307, the write operation is ended.
As shown in fig. 4, the present application provides a flow chart of a read operation, where a process of the read operation includes:
s401, judging whether the offset address is legal or not.
If the offset address is legal, S402 is performed, and if the offset address is not legal, S404 is performed.
S402, acquiring a head address of a second configuration space of the shadow PCIe device.
S403, reading the value at the address after summing the head address and the offset address.
S404, ending the reading operation.
After the hook is hung, the device number and the function number of the target device on the external device interconnection bus can be retrieved, and if the device number is a preset device number and the function number is a preset function number, the interface function of the shadow PCIe device is called. The preset device number may be a device number of the shadow PCIe device, and the preset function number may be a function number of the shadow PCIe device. As shown in fig. 5, the figure is a flowchart of a function call procedure provided in an embodiment of the present application, where the procedure includes:
s501, judging whether the target device is a shadow PCIe device.
If the target device is a shadow PCIe device, S502 is performed, and if the target device is not a shadow PCIe device, S503 is performed.
S502, calling an interface function of the shadow PCIe device.
S503, calling an interface function of the target device.
Therefore, the modification to the original system can be reduced, and the influence on the original flow in the original system is reduced after the hooks are hung.
S203, establishing a data channel between the second address of the base address register of the shadow PCIe device and the first address of the base address register of the shared PCIe device through a non-transparent bridge.
After the PCI subsystem enumerates the shadow PCIe device, the structure of the shadow PCIe device is already generated, and the BAR resource of the shadow PCIe device is illegal and needs to be redistributed. After the reassignment, the entry address resource of the data channel is part of the NTB BAR resource, and the BAR resource in the structure of the modified shadow PCIe device is described as a corresponding part of the NTB BAR resource.
In some examples, a data channel is established over the NTB with a second address of a base address register (BAR, base Address Register) of the shadow PCIe device and a first address of the BAR of the shared PCIe device. Specifically, an NTB Memory Window (MW) is set, and an address of an NTB BAR corresponding to the MW is reassigned to a BAR of a shadow PCIe device, and then, when a device driver of a server host accesses the BAR of the shadow PCIe device, the device driver directly enters the MW corresponding to the NTB, thereby accessing the BAR of the shared PCIe device.
S204, controlling the shared PCIe device through the data channel.
After the creation of the shadow PCIe device and the creation of the data channel are completed, the shared PCIe device may be controlled through the data channel. And after the shadow PCIe devices are all created by the plurality of server hosts in the server cluster and the shared PCIe devices are used for establishing data channels through the NTB, the shared PCIe devices can be commonly used by the plurality of server hosts in the server cluster, so that the shared use of the PCIe devices is realized.
Based on the above description, the present application provides a method for sharing PCIe devices, where the method is applied to a server cluster, and the server cluster includes at least one server host, where the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, and the method includes: and acquiring the content of a first configuration space of the shared PCIe device, creating a shadow PCIe device according to the content of the first configuration space, then establishing a data channel between a second address of a base address register of the shadow PCIe device and a first address of a base address register of the shared PCIe device through a non-transparent bridge, and finally controlling the shared PCIe device through the established data channel, thereby realizing the sharing of the PCIe device. Based on the method, the shared PCIe devices can be shared by the multiple server hosts through the respective shadow PCIe devices, so that the actual use quantity of the PCIe devices is reduced, and the cost is lowered.
Further, the principle of the sharing method of PCIe devices provided in the embodiment of the present application is: and creating a shadow PCIe device on the server host of the server cluster, wherein the shadow PCIe device establishes a data channel with the shared PCIe device of the data center through the NTB, so that the server host in the server cluster directly drives the shared PCIe device of the remote data center by taking the shadow PCIe device as a medium through PCIe driving, thereby achieving the purpose that the server host in the server cluster shares the shared PCIe device of the data center.
The method for sharing PCIe devices provided in the embodiments of the present application is described in detail above with reference to fig. 1 to 5, and the apparatus, the device, and the medium provided in the embodiments of the present application are described below with reference to the accompanying drawings.
As shown in fig. 6, the schematic structural diagram of a PCIe device sharing apparatus provided in the embodiment of the present application is applied to a server cluster, where the server cluster includes at least one server host, and the server host is connected to a PCIe device to be shared of a data center through a non-transparent bridge, and the apparatus includes:
an obtaining module 601, configured to obtain content of a first configuration space of the shared PCIe device;
a creating module 602, configured to create a shadow PCIe device according to the content of the first configuration space;
a configuration module 603, configured to establish a data channel through the non-transparent bridge with the second address of the base address register of the shadow PCIe device and the first address of the base address register of the shared PCIe device;
and the control module 604 is configured to control the shared PCIe device through the data channel.
In some possible implementations, the content of the second configuration space of the shadow PCIe device is the same as the content of the first configuration space of the shared PCIe device.
In some possible implementations, the apparatus further includes an add module;
the creating module 602 is further configured to create an interface function of the shadow PCIe device, where the interface function is used to access content of the second configuration space;
the adding module is configured to add the interface function to an interface function set of an interconnection bus of the external device.
In some possible implementations, the apparatus further includes: a retrieval module and a calling module;
the retrieval module is used for retrieving the device number and the function number of the target device on the external device interconnection bus;
and the calling module is used for calling the interface function if the equipment number is a preset equipment number and the function number is a preset function number.
In some possible implementations, the control module 604 is further configured to perform a write operation on the second configuration space of the shadow PCIe device through a write access function of the shadow PCIe device; if the address corresponding to the writing operation is a preset address and the writing value is a preset value, or the address corresponding to the writing operation is a second address, ending the writing operation.
The sharing device of the PCIe device according to the embodiments of the present application may correspond to performing the methods described in the embodiments of the present application, and the foregoing and other operations and/or functions of each module/unit of the sharing device of the PCIe device are respectively for implementing the corresponding flows of each method in the embodiments shown in fig. 2-5, which are not repeated herein for brevity.
The embodiment of the application also provides a computing device. The computing device may be a server host in a server cluster, and is specifically configured to implement the functionality of the sharing means of the PCIe device in the embodiment shown in fig. 6.
As shown in fig. 7, which is a schematic structural diagram of a computing device according to an embodiment of the present application, as shown in fig. 7, a computing device 700 includes a bus 701, a processor 702, a communication interface 703, and a memory 704. Communication between processor 702, memory 704 and communication interface 703 is via bus 701.
Bus 701 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 7, but not only one bus or one type of bus.
The processor 702 may be any one or more of a central processing unit (central processing unit, CPU), a graphics processor (graphics processing unit, GPU), a Microprocessor (MP), or a digital signal processor (digital signal processor, DSP).
The communication interface 703 is used for communication with the outside. For example, the communication interface 703 may obtain the content of the first configuration space of the shared PCIe device.
The memory 704 may include volatile memory (RAM), such as random access memory (random access memory). The memory 704 may also include a non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, hard Disk Drive (HDD), or solid state drive (solid state drive, SSD).
The memory 704 has stored therein executable code that the processor 702 executes to perform the aforementioned sharing methods of PCIe devices.
In particular, in the case where the embodiment shown in fig. 6 is implemented, and where the modules or units of the shared device of the PCIe apparatus described in the embodiment of fig. 6 are implemented by software, software or program code required to perform the functions of the modules/units in fig. 6 may be partially or entirely stored in the memory 704. The processor 702 executes program codes corresponding to the respective units stored in the memory 704, and performs the aforementioned PCIe device sharing method.
Embodiments of the present application also provide a computer-readable storage medium. The computer readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc. The computer-readable storage medium includes instructions that instruct a computing device to perform the above-described method of sharing a PCIe device applied to a sharing apparatus of the PCIe device.
Embodiments of the present application also provide a computer program product comprising one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the processes or functions described in accordance with the embodiments of the present application are produced in whole or in part.
The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).
The computer program product, when executed by a computer, performs any of the methods of sharing the PCIe device described above. The computer program product may be a software installation package that can be downloaded and executed on a computer in the event that any of the aforementioned methods of sharing PCIe devices are desired.
The descriptions of the processes or structures corresponding to the drawings have emphasis, and the descriptions of other processes or structures may be referred to for the parts of a certain process or structure that are not described in detail.
The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application.
Claims (10)
1. A method for sharing PCIe devices, applied to a server cluster, where the server cluster includes at least one server host, and the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, the method comprising:
acquiring the content of a first configuration space of the shared PCIe device;
creating a shadow PCIe device according to the content of the first configuration space;
establishing a data channel through the non-transparent bridge by the second address of the base address register of the shadow PCIe device and the first address of the base address register of the shared PCIe device;
and controlling the shared PCIe device through the data channel.
2. The method of claim 1, wherein the second configuration space of the shadow PCIe device has the same content as the first configuration space of the shared PCIe device.
3. The method of claim 2, wherein after the creating a shadow PCIe device from the content of the first configuration space, the method further comprises:
creating an interface function of the shadow PCIe device, the interface function being used to access content of the second configuration space;
and adding the interface function to an interface function set of an interconnection bus of the external equipment.
4. A method according to claim 3, characterized in that the method further comprises:
retrieving a device number and a function number of a target device on the external device interconnection bus;
and if the equipment number is a preset equipment number and the function number is a preset function number, calling the interface function.
5. The method according to claim 4, wherein the method further comprises:
performing write operation on a second configuration space of the shadow PCIe device through a write access function of the shadow PCIe device;
if the address corresponding to the writing operation is a preset address and the writing value is a preset value, or the address corresponding to the writing operation is a second address, ending the writing operation.
6. A sharing apparatus of PCIe devices, applied to a server cluster, where the server cluster includes at least one server host, and the server host is connected to a shared PCIe device of a data center through a non-transparent bridge, the apparatus comprising:
the acquisition module is used for acquiring the content of the first configuration space of the shared PCIe device;
the creating module is used for creating a shadow PCIe device according to the content of the first configuration space;
a configuration module, configured to establish a data channel through the non-transparent bridge with a second address of a base address register of the shadow PCIe device and a first address of a base address register of the shared PCIe device;
and the control module is used for controlling the shared PCIe equipment through the data channel.
7. The apparatus of claim 6, wherein the second configuration space of the shadow PCIe device has the same content as the first configuration space of the shared PCIe device.
8. The apparatus of claim 7, further comprising an add-on module;
the creating module is further configured to create an interface function of the shadow PCIe device, where the interface function is used to access content of the second configuration space;
the adding module is configured to add the interface function to an interface function set of an interconnection bus of the external device.
9. A computing device comprising a memory and a processor;
wherein one or more computer programs are stored in the memory, the one or more computer programs comprising instructions; the instructions, when executed by the processor, cause the computing device to perform the method of any of claims 1 to 5.
10. A computer readable storage medium for storing a computer program for performing the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311773957.5A CN117725012A (en) | 2023-12-21 | 2023-12-21 | Sharing method, device, equipment and storage medium of PCIe equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311773957.5A CN117725012A (en) | 2023-12-21 | 2023-12-21 | Sharing method, device, equipment and storage medium of PCIe equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117725012A true CN117725012A (en) | 2024-03-19 |
Family
ID=90201361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311773957.5A Pending CN117725012A (en) | 2023-12-21 | 2023-12-21 | Sharing method, device, equipment and storage medium of PCIe equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117725012A (en) |
-
2023
- 2023-12-21 CN CN202311773957.5A patent/CN117725012A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107305534B (en) | Method for simultaneously carrying out kernel mode access and user mode access | |
| CN106445398B (en) | A kind of embedded file system and its implementation based on novel memory devices | |
| US9875208B2 (en) | Method to use PCIe device resources by using unmodified PCIe device drivers on CPUs in a PCIe fabric with commodity PCI switches | |
| RU2649771C2 (en) | Mass storage virtualization for cloud computing | |
| WO2021244194A1 (en) | Register reading/writing method, chip, subsystem, register group, and terminal | |
| CN109324991A (en) | A kind of hot plug device of PCIE device, method, medium and system | |
| CN104731635A (en) | Virtual machine access control method and virtual machine access control system | |
| TWI416418B (en) | Controllers, apparatuses, and methods for transferring data | |
| CN111399771B (en) | Protocol configuration method, device and equipment of MCS storage system | |
| US20150326684A1 (en) | System and method of accessing and controlling a co-processor and/or input/output device via remote direct memory access | |
| WO2020177567A1 (en) | Method, apparatus, and system for migrating data | |
| CN110765060B (en) | MDIO bus-to-parallel bus conversion method and device, equipment and medium | |
| CN114579319B (en) | Video memory management method, video memory management module, SOC and electronic equipment | |
| CN117725012A (en) | Sharing method, device, equipment and storage medium of PCIe equipment | |
| CN111666579B (en) | Computer device, access control method thereof and computer readable medium | |
| CN113031863B (en) | SSD command correlation management method, SSD command correlation management device, computer equipment and storage medium | |
| US7418545B2 (en) | Integrated circuit capable of persistent reservations | |
| US20060277326A1 (en) | Data transfer system and method | |
| US11307999B2 (en) | Data cache segregation for spectre mitigation | |
| CN116418848A (en) | Method and device for processing configuration and access requests of network nodes | |
| CN116670661A (en) | Cache access method of graphics processor, graphics processor and electronic device | |
| US8667188B2 (en) | Communication between a computer and a data storage device | |
| US7694004B2 (en) | Bit-granular writes of control registers | |
| US20020178316A1 (en) | System and method for defining private functions of a multi-function peripheral device | |
| CN216014148U (en) | Server and server backboard |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |