CN110581869B

CN110581869B - Resource allocation method, TLP packet processing method, equipment and storage medium

Info

Publication number: CN110581869B
Application number: CN201810582207.2A
Authority: CN
Inventors: 尹旭全
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2018-06-07
Filing date: 2018-06-07
Publication date: 2022-04-08
Anticipated expiration: 2038-06-07
Also published as: CN110581869A

Abstract

The embodiment of the invention discloses a resource allocation method, a TLP message processing method, equipment and a storage medium, and belongs to the technical field of hot plugging. The resource allocation method comprises the following steps: after detecting that a hot plug bus interface is inserted into new equipment, acquiring BOOT reserved resources; when the reserved resources do not meet the requirements of the new equipment, allocating resources for the new equipment from idle resources of an operating system; and when the allocated resource address is not continuous with the resource address of the upper-level bridge where the new equipment is positioned, allocating an idle register group for the new equipment, and writing the allocated resource address into the allocated idle register group. The embodiment of the invention does not require the resources distributed by the new equipment to be continuous with the resources of the previous-level bridge, and does not need to distribute the resources to all the equipment of the previous-level bridge again, thereby realizing the dynamic resource distribution of the new equipment under the condition of not influencing the normal work of the original equipment when the BOOT does not reserve the resources or the reservation is insufficient.

Description

Resource allocation method, TLP packet processing method, equipment and storage medium

Technical Field

The present invention relates to the field of PCIe (peripheral component interconnect express) hot plug, and in particular, to a resource allocation method, a TLP (Transaction Layer Packet) Packet processing method, a device, and a storage medium.

Background

PCIe is a high-speed serial computer expansion bus standard, is designed to support a hot plug function at the beginning, defines the standard use mode of the hot plug and the requirement of a register group, allows a user to remove a damaged module without closing the system, and can update or expand the capacity without influencing the operation of the system under the startup condition, thereby improving the timely recovery capability, expansibility and flexibility of the system to disasters.

In order to implement the hot-plug function of PCIe devices, when a system is started, a BOOT (system BOOT, an X86 system is a BIOS, and an embedded system is a UBOOT) reserves a resource with a fixed size, generally a memory address, for each PCI slot of an unplugged device, after the system is powered on, if a new device is plugged in, the new device applies for the resource reserved by the BIOS according to the resource required by the device itself, and if the new device is not applied, the new device is applied for (some systems do not support) and allocated to the device from an operating system, at present, the following problems exist:

firstly, when the BOOT is started, the user does not know what device needs to be inserted into the PCI slot and how many resources are needed, or the device is pulled out and replaced with a device that needs more resources, the reserved resources are insufficient, and the device cannot work normally.

Secondly, if the operating system supports reallocation of resources, if resources are allocated only for newly inserted devices, the new resources and root bridge resources are discontinuous, the current PCIe technology cannot support the resources, all the devices under the root bridge must be reallocated, all the device address resources are added up to be a large address, and it is difficult to apply for a continuous large address after the operating system runs for a long time.

In addition, even if a continuous big block address can be applied, the new address and the original address of the root bridge are probably two independent sections of spaces, the new address needs to be reconfigured to the root bridge, all the sub-bridges and the devices under the root bridge reconfigure their own resource addresses, and all the devices stop working.

Therefore, when the BOOT does not reserve the resources or the reservation is insufficient, the resources can not be dynamically allocated to the inserted new equipment under the condition that the normal working condition of the original equipment is not influenced.

Disclosure of Invention

In view of this, embodiments of the present invention provide a resource allocation method, a TLP processing method, a device and a storage medium, so as to solve the technical problem of dynamically allocating resources to a new device without affecting normal operation of an original device when resources are not reserved or reserved insufficiently at a BOOT.

The technical scheme adopted by the invention for solving the technical problems is as follows:

according to an aspect of an embodiment of the present invention, there is provided a resource allocation method, including:

after detecting that a hot plug bus interface is inserted into new equipment, acquiring BOOT reserved resources;

when the reserved resources do not meet the requirements of the new equipment, allocating resources for the new equipment from idle resources of an operating system;

when the allocated resource address is not continuous with the resource address of the upper-level bridge where the new device is located, allocating an idle register group for the new device, and writing the allocated resource address into the allocated idle register group.

According to another aspect of the embodiments of the present invention, a TLP processing method is provided, where the method includes:

after receiving a TLP (TLP packet), an uplink port searches a register group in which an address carried by the TLP packet falls;

when the message falls into the BAR space of the base address register group of the uplink port, the message is analyzed and processed;

and when the TLP message falls into the space of the segment address register group of the downlink port, forwarding the TLP message downwards through the downlink port.

According to another aspect of the embodiments of the present invention, there is provided a device supporting a peripheral component interconnect express (PCIe) bus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the computer program implements the steps of the above resource allocation method when executed by the processor; or the steps of the TLP processing method implemented above.

According to still another aspect of the embodiments of the present invention, there is provided a storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of the above-mentioned resource allocation method; or, the steps of the TLP processing method are implemented.

In the resource allocation method, the TLP processing method, the device and the storage medium provided in the embodiments of the present invention, when the resource address allocated by the new device is not consecutive with the resource address of the previous bridge, the allocated resource address is written into the allocated idle register group by allocating the idle register group to the new device, and resources allocated by the new device and resources of the previous bridge are not required to be consecutive, and resource allocation to all devices of the previous bridge is not required to be performed again, so that when the BOOT does not reserve resources or reserves insufficient resources, resources are dynamically allocated to the new device without affecting normal operation of the original device.

Drawings

FIG. 1 is a basic PCIe topology according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating a register set according to an embodiment of the present invention;

fig. 3 is a flowchart of a resource allocation method according to an embodiment of the present invention;

fig. 4 is a flowchart of another resource allocation method according to an embodiment of the present invention;

fig. 5 is a flowchart of a TLP processing method provided by the embodiment of the present invention;

fig. 6 is a flowchart of a TLP processing method provided by the embodiment of the invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As shown in fig. 1, the PCIe system structure is described by taking a computer system using a PCIe interface as an example, a common PCIe device is represented by an endpoint device, a hexagon is a PCIe Switch device (e.g., PCIe Switch chip), P2P is a PCI bridge inside the Switch device, P2P connected to an upper level bridge is referred to as an upstream port, and the other P2P is referred to as a downstream port. A root bridge of a CPU of the system is connected to an uplink port of PCIe switching equipment A, and two downlink ports of the PCIe switching equipment A are respectively connected to an uplink port of a destination equipment 1 and an uplink port of PCIe switching equipment B. Two downstream ports of the PCIe switch chip B are connected to the endpoint device 2 and the endpoint device 3, respectively.

As shown in fig. 2, the PCIe switch device a/and the PCIe switch device B both implement N resource register sets, each register set includes a segment address register set, and the segment address register set includes a memory base address register base and a memory final address register limit, and a prefetcheable memory base address base and a prefetcheable memory final address register limit. Wherein, base is the base address of the segment, is the start address limit saving length register of the segment in the linear address space, and is the boundary of the segment. The segment address register set is used for controlling the forwarding of the TLP.

Based on the above system configuration, the following embodiments of the present invention are proposed.

Example one

As shown in fig. 3, an embodiment of the present invention provides a resource allocation method, where the method includes:

s301, after detecting that PCIe is inserted into the new device, obtaining the reserved resources of BOOT.

Specifically, when the operating system is started, the BOOT may reserve resources for each root bridge, or may not reserve resources. When the operating system detects that a new device is hot-plugged, firstly, the resources required by the device are checked, and the number of the resources reserved by the BOOT is obtained to determine whether the resources reserved by the BOOT meet the resource requirements of the new device. Resources include memory addresses, or input/output (I/O) addresses.

S302, when the reserved resources do not meet the requirements of the new equipment, allocating resources for the new equipment from idle resources of an operating system.

Specifically, when the reserved resources of the BOOT meet the requirements of the new device, the resources can be allocated to the new device from the reserved resources, and the resources can also be dynamically allocated to the new device from the operating system; when the reserved resources of the BOOT do not meet the requirements of the new device, the resources can only be allocated to the new device from the idle resources of the operating system.

S303, when the allocated resource address is not continuous with the resource address of the upper-level bridge, allocating an idle register group for the new device, and writing the allocated resource address into the allocated idle register group.

Specifically, when dynamically allocating resources to the new device from the operating system, determining whether the address of the allocated resources is continuous with the resource address of the previous bridge where the new device is located, that is, comparing the allocated resources with a Base/Limit (Base/Limit) of the previous bridge of the new device, and if so, writing the allocated resource addresses into the resource register set of the previous bridge; if not, allocating an idle register group for the new device, writing the allocated resources into the idle register group, and issuing the resources. The resource addresses continuously comprise the head-to-tail connection of the resource addresses and the overlapping of the resource addresses.

In the embodiment of the invention, when the resource address allocated by the new equipment is not continuous with the resource address of the previous-level bridge, the allocated resource address is written into the allocated idle register group by allocating the idle register group to the new equipment, so that the resources allocated by the new equipment are not required to be continuous with the resources of the previous-level bridge, and the resources are not required to be allocated to all the equipment of the previous-level bridge again. Therefore, when the BOOT does not reserve the resources or the reservation is insufficient, the resources are dynamically allocated to the new equipment under the condition that the normal work of the original equipment is not influenced.

Example two

As shown in fig. 4, another method for resource allocation is provided in an embodiment of the present invention, where the method includes:

s401, after PCIe insertion into a new device is detected, BOOT reserved resources are obtained;

s402, judging whether the reserved resources meet the requirements of the new equipment, if so, executing a step S403, otherwise, executing a step S404;

s403, distributing resources for the new device from the reserved resources, and turning to step S405;

s404, distributing resources for the new equipment from idle resources of an operating system;

s405, judging whether the allocated resource address is continuous with the resource address of the upper-level bridge where the new equipment is located, if so, executing the step S406, otherwise, executing the step S407;

s406, writing the allocated resource address into a resource register group of the upper-level bridge, and turning to S408;

specifically, the allocated resource address is written into the resource register group of the previous bridge where the new device is located, and is configured to the resource register group of the previous bridge where the new device is located. And traversing all the upper-level bridges to the root bridge in turn upwards, and performing the same treatment on each-level bridge.

S407, allocating an idle register group for the new device, and writing the allocated resource address into the allocated idle register group;

and S408, ending the flow.

In the embodiment of the invention, when the system is started, BOOT reserves resources for each root bridge, when PCIe equipment is inserted in a hot manner, the PCIe equipment applies for from the reserved resources, if dynamic allocation from an operating system is not applied, newly allocated resources are not required to be continuous with resources of a previous-level bridge, the allocated resources are compared with the previous-level bridge where the new equipment is located, and when the allocated resource address is overlapped or continuous with the resource address of the previous-level bridge, the allocated resource address is written into a resource register group of the previous-level bridge and is allocated to an original resource register group of the previous-level bridge, otherwise, the new resources are allocated to an idle register group. And traversing all the upper-level bridges to the root bridge in turn upwards, and performing the same treatment on each-level bridge. And finally, writing the resources into the configuration space of the PCIe equipment to complete the resource allocation and configuration process. The method supports multiple discontinuous address resources on the chip level, can still dynamically allocate resources for the equipment when the BIOS does not reserve resources or the reservation is insufficient, and does not influence the normal work of other equipment.

EXAMPLE III

As shown in fig. 5, a TLP processing method according to an embodiment of the present invention includes:

s501, after receiving a TLP, the uplink port searches for a register group in which an address carried by the TLP falls.

Specifically, the TLP uses a group of important packets routed based on the ID, and the TLP mainly functions to read and write configuration registers of the EP, the switch chip, and the PCIe bridge of the PCIe bus to complete the configuration of the PCIe bus. As shown in fig. 2, the PCIe switch device implements a plurality of register sets, each register set records one resource segment, and each register set includes a memory base/limit and a replaceable memory base/limit. And the address of the destination device will be carried in the TLP, so the bridge can search the register set according to the address of the destination device when forwarding the PCIe TLP. Whether the address carried by the TLP falls into the two BAR spaces of the uplink port or not can be determined, and when the address does not fall into the BAR space of the uplink port, all the segment address register sets of all the downlink ports are traversed until the segment address register set meeting the requirement is found. Of course, when all the segment address register sets of all the downstream ports do not satisfy the requirement, the information that the TLP is not supported may be returned.

And S502, when the packet falls into the BAR space of the uplink port, analyzing and processing the packet.

Specifically, if the address carried by the TLP falls into the upstream port BAR space, it indicates that the packet is sent to the PCIe switching device itself, and therefore the packet does not need to be forwarded, and the PCIe switching device analyzes and processes the packet.

And S503, when the TLP packet falls into the segment address register set space of the downlink port, forwarding the TLP packet downwards through the downlink port.

Specifically, when the address carried by the TLP falls into a certain resource segment, it indicates that the TLP should be forwarded to the device corresponding to the downlink port, so that the packet is forwarded to the next-level bridge or the destination device through the downlink port, and the next-level bridge or the destination device performs corresponding processing.

In this embodiment, all segment address register sets are traversed during TLP forwarding, since a new device is inserted to use a set of idle registers, the content of the original working register set is not changed, a TLP sent to the original device is matched with the original segment address register and correctly forwarded to the device, and the newly inserted device has no influence on the operation of the original device.

It will be understood by those of ordinary skill in the art that all or some of the steps of the disclosed methods of the present embodiments may be implemented as software, firmware, hardware, or any suitable combination thereof.

Example four

As shown in fig. 6, in this embodiment, a PCIE switch chip is taken as an example to describe a forwarding process of a TLP after supporting multiple resource register groups, so as to describe that the insertion device has no influence on an original working register group after using a group of idle registers, and further does not influence normal operations of other devices. The TLP processing method provided by the embodiment of the invention includes:

s601, the PCIe switching chip upper line port receives the TLP message.

S602, determining whether the address carried by the TLP message falls into the BAR space of the uplink port, if so, executing the step S603, otherwise, executing the step.

And S603, the PCIe switching chip analyzes and processes the message.

Specifically, if the packet falls into the BAR space of the uplink port, the packet is sent to the PCIe switch chip itself, and the packet does not need to be forwarded, and the PCIe switch chip analyzes and processes the packet.

S604, traversing all the segment address register groups of all the downlink ports until finding the segment address register group meeting the requirement.

Specifically, if the address carried by the TLP does not fall into the BAR space of the upstream port, whether the address carried by the TLP falls into the Base/Limit space of the downstream port is checked, if the address falls into the Base/Limit space, step S605 is executed, otherwise, a group of Base/Limit is taken down and compared until all Base/Limit groups in one downstream port are processed. A set of Base/Limit for the next downstream port is removed and S604 is repeated, and if all Base/Limit for all downstream ports do not satisfy the condition, S608 is passed to.

S605, determining whether a segment address register meeting the requirement is found, if so, executing S606, otherwise, turning to the step S609.

S606, forward the TLP from the downlink port to the destination device.

S607, the destination device checks whether the address carried by the TLP falls into its BAR space, if so, then step S607 is executed, otherwise, the process goes to step S608.

And S608, the terminal equipment receives and processes the message, and the process is ended.

And S609, the message return does not support.

And S610, ending the process.

From step 4, it can be seen that, when the TLP is forwarded, all the Base/Limit register sets are traversed, the new device is inserted to use one set of idle registers, the content of the original working register set is not changed, the TLP sent to the original device is matched with the original Base/Limit register and correctly forwarded to the device, and the new inserted device has no influence on the working of the original device.

EXAMPLE five

The device provided by the embodiment of the invention supports peripheral component interconnect express (PCIe), and comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is executed by the processor, the steps of the resource allocation method are realized.

Wherein the device is most typically a computer supporting a PCIE interface.

The device of the embodiment of the present invention and the resource allocation method of the first and second embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable in the device embodiments, which are not described herein again.

EXAMPLE six

The device supporting peripheral component interconnect express (PCIe) provided by the embodiment of the invention comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is executed by the processor, the steps of the TLP message processing method are realized.

Among them, the device is most typically a PCIe switch chip.

The device of the embodiment of the present invention and the TLP processing method of the third and fourth embodiments belong to the same concept, and specific implementation processes thereof are detailed in the corresponding method embodiments, and technical features in the method embodiments are correspondingly applicable in the device embodiment, which is not described herein again.

EXAMPLE seven

An embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the resource allocation method according to the first embodiment or the second embodiment are implemented, or the steps of the TLP processing method according to the third embodiment or the fourth embodiment are implemented.

The computer-readable storage medium of the embodiment of the present invention and the method of the first to fourth embodiments belong to the same concept, and specific implementation processes thereof are described in detail in the corresponding method embodiments, and technical features in the method embodiments are applicable in the computer-readable storage medium embodiments, and are not described herein again.

It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.

In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, and are not to be construed as limiting the scope of the invention. Any modifications, equivalents and improvements which may occur to those skilled in the art without departing from the scope and spirit of the present invention are intended to be within the scope of the claims.

Claims

1. A method for resource allocation, the method comprising:

after a device supporting peripheral component interconnect express (PCIe) detects that a hot plug bus interface is inserted into a new device, when the acquired resource reserved by a system BOOT program BOOT does not meet the requirement of the new device or the resource reserved by the system BOOT program BOOT is not reserved, the device allocates resources for the new device from idle resources of an operating system;

when the allocated resource address is not continuous with the resource address of the upper-level bridge where the new device is located, the device allocates an idle register group for the new device, and writes the allocated resource address into the allocated idle register group.

2. The method of claim 1, further comprising: and when the reserved resources meet the requirements of the new equipment, allocating resources for the new equipment from the reserved resources.

3. The method of claim 1, further comprising: and when the allocated resource address is continuous with the resource address of the previous-level bridge where the new equipment is positioned, writing the allocated resource address into the resource register group of the previous-level bridge.

4. The method according to any of claims 1-3, wherein the resource comprises a memory space or an input/output space.

5. A transaction layer packet TLP packet processing method is characterized by comprising the following steps:

after receiving a TLP, an upstream port supports peripheral component interconnect express (PCIe) devices to find a register group in which an address carried by the TLP falls, where the devices are devices that implement the resource allocation method according to any one of claims 1 to 4;

when the device falls into the BAR space of the base address register group of the uplink port, the device analyzes and processes the message;

and when the TLP message falls into the space of the segment address register group of the downlink port, the equipment forwards the TLP message downwards through the downlink port.

6. The TLP processing method according to claim 5, wherein the searching for the register group into which the address carried by the TLP falls includes:

determining whether the address carried by the TLP message falls into a BAR space of an uplink port;

and when the address does not fall into the BAR space of the uplink port, traversing all the segment address register groups of all the downlink ports until finding the segment address register group meeting the requirement.

7. The TLP processing method according to claim 6, wherein after traversing all the segment address register sets of all the downstream ports, further comprising:

and when all the segment address register groups of all the downlink ports do not meet the requirements, returning the information that the TLP message is not supported, and ending the process.

8. The TLP processing method according to any of claims 5-7, further comprising, after the method:

and when the destination device receives the TLP, determining whether the address carried by the TLP falls into a BAR space of the destination device, and when the address falls into the BAR space, analyzing and processing the packet.

9. An apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the resource allocation method according to any one of claims 1 to 4; or implementing the TLP processing method of any of claims 5 to 8.

10. A storage medium, characterized in that the storage medium has stored thereon a computer program which, when executed by a processor, carries out the steps of the resource allocation method according to any one of claims 1 to 4; and/or implementing the TLP processing method of any of claims 5 to 8.