CN116724297A

CN116724297A - Fault processing method, device and system

Info

Publication number: CN116724297A
Application number: CN202180090841.4A
Authority: CN
Inventors: 胡成; 董钰山
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-01-22
Filing date: 2021-01-22
Publication date: 2023-09-08
Also published as: WO2022155919A1

Abstract

A fault handling method, apparatus and system for maintaining high reliability of a peripheral component interconnect express (peripheral component interconnect express, PCIe) system. The method comprises the following steps: the central processing unit acquires the abnormal interrupt information corresponding to the abnormal PCIe port, determines the fault type corresponding to the abnormal PCIe port according to the abnormal interrupt information, and resets the abnormal PCIe port and the communication link of the abnormal PCIe port when the fault type is determined to be a recoverable fault. The fault processing scheme not only can timely restore the communication service capability of the abnormal PCIe port, but also can not influence other PCIe ports in the PCIe system, thereby being beneficial to maintaining the high reliability of the PCIe system.

Description

Fault processing method, device and system

Technical Field

The present application relates to the field of communications technologies, and in particular, to a fault processing method, device, and system.

Background

Peripheral component interconnect express (peripheral component interconnect express, PCIe) is a high-speed short-range communication interface. The communication interface can rapidly read and write the memory and can support ultra-high broadband communication, and is widely applied to various fields such as networks, communication, storage, industry, consumer electronic products and the like.

The main constituent elements of PCIe systems include Root Complexes (RCs), switch nodes (switches), and end nodes (EPs). The root complex is used for managing all buses and all nodes in the PCIe system, and is a bridge for communication between nodes in the PCIe system. A root complex may include multiple PCIe ports through which the root complex is connected to multiple nodes, such as multiple end nodes or multiple switching nodes, respectively. One switching node may be used to communicate the root complex with other switching nodes, or to communicate the root complex with an end node, and is a forwarding node for data in a PCIe system. An end node is an end device, such as a peripheral device (peripheral) or the like, that receives data or transmits data.

In the existing scheme, when an uncorrectable error occurs in a certain PCIe port, in order to maintain the availability of the PCIe port, the whole PCIe system recovers the PCIe port through a restarting mode. However, this means that all PCIe ports of the entire PCIe system are in an unavailable state for the period of time of restarting, which obviously reduces the reliability of the PCIe system.

Disclosure of Invention

The application provides a fault processing method, device and system, which are used for solving the technical problem of lower reliability of a PCIe system caused by restarting the whole PCIe system to recover an abnormal PCIe port in the prior art.

In a first aspect, the present application provides a method of fault handling that is applicable to a central processor that may be directly or indirectly connected to each PCIe port in a PCIe system. The method comprises the following steps: after receiving the abnormal interrupt information reported by the abnormal PCIe port, the central processing unit determines the fault type corresponding to the abnormal PCIe port according to the abnormal interrupt information, and resets the abnormal PCIe port and the communication link of the abnormal PCIe port when the fault type is determined to be a recoverable fault. The communication link of the abnormal PCIe port is used for communicating the abnormal PCIe port and the PCIe node. In the design, by resetting the recoverable abnormal PCIe port and the corresponding communication link, the availability of the abnormal PCIe port can be recovered in time, the communication service capability of the abnormal PCIe port is maintained, and the whole PCIe system does not need to be restarted, so that the fault processing scheme does not influence other PCIe ports in the PCIe system while recovering the abnormal PCIe port, and is beneficial to maintaining the high reliability of the PCIe system.

In one possible design, an aberrant PCIe port is provided in the root complex, and a PCIe node may be any node to which the root complex is connected, such as an end node, a switch node, or a bridge node. By the design, the scheme can detect the link abnormality of the end node, can detect the link abnormality of other types of nodes (such as a switching node or a bridge node), and can maintain the availability of the PCIe system as possible by identifying and repairing various types of link abnormality of the whole PCIe system.

In one possible design, the central processor may implement the reset of the aberrant PCIe port by resetting the media access control layer (media access control, MAC) logic of the PCIe core in which the aberrant PCIe port is located. In the design, the exception occurred by the abnormal PCIe port can be recovered in a targeted way by resetting only the MAC layer logic related to the port and not resetting other logic unrelated to the port, and the processing resources of the central processing unit and the PCIe core are saved on the basis of improving the resetting efficiency of the PCIe port.

In one possible design, the central processor may implement the reset of the communication link of the aberrant PCIe port by resetting the serializer/deserializer (SerDes) link parameters corresponding to the PCIe core in which the aberrant PCIe port is located. Thus, the design can calibrate the current SerDes link parameters of the abnormal PCIe port when the surrounding environment changes, and restore the communication quality of the abnormal PCIe port by adjusting the current SerDes link parameters to the parameters suitable for the current environment.

In one possible design, the central processor may also disconnect the communication link between the abnormal PCIe port and the PCIe node mounted on the abnormal PCIe port before resetting the serializer/demodulator SerDes link parameters corresponding to the PCIe core where the abnormal PCIe port is located, and reestablish the communication link between the abnormal PCIe port and the PCIe node mounted on the abnormal PCIe port after resetting the serializer/demodulator SerDes link parameters corresponding to the PCIe core where the abnormal PCIe port is located. In the design, by removing each node mounted on the abnormal PCIe port before resetting the abnormal PCIe port, the abnormal PCIe port and other nodes in the PCIe core can be decoupled, and independent resetting of the abnormal PCIe port is facilitated.

In one possible design, if the central processor determines that the fault type corresponding to the abnormal PCIe port is an unrecoverable fault, the abnormal PCIe port and a communication link of the abnormal PCIe port may be disabled, so as to save resources of the PCIe core, and avoid a phenomenon that the unrecoverable fault of the abnormal PCIe port is spread to the whole PCIe core to cause a service fault of the whole PCIe core as much as possible.

In one possible design, the recoverable fault may include one or more of the following: data link layer packet transmission timeout errors, excessive retry times errors of a transaction layer packet write configuration space, two-bit data errors, response errors of an advanced extensible interface AXI bus. Thus, the design can repair various types of errors related to the PCIe port, such as overtime errors of data link layer packet transmission and excessive retry times of transaction layer packet write configuration space related to PCIe node communication, two-bit data errors related to self data storage and AXI bus response errors related to CPU transmission, and is helpful for more comprehensively maintaining the availability of the PCIe port.

In one possible design, the abort information corresponding to the abort PCIe port may include one or more of the following: the method comprises the steps of identifying an abnormal PCIe port, a fault type corresponding to the abnormal PCIe port, identifying a PCIe core in which the abnormal PCIe port is located, and identifying a Central Processing Unit (CPU) connected with the PCIe core. Thus, the central processing unit can acquire some characteristics related to the current abnormality by analyzing the abnormal interrupt information corresponding to the abnormal PCIe port so as to calculate the fault type corresponding to the abnormal PCIe port.

In one possible design, the central processor may obtain the abort information corresponding to the abort PCIe port from a preset work queue in which the abort information corresponding to the abort PCIe port in each PCIe core is stored. Therefore, the design can intensively manage the port abnormality occurring in each PCIe core through the preset work queue, and effectively improve the flexibility of recovering the abnormal PCIe port in each PCIe core.

In a second aspect, the present application provides a fault handling apparatus comprising a processor and a memory, the memory storing a computer program. In implementation, the processor may perform the following by invoking a computer program stored in memory: obtaining abnormal interrupt information corresponding to the abnormal peripheral component interconnect transfer PCIe port, determining a fault type corresponding to the abnormal PCIe port according to the abnormal interrupt information, and resetting the abnormal PCIe port and a communication link of the abnormal PCIe port under the condition that the fault type corresponding to the abnormal PCIe port is a recoverable fault. The communication link of the abnormal PCIe port is used for communicating the PCIe port and the PCIe equipment.

In one possible design, the PCIe node may be one or more of an end node, a switch node, or a bridge node.

In one possible design, the fault handling apparatus may also include an advanced configuration and power management interface (advanced configuration and power management interface, ACPI). The processor may execute the media access control layer MAC logic that resets the PCIe core where the abnormal PCIe port is located by invoking ACPI.

In one possible design, the fault handling device may further include serial demodulator SerDes firmware, where the processor may reset the SerDes link parameters corresponding to the PCIe core where the abnormal PCIe port is located by invoking ACPI, and may also invoke the SerDes firmware after resetting the MAC logic of the PCIe core where the abnormal PCIe port is located.

In one possible design, the fault handling device may also include a PCIe drive. The processor can disconnect a communication link between the abnormal PCIe port and the PCIe node mounted on the abnormal PCIe port by calling the PCIe driver, call ACPI, reset a SerDes link parameter corresponding to the PCIe core where the abnormal PCIe port is located by calling the SerDes firmware, return to call the PCIe driver, and reestablish the communication link between the abnormal PCIe port and the PCIe node mounted on the abnormal PCIe port by returning to call the PCIe driver.

In one possible design, the memory may include public registers and private registers:

in one case, PCIe drivers and ACPI may be stored in public registers, and SerDes firmware may be stored in private registers, so as to privacy the link reset method in the SerDes firmware, and effectively protect implementation logic of SerDes link parameter reset;

in another case, PCIe drivers may be stored in public registers and ACPI and SerDes firmware may be stored in private registers to privacy the port reset method in ACPI and link reset method in SerDes firmware, effectively protecting the overall logic of fault handling.

In one possible design, the processor may also perform the following by invoking a computer program stored in memory: and under the condition that the fault type corresponding to the abnormal PCIe port is an unrecoverable fault, disabling the abnormal PCIe port and the communication link of the abnormal PCIe port.

In one possible design, the recoverable fault may include one or more of the following: data link layer packet transmission timeout errors, excessive retry times errors of a transaction layer packet write configuration space, two-bit data errors, response errors of an advanced extensible interface AXI bus.

In one possible design, the abort information corresponding to an abort PCIe port may include one or more of the following: the method comprises the steps of identifying an abnormal PCIe port, a fault type corresponding to the abnormal PCIe port, identifying a PCIe core in which the abnormal PCIe port is located, and identifying a Central Processing Unit (CPU) connected with the PCIe core.

In one possible design, the fault handling device may further include a communication interface, and the processor may perform the following operations by invoking a computer program stored in the memory: the processor receives the abnormal interrupt information corresponding to the abnormal PCIe port through the communication interface, adds the abnormal interrupt information corresponding to the abnormal PCIe port to a preset work queue, and acquires the abnormal interrupt information corresponding to the abnormal PCIe port from the preset work queue. The preset work queue is used for storing abnormal interrupt information corresponding to an abnormal PCIe port in each PCIe core.

In a third aspect, the application provides a fault handling apparatus comprising a module, unit or circuit to perform the method of any one of the possible designs of the above aspect. These modules, units or circuits may be implemented in hardware or may be implemented by hardware executing corresponding software.

In a fourth aspect, the present application provides a chip, which may comprise a processor and a communication interface, the processor being configured to read instructions via the communication interface to perform the fault handling method according to any of the first aspects.

In a fifth aspect, the present application provides a fault handling system, comprising a central processor and a peripheral component interconnect passing PCIe core, the PCIe core comprising a root complex and at least one PCIe node, the central processor being coupled to the root complex, the root complex comprising at least one PCIe port, the root complex being coupled to the at least one PCIe node through the at least one PCIe port. The root complex may be configured to generate abort information corresponding to an abnormal PCIe port in the at least one PCIe port and report the abort information to the central processor, where the central processor may be configured to perform fault processing on the abnormal PCIe port according to the fault processing method according to any one of the first aspect.

In a sixth aspect, the present application provides a computer readable storage medium storing program code which, when run on a computer, causes the computer to perform the fault handling method as described in any one of the first aspects above.

In a seventh aspect, the present application provides a computer program product comprising computer program code which, when run on a computer, causes the computer to perform the fault handling method as claimed in any one of the first aspects above.

The advantages corresponding to any of the second to seventh aspects of the present application may be specifically referred to the advantages described in any of the first aspects, and the detailed description is not repeated here.

Drawings

FIG. 1 schematically illustrates a system architecture to which embodiments of the present application are applicable;

fig. 2 schematically illustrates a flow chart of a fault handling method according to an embodiment of the present application;

fig. 3 is a schematic flow diagram corresponding to a reset method according to an embodiment of the present application;

fig. 4 is a schematic diagram schematically illustrating a software and hardware architecture of fault handling logic according to an embodiment of the present application;

FIG. 5 schematically illustrates a flow chart of another fault handling method provided by an embodiment of the present application;

fig. 6 is a schematic structural diagram of a fault handling apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of another fault handling apparatus according to an embodiment of the present application.

Detailed Description

The fault processing method disclosed by the application can be applied to electronic equipment for communication based on a PCIe system. In some embodiments of the application, the fault handling means may be an electronic device or a stand alone unit. When the fault handling device is a stand-alone unit, the unit may be embedded in the electronic device and may be capable of performing fault handling on the PCIe port of the electronic device to maintain high reliability of the PCIe system. In other embodiments of the present application, the fault handling device may also be a unit that is encapsulated inside the electronic device, for implementing the fault handling function of the PCIe port of the electronic device. The electronic device may be a server, memory, test equipment, or contain, for example, a personal digital assistant and/or music playingPortable electronic devices with functions such as a mobile phone, a tablet computer, a wearable device (such as a smart watch) with a wireless communication function, or a vehicle-mounted device. Exemplary embodiments of portable electronic devices include, but are not limited to, piggy-back Or other operating system, such as a Laptop (Laptop) or desktop computer having a touch-sensitive surface (e.g., a touch panel).

The present application will be described in further detail with reference to the accompanying drawings. In the description of the present application, "at least one" means one or more, wherein a plurality means two or more. In view of this, the term "plurality" may also be understood as "at least two" in embodiments of the present application. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/", unless otherwise specified, generally indicates that the associated object is an "or" relationship.

Fig. 1 schematically illustrates a system architecture to which an embodiment of the present application is applicable. As shown in fig. 1, the system architecture includes at least one central processing unit (central processing unit, CPU) and at least one PCIe core, where the at least one central processing unit and the at least one PCIe core may be in one-to-one correspondence, and as shown in fig. 1, the central processing unit 1 corresponds to the PCIe core 1, and the central processing unit 2 corresponds to the PCIe core 2. PCIe cores are also known as PCIe systems. Each PCIe core may include a root complex and at least one end node, and may also include at least one switch node and at least one bridge node. The root complex is used for carrying out system initialization and configuring communication links among nodes when the PCIe core is constructed so as to connect the central processor corresponding to the PCIe core with one or more of the switching nodes, the end nodes and the bridge nodes in the PCIe core one by one. The central processor corresponding to the PCIe core may implement communication interactions with the nodes in the PCIe core by connecting the root complexes in the PCIe core. The switching node is respectively connected with the upstream root complex and one or more of the downstream end nodes, the switching node or the bridge nodes, and is used for routing the data of the upstream root complex to one or more downstream nodes, or respectively routing the data of each downstream node to the upstream unique root complex, or flexibly routing the data of one downstream node to another downstream node in a point-to-point manner. The bridge node is used for realizing communication connection between the PCIe core and other PCI or other PCIe cores adopting other bus standards through non-transparent bridges (non-transparent bridge, NTB) arranged in different bus systems. The end node is typically located within the Application (APP) of the terminal, and is responsible for connecting the terminal APP with other nodes within the PCIe core and completing PCIe-based transaction transmissions. In general, the number of end nodes in one PCIe core is greater than the number of other types of nodes.

Taking PCIe core 1 shown in fig. 1 as an example, the node connection manner in each PCIe core is further described below:

the PCIe core 1 includes a root complex 1, a switch node 1, a bridge node 1, and four end nodes, namely, end node 1, end node 2, end node 3, and end node 4. Wherein the switching node 1, the end node 1 and the bridge node 1 belong to the downstream node of the root complex 1 (the root complex 1 belongs to the upstream node of the switching node 1, the end node 1 and the bridge node 1), and the end node 2, the end node 3 and the end node 4 belong to the downstream node of the switching node 1 (the switching node 1 belongs to the upstream node of the end node 2, the end node 3 and the end node 4). The upstream node and the corresponding downstream node may be connected by a PCI bus (see thick black lines shown in fig. 1). One or more PCIe ports, such as Root Ports (RP) 1, RP2, and RP3, may also be included in root complex 1. The root complex 1 may connect downstream switching node 1, end node 1, and bridge node 1 through RP1, RP2, and RP3, respectively, such that the root complex 1 may implement data routing with switching node 1, and downstream end node 2, end node 3, and end node 4 thereof through RP1, may implement data routing with end node 1 through RP2, and may implement data routing with nodes in PCIe core 2 (e.g., bridge node 2) through RP3.

It should be noted that the above is merely an exemplary illustration, and that more or fewer nodes than illustrated in fig. 1 may be included in a PCIe core, such as a greater number of switch nodes, end nodes, or bridge nodes, or other types of nodes other than a root complex, switch node, end node, and bridge node. The root complexes and the central processing units may be connected one to one in the manner illustrated in fig. 1, or may be connected in a one to many or many to one manner, for example, one root complex may be connected to at least two central processing units respectively, or one central processing unit may be connected to at least two root complexes respectively, or the like. In addition, the central processor and the PCIe core may be disposed in the same physical entity, or may be disposed in different physical entities, or may be disposed in the same physical entity by a part of nodes in the central processor and the PCIe core, and another part of nodes in the PCIe core may be disposed in another physical entity, which is not limited in particular.

As is clear from what is illustrated in fig. 1, the root complex realizes communication connection with other nodes in the PCIe core through each PCIe port provided internally, in which case, traffic between the root complex and the terminal APP is actually distributed on a communication link corresponding to each PCIe port to be processed. Therefore, ensuring the connection between each PCIe port and each node is of great importance to maintain the service processing capability of the whole PCIe core. At this stage, when a connection problem is detected between a certain PCIe port and a downstream end node, it is common practice to restart the entire PCIe core. However, by the method, all PCIe ports in the PCIe core are in an unavailable state, so that services of the PCIe ports with the connection problem cannot be recovered, services of other PCIe ports in the PCIe core can be influenced, and reliability of the whole PCIe core and even the whole system is reduced. To address this problem, in an alternative embodiment, the PCIe core may not be restarted, but rather only the PCIe port that has a connection problem with the downstream end node may be disabled. However, this way, the disabled PCIe port and all downstream nodes mounted on the PCIe port are in an unavailable state, and although the traffic of other PCIe ports is not affected, the traffic of the disabled PCIe port cannot be recovered for a long time. In addition, both of the above two methods can only handle the connection failure when the downstream node is an end node, but cannot handle the connection failure when the downstream node is other types of nodes (such as a switching node or a bridge node), resulting in poor versatility of failure handling.

In view of this, the present application provides a fault handling method for quickly recovering traffic of an abnormal PCIe port without affecting traffic of other PCIe ports, and further implementing handling of connection faults of more types of downstream nodes.

The following describes a specific implementation procedure of the fault handling scheme in the present application by using a specific embodiment.

[ embodiment one ]

Fig. 2 is a schematic flow chart of a fault handling method according to an embodiment of the present application, where the method is applicable to the central processing unit in fig. 1, and the central processing unit 1 or the central processing unit 2 is illustrated in fig. 1. As shown in fig. 2, the method includes:

in step 201, the central processing unit obtains the abort information corresponding to the abort PCIe port.

In an optional implementation manner, the root complex may detect, in real time or periodically, a service processing condition between each PCIe port and a downstream node mounted on each PCIe port, and when detecting that an abnormality occurs in service processing between a PCIe port and a mounted downstream node, in order to avoid that service accuracy of a PCIe core is affected due to a current service of continuing to execute the abnormality, the root complex may interrupt the current service of the abnormal PCIe port first, then generate, according to relevant information of the current fault, an abnormal interrupt information corresponding to the abnormal PCIe port, and finally report the abnormal interrupt information to a connected central processor. The abnormal interrupt information corresponding to the abnormal PCIe port may include a fault type, an identifier of the abnormal PCIe port, an identifier of a PCIe core where the abnormal PCIe port is located, and one or more items of a type of a current service, a service processing progress, or contact information.

In the above embodiment, each interface in the PCIe core is PCIe, and the interface between the central processor and the root complex is not PCIe, and does not belong to the PCIe core. In this case, the root complex may transmit the abort information to the central processor via a non-PCIe bus, such as via a file transfer protocol (file transfer protocol, FTP).

In the embodiment of the present application, the failure type of the abnormal PCIe port may include one or more of the following:

error type one: two-bit data error

An error correction program (error correcting code, ECC) can also be stored in the root complex, which can correct 1 bit error (belonging to a correctable error (correctable errors, CE)) and detect 2 bit errors (belonging to an uncorrectable error (uncorrectable errors, UE)), i.e. can correct data that only have 1 bit error to correct data, and can detect data that have 2 bit errors but cannot correct data. In implementation, when detecting that the service processing of a certain PCIe port is abnormal, the root complex may first invoke ECC to pre-process the error, if only 1 bit of data is wrong, the root complex may self-locate the wrong 1 bit of data in a manner pointed by the ECC and directly correct the data into correct data, and then continue to execute the current service of the PCIe port, where the PCIe port is still a normal PCIe port. However, if there is an error in the 2-bit data, the root complex can only detect which two-bit error but cannot correct the error by itself, and in this case, the PCIe port is an abnormal PCIe port, and the root complex can suspend the current service of the PCIe port first, and then assemble and report the corresponding abnormal interrupt information to the central controller based on the 2-bit error location located by the ECC. In this case, the root complex may label the type of fault in the generated abort information as "two-bit data error".

Error type two: data link layer packet transmission timeout error

The root complex may be composed of an application layer, a transaction Transport Layer (TL), a Data Link Layer (DLL), and a physical layer. When executing the transaction between the central processor and the terminal APP, the application layer initiates a transaction transmission request to the transaction transmission layer, the transaction transmission layer generates a corresponding transaction transmission layer packet (transport layer package, TLP) and sends the corresponding transaction transmission layer packet to the data link layer, the data link layer adds a sequence number and a link cyclic redundancy check (link cyclic redundancy check, LCRC) code to the TLP to generate a corresponding data link layer packet (data link layer package, DLLP) and sends the corresponding data link layer packet to the physical layer, and the transaction is transmitted on a PCIe link corresponding to the PCIe port of the physical layer. After the data link layer sends out the DLLP, the data link layer waits to receive response information which is successfully transmitted by the physical layer, and only when the response information is received within a preset time period, the data link layer can confirm that the bidirectional transaction between the data link layer and the physical layer is correctly received. According to the transaction flow, in implementation, if the data link layer in the root complex has not received the response information of successful transmission returned by the physical layer after sending the DLLP for more than a preset period of time, the root complex may confirm that the transmission between the data link layer and the physical layer is problematic, which may be an uncorrectable error caused by network delay, and the PCIe port in this case is an abnormal PCIe port. The root complex may generate abort information corresponding to the PCIe port for the PCIe port and may label the failure type as "data link layer packet transmission timeout error".

Error type three: excessive number of retries in write configuration space

A node in the PCIe core (e.g., a root complex, a switching node, or an end node) may support up to 8 functions, such as audio, video, etc. When a node supports a plurality of functions at the same time, each function of the node corresponds to its own configuration space in which relevant information of the function is stored. The configuration space may be a section of independent memory unit in the node, for example the configuration space may be 256k in size. Other nodes except the root complex can only see the relevant information of the configuration space of the node, and the root complex has the authority of reading and writing the configuration space of each node. For example, the root complex may read information in the configuration space of any node through the transaction layer packet to determine the functions supported by the node, or may write the configuration space of any node through the transaction layer packet to complete initialization and function configuration of the node. However, if the written node is not ready to respond to the request for the root complex to write configuration space, the written node will return a transaction layer response packet to the root complex with a status of "configuration retry status (configuration retry status, CRS)". This marks that the root complex failed to successfully write the configuration space of the node. When the number of times of writing failure does not exceed the preset number of times, the writing failure belongs to a correctable error, and the root complex can also continuously rewrite. When a transaction layer response packet with a certain number of CRS states is received, the root complex is indicated to be not successfully written in a certain number of rewrites, uncorrectable errors occur in PCIe ports on a PCIe link where the abnormal node is located, and the PCIe ports in the case are abnormal PCIe ports. The root complex may generate corresponding abort information for the PCIe port and may label the failure type as "data link layer packet transmission timeout error".

Error type four: AXI bus response error

The root complex in the PCIe core is connected to the corresponding central processor through an advanced extensible interface (advanced eXtensible interface, AXI) bus. Meaning that the root complex and the central processor need to send and receive data in the manner specified by the AXI bus protocol. In this case, after the root complex receives a data processing request for a PCIe port issued by the central processor through the AXI bus, if a response cannot be returned to the central processor within a time specified by the AXI bus protocol, or the returned response does not conform to a format specified by the AXI bus protocol, the root complex may determine that a problem occurs in the PCIe port. The problem may be an uncorrectable error due to a PCIe port anomaly, or an uncorrectable error due to a transmission link anomaly between the PCIe port and a downstream node, in which case the PCIe port is an anomalous PCIe port. The root complex may generate corresponding abort information for the PCIe port and may label the failure type as an AXI bus response error. It should be appreciated that the above description merely describes this type of error by taking the example of the root complex detecting an AXI bus response error, which in other examples may also be detected by the central processor itself, without being reported by the root complex.

It should be noted that the foregoing list just a few possible failure types, and the failure type of an abnormal PCIe port may also be uncorrectable errors at any other port level. In addition, the failure of the abnormal PCIe port may be caused by a hardware failure of the PCIe port and the downstream node, or may be caused by a software failure of the PCIe port and the downstream node, which is not particularly limited in the present application.

Step 202, the central processing unit determines the fault type according to the abort information corresponding to the abort PCIe port: in case the fault type is a non-recoverable fault, step 203 is performed; in case the fault type is a recoverable fault, step 204 is performed.

In the embodiment of the application, the unrecoverable fault refers to a fault which cannot be directly or indirectly repaired by the central processing unit and can be solved only by a special person for debugging, such as hardware damage or recording medium defect. A recoverable failure refers to a failure that a central processing unit can repair directly or indirectly, such as by resetting, upgrading, updating, downloading a patch, or restarting.

In step 203, the central processor disables the aberrant PCIe port and the corresponding communication link.

In the embodiment of the application, the central processing unit can also store a recoverable fault record table, and the recoverable fault record table records the recoverable fault type which can be recovered directly or indirectly by the central processing unit. The recoverable failures in the recoverable failure record table may include, for example, one or more of the two-bit data errors illustrated above, data link layer packet transmission timeout errors, excessive number of retries of the write configuration space, AXI bus response errors, or other recoverable port level errors. The types of the recoverable faults can be preset in a recoverable fault record table according to experience by a developer, can be learned by a central processing unit during the process of executing the service and stored in the recoverable fault record table in real time, and can be obtained by interaction information and the like acquired by the central processing unit from other central processing units or network equipment and the like, and the type of the recoverable faults is not particularly limited.

In an implementation, the central processor may obtain the failure type of the abnormal PCIe port from the abort information first, then match the failure type of the abnormal PCIe port with the recoverable failure type in the recoverable failure record table, and if all the recoverable failure types in the recoverable failure record table do not match the failure type of the abnormal PCIe port, the central processor may locate the failure of the abnormal PCIe port as an unrecoverable failure. Because the unrecoverable failure can not be recovered by a non-computer program or an operating technology, nor can the unrecoverable failure be corrected by an error check code or other technologies, once the central processor detects that a certain PCIe port has an unrecoverable failure, the PCIe port can be directly disabled, so as to save resources of the PCIe core, and avoid the phenomenon that the unrecoverable failure of the abnormal PCIe port spreads to the whole PCIe core to cause the business failure of the whole PCIe core.

In the embodiment of the present application, the central processor may disable the abnormal PCIe port and the corresponding PCIe link in a variety of ways, for example:

in one mode, the central processing unit can call out a configuration interface of a basic input/output system (basic input output system, BIOS) through a hot key or an instruction and the like, an abnormal PCIe port is selected on the configuration interface, and a forbidden command is issued, wherein the forbidden command can enable the root complex to disable the port functions of the abnormal PCIe port and the node functions of all nodes mounted on the abnormal PCIe port according to the instruction or the write configuration space and the like;

in another way, the cpu may call out the on-board interface of the RAM chip (e.g., the complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) chip) in the motherboard, and set the slot of the abnormal PCIe port in the on-board interface to be "enabled" to disable the root complex to the abnormal PCIe port, and further offload all the nodes mounted on the abnormal PCIe port.

When the abort information further includes contact information, the central processor may generate a corresponding alarm message after disabling the PCIe port and the corresponding PCIe link, and push the alarm message to the user according to the contact information in the abort information, so that the user can timely learn and repair the failed PCIe port, and resume the service of the PCIe port as soon as possible.

In step 204, the CPU resets the abnormal PCIe port and corresponding communication link.

In the embodiment of the present application, the central processor may further store a reset manner of each recoverable fault recorded in the recoverable fault record table, where any one of the reset manners of the recoverable faults may be one or more of reset, upgrade, update, patch download, or restart. In an implementation, if the target recoverable fault type matched with the fault type of the abnormal PCIe port exists in the recoverable fault record table, the central processor may locate the fault of the abnormal PCIe port as a recoverable fault, and may reset the abnormal PCIe port and the communication link corresponding to the abnormal PCIe port by using a reset mode corresponding to the target recoverable fault type. Therefore, the central processing unit not only can recover the services of the abnormal PCIe port as soon as possible in a resetting mode, but also can not influence the services of other PCIe ports, and the reliability of the PCIe core is effectively maintained.

It should be noted that, in general, only the configuration space of the port and the node in the PCIe core can be directly written by the root complex in the PCIe core, but the configuration space of the port or the node in the PCIe core cannot be directly written by the central processor, so in order to reset the abnormal PCIe port and the corresponding communication link, the method can synchronize the authority of writing the configuration space of the port and the node in the PCIe core to the central processor in advance, complete the reset by directly writing the configuration space by the central processor, and also can drive the root complex to write the configuration space by sending a corresponding instruction to the root complex by the central processor, which is not limited in detail.

In the following, a specific implementation manner of resetting an abnormal PCIe port and a corresponding communication link is described taking a resetting manner as an example.

Fig. 3 is a schematic flow diagram corresponding to a reset method according to an embodiment of the present application, where the method is applicable to a central processing unit, such as the central processing unit 1 or the central processing unit 2 illustrated in fig. 1. In this example, assume that the central processor writes the configuration space of ports and nodes in the PCIe core in an indirect manner. As shown in fig. 3, the method includes:

in step 301, the central processor disables the abnormal PCIe port.

In step 301 described above, the central processor may send a disable instruction for the aberrant PCIe port to the root complex to indicate that the root complex is no longer continuing current traffic for the aberrant PCIe port. The root complex may also record, after disabling the abnormal PCIe port, a traffic handling progress of the abnormal PCIe port before disabling, so as to facilitate resuming execution of the traffic after recovery. By disabling the abnormal PCIe port before resetting the abnormal PCIe port, the executed service can be prevented from being influenced in the process of resetting the abnormal PCIe port, and the accuracy of service execution before and after resetting the abnormal PCIe port is ensured.

In step 302, the central processor disconnects the communication link between the abnormal PCIe port and the node mounted on the abnormal PCIe port.

In step 302, the central processor may send a removal instruction for the abnormal PCIe port to the root complex, so as to drive the root complex to write the configuration space of each node mounted on the abnormal PCIe port, remove each node mounted on the abnormal PCIe port from the current communication link, and disconnect the connection relationship between the abnormal PCIe port and each node mounted on the abnormal PCIe port. By removing each node mounted on the abnormal PCIe port before resetting the abnormal PCIe port, the abnormal PCIe port and other nodes in the PCIe core can be decoupled, and independent resetting of the abnormal PCIe port can be realized.

In step 303, the central processor disables the isochronous PCIe port.

In step 303, the central processor may send a failure instruction for the abnormal PCIe port to the root complex, so as to drive the root complex to write the configuration space of the abnormal PCIe port, and switch the abnormal PCIe port from an enabled (enabled) state to a disabled (disabled) state. Wherein, "enabling" refers to "enabling …". When the abnormal PCIe port is in an enabled state, the abnormal PCIe port has the processing capability for data issued by an upstream root complex or data reported by a downstream node. When the abnormal PCIe port is in a disabled state, the abnormal PCIe port does not have the processing capability for data issued by an upstream root complex or data reported by a downstream node.

In step 304, the central processor resets the media access control (media access control, MAC) logic of the PCIe core.

In step 304, the central processor may send a MAC reset instruction for the abnormal PCIe port to the root complex to drive the root complex to initialize the MAC logic of the entire PCIe core, restoring the MAC layer communication mechanism of the abnormal PCIe port. By resetting only the MAC layer logic related to the port, but not resetting other logic unrelated to the port, the exception occurred by the abnormal PCIe port can be recovered in a targeted manner, and the processing resources of the central processing unit and the PCIe core are saved on the basis of improving the resetting efficiency of the PCIe port.

In step 305, the central processor resets the SerDes link parameters corresponding to the PCIe core.

In the embodiment of the application, each communication link in the PCIe core is managed by a serializer/deserializer (SerDes), default SerDes link parameters are preset in the SerDes, and the SerDes converts parallel transmission data into serial transmission data according to the default SerDes link parameters, or converts serial reception data into parallel reception data according to the default SerDes link parameters. However, when the surrounding environment changes, the default SerDes link parameters may no longer be adapted to the PCIe core, resulting in anomalies in the communication links where portions of the PCIe ports in the PCIe core are located. In this case, the SerDes link parameters in the SerDes need to be adjusted to accommodate the current environment.

In implementations, the central processor may send a link reset instruction for the aberrant PCIe port to the root complex to drive the root complex to adaptively calibrate the SerDes link parameters corresponding to the aberrant PCIe port. The calibrated SerDes link parameter may be obtained by obtaining a current environmental parameter (such as temperature) and substituting the current environmental parameter into a preset formula for calculation, or may be obtained by performing closed-loop feedback adjustment according to an adjusted execution effect until the adjustment is converged, or may be selected randomly, which is not particularly limited.

At step 306, the central processor reconstructs the communication link between the anomalous PCIe port and the node mounted on the anomalous PCIe port.

In step 306, the cpu may send a rebuilding chain instruction for the entire PCIe core to the root complex, so that the root complex rebuilds the entire topology of the PCIe core, or may send a rebuilding chain instruction for only the abnormal PCIe port to the root complex, so that the root complex directly detects the abnormal PCIe port and all the nodes mounted on the abnormal PCIe port, and adds the rebuilding chain instruction to the topology of the existing PCIe core.

In an alternative embodiment, with continued reference to FIG. 1, assuming that the entire PCIe core 1 is to be rebuilt, the root complex 1 may sequentially traverse the bus paths on which the various internally configured PCIe ports RP1, RP2, and RP3 reside:

Traversing the downstream bus of RP1 by the root complex 1, discovering the switching node 1, allocating a bus address (which may include a bus number, a node number, a function number (bus device and function number, BDF), etc.) to the switching node 1; the root complex 1 continuously traverses nodes connected with a downstream bus of the switching node 1 according to the depth priority rule, discovers the end node 2, and distributes bus addresses for the end node 2; since the end node 2 does not have a downstream bus, the root complex 1 continues to traverse the nodes connected by the downstream bus of the switching node 1, discovers the end node 3, and allocates a bus address to the end node 3; since the end node 3 does not have a downstream bus, the root complex 1 continues to traverse the nodes connected by the downstream bus of the switching node 1, discovers the end node 4, and allocates a bus address to the end node 4; so far, the bus link where RP1 is located is traversed;

traversing the downstream bus of RP2 by the root complex 1, finding out the end node 1, and distributing a bus address for the end node 1; so far, the bus link where RP2 is located is traversed;

traversing the downstream bus of RP3 by the root complex 1, finding out a bridge node 1, and distributing a bus address for the bridge node 1; the root complex 1 records the bridge address of the bridge node 1 at the same time so as to establish association with the topological structure obtained by traversing the PCIe core 2; so far, the bus link traversal where RP3 is located is completed.

Through the above flow, the root complex 1 can allocate bus addresses for all nodes in the PCIe core 1, and can construct a topology structure of the whole PCIe core 1.

It should be noted that, since only the node mounted on the abnormal PCIe port that was removed before is not in the topology structure corresponding to the PCIe core, but the nodes mounted on other PCIe ports except the abnormal PCIe port are themselves in the topology structure corresponding to the PCIe core, even if the rebuilding chain is performed for the whole PCIe core, the node mounted on the non-abnormal PCIe port traversed by the root complex is not added to the topology structure of the PCIe core again, but only the node mounted on the abnormal PCIe port that is not in the current topology structure is added, so that rebuilding of the communication link between the abnormal PCIe port and the node mounted on the abnormal PCIe port is completed by the way of supplementing the link.

In step 307, the central processor validates the abnormal PCIe port.

In step 307, the central processor may send an effective instruction for the abnormal PCIe port to the root complex, so as to drive the root complex to write the configuration space of the abnormal PCIe port, and switch the abnormal PCIe port from the disable state to the disable state, so as to restore the processing capability of the abnormal PCIe port to the data issued by the upstream root complex or the data reported by the downstream node.

In step 308, the central processor enables an aberrant PCIe port.

In step 308 described above, the central processor may send an enable instruction for the aberrant PCIe port to the root complex to instruct the root complex to enable traffic processing for the aberrant PCIe port. The root complex may re-execute the current service of the abnormal PCIe port, or may continue to execute the current service of the abnormal PCIe port from the service processing progress recorded when the root complex is previously deactivated, so as to improve the efficiency of service processing while saving the processing resources of the PCIe core, or may not execute the current service any more, but directly execute the new service after waiting for the arrival of the subsequent new service, which is not specifically limited.

In the first embodiment, by resetting the recoverable abnormal PCIe port and the corresponding communication link, not only the availability of the abnormal PCIe port can be recovered in time, the communication service capability of the PCIe port is maintained, but also the whole PCIe core is not required to be restarted, so that the processing of the abnormal PCIe port does not affect the service processing of other PCIe ports, thereby helping to maintain the high reliability of the PCIe system. Furthermore, the scheme can also process any recoverable fault of any PCIe node (such as an end node, a switching node or a bridge node) in the PCIe core, not only limited to the end node or a certain fault type, so that the method can also effectively detect and repair various faults in the PCIe core, and further improve the high reliability of the whole PCIe core.

It should be noted that, in the foregoing embodiment, the specific implementation process of fault handling is actually described by taking the central processing unit as an execution body as an example, which is just an alternative implementation manner. In another alternative embodiment, the fault handling scheme may also be performed directly by the root complex. In this embodiment, if the root complex detects that an abnormality occurs in a certain PCIe port inside, the PCIe port and the corresponding communication link may be directly reset, and no report is required to the central processor for processing. This embodiment, while increasing the working pressure of the root complex, saves communication overhead of the PCIe core and the central processor and handles faults faster.

In the embodiment of the application, the central processing unit can run in a Linux operating system. Virtual memory space in Linux operating systems is divided into kernel space (kernel space) and user space (user space). The kernel space is the running space of the kernel in the Linux operating system, and the user space is the running space of the user program. The kernel space and the user space are isolated from each other, and even if the user program crashes, the kernel space is not affected.

Based on the Linux operating system, fig. 4 illustrates a schematic diagram of a software and hardware architecture of a fault handling logic according to an embodiment of the present application, and as shown in fig. 4, a fault handling scheme may be implemented in hardware by a central processor and a root complex that are disposed in chip hardware. The chip hardware may also be connected to other peripheral devices such as a memory, an input output device, or a drive device. The central processor encapsulates fault handling software global logic consisting of reliability, availability and serviceability (RAS) firmware, PCIe drivers, advanced configuration and power management interface (advanced configuration and power management interface, ACPI) and SerDes firmware. The programs of the RAS firmware or the SerDes firmware can be written into the memory in advance, the RAS firmware is mainly used for processing the interrupt, and the SerDes firmware is mainly used for reconstructing the chain. The central processing unit can execute the method logic corresponding to the RAS firmware or the SerDes firmware by calling the RAS firmware or the SerDes firmware in the memory. ACPI defines various working interfaces between an operating system, BIOS, and system hardware. ACPI may be implemented in BIOS or system hardware and may be invoked or triggered by an operating system. The PCIe driver is positioned in a kernel space of the Linux system and is used for managing the enabling or disabling of each port in the PCIe kernel and managing the connection relation between each port and each node. PCIe drivers may be open-sourced from communities.

It will be appreciated that the memory in embodiments of the application may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and direct memory bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

It should be understood that fig. 4 is presented by way of example only, and in other examples, the central processor may be located on a different chip hardware than the root complex. In addition, the memory and the central processing unit may be in the same physical entity (such as chip hardware), or may be in different physical entities.

From the execution point of view of the above-mentioned software and hardware architecture, the specific implementation of the fault handling scheme is further described in the second embodiment.

[ example two ]

FIG. 5 is a schematic flow chart of another fault handling method according to an embodiment of the present application, where the method is applicable to chip hardware, RAS firmware, ACPI, PCIe drivers and SerDes firmware, and the chip hardware is provided with a central processor and an abnormal PCIe port, and the PCIe drivers are located in kernel space, and the kernel space interacts with user space. As shown in fig. 5, the method includes:

in step 501, when an abnormal PCIe port fails, an RAS interrupt is triggered.

In step 501 described above, when the failure of the abnormal PCIe port is a correctable failure, the failure may be repaired by the root complex before triggering the RAS interrupt. However, when the failure of the abnormal PCIe port is an uncorrectable failure, the root complex cannot repair the failure, so the root complex may trigger the root complex to interrupt the current service of the abnormal PCIe port, and report relevant information of the current RAS interrupt to the central processor, such as the identifier (core port ID, such as a number) of the abnormal PCIe port, the type of failure (error type), and the identifier (core ID, such as a number) of the PCIe core where the abnormal PCIe port is located.

Step 502, the RAS firmware calls the ACPI to generate a corresponding ACPI interrupt event according to the RAS interrupt and reports the ACPI interrupt event to the PCIe driver.

In the step 502, the central processing unit invokes the RAS firmware to process the RAS interrupt, generates a corresponding ACPI interrupt event according to the relevant information of the current RAS interrupt and the relevant information (such as the identifier) of the central processing unit, and reports the ACPI interrupt event to the PCIe driver in the kernel space.

In step 503, the PCIe driver generates fault information corresponding to the abnormal PCIe port according to the ACPI interrupt event, and adds the fault information to a work queue (i.e., a preset work queue).

In step 503, the PCIe driver responds to the ACPI interrupt event related to the PCIe core, first extracts the identity of the abnormal PCIe port, the fault type and the identity of the PCIe core where the abnormal PCIe port is located from the ACPI interrupt event, and then assembles the fault information according to the information and the identity (socket ID, such as number) of the chip set in the PCIe core according to the set message structure, and further adds the fault information to the work queue. The work queue is located in kernel space for storing various fault information occurring in the PCIe core. In an implementation, the central processing unit may set a work queue corresponding to each PCIe core in the kernel space, where the PCIe driver in any central processing unit adds fault information in the PCIe core connected to the central processing unit to the work queue corresponding to the PCIe core, or the central processing unit may set the same work queue for all PCIe cores in the kernel space, where the PCIe driver in each central processing unit adds fault information in the PCIe core connected to each central processing unit to the same work queue, or the central processing unit may set one work queue for some PCIe cores in the kernel space, set another work queue for another PCIe core, and so on.

In step 504, the pcie driver fetches a piece of failure information from the work queue, determines whether the failure type in the failure information belongs to a recoverable failure, if not, executes step 505, if so, executes step 506.

In step 504, the PCIe driver may fetch the failure information from the work queue in a first-in first-out order to sequentially recover each PCIe port in order from early to late when the failure occurs, may fetch the failure information from the work queue in a first-in last-out manner to recover the failure of the PCIe port that has occurred recently and then recover the failure of the PCIe port that has occurred earlier in time, and may sequentially fetch the failure information from the work queue in order from heavy to light in degree of failure, which is not particularly limited.

Illustratively, the central processor includes a plurality of processes, such as a plurality of central processor cores, and the PCIe driver may further invoke the plurality of central processor cores to comprehensively process each piece of fault information. For example, the PCIe driver may uniformly pre-allocate each piece of fault information to a plurality of central processing units, may call the most idle central processing unit to process one piece of fault information to be processed, may allocate one piece of fault information to a central processing unit that is most good at processing the current service, and the like, and is not particularly limited.

In step 505, the PCIe driver pushes the failure information to the user space.

In step 505, when the PCIe driver cannot recover the abnormal PCIe fault by itself, the PCIe driver can timely notify the user of the fault by pushing the fault to the user, so as to facilitate early manual maintenance, and avoid that the PCIe port is in an unavailable state for a long time.

For example, when the PCIe driver adds one piece of fault information to the work queue, one piece of log information may also be pushed to the user space, so that the user can timely learn about the abnormal situation of the whole PCIe core and the current working pressure of the central processing unit.

At step 506, the pcie driver joins the ACPI and SerDes firmware to perform overall logic for fault handling, the execution including the steps of:

step 5061, the PCIe driver invokes the PCIe public interface to disable the abnormal PCIe port;

step 5062, the PCIe driver calls the PCIe public interface to remove all nodes mounted on the abnormal PCIe port;

in steps 5061 and 5062 described above, the PCIe common interface may be located in a common register where methods may originate from the community and be visible to other users. The implementation of PCIe common interface may refer to existing logic and is not described here in detail.

In step 5063, the pcie driver invokes a port reset method in the ACPI interface.

In step 5063, the port reset method is programmed according to the programming language specified by the ACPI interface and is added as an interface to the ACPI interface logic. In this way, the PCIe driver may execute the corresponding port reset logic by calling the interface name corresponding to the port reset method. The interface name of the port reset method can be set by itself, for example, set to RP reset.

Step 5064, the PCIe driver resets the MAC logic of the PCIe core according to the port reset method;

step 5065, resetting the SerDes link parameters corresponding to the PCIe core by the PCIe driver according to the port resetting method and calling SerDes firmware;

the port reset method can be located in a common register of the chip, and the PCIe driver directly invokes the PCIe common interface and the port reset method through intra-chip messages in the common register of the chip, so that message transmission channels between chips are reduced. Or, the port resetting method can also be located in a private register of the chip, and after the PCIe driver calls the PCIe public interface in the public register of the chip, the port resetting method in the private register of the chip is called by the message calling method in the chip so as to privacy the port resetting method.

Step 5066, resetting the SerDes link parameters corresponding to the PCIe core by the PCIe driver according to the SerDes firmware;

in step 5066 described above, the SerDes firmware may be programmed in any operating language, such as C++, phython, and the like. When resetting the SerDes link, the SerDes link arranged in the chip receives a reset command of the PCIe drive, then the parameter of the SerDes link is calibrated according to the current environment, and the parameter is adaptively calibrated again according to the execution effect of the calibrated parameter, so as to restore the SerDes link corresponding to the abnormal PCIe port in the PCIe core to the optimal state as much as possible.

Illustratively, the SerDes firmware may be located in a private register of the chip. Because PCIe driver realizes the SerDes link reset method by calling SerDes firmware in the port reset method, the SerDes link reset method provided by the SerDes firmware is invisible to the outside even if the port reset method is started in the community, which is helpful for maximally ensuring the security of the SerDes link reset method.

Step 5067, the pcie driver determines that the SerDes firmware call is completed, and returns to the port reset method;

step 5068, the PCIe driver determines that the port reset method is called and returns to the PCIe public interface;

In step 5069, the PCIe driver reconstructs the topology of the PCIe core by means of enumeration traversal.

In the second embodiment, through the cooperation of the chip hardware, the chip firmware (including the RAS firmware and the SerDes firmware), the PCIe driver and the ACPI, the cpu can complete fault sensing, resetting and service recovery of the PCIe port through a soft-hard combination mode. In this approach, recovery from failure of one PCIe port does not affect normal traffic of the other PCIe ports. And, even if the driver for PCIe failure recovery is powered on to the community, the SerDes firmware set in the chip private register, or the port reset method and SerDes firmware will not be exposed. By protecting the port reset logic as much as possible, the probability that the reset flow of the abnormal PCIe port is interfered by the outside can be reduced, and the reset accuracy is improved.

According to the foregoing method, fig. 6 is a schematic structural diagram of a fault handling apparatus 600 according to an embodiment of the present application, where the fault handling apparatus 600 may be a chip or a circuit, such as a chip or a circuit that may be disposed in a central processing unit. The fault handling apparatus 600 may correspond to a central processor in the above-described method. The fault handling apparatus 600 may implement the steps of the method as shown in any one or more of the above figures 2 to 5. As shown in fig. 6, the fault handling apparatus 600 may include a monitoring circuit 601 and a processing circuit 602. Further, the fault handling apparatus 600 may further comprise a bus system, through which the monitoring circuit 601 and the processing circuit 602 may be connected. Furthermore, the monitor circuit 601 may also connect to each PCIe port in the PCIe core through a bus system, and the processing circuit 602 may also connect to the root complex in the PCIe core through the bus system.

In the embodiment of the present application, the monitoring circuit 601 may receive the abort information reported by the abort PCIe port and send the abort information to the processing circuit 602. Correspondingly, the processing circuit 602 may determine, first, a fault type corresponding to the abnormal PCIe port according to the abort information, and reset the abnormal PCIe port and the communication link of the abnormal PCIe port to restore the connectivity relationship between the abnormal PCIe port and the PCIe node when the fault type corresponding to the abnormal PCIe port is a recoverable fault.

The concepts related to the technical solutions provided by the embodiments of the present application related to the fault handling apparatus 600, explanation and detailed description and other steps refer to the descriptions of the foregoing methods or other embodiments, and are not repeated herein.

According to the foregoing method, fig. 7 is a schematic structural diagram of still another fault handling apparatus 700 according to an embodiment of the present application, where the fault handling apparatus 700 may be a chip or a circuit, such as a chip or a circuit that may be disposed in a central processing unit. The fault handling apparatus 700 may correspond to a central processor in the above-described method. The fault handling apparatus 700 may implement the steps of the method as shown in any one or more of the above figures 2 to 5. As shown in fig. 7, the fault handling apparatus 700 may include a communication interface 701, a determination unit 702, and a processing unit 703.

In the embodiment of the present application, the communication interface 701 may be a receiving unit or a receiver when receiving information, and the receiving unit or the receiver may be a radio frequency circuit. In a specific implementation, the communication interface 701 may receive the abort information reported by the abort PCIe port, the determining unit 702 may determine a fault type corresponding to the abort PCIe port according to the abort information, and in a case that the fault type corresponding to the abort PCIe port is a recoverable fault, the processing unit 703 may reset the abort PCIe port and a communication link of the abort PCIe port to restore a connection relationship between the abort PCIe port and the PCIe node.

The concepts related to the technical solutions provided by the embodiments of the present application, explanation and detailed description of the concepts related to the fault handling apparatus 700 and other steps are referred to in the foregoing methods or descriptions related to the other embodiments, and are not repeated herein.

It can be appreciated that the functions of the respective units in the fault handling apparatus 700 may refer to implementation of the corresponding method embodiments, which are not described herein.

It should be understood that the above division of the units of the fault handling apparatus 700 is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. In the embodiment of the present application, the communication interface 701 may be implemented by the monitoring circuit 601 of fig. 6, and the determining unit 702 and the processing unit 703 may be implemented by the processing circuit 602 of fig. 6.

According to the method provided by the embodiment of the application, the application further provides a fault processing system, which comprises the central processing unit and the PCIe core. The PCIe core comprises a root complex and at least one PCIe node, and the root complex is connected with the downstream PCIe node through at least one PCIe port arranged inside. The central processor may perform the method of any of the embodiments shown in fig. 1-5 to implement fault handling for an anomalous PCIe port of the at least one PCIe port.

According to a method provided by an embodiment of the present application, the present application also provides a computer program product, including: computer program code which, when run on a computer, causes the computer to perform the method of any of the embodiments shown in fig. 1 to 5.

According to the method provided by the embodiment of the present application, the present application further provides a computer readable storage medium storing a program code, which when executed on a computer, causes the computer to perform the method of any one of the embodiments shown in fig. 1 to 5.

As used in this specification, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. Furthermore, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from two components interacting with one another in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal).

Those of ordinary skill in the art will appreciate that the various illustrative logical blocks (illustrative logical block) and steps (steps) described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A method of fault handling, the method comprising:

obtaining abnormal interrupt information corresponding to an abnormal peripheral component interconnect express (PCIe) port;

determining a fault type corresponding to the abnormal PCIe port according to the abnormal interrupt information;

resetting the abnormal PCIe port and a communication link of the abnormal PCIe port under the condition that the fault type corresponding to the abnormal PCIe port is a recoverable fault;

the communication link of the abnormal PCIe port is used for communicating the abnormal PCIe port and the PCIe node.
The method of claim 1, wherein the PCIe node is an end node, a switch node, or a bridge node.
The method of claim 1 or 2, wherein the resetting the aberrant PCIe port comprises:

resetting the media access control layer (MAC) logic of the PCIe core in which the abnormal PCIe port is located.
The method of any of claims 1-3, wherein the resetting the communication link of the aberrant PCIe port comprises:

resetting the SerDes link parameters of the corresponding serializer/demodulator of the PCIe core where the abnormal PCIe port is located.
The method of claim 4, wherein,

Before resetting the serializer/demodulator SerDes link parameters corresponding to the PCIe core where the abnormal PCIe port is located, the method further includes:

disconnecting a communication link between the abnormal PCIe port and a PCIe node mounted on the abnormal PCIe port;

after resetting the serializer/demodulator SerDes link parameters corresponding to the PCIe core where the abnormal PCIe port is located, the method further includes:

reconstructing a communication link between the abnormal PCIe port and a PCIe node mounted on the abnormal PCIe port.
The method of any one of claims 1 to 5, wherein the method further comprises:

and under the condition that the fault type corresponding to the abnormal PCIe port is an unrecoverable fault, disabling the abnormal PCIe port and the communication link of the abnormal PCIe port.
The method of any of claims 1 to 6, wherein the recoverable failure comprises one or more of:

data link layer packet transmission timeout errors, excessive retry times errors of a transaction layer packet write configuration space, two-bit data errors, response errors of an advanced extensible interface AXI bus.
The method of any of claims 1-7, wherein the abort information corresponding to the abort PCIe port includes one or more of:

The identification of the abnormal PCIe port, the fault type corresponding to the abnormal PCIe port, the identification of the PCIe core where the abnormal PCIe port is located, and the identification of the CPU (central processing unit) connected with the PCIe core.
The method of any of claims 1-8, wherein the acquiring abort information corresponding to a PCIe port for an abort peripheral component interconnect transfer, comprises:

obtaining abnormal interrupt information corresponding to the abnormal PCIe port from a preset work queue;

the preset work queue is used for storing abnormal interrupt information corresponding to an abnormal PCIe port in each PCIe core.
A fault handling device comprising a processor and a memory, the memory having a computer program stored therein;

the processor performs the following operations by calling the computer program stored in the memory:

obtaining abnormal interrupt information corresponding to an abnormal peripheral component interconnect express (PCIe) port;

determining a fault type corresponding to the abnormal PCIe port according to the abnormal interrupt information;

resetting the abnormal PCIe port and a communication link of the abnormal PCIe port under the condition that the fault type corresponding to the abnormal PCIe port is a recoverable fault;

The communication link of the abnormal PCIe port is used for communicating the PCIe port and the PCIe device.
The apparatus of claim 10, wherein the PCIe node is an end node, a switch node, or a bridge node.
The apparatus of claim 10 or 11, wherein the fault handling apparatus further comprises an advanced configuration and power management interface ACPI;

the processor performs the following operations in particular by calling the computer program stored in the memory:

by invoking the ACPI, performing: resetting the media access control layer (MAC) logic of the PCIe core in which the abnormal PCIe port is located.
The apparatus of claim 12, wherein the fault handling means further comprises serial demodulator SerDes firmware;

the processor further performs the following operations by calling the computer program stored in the memory:

by invoking the ACPI, performing: after resetting the media access control layer (MAC) logic of the PCIe core in which the abnormal PCIe port is located, calling the SerDes firmware;

by invoking the SerDes firmware, execute: resetting the SerDes link parameters corresponding to the PCIe core in which the abnormal PCIe port is located.
The apparatus of claim 13, wherein the fault handling means further comprises a PCIe drive;

the processor further performs the following operations by calling the computer program stored in the memory:

by invoking the PCIe driver, performing: disconnecting a communication link between the abnormal PCIe port and a PCIe node mounted on the abnormal PCIe port, and calling the ACPI;

by invoking the SerDes firmware, execute: resetting the SerDes link parameters corresponding to the PCIe core where the abnormal PCIe port is located, and then returning to call the PCIe driver;

after the PCIe driver is called back, executing: reconstructing a communication link between the abnormal PCIe port and a PCIe node mounted on the abnormal PCIe port.
The apparatus of claim 14, wherein the memory comprises public registers and private registers;

the PCIe driver and the ACPI are stored in the public register, and the SerDes firmware is stored in the private register; or alternatively, the process may be performed,

the PCIe driver is stored in the public register, and the ACPI and SerDes firmware are stored in the private register.
The apparatus of any of claims 10 to 15, wherein the processor further performs the following by invoking the computer program stored in the memory:

And under the condition that the fault type corresponding to the abnormal PCIe port is an unrecoverable fault, disabling the abnormal PCIe port and the communication link of the abnormal PCIe port.
The apparatus of any of claims 10 to 16, wherein the recoverable failure comprises one or more of:

data link layer packet transmission timeout errors, excessive retry times errors of a transaction layer packet write configuration space, two-bit data errors, response errors of an advanced extensible interface AXI bus.
The apparatus of any of claims 10-17, wherein the abort information corresponding to the abort PCIe port includes one or more of:

the identification of the abnormal PCIe port, the fault type corresponding to the abnormal PCIe port, the identification of the PCIe core where the abnormal PCIe port is located, and the identification of the CPU (central processing unit) connected with the PCIe core.
The apparatus of any one of claims 10 to 18, further comprising a communication interface;

the processor performs the following operations in particular by calling the computer program stored in the memory:

receiving the abnormal interrupt information corresponding to the abnormal PCIe port through the communication interface;

Adding the abnormal interrupt information corresponding to the abnormal PCIe port to a preset work queue; the preset work queue is used for storing abnormal interrupt information corresponding to an abnormal PCIe port in each PCIe core;

and acquiring abnormal interrupt information corresponding to the abnormal PCIe port from the preset work queue.
A fault handling system comprising a central processor and peripheral component interconnect passing PCIe cores, said PCIe cores comprising a root complex and at least one PCIe node, said central processor connected to said root complex; the root complex comprises at least one PCIe port, and the root complex is connected with the at least one PCIe node through the at least one PCIe port;

the root complex is configured to generate abort information corresponding to an abort PCIe port in the at least one PCIe port and report the abort information to the central processor;

the central processing unit is configured to perform fault processing on the abnormal PCIe port according to the fault processing method according to any one of claims 1 to 9.
A computer readable storage medium, characterized in that the computer readable medium stores a program code which, when run on a computer, causes the computer to perform the method of any of claims 1 to 9.
A computer program product comprising computer program code which, when run on a computer, causes the computer to perform the method of any one of claims 1 to 9.