CN114253877A - Scanning method and device for Peripheral Component Interconnect Express (PCIE) equipment under Liunx system - Google Patents

Scanning method and device for Peripheral Component Interconnect Express (PCIE) equipment under Liunx system Download PDF

Info

Publication number
CN114253877A
CN114253877A CN202011026067.4A CN202011026067A CN114253877A CN 114253877 A CN114253877 A CN 114253877A CN 202011026067 A CN202011026067 A CN 202011026067A CN 114253877 A CN114253877 A CN 114253877A
Authority
CN
China
Prior art keywords
pcie
link
state
port
scanning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011026067.4A
Other languages
Chinese (zh)
Other versions
CN114253877B (en
Inventor
邵晓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Datang Mobile Communications Equipment Co Ltd
Original Assignee
Datang Mobile Communications Equipment Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Datang Mobile Communications Equipment Co Ltd filed Critical Datang Mobile Communications Equipment Co Ltd
Priority to CN202011026067.4A priority Critical patent/CN114253877B/en
Publication of CN114253877A publication Critical patent/CN114253877A/en
Application granted granted Critical
Publication of CN114253877B publication Critical patent/CN114253877B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/10Program control for peripheral devices
    • G06F13/102Program control for peripheral devices where the programme performs an interfacing function, e.g. device driver
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0026PCI express

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Systems (AREA)

Abstract

The embodiment of the invention provides a scanning method and a scanning device for high-speed Peripheral Component Interconnect (PCIE) equipment in a Liunx system, which are applied to an RC (remote control) in a PCIE (peripheral component interconnect express) data transmission system, wherein the PCIE data transmission system also comprises at least one PCIE device. The method comprises the following steps: detecting a link state of each PCIE device corresponding to a PCIE link in at least one PCIE device, wherein the PCIE link corresponding to the PCIE device is a link which takes an upstream PCIE port of the PCIE device as an end point, the link state comprises a first state and a second state, when the PCIE link is in the first state, the PCIE link cannot transmit data, and when the PCIE link is in the second state, the PCIE link normally transmits data; when the link state of any PCIE equipment corresponding to the PCIE link is changed from the first state to the second state, PCIE scanning is carried out on the PCIE data transmission system. The embodiment of the invention realizes the automatic PCIE scanning operation, avoids the triggering of PCIE scanning by manual intervention and improves the PCIE scanning efficiency.

Description

Scanning method and device for Peripheral Component Interconnect Express (PCIE) equipment under Liunx system
Technical Field
The invention relates to the technical field of communication, in particular to a scanning method and a scanning device for Peripheral Component Interconnect Express (PCIE) equipment under a Liunx system.
Background
With the development of embedded technology, a Linux system (a computer operating system) is widely applied to the development of embedded devices, and the communication problem of multiple devices is caused, so that a Peripheral Component Interconnect Express (PCIE) bus and Interface standard, that is, a high-speed serial computer expansion bus standard, arises. The PCIE provides a serial interconnection standard between two PCIE devices, and allocates an independent channel bandwidth to each device by using a point-to-point technology, so that the devices that use the PCIE standard for transmission have the characteristics of high transmission rate, support of a multi-channel data transfer mode, controllable transmission traffic, and the like.
The current PCIE data transmission system is in a Linux system, and its network topology is usually a tree network topology from a Root Complex (RC) to one or more node devices (EndPoint, EP). The RC may be directly connected to one or more EPs, or the RC may be connected to one or more EPs through a Switch (SW), and the EPs and the SW may be collectively referred to as PCIE devices. When the Linux system kernel is started, the RC may perform a PCIE scan once to determine PCIE devices included in the current PCIE network topology, and allocate system bus numbers, system address spaces, interrupts, and other system and hardware resources to the PCIE devices.
However, if a PCIE device needs to be added during the system operation process, it is necessary to manually provide sysfs file subsystem input and send a command through the Linux system, so that the Linux system checks the command for analysis, and performs a PCIE scanning operation. This manner of manually triggering PCIE scanning operations is inefficient and is not conducive to maintenance of system devices.
Disclosure of Invention
The embodiment of the invention provides a scanning method and a scanning device for high-speed Peripheral Component Interconnect (PCIE) equipment under a Liunx system, which can automatically carry out PCIE scanning operation on a PCIE data transmission system, avoid manual intervention to trigger PCIE scanning and improve the PCIE scanning efficiency.
The embodiment of the invention provides a PCIE equipment scanning method under a Liunx system, which is characterized in that the method is applied to an RC in a PCIE data transmission system, the PCIE data transmission system also comprises at least one PCIE equipment, and the method comprises the following steps:
detecting a link state of each PCIE device in the at least one PCIE device corresponding to a PCIE link, wherein the PCIE link corresponding to the PCIE device is a link which takes an upstream PCIE port of the PCIE device as an end point, the link state comprises a first state and a second state, when the PCIE link is in the first state, the PCIE link cannot transmit data, and when the PCIE link is in the second state, the PCIE link normally transmits data;
when the link state of any PCIE equipment corresponding to the PCIE link is changed from the first state to the second state, PCIE scanning is carried out on the PCIE data transmission system.
The embodiment of the invention provides a PCIE equipment scanning device under a Liunx system, which is characterized in that the scanning device is applied to an RC in a PCIE data transmission system, the PCIE data transmission system also comprises at least one PCIE equipment, and the scanning device comprises:
the detection module is configured to detect a link state of a PCIE link corresponding to each PCIE device in the at least one PCIE device, where the PCIE link corresponding to the PCIE device is a link using an upstream PCIE port of the PCIE device as an endpoint, and the link state includes a first state and a second state, where the PCIE link cannot transmit data when the PCIE link is in the first state, and the PCIE link normally transmits data when the PCIE link is in the second state;
and the scanning module is used for carrying out PCIE scanning on the PCIE data transmission system when the link state of the PCIE link corresponding to any one of the PCIE equipment is changed from the first state to the second state.
The embodiment of the invention has the following advantages:
in the PCIE device scanning method in the Liunx system provided in the embodiment of the present invention, when the link state of the PCIE link corresponding to any PCIE device is changed from the first state to the second state, it is determined that the added PCIE device exists in the current PCIE data transmission system, and the PCIE scanning is performed on the PCIE data transmission system by detecting the link state of the PCIE link corresponding to each PCIE device in at least one PCIE device. Therefore, automatic PCIE scanning operation is realized, manual intervention is avoided to trigger PCIE scanning, and the efficiency of PCIE scanning is improved. Meanwhile, the PCIE scanning operation is automatically carried out, so that the system resources are automatically distributed to the added PCIE devices (for example, the PCIE devices which are newly and ready or the PCIE devices which are recovered from the fault), the normal communication of each PCIE device in the PCIE data transmission system can be automatically maintained, the maintenance of the system devices is facilitated, and the operation and maintenance capacity of the system is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
Fig. 1 shows a schematic structural diagram of a PCIE data transmission system provided in an embodiment of the present invention;
fig. 2 is a schematic structural diagram of another PCIE data transmission system provided in the embodiment of the present invention;
fig. 3 shows a device structure diagram of PCIE communication in the PCIE standard;
FIG. 4 shows a state transition diagram of DLCMSM;
fig. 5 shows a flowchart of a PCIE device scanning method in a Liunx system according to an embodiment of the present invention;
fig. 6 shows a flowchart of a PCIE device scanning method in another Liunx system according to an embodiment of the present invention;
fig. 7 shows a flowchart of a PCIE device scanning method in a Liunx system according to another embodiment of the present invention;
fig. 8 is a schematic structural diagram illustrating a PCIE device scanning apparatus in a Liunx system according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
With the development of embedded technology, a Linux system (a computer operating system) is widely applied to the development of embedded devices, and along with the problem of communication among multiple devices, a Peripheral Component Interconnect Express (PCIE) is a bus and Interface standard, that is, a high-speed serial computer expansion bus standard, and the PCIE standard gradually replaces a Peripheral Component Interconnect standard (PCI). Different from the PCI standard, the PCIE provides a serial interconnection standard between two PCIE devices, and allocates an independent channel bandwidth to each device by using a point-to-point technology, so that the devices that transmit using the PCIE standard have the characteristics of high transmission rate, support of a multi-channel data transmission mode, controllable transmission traffic, and the like. For example, the generation 1 PCIE standard supports 2.5G/Second/Lane/direction, the generation 2 PCIE standard supports 5.0G, and the generation 3 PCIE standard supports 8.0G. And the PCIE standard may be compatible with the PCI standard.
The current PCIE data transmission system is in a Linux system, and its network topology is usually a tree network topology from a Root Complex (RC) to one or more node devices (EndPoint, EP). The RC may be directly connected to one or more EPs, or the RC may be connected to one or more EPs through a Switch (SW), and the EPs and the SW may be collectively referred to as PCIE devices. When the Linux system kernel is started, the RC may perform a PCIE scan once to determine PCIE devices included in the current PCIE network topology, and allocate system bus numbers, system address spaces, interrupts, and other system and hardware resources to the PCIE devices.
However, under the condition that the PCIE data transmission system does not support the hot plug function, if PCIE devices need to be added in the system operation process, the scanning operation of the RC device may only occur when the Linux system kernel is started, so that unless the RC device is restarted, the PCIE devices cannot be scanned again, and thus the PCIE devices cannot perform PCIE communication even after the failure is recovered, and cannot perform normal operation. At this time, a command is manually provided by the Linux system for the sysfs file subsystem to input and send, so that the Linux system checks the command and analyzes the command, and PCIE scanning operation is executed. This manner of manually triggering PCIE scanning operations is inefficient and is not conducive to maintenance of system devices.
The PCIE devices that need to be added in the system operation process may include:
a first PCIE device. The first PCIE device is a device in an abnormal state where connection cannot be performed in a PCIE scanning process when a kernel of the Linux system is started. If the PCIE device is in an abnormal state where the PCIE device cannot be connected in a PCIE scanning process when the kernel of the Linux system is started, the PCIE device cannot be obtained through the PCIE scanning at this time, and thus the PCIE device cannot perform PCIE communication with other PCIE devices. For example, the reason that the first PCIE device is in the abnormal state of being unable to connect may include that the PCIE device is not ready or a physical link communicating with the PCIE device is not successfully negotiated.
A second type of PCIE device. The second PCIE device may refer to a device that is repaired after a failure in normal operation. If the PCIE device fails in normal operation, after the failure is recovered, the RC device needs to scan the PCIE device again, and allocate system and hardware resources to the PCIE device.
Certainly, the PCIE device that needs to be added in the system operation process may also be a device in other situations, for example, a PCIE device that needs to be added according to a requirement, which is not limited in this embodiment of the present invention.
The embodiment of the invention provides a PCIE data transmission system, which can comprise an RC and at least one PCIE device connected with the RC. The PCIE device may include only an EP, or the PCIE device may include an EP and a SW.
Please refer to fig. 1, which illustrates a schematic structural diagram of a PCIE data transmission system according to an embodiment of the present invention. As shown in fig. 1, the PCIE data transmission system includes: an RC101 and at least one EP102 connected to the RC 101. The at least one EP102 refers to one or more EPs 102, and the PCIE data transmission system including the first EP102A and the second EP102B is illustrated in fig. 1 as an example. In the PCIE standard, point-to-point communication modules are used, and only a unique path exists between the RC and each EP. For example, a Central Processing Unit (CPU) may include one or more RCs, each of which is configured to allocate system and hardware resources (collectively, system resources) such as a system bus number, a system address space, and interrupts to an EP coupled thereto. EP is a standard function device defined in the PCIE standard, such as an ethernet, an FC network card, a wireless broadband (IB) network card, and the like.
Optionally, please refer to fig. 2, which shows a schematic structural diagram of another PCIE data transmission system according to an embodiment of the present invention. As shown in fig. 2, based on the illustration in fig. 1, the PCIE data transmission system further includes: at least one SW103, which may include a PCIE to Peripheral Component Interconnect standard (PCI) bridge or a PCIE to PCI-X (a PCI bus based expansion architecture) bridge, etc. The one or more SWs 103 are each connected to an RC101 and at least one EP102, respectively, i.e. RC101 may be directly connected to EP102, and RC101 may also be connected to EP102 via SW 103. Fig. 2 illustrates an example in which one SW103 of the SW apparatuses 103 is connected to a third EP102C and a fourth EP 102D.
Certainly, the PCIE data transmission system provided in the embodiment of the present invention may also only include a case where the RC is connected to the EP through the SW, and the PCIE data transmission systems shown in fig. 1 and fig. 2 do not limit the PCIE data transmission system provided in the present invention.
For the convenience of readers to understand the present invention, the embodiment of the present invention first introduces the structure of a device for PCIE communication under the PCIE standard.
Please refer to fig. 3, which shows a structure diagram of a device for PCIE communication under the PCIE standard. As shown in fig. 3, a first PCIE device performs PCIE communication with a second PCIE device, and the dividing of each PICE device (the first PCIE device and the second PCIE device) by the PCIE standard includes: the system comprises a transaction layer, a data link layer and a physical layer, wherein the data link layer is arranged between the transaction layer and the physical layer. The Transaction Layer of the PCIE device at the sending end may receive a data request from the PCIE device core Layer, convert the data request into a Transaction Layer Packet (TLP), and store the TLP in a buffer to send the TLP to the data link Layer. The Data Link Layer may translate a Data Link Layer Packet (DLLP) based on the TLP to send to the physical Layer. The Physical Layer can be converted into a Physical Layer Packet (PLP) based on DLLP to be sent to the peer PCIE device. And the opposite PCIE equipment receives the PLP, performs reverse data packet conversion processing of a data link layer and a transaction layer, and sends the converted data packet to a core layer of the PCIE equipment to realize communication of the two PCIE equipment.
The data link layer in each PCIE device manages a point-to-point (between two PCIE devices performing PCIE communication) link state. The data link layer may obtain a current link state of the PCIE port from the physical layer, and report the link state to the transaction layer. The link state of the PCIE port includes a data link on state (DL _ Up state) and a data link port state (DL _ Down state). The DL _ Up state indicates that the PCIE port is detected to be connected to another device. Then it can be considered that the PCIE link to which the PCIE port belongs can normally transmit data. The DL _ Down state indicates that no other devices are detected to be connected to the PCIE port. Then it can be considered that the PCIE link to which the PCIE port belongs cannot transmit data.
The data link layer includes: a Data Link Control and Management State Machine (dlcms), where the dlcms is a Data Link layer State of a PCIE port, and the dlcms may correspond to a Link State of the PCIE port to reflect a Link State of a PCIE Link between two PCIE devices.
Specifically, as shown in fig. 4, a state transition diagram of the DLCMSM is shown. The states of the DLCMSM include:
data link invalid state (DL _ Inactive state): the physical layer notifies the data link layer that the current PCIE link is unavailable, and at this time, the opposite end of the PCIE link may not be connected to the PCIE device or the PCIE device of the opposite end is not detected. When the PCIE device is reset, the state of the DLCMSM is a DL _ Inactive state. At this time, the PCIE device discards the TLP stored in the buffer, and stops sending the DLLP. And the data link layer submits the link state of the PCIE port to the transaction layer as DL _ Down state information. When the link training performed by the physical layer is finished, the current PCIE link state is normal and the transaction layer does not disable the current PCIE link, the DL _ Inactive state is transferred to the DL _ Init state.
Several link initialization state (DL _ Init state): after the PCIE device enters the DL _ Init state, the physical layer notifies the data link layer that the current PCIE link is available, but the PCIE device may not receive and transmit DLLPs and TLPs. At this time, the data link layer needs to perform flow control initialization first, and after the initialization is completed, it is checked whether the physical layer link state is normal. If the physical layer link state is normal, submitting the link state of the PCIE port to the transaction layer as DL _ Up state information, and migrating the data link layer state from DL _ Init state to DL _ Active state. If the physical layer link state is abnormal, the data link layer state is transferred from the DL _ Init state to the DL _ Inactive state.
Data link Active state (DL _ Active state): after the PCIE device enters the DL _ Active state, the physical layer notifies the data link layer that the current PCIE link is normal, and the PCIE link may receive and send DLLPs and TLPs normally. At this time, the data link layer submits the link state of the PCIE port as DL _ Up state information to the transaction layer. If an abnormal condition occurs, for example, a PCIE device fails, a Link state of the PCIE device is set to a Link disabled state (Link disabled state) by software, or the PCIE device receives a PME _ Turn _ Off message, the data Link layer state is migrated from the DL _ Active state to the DL _ Inactive state.
Method embodiment one
Fig. 5 is a flowchart illustrating a PCIE device scanning method in a Liunx system according to an embodiment of the present invention. The method can be applied to the RC in the PCIE data transmission system shown in fig. 1 or fig. 2. As shown in fig. 5, the method includes:
step 201, detecting a link state of a PCIE link corresponding to each PCIE device in at least one PCIE device.
After the Linux system kernel is normally started, the RC may periodically and regularly detect a link state of a PCIE link corresponding to each PCIE device in the at least one PCIE device. The PCIE link corresponding to the PCIE device is a link using an upstream PCIE port of the PCIE device as an endpoint. A PCIE port refers to a port in a PCIE device for PCIE communication. An upstream PCIE port of the PCIE device is directly connected or indirectly connected to the RC. The link state includes a first state and a second state, when the PCIE link is in the first state, the PCIE link cannot transmit data, and when the PCIE link is in the second state, the PCIE link normally transmits data. A PCIE device includes an EP, or a PCIE device includes a switch and an EP.
For example, in the case that the PCIE data transmission system is the PCIE data transmission system shown in fig. 1, the PCIE device includes an EP; in the case that the PCIE data transmission system is the PCIE data transmission system shown in fig. 2, the PCIE device includes SW and EP. Taking the PCIE data transmission system shown in fig. 2 as an example, the upstream PCIE port of the SW103 is a port directly connected to the RC, the corresponding PCIE link is the link 1031, the upstream PCIE port of the first EP102A is a port directly connected to the RC, the corresponding PCIE link is the link 102a1, the upstream PCIE port of the third EP102C is a port indirectly connected to the RC, and the corresponding PCIE link is the link 102C 1.
Optionally, the link state of the PCIE link may be represented by a state of a PCIE port corresponding to at least one endpoint in the PCIE link. The process of detecting the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device includes: the method includes detecting a state of a PCIE port of each PCIE device in at least one PCIE device.
For example, the state of the PCIE port may include a link state of the PCIE port, or a data link layer state of the PCIE port, or a link state of the PCIE port and a data link layer state of the PCIE port. The link state of the PCIE port includes a DL _ Up state and a DL _ Down state. The DL _ Up state indicates that when it is detected that the PCIE port is connected to another device, it may be considered that the PCIE link to which the PCIE port belongs may normally transmit data. The DL _ Down state indicates that no other devices are detected to be connected to the PCIE port. Then it can be considered that the PCIE link to which the PCIE port belongs cannot transmit data.
The data link layer state of the PCIE port includes a DL _ Inactive state, a DL _ Init state, and a DL _ Active state. In the case that the data link layer is in the DL _ Active state, the PCIE link may normally receive and send DLLP and TLP, and then the PCIE link may be considered to be capable of normally transmitting data. In the DL _ Inactive state of the data link layer, the opposite end of the PCIE link may not be connected to the PCIE device or detect the PCIE device of the opposite end, and then the PCIE link may be considered to be capable of normally transmitting data. In the state that the data link layer is DL _ Init, if the PCIE link cannot receive and send DLLP and TLP, it may be considered that the PCIE link cannot transmit data.
It should be noted that the link state of a PCIE link may be represented by the state of a PCIE port corresponding to at least one endpoint in the PCIE link, and it is not difficult to understand the PCIE link corresponding to the RC and the PCIE device directly connected to the RC, for example, the link 1031 and the link 102a1 in fig. 2, and the link state of the PCIE link may also be represented by the state of a downstream PCIE port of the RC. Then, in this embodiment of the present application, the detection of the link state of the PCIE link corresponding to the PCIE device directly connected to the RC may also be implemented by detecting the state of the PCIE port of the RC.
Step 202, when the link status of the PCIE link corresponding to any PCIE device changes from the first status to the second status, performing PCIE scanning on the PCIE data transmission system.
When the link state of the PCIE device corresponding to the PCIE link is changed from the first state to the second state, it indicates that the PCIE link connected to the PCIE device is changed from a state in which data transmission cannot be performed to a state in which data transmission can be performed. Therefore, when it is detected that the link state of the PCIE link corresponding to any PCIE device is changed from the first state to the second state, it indicates that the PCIE device whose link state changes in the current PCIE data transmission system is the added PCIE device, and at this time, PCIE scanning may be performed on the PCIE data transmission system to obtain the added PCIE device, and system resources are allocated to the added PCIE device. The system resources may be system and hardware resources such as system bus numbers, system address spaces, interrupts, and the like. The PCIE device whose link status changes may be the first PCIE device, that is, a PCIE device that is ready newly. Of course, the PCIE device with the link state being changed may also be the aforementioned second PCIE device, that is, the PCIE device after the failure recovery.
For example, when it is detected that the link state of the PCIE link corresponding to any PCIE device is changed from the first state to the second state, the RC may implement PCIE scanning on the PCIE data transmission system again by calling a rescan function provided by the Linux system. The RC may call a rescan function by calling a rescan command (system ("echo 1>/sys/bus/pci/rescan")) in the sysfs file subsystem in the Linux system. Or the RC may also implement the invocation of the rescan function by invoking a rescan function.
In summary, in the PCIE device scanning method in the Liunx system provided in the embodiment of the present invention, by detecting the link state of each PCIE device in at least one PCIE device corresponding to the PCIE link, when the link state of any PCIE device corresponding to the PCIE link is changed from the first state to the second state, it is determined that there is an additional PCIE device in the current PCIE data transmission system, and the PCIE scanning is performed on the PCIE data transmission system. Therefore, automatic PCIE scanning operation is realized, manual intervention is avoided to trigger PCIE scanning, and the efficiency of PCIE scanning is improved. Meanwhile, the PCIE scanning operation is automatically carried out, so that the system resources are automatically distributed to the added PCIE devices (for example, the PCIE devices which are newly and ready or the PCIE devices which are recovered from the fault), the normal communication of each PCIE device in the PCIE data transmission system can be automatically maintained, the maintenance of the system devices is facilitated, and the operation and maintenance capacity of the system is improved.
Please refer to fig. 6, which illustrates a flowchart of a PCIE device scanning method in a Liunx system according to another embodiment of the present invention. The method can be applied to the RC in the PCIE data transmission system shown in fig. 1 or fig. 2. As shown in fig. 6, the method includes:
step 301, determine whether the PCIE data scanning system completes PCIE scanning currently. When it is determined that the PCIE scanning system currently completes PCIE scanning, step 302 is executed; when it is determined that the PCIE data scanning system does not currently complete PCIE scanning, step 306 is executed.
After the Linux system kernel is normally started, the RC may periodically determine whether the PCIE scanning of the PCIE data scanning system is currently completed. Optionally, the RC stores a PCIE SCAN completion FLAG (PCIE _ DEVICE _ SCAN _ FLAG). The PCIE scan completion flag has a first value or a second value. The first value indicates that the PCIE scanning is not currently completed by the PCIE data transmission system. That is, the first value indicates that the PCIE scanning is currently performed or the PCIE scanning is started by the PCIE data transmission system. The second data indicates that the PCIE scanning is currently completed by the PCIE data transmission system. Illustratively, the first value is TRUE and the second value is FALSE. The RC may determine that the current PCIE scan completion flag is the first value or the second value by obtaining the PCIE scan completion flag. And when the PCIE scanning completion flag is a second numerical value, determining that the PCIE data transmission system currently completes the PCIE scanning. When the PCIE scanning completion flag is a first value, it is determined that the PCIE scanning of the PCIE data transmission system is not completed currently.
For example, the PCIE scan complete flag may be added to the Linux PCIE bus driver code. For example, a PCIE scan completion flag that can be used for the user layer and the kernel layer to read and write may be added to the Probe driving function, and after the Probe driving function is executed, the initial value of the PCIE scan completion flag is set to be the second data. The RC may obtain a PCIE scanning completion flag in the Probe's driving function, so as to determine whether the PCIE scanning is currently completed by the PCIE data transmission system based on the PCIE scanning completion flag.
Step 302, detecting a link state of a PCIE link corresponding to each PCIE device in the at least one PCIE device.
The RC may periodically detect a link state of a PCIE link corresponding to each PCIE device in the at least one PCIE device at regular time. The PCIE link corresponding to the PCIE device is a link using an upstream PCIE port of the PCIE device as an endpoint. A PCIE port refers to a port in a PCIE device for PCIE communication. An upstream PCIE port of the PCIE device is directly connected or indirectly connected to the RC. The link state includes a first state and a second state, when the PCIE link is in the first state, the PCIE link cannot transmit data, and when the PCIE link is in the second state, the PCIE link normally transmits data. A PCIE device includes an EP, or a PCIE device includes a switch and an EP.
Optionally, the link state of the PCIE link includes a state of a PCIE port corresponding to at least one endpoint in the PCIE link, that is, the link state of the PCIE link may be represented by a state of a PCIE port corresponding to at least one endpoint in the PCIE link. The at least one endpoint may comprise either of the two endpoints of the PCIE link, or both endpoints of the PCIE link. The state of the PCIE port includes a link state of the PCIE port and/or a data link layer state of the PCIE port. For the explanation of the link status of the PCIE port and the data link layer status of the PCIE port, reference may be made to the relevant explanation in step 201, which is not described in detail in this embodiment of the present invention.
For example, a link state of a PCIE link is described by taking a data link layer state representation of a PCIE port as an example, where the data link layer state of the PCIE port includes a DL _ Inactive state, a DL _ Init state, and a DL _ Active state. The process of detecting, by the RC, the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device may include: the RC detects the current state of the data link control and management state machine of all PCIE ports of all PCIE equipment in the data transmission system.
For example, the RC may read the configuration space of each PCIE device in all PCIE devices to obtain the current state of the DLCMSM of each PCIE port in the PCIE device, that is, obtain the current state of the data link layer of each PCIE port. Alternatively, each PCIE device may send a data link layer status message to the RC periodically or at certain times. And when receiving the data link layer state message, the RC acquires the current state of the data link control and management state machine of each PCIE port in the PCIE device. The data link layer state message may be a PME message, where the PME message includes a current state of the data link control and management state machine of each PCIE port in the PCIE device.
In an optional embodiment of the present invention, before performing step 302 to detect a link status of a PCIE link corresponding to each PCIE device in at least one PCIE device, the method further includes steps 401 to 402.
In step 401, it is determined whether to detect the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device for the first time. When the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device is detected for the first time, step 402 is executed; when the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device is not detected for the first time, step 302 is executed to detect the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device.
The RC determines whether to detect the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device for the first time. When the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device is detected for the first time, indicating that the last completed PICE scan is PCIE scan performed when the Linux kernel is started, the link state of the PCIE link corresponding to the PCIE device needs to be recorded, so that it can be determined whether the link state of the PCIE link is changed from the first state to the second state subsequently.
Optionally, the RC may store a set of link state information (PCIE _ PortLinkInfo). The link state information set may be used to record the device identifier of each PCIE device in the PCIE data transmission system and the link state of the PCIE link corresponding to the PCIE device. The process of determining whether to detect the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device for the first time may include: the RC may obtain the stored link state information set, and if the link state information set is empty, determine to detect the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device for the first time. If the link state information set is not empty, determining that the link state of the PCIE link corresponding to each PCIE device in at least one PCIE device is not detected for the first time.
For example, a description is given by taking a link state of a PCIE link and a data link layer state representation of a PCIE port as an example, the link state information set includes a correspondence between port identifiers of all PCIE ports and states of the PCIE ports. For example, the port identifier of the PCIE port may be a PCIE port address, and the PCIE port address may include a domain number, a bus number, a device number, and a function number.
In step 402, the device identifier of each PCIE device and the link state of the PCIE link corresponding to the PCIE device in the PCIE data transmission system are acquired and stored in the link state information set.
For example, the description continues by taking the link state of the PCIE link as represented by the data link layer state of the PCIE port, and the port identifier of the PCIE port is a PCIE port address as an example, the obtaining the device identifier of each PCIE device in the PCIE data transmission system and the link state of the PCIE link corresponding to the PCIE device, and the process of saving the link state information set may include:
the PCIE scanning configuration information of all PCIE ports of all PCIE devices in the PCIE data transmission system and the data link layer state of the PCIE ports are read, that is, the PCIE scanning results of all PCIE ports and the current state of the DLCMSM are read. And recording PCIE scanning configuration information of all PCIE ports and data link layer states of the PCIE ports to a link state information set. The PCIE scanning result of the PCIE port includes: a PCIE port address. And storing the port addresses of all PCIE ports and the data link layer states of the PCIE ports to a link state information set.
Step 303, determining whether the link state of any PCIE device in the PCIE data transmission system corresponding to the PCIE link is changed from the first state to the second state. When the link status of any PCIE device corresponding to the PCIE link changes from the first status to the second status, step 304 is executed; when the link status of any PCIE device corresponding to the PCIE link is not changed from the first status to the second status, step 305 is executed.
The RC may determine whether a link state of a PCIE link corresponding to any PCIE device in the PCIE data transmission system is converted from the first state to the second state. When the link state of the PCIE link corresponding to the PCIE device is changed from the first state to the second state, it indicates that the PCIE link connected to the PCIE device is changed from the state where data transmission cannot be performed to the state where data transmission can be performed. Therefore, when it is detected that the link state of the PCIE link corresponding to any PCIE device is changed from the first state to the second state, it is indicated that the PCIE device whose link state changes in the current PCIE data transmission system is the added PCIE device. When it is not detected that the link state of the PCIE link corresponding to any PCIE device is changed from the first state to the second state, that is, when the link state of the PCIE link corresponding to any PCIE device is not changed from the first state to the second state, it indicates that no PCIE device is added in the current PCIE data transmission system.
For example, taking the link state of the PCIE link and the data link layer state representation of the PCIE port as an example, the process of determining whether the link state of the PCIE link corresponding to any PCIE device in the PCIE data transmission system is changed from the first state to the second state may include: and judging whether the current state of any PCIE port in the PCIE data transmission system is converted from the non-DL _ Active state to the DL _ Active state. When the current state of any PCIE port is changed from the non-DL _ Active state to the DL _ Active state, that is, when the link state of any PCIE port is changed from the DL _ Down state to the DL _ Up state, step 304 is executed; when the current state of any PCIE port is not converted from the non-DL _ Active state to the DL _ Active state, step 305 is executed. The non-DL _ Active state refers to a DL _ Inactive state or a DL _ Init state.
In an optional embodiment of the present invention, the process of determining whether the link state of the PCIE link corresponding to any PCIE device in the PCIE data transmission system is converted from the first state to the second state may include acquiring the link states of the PCIE links corresponding to all PCIE devices in the link state information. And judging whether the detected link state of the PCIE link corresponding to any PCIE equipment is in a second state or not, and whether the link state of the PCIE equipment corresponding to any PCIE equipment in the link state information is in a first state or not.
For example, when the link state of the PCIE link is the data link layer state of the PCIE port, the process of determining whether the link state of the PCIE link corresponding to any PCIE device in the PCIE data transmission system is changed from the first state to the second state may include steps 501 to 502.
In step 501, the states of all PCIE ports in the link state information are acquired.
The link state information records the current states of the DLCMSMs of all PCIE ports of all PCIE devices after PCIE scanning is performed when the Linux kernel is started. Or the link state information records the current states of the DLCMSM of all the PCIE ports of all the PCIE devices detected last time, and the port identifiers of all the PCIE ports identify the data link layer states of the corresponding PCIE ports. The RC acquires the data link layer states of the PCIE ports corresponding to the port identifications of all the PCIE ports of the link state information.
In step 502, it is determined whether the detected current state of the DLCMSM of any PCIE port is a DL _ Active state, and whether the state of the PCIE port corresponding to any PCIE port in the link state information is a non-DL _ Active state.
And the RC compares whether the current state of any detected PCIE port is a DL _ Active state or not, and compares whether the data link layer state of the PCIE port corresponding to the PCIE port is a non-DL _ Active state or not in the acquired link state information.
When the detected current state of any PCIE port is a DL _ Active state and the state of the corresponding PCIE port in the link state information is a non-DL _ Active state, the data link layer state of the PCIE port is changed from the non-DL _ Active state to a DL _ Active state, and the link state of the PCIE port is changed from a DL _ Down state to a DL _ Up state. When the detected current state of any PCIE port is not the DL _ Active state, or the state of the corresponding PCIE port in the link state information is not the non-DL _ Active state, it indicates that the data link layer state of the PCIE port is not changed from the non-DL _ Active state to the DL _ Active state, that is, the link state of the PCIE port is not changed from the DL _ Down state to the DL _ Up state.
Step 304, performing PCIE scanning on the PCIE data transmission system.
When the link state of any PCIE equipment corresponding to the PCIE link is changed from the first state to the second state, the RC carries out PCIE scanning on the PCIE data transmission system.
Optionally, the process of performing PCIE scanning on the PCIE data transmission system by the RC may include the following steps 601 to 603.
In step 601, it is determined whether the target PCIE port is a failure recovery port. When the target PCIE port is the failure recovery port, step 602 is executed; when the target PCIE port is not the failure recovery port, step 603 is executed.
The target PCIE port is a port of which the current state is converted from a non DL _ Active state to a DL _ Active state. The failure recovery port refers to the PCIE port of the second PCIE device, and specifically refers to a port of the PCIE device after failure recovery. Since the PCIE device after the failure recovery does not operate the change of the total number of PCIE devices in the PCIE data transmission system compared to the newly added PCIE device (e.g., the first device), when the target PCIE is the failure recovery port, it is not necessary to change the system resources allocated to other PCIE devices in the current PCIE data transmission system except the target PCIE device, and it is only necessary to allocate the system resources to the target PCIE device again. When the target PCIE is not the failure recovery port, the system resources allocated to other PCIE devices in the current PCIE data transmission system except the target PCIE device need to be changed to reallocate the system resources of all PCIE devices in the current PCIE data transmission system. Therefore, different operations can be executed under different conditions by judging whether the target PCIE device is the failure recovery port, thereby avoiding that the device overhead is large and the resources are wasted due to the fact that the system resources of all the PCIE devices in the current PCIE data transmission system are reallocated no matter whether the target PCIE device is the failure recovery port.
Optionally, the link state information set may further include a correspondence between a port identifier of each PCIE port in the PCIE data transmission system and the resource allocation information. The resource allocation information includes an allocation identifier or an unallocated identifier, where the allocation identifier is used to indicate that system resources have been allocated to a PCIE port, that is, to indicate that the PCIE port corresponding to the allocation identifier is a failure recovery port. The unallocated identifier is used to indicate that system resources have not been allocated to the PCIE port, that is, to indicate that the PCIE port corresponding to the allocated identifier is not a failure recovery port.
The process of determining whether the target PCIE device is the failure recovery port may include: and acquiring a port identifier and a link state information set of the target PCIE port. And determining the resource allocation information corresponding to the port identification of the target PCIE port in the link state information set. And when the resource allocation information of the target PCIE port is the allocation identification, determining that the target PCIE port is a fault recovery port. And when the allocated resource information of the target PCIE port is the unallocated identifier, determining that the target PCIE port is not the failure recovery port.
Step 602, performing PCIE scanning on the PCIE data transmission system, and allocating system resources to the PCIE device where the target PCIE port is located.
For example, the RC may implement PCIE scanning again for the PCIE data transmission system by calling a rescan function provided by the Linux system. And allocating system resources for the PCIE device where the target PCIE port is located. The RC can call the rescan function by calling a rescan command in a sysfs file subsystem in the Linux system. Or the RC may also implement the invocation of the rescan function by invoking a rescan function.
Step 603, releasing the system resources of each PCIE device in the at least one PCIE device, performing PCIE scanning on the PCIE data transmission system, and allocating the system resources to each PCIE device obtained by scanning.
In an example, the RC release allocates system bus numbers, system address spaces, interrupts, and other system and hardware resources to each PCIE device in the PCIE data transmission system. By calling the rescan function provided by the Linux system, the PCIE data transmission system is scanned again. And allocating system resources for each scanned PCIE device.
In an optional embodiment of the present invention, before the RC executes step 304 to perform PCIE scanning on the PCIE data transmission system, the method further includes: when the link state of any PCIE equipment corresponding to the PCIE link is changed from the first state to the second state, the RC updates the PCIE scanning completion mark to be a second numerical value.
Correspondingly, after the RC performs PCIE scanning on the PCIE data transmission system in step 304, the method further includes: the PCIE scan completion flag is updated to a first value. The PCIE scanning completion flag is updated in time in this way, so that the PCIE scanning completion flag can truly reflect the PCIE scanning state of the current PCIE data transmission system, and it is convenient to determine whether the PCIE scanning of the PCIE data transmission system is currently completed based on the PCIE scanning completion flag.
Step 305, recording the detected link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device.
When detecting the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device, the RC records the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device detected in step 302. After the RC performs the PCIE scanning in step 304, record the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device detected in step 302.
Optionally, when the RC stores the link state information set, the process of recording the detected link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device may include:
and updating the state of the corresponding PCIE port in the link state information set based on the current states of all the PCIE ports. For example, based on the port identifier of each PCIE port and the current state of the dlcms that obtains the PCIE port, the state corresponding to the port identifier in the link state information set is updated, and in the updated link state information set, the state corresponding to the port identifier of each PCIE port is the current state of the obtained dlcms that corresponds to the PCIE port.
Optionally, the link state information set includes a correspondence between port identifiers of all PCIE ports and states of the PCIE ports, and the states of the PCIE ports may also be divided into a history state and a current state. The set of link state information may include: and the port identifications of all PCIE ports are in corresponding relation with the historical state and the current state of the PCIE ports.
When the RC stores the link state information set, the process of recording the detected link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device may include:
updating the historical state of each PCIE port in the link state information set into the current state corresponding to each PCIE port in the link state information set; and updating the current state of each PCIE port in the link state information set into the current state of the DLCMSM corresponding to the PCIE port.
In an optional embodiment of the present invention, after the RC performs the PCIE scanning in step 304, a process of recording a detected link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device may be replaced with: and updating the state corresponding to the port identifier of the target PCIE port in the link state information set based on the current state of the DLCMSM of the target PCIE port and the port identifier of the target PCIE port. And the state corresponding to the port identification of the target PCIE port in the updated link state information set is the current state of the DLCMSM of the target PCIE port. Therefore, the state corresponding to the target PCIE port is updated only in the link state information set, and compared with the state corresponding to each PCIE port in the link state information set, the method saves equipment overhead, improves the updating rate and further improves the performance of the method.
And step 306, ending the operation.
The RC finishes the operation, and waits for the next cycle to execute the above step 301.
In summary, in the PCIE device scanning method in the Liunx system provided in the embodiment of the present invention, by detecting the link state of each PCIE device in at least one PCIE device corresponding to the PCIE link, when the link state of any PCIE device corresponding to the PCIE link is changed from the first state to the second state, it is determined that there is an additional PCIE device in the current PCIE data transmission system, and the PCIE scanning is performed on the PCIE data transmission system. Therefore, automatic PCIE scanning operation is realized, manual intervention is avoided to trigger PCIE scanning, and the efficiency of PCIE scanning is improved. Meanwhile, the PCIE scanning operation is automatically carried out, so that the system resources are automatically distributed to the added PCIE devices (for example, the PCIE devices which are newly and ready or the PCIE devices which are recovered from the fault), the normal communication of each PCIE device in the PCIE data transmission system can be automatically maintained, the maintenance of the system devices is facilitated, and the operation and maintenance capacity of the system is improved.
Fig. 7 is a flowchart illustrating a PCIE device scanning method in a Liunx system according to another embodiment of the present invention. The method can be applied to the RC in the PCIE data transmission system shown in fig. 1 or fig. 2. In the method, a link state of the PCIE link is taken as a data link layer state of the PCIE port as an example for description. As shown in fig. 7, the method includes:
step 701, determining whether the PCIE data scanning system completes PCIE scanning currently. When it is determined that the PCIE scanning system currently completes PCIE scanning, step 702 is executed; and when the PCIE data scanning system is determined not to complete the PCIE scanning currently, ending the detection operation.
For the explanation of step 701, reference may be made to the explanation of step 301, which is not described in detail in this embodiment of the present invention.
Step 702, determine whether it is the first detection after PCIE scanning. When the detection is the first detection, step 703 and step 704 are executed; when it is not the first detection, step 704 is performed.
And judging whether the link state of the PCIE link corresponding to each PCIE device in at least one PCIE device is detected for the first time after PCIE scanning. When the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device is detected for the first time, step 703 is executed; when the link status of the PCIE link corresponding to each PCIE device in the at least one PCIE device is not detected for the first time, step 704 is executed. For the explanation of step 702, reference may be made to the explanation of step 401, which is not described in detail in this embodiment of the present invention.
Step 703, recording the scanning configuration information of each PCIE port and the status of the data link layer. Step 704 is performed.
For the explanation of step 703, reference may be made to the explanation of step 402, which is not described in detail in this embodiment of the present invention.
Step 704, sequentially detecting the data link layer states of each PCIE port in the PCIE data transmission system.
For the explanation of step 704, reference may be made to the explanation of step 302, which is not described in detail in the embodiment of the present invention.
Step 705, determining whether the current state of any PCIE port in the PCIE data transmission system is converted from the non-DL _ Active state to the DL _ Active state. When the current state of any PCIE port is changed from the non-DL _ Active state to the DL _ Active state, step 706 is executed; when the current state of any PCIE port is not converted from the non-DL _ Active state to the DL _ Active state, step 710 is executed.
For the explanation of step 705, reference may be made to the explanation of step 303, which is not described in detail in this embodiment of the present invention.
Step 706, a PCIE scan completion flag is set.
Updating the PCIE _ DEVICE _ SCAN _ FLAG to a second value to indicate that the current PCIE data transmission system does not complete PCIE scanning. For the explanation of the PCIE scan completion flag, refer to the explanation of the PCIE scan completion flag in step 301.
Step 707, determine whether the target PCIE port is a failure recovery port. When the target PCIE port is the failure recovery port, step 708 is executed; when the target PCIE port is not the failure recovery port, step 709 is executed.
The target PCIE port is a port of which the current state is converted from a non DL _ Active state to a DL _ Active state. For the explanation of step 707, reference may be made to the explanation of step 601, which is not described in detail in this embodiment of the present invention.
Step 708, performing PCIE scanning on the PCIE data transmission system again. Step 710 is performed.
For the explanation of step 708, reference may be made to the explanation of step 602, which is not described in detail in this embodiment of the present invention.
Step 709, releasing system resources allocated to all PCIE devices in the PCIE data transmission system, and performing PCIE scanning on the PCIE data transmission system again.
The method includes releasing system resources allocated to all PCIE devices in the PCIE data transmission system, that is, releasing system resources allocated to each PCIE device in at least one PCIE device. For the explanation of step 709, refer to the explanation of step 603, which is not described in detail in this embodiment of the present invention.
And step 710, recording the data link layer state of the current PCIE port.
The current PCIE port data link layer state is the data link layer state of the PCIE port detected in step 704.
In summary, in the PCIE device scanning method in the Liunx system provided in the embodiment of the present invention, by detecting the link state of each PCIE device in at least one PCIE device corresponding to the PCIE link, when the link state of any PCIE device corresponding to the PCIE link is changed from the first state to the second state, it is determined that there is an additional PCIE device in the current PCIE data transmission system, and the PCIE scanning is performed on the PCIE data transmission system. Therefore, automatic PCIE scanning operation is realized, manual intervention is avoided to trigger PCIE scanning, and the efficiency of PCIE scanning is improved. Meanwhile, the PCIE scanning operation is automatically carried out, so that the system resources are automatically distributed to the added PCIE devices (for example, the PCIE devices which are newly and ready or the PCIE devices which are recovered from the fault), the normal communication of each PCIE device in the PCIE data transmission system can be automatically maintained, the maintenance of the system devices is facilitated, and the operation and maintenance capacity of the system is improved.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Device embodiment
Fig. 8 shows a PCIE device scanning apparatus in a Liunx system according to an embodiment of the present invention, which is applied to an RC in a PCIE data transmission system, where the PCIE data transmission system further includes at least one PCIE device. As shown in fig. 8, the apparatus includes:
the detection module 801 is configured to detect a link state of each PCIE device in the at least one PCIE device corresponding to the PCIE link, where the PCIE link corresponding to the PCIE device is a link using an upstream PCIE port of the PCIE device as an endpoint, the link state includes a first state and a second state, the PCIE link cannot transmit data when the PCIE link is in the first state, and the PCIE link normally transmits data when the PCIE link is in the second state, and the PCIE device includes a node device EP or the PCIE device includes an exchanger and an EP.
The scanning module 802 is configured to perform PCIE scanning on the PCIE data transmission system when a link state of a PCIE link corresponding to any PCIE device changes from the first state to the second state.
Optionally, the link state of the PCIE link includes a state of a PCIE port corresponding to at least one endpoint in the PCIE link, and the state of the PCIE port includes a link state of the PCIE port and/or a data link layer state of the PCIE port.
Optionally, when the link state of the PCIE link includes a data link layer state of the PCIE port, the detecting module 801 is further configured to: the current states of the data link control and management state machines of all PCIE ports in the data transmission system are detected, and the states of the data link control and management state machines comprise the effective states of the data links.
A scanning module 802, further configured to: when the current state of any PCIE port is converted from the non-data link effective state to the data link effective state, PCIE scanning is carried out on the PCIE data transmission system.
Optionally, the scanning module 802 is further configured to:
when the target PCIE port is a failure recovery port, performing PCIE scanning on the PCIE data transmission system, and distributing system resources for PCIE equipment where the target PCIE port is located, wherein the target PCIE port is a port of which the current state is converted from a non-data link effective state to a data link effective state;
when the target PCIE port is not the fault recovery port, releasing the system resources allocated to each PCIE device in at least one PCIE device, performing PCIE scanning on the PCIE data transmission system, and allocating the system resources to each PCIE device obtained through scanning.
Optionally, the apparatus further comprises:
the first obtaining module is configured to obtain a port identifier and a link state information set of a target PCIE port, where the link state information set includes a correspondence between the port identifier of each PCIE port in the PCIE data transmission system and allocation resource information, and the allocation resource information includes an allocation identifier or an unallocated identifier, where the allocation identifier is used to indicate that system resources are allocated to the PCIE port, and the unallocated identifier is used to indicate that system resources are not allocated to the PCIE port.
The first determining module is used for determining that the target PCIE port is a failure recovery port when the resource allocation information of the target PCIE port is an allocation identifier; and when the allocated resource information of the target PCIE port is the unallocated identifier, determining that the target PCIE port is not the failure recovery port.
Optionally, the link state information set further includes: the corresponding relation between the port identification of each PCIE port in the PCIE data transmission system and the state of the PCIE port, the device also comprises:
and the first updating module is used for updating the state corresponding to the port identifier of the target PCIE port in the link state information set based on the current state of the target PCIE port and the port identifier of the target PCIE port.
And the second acquisition module is used for acquiring the states of all PCIE ports in the link state information.
The second determining module is configured to determine that the current state of any PCIE port is changed from the non-data link active state to the data link active state when the detected current state of any PCIE port is the data link active state and the state of the corresponding PCIE port in the link state information is the non-data link active state.
The second updating module is further configured to update the state of the corresponding PCIE port in the link state information set based on the current states of all PCIE ports when the detected current state of any PCIE port is not converted from the non-data link active state to the data link active state.
Optionally, the detecting module 801 is further configured to: determining that a PCIE data transmission system currently completes PCIE scanning; and detecting the link state of a PCIE link corresponding to each PCIE device in at least one PCIE device.
Optionally, the detecting module 801 is further configured to:
acquiring a PCIE scanning completion mark, wherein the PCIE scanning completion mark has a first numerical value or a second numerical value, the first numerical value indicates that the PCIE scanning of the PCIE data transmission system is not completed currently, and the second data indicates that the PCIE scanning of the PCIE data transmission system is completed currently; and when the PCIE scanning completion flag is a second numerical value, determining that the PCIE data transmission system currently completes the PCIE scanning.
Optionally, the apparatus further comprises:
a third updating module, configured to update the PCIE scanning completion flag to a second numerical value when the link state of the PCIE link corresponding to any PCIE device is changed from the first state to the second state; and after the PCIE scanning is finished, updating the PCIE scanning finishing mark to be a first numerical value.
To sum up, in the PCIE device scanning apparatus in the Liunx system provided in the embodiment of the present invention, the link state of each PCIE device in the at least one PCIE device corresponding to the PCIE link is detected by the detection module in the RC, so that when the link state of any PCIE device corresponding to the PCIE link changes from the first state to the second state, the scanning module determines that the added PCIE device exists in the current PCIE data transmission system, and performs PCIE scanning on the PCIE data transmission system. Therefore, automatic PCIE scanning operation is realized, manual intervention is avoided to trigger PCIE scanning, and the efficiency of PCIE scanning is improved. Meanwhile, the PCIE scanning operation is automatically carried out, so that the system resources are automatically distributed to the added PCIE devices (for example, the PCIE devices which are newly and ready or the PCIE devices which are recovered from the fault), the normal communication of each PCIE device in the PCIE data transmission system can be automatically maintained, the maintenance of the system devices is facilitated, and the operation and maintenance capacity of the system is improved.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
An embodiment of the present invention also provides an electronic device, referring to fig. 9, including: a processor 901, a memory 902, and a computer program 9021 stored in the memory 902 and operable on the processor 901, where the processor 901 implements the PCIE device scanning method under the Liunx system of the foregoing embodiment when executing the program.
Embodiments of the present disclosure also provide a readable storage medium, where instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the PCIE device scanning method under the Liunx system of the foregoing embodiments.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present disclosure are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the embodiments of the present disclosure as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the embodiments of the present disclosure.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the disclosure, various features of the embodiments of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that is, claimed embodiments of the disclosure require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of an embodiment of this disclosure.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
The various component embodiments of the disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in a sequencing device according to embodiments of the present disclosure. Embodiments of the present disclosure may also be implemented as an apparatus or device program for performing a portion or all of the methods described herein. Such programs implementing embodiments of the present disclosure may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit embodiments of the disclosure, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the disclosure may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The above description is only for the purpose of illustrating the preferred embodiments of the present disclosure and is not to be construed as limiting the embodiments of the present disclosure, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the embodiments of the present disclosure are intended to be included within the scope of the embodiments of the present disclosure.
The above description is only a specific implementation of the embodiments of the present disclosure, but the scope of the embodiments of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the embodiments of the present disclosure, and all the changes or substitutions should be covered by the scope of the embodiments of the present disclosure. Therefore, the protection scope of the embodiments of the present disclosure shall be subject to the protection scope of the claims.

Claims (20)

1. A method for scanning a Peripheral Component Interconnect Express (PCIE) device in a Liunx system is characterized in that the method is applied to a root complex RC in a PCIE data transmission system, the PCIE data transmission system further comprises at least one PCIE device, and the method comprises the following steps:
detecting a link state of each PCIE device in the at least one PCIE device corresponding to a PCIE link, wherein the PCIE link corresponding to the PCIE device is a link which takes an upstream PCIE port of the PCIE device as an end point, the link state comprises a first state and a second state, when the PCIE link is in the first state, the PCIE link cannot transmit data, and when the PCIE link is in the second state, the PCIE link normally transmits data;
when the link state of any PCIE equipment corresponding to the PCIE link is changed from the first state to the second state, PCIE scanning is carried out on the PCIE data transmission system.
2. The method according to claim 1, wherein the link status of the PCIE link comprises a status of a PCIE port corresponding to at least one endpoint in the PCIE link, and the status of the PCIE port comprises a link status of a PCIE port and/or a data link layer status of the PCIE port.
3. The method of claim 2, wherein when the link state of the PCIE link comprises a data link layer state of the PCIE port,
detecting a link state of a PCIE link corresponding to each PCIE device in the at least one PCIE device, including:
detecting the current states of data link control and management state machines of all PCIE ports in the PCIE data transmission system, wherein the states of the data link control and management state machines comprise a data link effective state or a non-data link effective state;
when the link state of any PCIE device corresponding to the PCIE link is changed from the first state to the second state, performing PCIE scanning on the PCIE data transmission system, including:
and when the current state of any PCIE port is converted from a non-data link effective state to a data link effective state, PCIE scanning is carried out on the PCIE data transmission system.
4. The method of claim 3, wherein the performing PCIE scanning on the PCIE data transmission system comprises:
when a target PCIE port is a failure recovery port, carrying out PCIE scanning on the PCIE data transmission system, and distributing system resources for PCIE equipment where the target PCIE port is located, wherein the target PCIE port is a port of which the current state is converted from a non-data link effective state to a data link effective state;
when the target PCIE port is not the fault recovery port, releasing the system resource of each PCIE device in the at least one PCIE device, performing PCIE scanning on the PCIE data transmission system, and distributing the system resource for each PCIE device obtained by scanning.
5. The method of claim 4, wherein prior to the PCIE scanning of the PCIE data transmission system, the method further comprises:
acquiring a port identifier and a link state information set of a target PCIE port, wherein the link state information set comprises a corresponding relation between the port identifier of each PCIE port in the PCIE data transmission system and allocated resource information, the allocated resource information comprises an allocated identifier or an unallocated identifier, the allocated identifier is used for indicating that system resources are allocated to the PCIE port, and the unallocated identifier is used for indicating that system resources are not allocated to the PCIE port;
when the allocated resource information of the target PCIE port is an allocated identifier, determining that the target PCIE port is a fault recovery port;
and when the allocated resource information of the target PCIE port is the unallocated identifier, determining that the target PCIE port is not the failure recovery port.
6. The method of any of claims 2-5, wherein the set of link state information further comprises: the port identification of each PCIE port in the PCIE data transmission system is corresponding to the state of the PCIE port,
after the PCIE scanning is performed on the PCIE data transmission system, the method further includes: updating the state corresponding to the port identifier of the target PCIE port in the link state information set based on the current state of the target PCIE port and the port identifier of the target PCIE port;
before performing PCIE scanning on the PCIE data transmission system, the method further includes:
acquiring the states of all PCIE ports in the link state information;
when the detected current state of any PCIE port is a data link effective state and the state of the corresponding PCIE port in the link state information is a non-data link effective state, determining that the current state of any PCIE port is converted from the non-data link effective state to the data link effective state;
when the detected current state of any PCIE port is not converted from a non-data link effective state to a data link effective state, updating the state of the corresponding PCIE port in the link state information set based on the current states of all the PCIE ports.
7. The method according to any one of claims 1 to 5, wherein the detecting a link status of a PCIE link corresponding to each PCIE device in the at least one PCIE device comprises:
determining that the PCIE data transmission system currently completes PCIE scanning;
and detecting the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device.
8. The method of claim 7, wherein the determining that the PCIE data transmission system is currently finished with PCIE scanning comprises:
acquiring a PCIE scanning completion mark, wherein the PCIE scanning completion mark has a first numerical value or a second numerical value, the first numerical value indicates that the PCIE data transmission system does not complete the PCIE scanning currently, and the second data indicates that the PCIE data transmission system completes the PCIE scanning currently;
and when the PCIE scanning completion flag is a second numerical value, determining that the PCIE data transmission system currently completes the PCIE scanning.
9. The method of claim 7, further comprising:
when the link state of any PCIE equipment corresponding to the PCIE link is changed from a first state to a second state, updating a PCIE scanning completion mark to a second numerical value;
and after the PCIE scanning is finished, updating the PCIE scanning finishing mark to be a first numerical value.
10. A Peripheral Component Interconnect Express (PCIE) device scanning device under a Liunx system is characterized in that the device is applied to a root complex RC in a PCIE data transmission system, the PCIE data transmission system further comprises at least one PCIE device, and the device comprises:
the detection module is configured to detect a link state of a PCIE link corresponding to each PCIE device in the at least one PCIE device, where the PCIE link corresponding to the PCIE device is a link using an upstream PCIE port of the PCIE device as an endpoint, and the link state includes a first state and a second state, where the PCIE link cannot transmit data when the PCIE link is in the first state, and the PCIE link normally transmits data when the PCIE link is in the second state;
and the scanning module is used for carrying out PCIE scanning on the PCIE data transmission system when the link state of the PCIE link corresponding to any one of the PCIE equipment is changed from the first state to the second state.
11. The apparatus of claim 10, wherein the link status of the PCIE link comprises a status of a PCIE port corresponding to at least one endpoint in the PCIE link, and the status of the PCIE port comprises a link status of a PCIE port and/or a data link layer status of the PCIE port.
12. The apparatus of claim 11, wherein when the link state of the PCIE link comprises a data link layer state of the PCIE port,
the detection module is further configured to: detecting the current states of data link control and management state machines of all PCIE ports in the PCIE data transmission system, wherein the states of the data link control and management state machines comprise a data link effective state or a non-data link effective state;
the scanning module is further configured to: and when the current state of any PCIE port is converted from a non-data link effective state to a data link effective state, PCIE scanning is carried out on the PCIE data transmission system.
13. The apparatus of claim 12, wherein the scanning module is further configured to:
when a target PCIE port is a failure recovery port, carrying out PCIE scanning on the PCIE data transmission system, and distributing system resources for PCIE equipment where the target PCIE port is located, wherein the target PCIE port is a port of which the current state is converted from a non-data link effective state to a data link effective state;
when the target PCIE port is not the fault recovery port, releasing the system resource of each PCIE device in the at least one PCIE device, performing PCIE scanning on the PCIE data transmission system, and distributing the system resource for each PCIE device obtained by scanning.
14. The apparatus of claim 13, further comprising:
a first obtaining module, configured to obtain a port identifier and a link state information set of a target PCIE port, where the link state information set includes a correspondence between a port identifier of each PCIE port in the PCIE data transmission system and allocation resource information, the allocation resource information includes an allocation identifier or an unallocated identifier, the allocation identifier is used to indicate that system resources are allocated to the PCIE port, and the unallocated identifier is used to indicate that system resources are not allocated to the PCIE port;
a first determining module, configured to determine that the target PCIE port is a failure recovery port when the allocation resource information of the target PCIE port is an allocation identifier,
and when the allocated resource information of the target PCIE port is the unallocated identifier, determining that the target PCIE port is not the failure recovery port.
15. The apparatus of any of claims 11-14, wherein the set of link state information further comprises: the corresponding relationship between the port identifier of each PCIE port in the PCIE data transmission system and the state of the PCIE port, where the apparatus further includes:
a first updating module, configured to update a state corresponding to a port identifier of a target PCIE port in the link state information set based on the current state of the target PCIE port and the port identifier of the target PCIE port;
a second obtaining module, configured to obtain states of all PCIE ports in the link state information;
a second determining module, configured to determine that the current state of any PCIE port is changed from a non-data link active state to a data link active state when the detected current state of any PCIE port is a data link active state and a state of a corresponding PCIE port in the link state information is a non-data link active state;
a second updating module, configured to update the state of the corresponding PCIE port in the link state information set based on the current states of all PCIE ports when the detected current state of any PCIE port is not changed from a non-data link active state to a data link active state.
16. The apparatus according to any one of claims 10-14, wherein the detection module is further configured to:
determining that the PCIE data transmission system currently completes PCIE scanning;
and detecting the link state of the PCIE link corresponding to each PCIE device in the at least one PCIE device.
17. The apparatus of claim 16, wherein the detection module is further configured to:
acquiring a PCIE scanning completion mark, wherein the PCIE scanning completion mark has a first numerical value or a second numerical value, the first numerical value indicates that the PCIE data transmission system does not complete the PCIE scanning currently, and the second data indicates that the PCIE data transmission system completes the PCIE scanning currently;
and when the PCIE scanning completion flag is a second numerical value, determining that the PCIE data transmission system currently completes the PCIE scanning.
18. The apparatus of claim 16, further comprising:
a third updating module, configured to update the PCIE scanning completion flag to a second numerical value when the link status of the PCIE link corresponding to any PCIE device is changed from the first status to the second status,
and after the PCIE scanning is finished, updating the PCIE scanning finishing mark to be a first numerical value.
19. An electronic device, comprising: processor, memory and computer program stored on the memory and executable on the processor, characterized in that the processor implements the PCIE device scanning method under the Liunx system according to one or more of claims 1 to 9 when executing the program.
20. A readable storage medium, characterized in that instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform a PCIE device scanning method under the Liunx system as set forth in one or more of method claims 1-9.
CN202011026067.4A 2020-09-25 2020-09-25 Method and device for scanning Peripheral Component Interconnect Express (PCIE) equipment in Liune system Active CN114253877B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011026067.4A CN114253877B (en) 2020-09-25 2020-09-25 Method and device for scanning Peripheral Component Interconnect Express (PCIE) equipment in Liune system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011026067.4A CN114253877B (en) 2020-09-25 2020-09-25 Method and device for scanning Peripheral Component Interconnect Express (PCIE) equipment in Liune system

Publications (2)

Publication Number Publication Date
CN114253877A true CN114253877A (en) 2022-03-29
CN114253877B CN114253877B (en) 2024-01-26

Family

ID=80789312

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011026067.4A Active CN114253877B (en) 2020-09-25 2020-09-25 Method and device for scanning Peripheral Component Interconnect Express (PCIE) equipment in Liune system

Country Status (1)

Country Link
CN (1) CN114253877B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140156898A1 (en) * 2012-11-30 2014-06-05 Gang Luo Pci and pci express virtual hot plug systems and methods
CN104820646A (en) * 2015-05-25 2015-08-05 烽火通信科技股份有限公司 PCIE (peripheral component interface express) device dynamic scanning method supporting multi RC (remote control) in Linux system
CN106326167A (en) * 2015-06-17 2017-01-11 中兴通讯股份有限公司 PCIE sub-card-based hot plugging method and apparatus
CN110134446A (en) * 2019-04-18 2019-08-16 深圳市广和通无线股份有限公司 Start the method for PCIE device scanning
CN111273923A (en) * 2018-12-05 2020-06-12 华为技术有限公司 FPGA (field programmable Gate array) upgrading method based on PCIe (peripheral component interface express) interface
CN111371582A (en) * 2018-12-26 2020-07-03 大唐移动通信设备有限公司 PCIE link fault processing method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140156898A1 (en) * 2012-11-30 2014-06-05 Gang Luo Pci and pci express virtual hot plug systems and methods
CN104820646A (en) * 2015-05-25 2015-08-05 烽火通信科技股份有限公司 PCIE (peripheral component interface express) device dynamic scanning method supporting multi RC (remote control) in Linux system
CN106326167A (en) * 2015-06-17 2017-01-11 中兴通讯股份有限公司 PCIE sub-card-based hot plugging method and apparatus
CN111273923A (en) * 2018-12-05 2020-06-12 华为技术有限公司 FPGA (field programmable Gate array) upgrading method based on PCIe (peripheral component interface express) interface
CN111371582A (en) * 2018-12-26 2020-07-03 大唐移动通信设备有限公司 PCIE link fault processing method and device
CN110134446A (en) * 2019-04-18 2019-08-16 深圳市广和通无线股份有限公司 Start the method for PCIE device scanning

Also Published As

Publication number Publication date
CN114253877B (en) 2024-01-26

Similar Documents

Publication Publication Date Title
US8141093B2 (en) Management of an IOV adapter through a virtual intermediary in an IOV management partition
US8141094B2 (en) Distribution of resources for I/O virtualized (IOV) adapters and management of the adapters through an IOV management partition via user selection of compatible virtual functions
JP5305848B2 (en) Method, data processing system and computer program for managing input / output (I / O) virtualization within a data processing system
JP4157710B2 (en) Method and apparatus for performing logical partitioning of PCI I / O slots
US7707465B2 (en) Routing of shared I/O fabric error messages in a multi-host environment to a master control root node
US7457900B2 (en) Method for discovering and partitioning PCI devices
JP5427574B2 (en) Virtual computer migration management method, computer using the migration management method, virtualization mechanism using the migration management method, and computer system using the migration management method
JP4658122B2 (en) DMA controller, node, data transfer control method, and program
JP3953835B2 (en) Mechanism for safely executing system firmware update on logically partitioned (LPAR) computers
WO2012147116A1 (en) Computer system and virtual machine control method
FI86923C (en) Control mechanism for multiprocessor systems
JP2012203636A (en) Control method for virtual computer and computer
US8141084B2 (en) Managing preemption in a parallel computing system
KR102387922B1 (en) Methods and systems for handling asynchronous event request command in a solid state drive
JP4366336B2 (en) Method for managing trace data in logical partition data processing system, logical partition data processing system for managing trace data, computer program for causing computer to manage trace data, logical partition data Processing system
US8996734B2 (en) I/O virtualization and switching system
CN113821309B (en) Communication method, device, equipment and storage medium between microkernel virtual machines
CN116382913A (en) Resource allocation device, method, electronic device and storage medium
US8230117B2 (en) Techniques for write-after-write ordering in a coherency managed processor system that employs a command pipeline
CN115840621A (en) Interaction method and related device of multi-core system
US8139595B2 (en) Packet transfer in a virtual partitioned environment
US6898731B2 (en) System, method, and computer program product for preventing machine crashes due to hard errors in logically partitioned systems
CN114253877A (en) Scanning method and device for Peripheral Component Interconnect Express (PCIE) equipment under Liunx system
CN111767082A (en) Computing chip starting method and device and computer system
CN115454896A (en) SMBUS-based SSD MCTP control message verification method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant