CN113647058B

CN113647058B - PCIe-based communication method and device

Info

Publication number: CN113647058B
Application number: CN202080002612.8A
Authority: CN
Inventors: 万蕾; 朱杰作; 王学寰; 鲍鹏鑫
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-02-27
Filing date: 2020-02-27
Publication date: 2022-12-30
Anticipated expiration: 2040-02-27
Also published as: EP4099637A1; US20220405229A1; CN113647058A; WO2021168758A1; EP4099637A4

Abstract

A PCIe based communication method and device comprises the following steps: the root complex writes the identity information of the second node into the first node, and writes the routing table information into the third node, wherein the first node is a source node of the first data, the second node is a destination node of the first data, and the third node is a node from the first data to the second node. By writing the identity information of the destination node of the data into the sending node, the sending node can carry the identity information of the destination node of the data when sending the data. By writing the routing table information into the intermediate node, the intermediate node can determine a routing path according to the identity information of the destination node and the routing table information when receiving the data, and forward the data to the destination node based on the routing path. Therefore, the communication between the nodes in the PCIe system does not pass through the root complex, and the direct communication between the nodes is realized, so that the communication complexity between the nodes in the PCIe system can be reduced.

Description

PCIe-based communication method and device

Technical Field

The present application relates to the field of communications technologies, and in particular, to a peripheral component interconnect express (PCIe) -based communication method and apparatus.

Background

The main constituent elements of the PCIe system include a Root complex (Root), a switch node (switch), an end node (endpoint), and the like. The root complex is used for managing all buses and all nodes in the PCIe system and is a bridge for communication between the nodes in the PCIe system; a PCIe system may include one or more switching nodes, a switching node being a forwarding node for data in the PCIe system, a switching node may be connected to one or more of the root complex, other switching nodes, or end nodes; an end node is an end device, such as a peripheral device (peripheral) or the like, that receives data from, or sends data to, other nodes or root complexes.

In a PCIe system, only the root complex and the nodes (e.g., end nodes or switching nodes) themselves have the right to read the configuration space of the node, where the configuration space of a node is used to store information such as function information or address of the node, and if the current node is a storage device, no other node in the PCIe system has the right to read the configuration space of the node. For example, for node 1 of the PCIe system, only the root complex and node 1 have the right to read the configuration space of node 1, and for nodes other than node 1 in the PCIe system, no right to read the configuration space of node 1 is provided. This means that only the root complex in a PCIe system can know information such as the function or address of each node. Therefore, direct communication cannot be achieved between end nodes and end nodes, between switching nodes and switching nodes, and between end nodes and switching nodes, and it is necessary to pass through the root complex. Since the communication between the nodes in the PCIe system must pass through the root complex, the communication complexity between the nodes is greatly increased.

Disclosure of Invention

The embodiment of the application provides a PCIe-based communication method and a PCIe-based communication device, which are used for enabling communication between nodes in a PCIe system to not pass through a root complex, so that the communication complexity between the nodes can be reduced.

In a first aspect, an embodiment of the present application provides a PCIe-based communication method, including: and the root complex writes the identity information of the second node into the first node and writes the routing table information into the third node, wherein the first node is a source node of the first data, the second node is a destination node of the first data, and the third node is a node from the first data to the second node.

In one possible design, the method of the first aspect may be performed by the root complex itself, or may be performed by a chip in the root complex, etc.

In the embodiment of the present application, by writing the identity information of the destination node of the data into the sending node (for example, the first node), the sending node may carry the identity information of the destination node of the data when sending the data. By writing the routing table information to the intermediate node (e.g., the third node), the intermediate node, upon receiving the data, may determine a routing path based on the identity information of the destination node and the routing table information, and may send the data to the destination node (e.g., the second node) based on the routing path. Therefore, the communication among the end nodes, the communication among the switching nodes, the communication between the end nodes and the switching nodes and the like in the PCIe system can realize the direct communication among the nodes without passing through the root complex, thereby reducing the communication complexity among the nodes in the PCIe system.

In one possible design, the root complex determines that the first node supports the first mode of operation; or determining that the first node supports a first working mode and a second working mode; the first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex. In the design, when the first node only supports the first working mode, the communication between the nodes can be realized without passing through the root complex, so that the communication complexity of each node in the PCIe system is reduced; when the first node supports both the first working mode and the second working mode, the communication between the nodes can be realized without passing through the root complex, so that the communication complexity of each node in the PCIe system is reduced, the communication can be carried out according to the current communication mode, and the compatibility is realized.

In a possible design, when the first node is in the second working mode, the root complex configures a first memory address for the first node, where the first memory address is a physical address in an operating memory corresponding to a Central Processing Unit (CPU) to which a local memory space of the first node is mapped. In the above design, the first node can communicate according to the current communication mode by configuring the first memory address for the first node, so that compatibility between two working modes is achieved.

In one possible design, the root complex determines that the third node supports the first mode of operation; or determining that the third node supports the first working mode and the second working mode; the first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex. In the above design, when the third node only supports the first working mode, the communication between the nodes does not pass through the root complex, thereby reducing the communication complexity of each node in the PCIe system; when the third node supports both the first working mode and the second working mode, the communication between the nodes can be realized without passing through the root complex, so that the communication complexity of each node in the PCIe system is reduced, the communication can be carried out according to the current communication mode, and the compatibility is realized.

In one possible design, when the third node is in the second working mode, the root complex configures a second memory address for the third node, where the second memory address is a physical address in an operating memory corresponding to the CPU mapped to a local memory space of the third node. In the above design, the third node can communicate according to the current communication mode by configuring the second memory address for the third node, so that the compatibility of the two working modes is realized.

In one possible design, the root complex writes identity information of the first node to the first node. In the above design, by writing the identity information of the first node into the first node, the identity information of the first node may be sent to the destination node of the data, so that the destination node confirms the information of the source node of the data; the received data may also be verified to determine whether the received data is data sent to the node.

In one possible design, the root complex writes the identity information of the third node to the third node. In the above design, by writing the identity information of the third node into the third node, the identity information of the third node may be sent to the destination node of the data, so that the destination node confirms the information of the source node of the data; the received data may also be verified to determine whether the received data is data sent to the node.

In one possible design, the identity information is a bus number, a device and function number (BDF), or an Identity (ID) number. In the above design, the identity information may be a unique BDF assigned by the root complex to the node, or a unique ID number reassigned by the root complex to the node, so that the identity information of each node has uniqueness.

In a second aspect, an embodiment of the present application provides a PCIe-based communication method, including: the first node determines identity information of the second node according to information stored in the first node, and sends a first Transaction Layer Packet (TLP) to the third node, where the second node is a destination node of the first data, and the first TLP includes the first data and the identity information of the second node.

In one possible design, the first node may be an end node or a bridge node, the second node may be an end node or a bridge node, and the third node may be a bridge node.

In the embodiment of the application, a sending node (e.g. a first node) determines identity information of a receiving node (e.g. a second node) through stored information, and then encapsulates the first data and the identity information of the receiving node together and sends the encapsulated first data and the encapsulated identity information to an intermediate node (e.g. a third node), so that the intermediate node forwards the first data to the receiving node. Therefore, communication among end nodes, communication among switching nodes, communication between the end nodes and the switching nodes and the like in the PCIe system can be realized without passing through a root complex, and the communication complexity among the nodes in the PCIe system can be reduced.

In one possible design, the stored information may be configured by the root complex, or other network node.

In one possible design, the first node supports a first mode of operation, which is a mode of operation in which communication between nodes does not pass through the root complex. In the above design, the first node supports the first operating mode, and it may be implemented that communication between nodes does not pass through the root complex, thereby reducing the communication complexity of each node in the PCIe system.

In one possible design, the first node supports a first operating mode in which communication between the nodes does not pass through the root complex, and a second operating mode in which communication between the nodes needs to pass through the root complex. In the above design, the first node supports both the first operating mode and the second operating mode, so that communication between nodes does not pass through the root complex, thereby reducing the communication complexity of each node in the PCIe system, and communication can be performed according to the current communication mode, thereby achieving compatibility of the two operating modes.

In one possible design, the stored information further includes identity information of the first node. In the above design, the information stored in the first node includes identity information of the first node, and the identity information of the first node may be sent to a destination node of the data, so that the destination node confirms information of a source node of the data; the received data may also be verified to determine whether the received data is data sent to the node.

In one possible design, the first TLP further includes identity information of the first node. In the above design, the first data and the identity information of the first node are encapsulated as the first TLP and sent to the destination node, so that the destination node can confirm the information of the source node of the data.

In one possible design, the identity information is a BDF, or an ID number. In the above design, the identity information may be a unique BDF assigned by the root complex to the node, or a unique ID number reassigned by the root complex to the node, so that the identity information of each node has uniqueness.

In a third aspect, an embodiment of the present application provides a PCIe-based communication apparatus, which may be a communication device, and may also be a chip or a chipset within the communication device, where the communication device may be any one of a root complex or a first node, that is, the communication device may be either the root complex or an end node or a switch node. The apparatus may include a processing unit. When the apparatus is a communication device, the processing unit may be a processor; the apparatus may further include a transceiver module and a storage module, which may be a memory; the storage module is configured to store an instruction, and the processing unit executes the instruction stored in the storage module to enable the root complex to perform a corresponding function in the first aspect, or executes the instruction stored in the storage module to enable the first node to perform a corresponding function in the second aspect. When the apparatus is a chip or a chip set in a communication device, the processing unit may be a processor, and the transceiving unit may be an input/output interface, a pin, a circuit, or the like; the processing unit executes instructions stored by the storage module to cause the root complex to perform the corresponding functions in the first aspect, or the processing unit executes instructions stored by the storage module to cause the first node to perform the corresponding functions in the second aspect. The memory module may be a memory module (e.g., register, cache, etc.) within the chip or chipset, or may be a memory module (e.g., read-only memory, random access memory, etc.) external to the chip or chipset within the network device.

In a fourth aspect, an embodiment of the present application provides a PCIe-based communication apparatus, including: the processor may also include a communication interface and/or memory. The communication interface is used for transmitting information, and/or messages, and/or data between the device and other devices. The memory is configured to store computer executable instructions, and when the apparatus is running, the processor executes the computer executable instructions stored by the memory to cause the apparatus to perform the communication method as set forth in the first aspect or any of the designs of the first aspect, the second aspect or any of the second aspects.

In a fifth aspect, embodiments of the present application further provide a computer-readable storage medium for storing computer instructions, which, when executed on a computer, cause the computer to perform a communication method according to the first aspect or any one of the possible designs of the first aspect, the second aspect, or any one of the possible designs of the second aspect.

In a sixth aspect, the present application further provides a computer program product comprising instructions for storing computer instructions which, when executed on a computer, cause the computer to perform the communication method of any one of the above first aspect or any one of the possible designs of the first aspect, the second aspect or any one of the possible designs of the second aspect.

In a seventh aspect, the present application further provides a PCIe system, including a first node, a second node, and a third node, for example, the first node is an end node, the second node is a switch node, and the third node is an end node, where the first node may perform the corresponding function in the first aspect, and the second node may perform the corresponding function in the second aspect.

In an eighth aspect, a chip provided in an embodiment of the present application includes at least one processor and a communication interface, where the processor is coupled with a memory and configured to read a computer program stored in the memory to perform the communication method of any one of the first aspect or the first aspect, the second aspect, or the second aspect of the embodiments of the present application.

In a ninth aspect, an embodiment of the present application provides a chip, which includes a communication interface and at least one processor, where the processor is operative to execute a communication method according to any one of the possible designs of the first aspect or the first aspect, and any one of the possible designs of the second aspect or the second aspect of the embodiment of the present application.

Drawings

Fig. 1 is a schematic architecture diagram of a PCIe system according to an embodiment of the present application;

FIG. 2 is a block diagram of an alternative PCIe system architecture provided in the embodiments of the present application;

fig. 3 is a schematic flow chart of BDF allocation according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a TLP head according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of another TLP head provided by the embodiment of the application;

fig. 6 is a schematic structural diagram of another TLP head provided by the embodiment of the application;

FIG. 7 is a schematic structural diagram of another TLP head provided in this embodiment of the present application;

fig. 8 is a schematic structural diagram of another TLP head provided by the embodiment of the application;

FIG. 9 is a schematic structural diagram of another TLP head provided in this embodiment of the present application;

FIG. 10 is a flowchart illustrating a PCIe based communication method according to an embodiment of the present application;

FIG. 11 is a flowchart illustrating another PCIe based communication method according to an embodiment of the present application;

FIG. 12 is a flowchart illustrating another PCIe based communication method according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of a PCIe-based communication device according to an embodiment of the present application;

FIG. 14 is a schematic structural diagram of another PCIe based communication device according to an embodiment of the present application;

FIG. 15 is a schematic block diagram of another PCIe based communication device provided in the embodiment of the present application;

FIG. 16 is a schematic structural diagram of another PCIe based communication device according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of another TLP head provided by the embodiment of the application;

fig. 18 is a schematic structural diagram of another TLP head provided by the embodiment of the application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

In the present embodiment, "a plurality" means two or more, and in view of this, a plurality may also be understood as "at least two". "at least one" is to be understood as meaning one or more, for example one, two or more. For example, including at least one means including one, two, or more, and does not limit which ones are included, for example, including at least one of a, B, and C, then including may be a, B, C, a and B, a and C, B and C, or a and B and C. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" generally indicates that the preceding and following related objects are in an "or" relationship, unless otherwise specified.

Unless stated to the contrary, the embodiments of the present application refer to the ordinal numbers "first", "second", etc., for distinguishing between a plurality of objects, and do not limit the sequence, timing, priority, or importance of the plurality of objects.

For the convenience of the skilled person, the PCIe system according to the embodiments of the application will be described with reference to the drawings.

PCIe is a high-speed short-distance communication interface, and is widely applied to devices such as computers and test instruments. PCIe can read and write the memory quickly, support large-bandwidth communication, and some vehicle enterprises lengthen the transmission distance thereof and can be used as a large-data-volume communication interface in the vehicle. Fig. 1 is a schematic structural diagram of a PCIe system according to an embodiment of the present application. As shown in FIG. 1, a PCIe system may include a root complex (shown in FIG. 1 as the root complex), at least one switch node (FIG. 1 includes switch node 1), and at least one end node (FIG. 1 includes end node 1, end node 2, and end node 3).

The root complex is responsible for managing all buses (shown by bold black lines in fig. 1), switching nodes, and end nodes in the PCIe system 100, and is a bridge for communications between nodes in the PCIe system 100, and is also a bridge for communications between nodes and CPUs in the PCIe system 100. The switching nodes, which are one or more of a bridge connection root complex, other switching nodes, and end nodes, are nodes for data forwarding, e.g., switching node 1 connects the root complex, end node 1, and end node 2. An end node is an end device, such as a peripheral device, responsible for sending data or receiving data.

Note that the interface between the CPU and the root complex is not PCIe and does not belong to the PCIe system 100. The CPU may establish a connection relationship with an entity (e.g., a computer device, a vehicle, etc.) where the PCIe system 100 is located, may also be located in the same entity as the PCIe system 100, and is configured to receive data sent by each node in the PCIe system 100, and may also process the received data, for example, perform format conversion, image rendering, and the like.

A device (e.g., root complex, switch node 1, end node 2, or end node 3) in the PCIe system 100 can support up to 8 functions (functions), e.g., audio, video, etc. Wherein the bridge node is used for connecting other devices. When a device supports multiple functions, each function of the device has its own configuration space, and the configuration space stores information of the function. The configuration space may be a separate piece of storage unit of the device, for example the size of the configuration space may be 256k. The root complex has the authority to read and write configuration spaces for each device in the PCIe system 100. For example, the root complex may obtain information of the functions supported by the end node 1 by reading information in the configuration space of the end node 1. As another example, the root complex may also complete initialization and configuration of the end node 1 by writing the configuration space of the end node 1. Other devices and functions of the devices except the root complex can only see the information of the self configuration space.

Please refer to fig. 2, which is a schematic diagram of another structure of the PCIe system 100. As shown, the functions supported by the root complex include a host bridge, a bridge 1, and a bridge 2, and the host bridge, the bridge 1, and the bridge 2 are connected to each other via a bus (shown in FIG. 2 as a bold black line). The functions supported by the switching node 1 include a bridge node 3, a bridge node 4 and a bridge node 5, and the bridge node 3, the bridge node 4 and the bridge node 5 are connected with each other through a bus. The main bridge node is used for establishing a connection relation with a CPU, the bridge node 1 establishes a connection relation with a bridge node 3 in the switching node 1 through a bus, the bridge node 2 establishes a connection relation with an end node 3 through a bus, the bridge node 4 establishes a connection relation with the end node 1 through a bus, and the bridge node 5 establishes a connection relation with the end node 2 through a bus.

When the PCIe system 100 is initialized, the root complex needs to perform enumeration traversal on the buses, devices, and functions in the devices in the PCIe system 100 to complete initialization and configuration of the devices. Referring to fig. 3, a flow chart of BDF assignment for root complexes is shown.

S1: the root complex sets the Bus connected to the host bridge node to Bus 0.

S2: the root complex discovers bridge node 1 via Bus 0, assigns a Bus number, a device number, and a function number (BDF) to bridge node 1, and sets the downstream Bus of bridge node 1 to Bus 1.

The root complex traverses the nodes connected with the Bus 0, firstly finds the bridge node 1, then traverses the nodes connected with the downstream Bus of the bridge node 1 according to the depth priority rule, and sets the Bus as the Bus 1. While the root complex assigns a BDF to bridge node 1. For a switching node (or bridge node), the bus number of the switching node (or bridge node) includes the upstream bus number of the switching node (or bridge node), the downstream bus number of the switching node (or bridge node), and the maximum bus number of the branch in which the switching node (or bridge node) is located. The upstream Bus of the bridge node 1 is Bus 0, and the downstream Bus number of the bridge node 1 is Bus 1. Since the branch of bridge node 1 has not been traversed, and the maximum bus number of the branch of bridge node 1 cannot be determined, the maximum bus number of the branch of bridge node 1 may be temporarily set to 0xFF, and the bus number of bridge node 1 may be 0,1,0xFF.

It should be noted that, for a switching node (or a bridge node), after the BDF is allocated to the switching node (or the bridge node), the root complex writes the bus number (bus number) in the BDF into the switching node (or the bridge node), so that the switching node (or the bridge node) performs ID routing according to the bus number. For example, the root complex may write a bus number to the configuration space of the switch node (or bridge node). For another example, after the switching node (or bridge node) receives the data, the root complex may determine a next-hop node to forward the data according to the bus number and the bus number of the destination node. The specific implementation of routing according to the bus number may refer to the prior art, and is not described herein.

S3: the root complex discovers bridge node 3 by Bus 1, assigns BDF to bridge node 3, and sets the Bus downstream of bridge node 3 to Bus 2.

The root complex traverses nodes connected to Bus 1, finds bridge node 3, then traverses nodes connected to a downstream Bus of bridge node 3 according to a depth-first rule, and sets the Bus to Bus 2. While the root complex assigns a BDF to bridge node 3, where bridge node 3 may have a bus number of 1,2,0xFF.

S4: the root complex discovers bridge node 4 via Bus 2, assigns a BDF to bridge node 4, and sets the Bus downstream of bridge node 4 to Bus 3.

The root complex traverses nodes connected to Bus 2, first discovers bridge node 4, then traverses nodes connected to buses downstream of bridge node 4 according to a depth-first rule, and sets the Bus to Bus 3. While the root complex assigns a BDF to bridge node 4, where bridge node 4 may have a bus number of 2,3,0xFF.

S5: the root complex discovers end node 1 by Bus 3 and assigns BDF to end node 1.

The root complex traverses the nodes connected to Bus 3, discovers end node 1, and then the root complex assigns BDF to end node 1. For an end node (or end node's function) the bus number of the end node (or end node's function) comprises the upstream bus number of the end node (or end node's function). The upstream Bus of end node 1 is Bus 3, so the Bus number of end node 1 may be 3.

It should be noted that, for an end node (or a function of an end node), after the end node (or the function of the end node) is allocated with the BDF, the root complex does not write the BDF of the end node (or the function of the end node) into the end node (or the function of the end node). The BDFs of the end nodes (or functions of end nodes) are managed by the root complex, and the end nodes (or functions of end nodes) are not aware of their own BDF.

At this point, the branch where the bridge node 4 is located is traversed. The root complex determines that the maximum bus number of the branch where the bridge node 4 is located is 3, and updates the bus numbers of the nodes in the branch where the bridge node 4 is located. After the update, the bus number of the bridge node 4 is 2,3,3; the bus number of end node 1 is 3.

S6: the root complex discovers bridge node 5 via Bus 2, assigns BDF to bridge node 5, and sets the Bus downstream of bridge node 5 to Bus 4.

After traversing the branch where the bridge node 4 is located, the root complex discovers the bridge node 5 through the Bus 2, then traverses the nodes connected with the downstream buses of the bridge node 5 according to the depth-first rule, and sets the buses as the Bus 4. While the root complex assigns BDF to bridge node 5, where bridge node 5 may have a bus number of 2,4,0xFF.

S7: the root complex discovers end node 2 by Bus 4 and assigns BDF to end node 2.

The root complex traverses the nodes connected to Bus 4, discovers end node 2, and then the root complex assigns BDF to end node 2, where the Bus number of end node 2 may be 4.

At this point, after the traversal of the branch where the bridge node 5 is located is completed, the root complex determines that the maximum bus number of the branch where the bridge node 5 is located is 4, and updates the bus numbers of each node in the branch where the bridge node 5 is located. After the update, the bus number of bridge node 1 is 0,1,4; the bus number of the bridge node 3 is 1,2,4; the bus number of the bridge node 5 is 2,4,4; the bus number of end node 2 is 4.

S8: the root complex discovers bridge node 2 via Bus 0, assigns BDF to bridge node 2, and sets the downstream Bus with bridge node 2 to Bus 5.

After traversing the branch where the bridge node 1 is located, the root complex discovers the bridge node 2 through the Bus 0, then traverses the nodes connected with the downstream buses of the bridge node 2 according to the depth-first rule, and sets the buses as Bus 5. While the root complex assigns a BDF to bridge node 2, where bridge node 2 may have a bus number of 0,5,0xFF.

S9: the root complex discovers end node 2 by Bus 5 and assigns BDF to end node 2.

The root complex traverses the nodes connected to Bus 5, discovers end node 2, and then the root complex assigns BDF to end node 2, where the end node's Bus number may be 5.

At this point, the branch where the bridge node 2 is located is traversed completely. And the root complex determines that the maximum bus number of the branch where the bridge node 2 is located is 5, and updates the bus number of each node in the branch where the bridge node 2 is located. After the update, the bus number of bridge node 2 is 0,5,5; the bus number of end node 2 is 5.

Through the process illustrated in fig. 3, the root complex may assign BDFs for all nodes in the PCIe system 100 and write the bus numbers of the switch nodes (or bridge nodes) to the switch nodes (or bridge nodes) for ID routing.

In the PCIe system, data is transmitted between nodes through a PCIe interface, and a packet for transmitting data is referred to as a Transaction Layer Packet (TLP) in the PCIe protocol. The PCIe system supports three routing modes for communication: address routing, identity (ID) routing, and implicit routing. Different routing modes, the routing address filled in the TLP Header (TLP Header) is different. When the length of the TLP header is 12 bytes (Byte), the fields associated with the routing are Byte8 through Byte11, as shown in fig. 4. When the length of the TLP header is 16 bytes, the fields related to the routing are Byte8 to Byte15, as shown in fig. 5.

Three routing modes supported by the PCIe system are described below.

1) Address routing refers to a manner of routing according to a memory address. The memory address is a physical address in the local memory space of the node mapped to the running memory corresponding to the CPU. The memory address includes a 32-bit memory address and a 64-bit memory address. Different memory addresses require the use of different TLP headers. For example, a TLP header with a length of 12 bytes is required for the 32-bit memory address, and Byte8 to Byte11 in the TLP header are used for indicating the 32-bit memory address, as shown in fig. 6. For another example, a TLP header with a length of 16 bytes is required to be used for the 64-bit memory address, the bytes 8 to 11 in the TLP header are used for indicating the upper 32-bit memory address, and the bytes 12 to 15 in the TLP header are used for indicating the lower 32-bit memory address, as shown in fig. 7.

The memory addresses of the nodes in the PCIe system are configured by the root complex and written into registers (e.g., bar) of each node. The node in the PCIe system has a local memory space, which is open to the CPU, but the CPU can only directly access the running memory corresponding to the CPU, but cannot directly access the local memory space of the node. Therefore, the root complex can map the local memory space of the node to the running memory corresponding to the CPU to obtain the memory address, so that the root complex can access the local memory space of the node according to the memory address, that is, the communication between the root complex and the node is realized.

When address routing is adopted, the routing table information of the switching node (or bridge node) is the operating memory range corresponding to the CPU, the operating memory range corresponding to the CPU covers the memory addresses of all nodes hung under the switching node (or bridge node), the memory address of the hung node is only known by the hung node itself and the root complex, the mapping relationship between the local memory space of the hung node and the operating memory corresponding to the CPU is also only known by the root complex, and the switching node (or bridge node) is unclear. Thus, when data is transmitted between two nodes via a switching node (or bridge node), the switching node (or bridge node) does not know to which node to forward the data to, and therefore communication between two nodes that need to pass through the switching node (or bridge node) must pass through the root complex. Thus, the address routing only supports data being sent by the root complex to the nodes or data being sent by the nodes to the root complex. While for node-to-node communication, the root complex must be traversed.

2) ID routing refers to the manner in which routing is performed according to the BDF. The ID routing scheme includes two TLP headers. For example, a TLP header with a length of 12 bytes may be used, and part of bytes from Byte8 to Byte11 in the TLP header are used to indicate the BDF, as shown in fig. 8. For another example, a TLP header with a length of 16 bytes may be used, and part of bytes in Byte8 to Byte11 in the TLP header are used to indicate the BDF, as shown in fig. 9. The BDF of each node is configured by the root complex for the node, and the BDF of the node is only known by the root complex and is unknown by the node itself or other nodes. This ID routing approach is typically used only for the root complex sending configuration messages to the nodes, and for the nodes sending configuration response messages to the root complex. Thus, ID routing supports only communication from the root complex to the nodes, or from the nodes to the root complex. And for the communication between nodes, the ID routing mode is not supported.

3) Implicit routing refers to other routing schemes that do not use address routing or ID routing. The implicit route is used to send data to the root complex, and the TLP sent by the nodes is sent by default and forwarded to the root complex by default. Thus, implicit routing is used only for communication from the node to the root complex.

At present, three routing modes supported by a PCIe system can not directly communicate between end nodes and end nodes, between switching nodes and between the end nodes and the switching nodes. If two nodes are to communicate, the root complex must be traversed. The working mechanism that the nodes in the PCIe system must communicate with each other through the root complex causes the complexity of communication between the nodes in the PCIe system to be greatly increased. As in fig. 1 or fig. 2, although the end node 1 and the end node 2 are connected to the switching node 1, but the two cannot communicate directly through the switching node 1, the end node 1 must first communicate with the root complex to find the end node 2 via the root complex, or the end node 2 must first communicate with the root complex to find the end node 1 via the root complex. For example, the end node 1 sends data to the end node 2, and the specific process may be: the end node 1 sends the data to the switching node 1, the switching node 1 sends the data to the root complex, which in turn sends the data to the end node 2 via the switching node 1.

In view of this, embodiments of the present disclosure provide a PCIe-based communication method and apparatus, which are used to enable communication between nodes in a PCIe system to not pass through a root complex, so as to reduce the communication complexity between nodes in the PCIe system.

Referring to fig. 10, a flowchart of a PCIe-based communication method provided in the embodiment of the present application is shown, and the method may be applied to a PCIe system in an intranet, particularly an intranet of an autonomous vehicle, and may also be applied to PCIe systems of other devices. For example, the method may be applied to the PCIe system 100 shown in fig. 1 or fig. 2. The method is described below by way of example as applied to a root complex in a PCIe system 100.

S1001: the root complex reads the information stored in the first node.

In particular, the root complex may read the information stored in the first node when the PCIe system 100 is initialized, or may read the information stored in the first node when the first node detects a need to send the first data. The stored information may be configured by the root complex, or configured by other nodes, or configured manually, etc., which is not limited in this embodiment.

The first data is data sent by the first node to the second node, that is, the first node is a source node of the first data. The first node may be an end node or a function of an end node in the PCIe system 100, or may be a switch node or a function in a switch node in the PCIe system 100. For example, the first node may be an end node 1 or a bridge node 4. The information stored in the first node may be information of a configuration space of the first node. The configuration space of a node may be a segment of physical memory addresses for storing configuration information.

The root complex reads the information stored in the first node from which the operating mode supported by the first node can be determined. For example, the reserved space of the configuration space of the first node may store indication information indicating the operation mode supported by the first node. The root complex may read the reserved space in the configuration space of the first node to determine the operating mode supported by the first node based on the read indication information. The reserved space is an unused storage address in the configuration space. The working modes supported by the first node comprise a first working mode and a second working mode, the first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex.

Further, the root complex determines the operational modes supported by the first node based on information stored in the first node. If the root complex determines that the first node supports only the first mode of operation, then only S1002 is performed; if the root complex determines that the first node only supports the second operating mode, executing S1003-S1004; if the root complex determines that the first node supports the first operation mode and the second operation mode, S1002 to S1004 are performed.

S1002: upon determining that the first node supports the first mode of operation, the root complex writes identity information of the second node to the first node.

After the root complex determines that the first node supports the first working mode, the root complex writes the identity information of a destination node (for example, called as a second node) corresponding to the first node into the first node. Specifically, the root complex may first determine a destination node corresponding to the first node, and then the root complex may write the identity information of the destination node corresponding to the first node into the first node. For example, the root complex may determine the destination node of the first data based on the type of the first data (e.g., video type or image type, etc.) and the functionality of the nodes in the PCIe system 100. For example, the first node is end node 1, the type of the first data is a video type, the end node 2 is a display, the end node 3 is a memory, and the root complex can determine that the destination node for the first data is end node 2. The root complex writes the identity information of the second node into the first node, for example, the identity information of the second node is written into information stored in the first node, or the identity information of the second node is written into a variable in the first node, or the identity information of the second node is written into a parameter in the first node, etc. For example, the root complex writes the identity information of the second node into the reserved space of the configuration space of the first node.

Wherein the second node is a destination node of the first data. The second node may be an end node or a function of an end node in the PCIe system 100, or may be a switch node or a function in a switch node in the PCIe system 100. The identity information of each node in the PCIe system 100 has uniqueness. The identity information of the node is assigned by the root complex, for example, one node identity information may be the BDF of the node assigned by the root complex, or the ID of the node assigned by the root complex, etc. For a specific allocation process, reference may be made to the flow shown in fig. 3, which is not described herein again.

In a specific implementation, if the root complex determines that there are multiple destination nodes corresponding to the first node, the root complex may write the identity information of each of the multiple destination nodes into the first node. For example, the first node is end node 1, the type of the first data is a video type, both end node 2 and end node 3 are displays, the root complex determines that both end node 2 and end node 3 can display the first data, and thus the root complex determines that both end node 2 and end node 3 are destination nodes for end node 1. The root complex may write both the end node 2 identity information and the end node 3 identity information to the end node 1.

In one possible implementation, the root complex may write identity information of the first node to the first node. In this way, when the first node sends the first data, the identity information of the first node, and the identity information of the second node may be encapsulated together as a TLP, so that the TLP may carry a source address, and the destination node may determine, according to the source address, that the sending node of the TLP is the first node. The first node may also determine whether to discard the received TLP according to whether a destination address carried in the received TLP is consistent with the identity information of the first node. For example, if the destination address carried by the TLP is consistent with the identity information of the first node, the first node determines that the TLP is sent to the first node; if the destination address carried by the TLP is inconsistent with the identity information of the first node, the first node determines that the TLP is not sent to the first node, and may discard the TLP.

The root complex writes the identity information of the first node into the first node, for example, the identity information of the first node is written into information stored in the first node, or the identity information of the first node is written into a variable in the first node, or the identity information of the first node is written into a parameter in the first node, and the like. For example, the root complex writes the identity information of the first node into a reserved space of the configuration space of the first node.

S1003: and when the first node is determined to support the second working mode, the root complex allocates a first memory address for the first node.

The root complex maps the local memory space of the first node to the running memory corresponding to the CPU to obtain a first memory address, wherein the first memory address refers to a physical address of the local memory space of the first node mapped to the running memory corresponding to the CPU. For example, the size of the local memory space of the first node is 2MB, the address of the local memory space is 0x0000 to 0x007f, the size of the operating memory corresponding to the CPU is 2GB (GigaByte), the address of the operating memory is 0x0000 to 0 xfffff, and the address of the operating memory mapped to the local memory space of the first node to the CPU is 0x0100 to 0x017F, that is, the first memory address is 0x0100 to 0x017F.

S1004: the root complex writes the first memory address to the first node.

The root complex writes the first memory address into the first node, for example, writes the first memory address into a register of the first node, writes the first memory address into information stored in the first node, writes the first memory address into a variable in the first node, writes the first memory address into a parameter in the first node, or the like.

The flow shown in fig. 10 may be implemented by the root complex, may be implemented by a chip in the root complex, or may be implemented by another device, for example, by a CPU. The embodiments of the present application do not limit this.

In an embodiment of the present application, the root complex writes the identity information of the destination node (e.g., the second node) to the sending node (e.g., the first node). When the sending node sends the first data to the destination node, the first data may be encapsulated together with the identity information of the destination node as a TLP to be sent to an intermediate node (e.g., a third node). The intermediate node may determine a destination address of the received TLP by decapsulating the TLP, so that forwarding may be performed according to the destination address of the TLP without going through the root complex. Therefore, communication among end nodes, communication among switching nodes, communication between the end nodes and the switching nodes and the like in the PCIe system do not pass through the root complex, direct communication among the nodes is achieved, and accordingly communication complexity among the nodes in the PCIe system can be reduced.

Furthermore, the root complex writes the identity information of the destination node into the first node and allocates the first memory address for the first node, so that the first node supports both the first working mode and the second working mode, and the compatibility of the two working modes is realized. When the two nodes communicate, the data transmission link passing through the root complex can be selected, and the data transmission link not passing through the root complex can also be selected, so that the redundancy backup of the data transmission link in the PCIe system is realized, and the stability of the data transmission in the PCIe system is improved. For example, if the root complex fails, a node may choose to communicate directly with other nodes without going through the root complex's data transport links.

Through the process shown in fig. 10, the root complex may write the identity information of the destination node corresponding to the node in the PCIe system into the node, so that the intermediate node may determine the destination address of the received TLP. In order to implement direct communication between nodes, an embodiment of the present application provides another PCIe-based communication method, and with this method, an intermediate node may forward a received TLP according to a destination address of the TLP in a manner that does not pass through a root complex, so as to implement direct communication between nodes. Please refer to fig. 11, which is a flowchart of a PCIe-based communication method provided in the embodiment of the present application, the method may be applied to a PCIe system in an intranet, particularly an intranet of an autonomous vehicle, and may also be applied to a PCIe system of other devices. For example, the method may be applied to the PCIe system 100 shown in fig. 1 or fig. 2. The method is described below by way of example as applied to a root complex in a PCIe system 100.

S1101: the root complex reads the information stored in the third node.

In a specific implementation, the root complex may be configured to read information stored in the third node when the PCIe system 100 is initialized, or may also read information stored in the third node when it is detected that the third node has a need to forward the first data, which is not limited in this embodiment of the present application.

Wherein the third node is the node through which the first data passes to the destination node. The third node may be a switching node or a function in a switching node in the PCIe system 100. For example, the third node may be the bridge node 4 or the switching node 1. The information stored in the third node may be information of a configuration space of the third node.

The root complex reads the information stored in the third node, and according to the stored information, the working mode supported by the third node can be determined. For example, the reserved space of the configuration space of the third node may store indication information indicating the operation mode supported by the third node. The root complex may read the reserved space of the configuration space of the third node to determine the operating mode supported by the third node according to the read indication information. The reserved space is an unused storage address in the configuration space, the working modes supported by the third node include a first working mode and a second working mode, the first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex.

The third node supports the first working mode, wherein the third node supports only the first working mode, or the third node supports both the first working mode and the second working mode.

When the third node supports the first working mode and the second working mode, the root complex may allocate a second memory address to the third node and write the second memory address into the third node. The second memory address refers to a physical address of the local memory space of the first node mapped to the running memory corresponding to the CPU. For example, the root complex maps the local memory space of the third node to the running memory corresponding to the CPU, and obtains the second memory address. The root complex writes the second memory address into the third node, for example, writes the second memory address into a register of the third node, or writes the second memory address into information stored in the third node, or writes the second memory address into a variable in the third node, or writes the second memory address into a parameter in the third node, or the like.

In a possible implementation manner, when it is determined that the third node supports the first working mode, the root complex writes the identity information of the destination node corresponding to the third node into the third node. For example, the root complex may determine a destination node corresponding to the third node according to the function of each node in the PCIe system 100, and then write the identity information of the destination node corresponding to the third node into the third node. If it is determined that the third node corresponds to a plurality of destination nodes, the root complex may write the identity information of each of the plurality of destination nodes to the third node. The root complex writes the identity information of the destination node corresponding to the third node into the third node, for example, the identity information of the destination node corresponding to the third node is written into information stored in the third node, or the identity information of the destination node corresponding to the third node is written into a variable in the third node, or the identity information of the destination node corresponding to the third node is written into a parameter in the third node, or the like. For example, the root complex writes the identity information of the destination node corresponding to the third node into the reserved space of the configuration space of the third node.

In another possible embodiment, upon determining that the third node supports the first mode of operation, the root complex writes the third node's identity information to the third node. In this way, the third node may carry the identity information of the third node in the sent or forwarded TLP, so that the destination node determines that the sending node of the TLP is the third node according to the source address. The third node may also determine whether to discard the received TLP according to whether the destination address carried by the received TLP is consistent with the identity information of the third node. For example, if the destination address carried by the received TLP is consistent with the identity information of the third node, it is determined that the received TLP is sent to the third node; if the destination address carried by the received TLP is inconsistent with the identity information of the third node, determining that the received TLP is not sent to the third node, and discarding the received TLP. The root complex writes the identity information of the third node into the third node, for example, the identity information of the third node is written into information stored in the third node, or the identity information of the third node is written into a variable in the third node, or the identity information of the third node is written into a parameter in the third node, and the like. For example, the root complex writes the identity information of the third node into the reserved space of the configuration space of the third node.

Further, the root complex determines the working mode supported by the third node according to the information stored in the third node. If the root complex determines that the third node supports the first working mode, executing S1102-S1103; if the root complex determines that the third node does not support the first mode of operation, the process ends.

S1102: upon determining that the third node supports the first mode of operation, the root complex determines routing table information.

The root complex may determine the routing table information of the third node based on the identity information or the function information of each node in the PCIe system 100. The routing table information includes identity information of the second node and identity information of a next hop node to reach the second node.

If the identity information allocated by the root complex for each node is BDF, the routing table information of the third node may be a bus number of the third node. After receiving the TLP, the third node determines a next-hop node of the TLP according to the bus number of the third node and the bus number of the destination node. For example, if the bus number of the destination node is greater than the maximum bus number in the branch where the third node is located, the third node discards the TLP. For another example, if the bus number of the destination node is greater than or equal to the downstream bus number of the third node and less than or equal to the maximum bus number in the branch where the third node is located, the third node determines that the next-hop node of the TLP is the node hung down by the downstream bus of the third node.

If the root complex sequentially assigns identity information to each node and the assigned identity information is an ID number, the routing table information of the third node may be an ID number range covering the ID numbers of all nodes hung by the third node. For example, the root complex assigns identity information to each node in order according to the depth-first principle. After receiving the TLP, the third node may determine the next-hop node of the TLP according to whether the ID number of the destination node is within the ID range. For example, if the ID number of the destination node is not within the ID range, the third node discards the TLP; if the ID number of the destination node is within the ID range, the third node determines that the next hop node of the TLP is the node hung by the third node.

If the root complex randomly allocates identity information to each node, and the allocated identity information is an ID number, the routing table information of the third node may include an ID number of at least one node hung down by the third node, an ID number of at least one destination node corresponding to each node in the at least one node, and an ID number of a next hop node to reach the at least one destination node. Taking the middle node as a bridge node 4 as an example, the node hung below the bridge node 4 is an end node 1, the destination node corresponding to the end node 1 comprises an end node 2 and an end node 3, the next hop node for the data to reach the end node 2 from the bridge node 4 is a bridge node 5, and the next hop node for the data to reach the end node 3 from the bridge node 4 is a bridge node 3. The routing table information for bridge node 4 includes the ID number of end node 1, the ID number of end node 2, the ID number of bridge node 5, the ID number of end node 3, and the ID number of bridge node 3.

S1103: the root complex writes the routing table information to the third node.

The root complex writes the routing table information into the third node, for example, the routing table information is written into information stored in the third node, or the routing table information is written into a variable in the third node, or the routing table information is written into a parameter in the third node. For example, the root complex writes the routing table information into the reserved space of the configuration space of the third node.

The flow shown in fig. 11 may be implemented by the root complex, may be implemented by a chip in the root complex, or may be implemented by another device, for example, by a third node. The embodiments of the present application do not limit this.

For example, after the root complex allocates the identity information to the nodes in the PCIe system 100, the third node determines that the third node supports the first working mode; the third node sends a broadcast signal to other nodes in the PCIe system 100 to obtain information such as identity information or functions of the other nodes; and then the third node establishes routing table information based on the identity information, the function and other information of other nodes.

In the above embodiment of the present application, the root complex writes the routing table information into an intermediate node (for example, a third node), so that the intermediate node may forward the received TLP in a manner that the TLP does not pass through the root complex according to the destination address in the received TLP and the routing table information. Therefore, communication among end nodes, communication among switching nodes, communication between the end nodes and the switching nodes and the like in the PCIe system do not pass through the root complex, direct communication among the nodes is achieved, and accordingly communication complexity among the nodes in the PCIe system can be reduced.

It should be noted that, the root complex may traverse all nodes in the PCIe system 100 according to the depth-first principle, and allocate identity information to all nodes according to the flow shown in fig. 3; for each node in at least one node supporting the first working mode or the first working mode and the second working mode in the PCIe system 100, writing the identity information of the node and the identity information of at least one destination node corresponding to the node into the node; the routing table information for the bridge node is then determined based on the identity information of all nodes in the PCIe system 100 and written to the bridge node.

Through the above-described flows shown in fig. 10 and fig. 11, the root complex writes the identity information of the destination node into the sending node, writes the routing table information into the intermediate node, and can implement communication between nodes without passing through the root complex according to the identity information of the destination node and the routing table information. When data transmission is performed, another PCIe-based communication method is provided in the embodiments of the present application, and the method can implement direct communication between nodes. Please refer to fig. 12, which is a flowchart of a PCIe-based communication method provided in the embodiment of the present application, the method may be applied to a PCIe system in an intranet, particularly an intranet of an autonomous vehicle, and may also be applied to PCIe systems of other devices. For example, the method may be applied to the PCIe system 100 shown in fig. 1 or fig. 2. The method will be described below by taking the first node as the end node 1, the second node as the end node 2, the third node as the bridge node 4, and the fourth node as the bridge node 5 as examples.

S1201: the first node determines identity information of the second node.

When determining to send the first data to the second node, the first node may determine identity information of the second node according to information stored in the first node. The stored information is configuration information, or a variable, or a parameter, etc. For example, the first node may determine the identity information of the second node from the reserved space of the configuration space of the first node. The second node is a destination node of the first data, and the first node supports the first working mode, or the first node supports the first working mode and the second working mode. The first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex.

And when the first node supports the second working mode, the first node stores the memory address allocated by the root complex. For example, the register of the first node stores the memory address.

S1202: the first node sends a TLP to the third node, and the third node receives the TLP.

Wherein the TLP may include the first data and identity information of the second node.

In one possible embodiment, the TLP may include the first data, identity information of the first node and identity information of the second node. For example, the first node may determine the identity information of the first node according to the information stored in the first node, and then encapsulate the identity information of the first node, the first data, and the identity information of the second node together into the TLP. Taking the identity information as the BDF as an example, if the length of the TLP header of the TLP is 12 bytes, the Byte8 to Byte11 in the TLP header may be used to indicate the BDF of the first node and the BDF of the second node, as shown in fig. 17; if the length of the TLP header of the TLP is 16 bytes, byte8 to Byte11 in the TLP header can be used to indicate the BDF of the first node and the BDF of the second node, as shown in fig. 18.

S1203: the third node determines first routing table information.

After receiving the TLP, the third node may determine the first routing table information according to information stored in the third node. The stored information is configuration information, or is a variable, or is a parameter, etc. For example, the third node may determine the first routing table information according to a reserved space of a configuration space of the third node. For example, the first routing table information may include identity information of the second node and identity information of the fourth node. And the third node supports the first working mode, or the third node supports the first working mode and the second working mode.

And when the third node supports the second working mode, the third node stores the memory address allocated by the root complex. For example, the register of the third node stores the memory address.

S1204: the third node determines a first routing path according to the first routing table information.

The third node may determine, according to the first routing table information and the identity information of the second node, a first routing path, where the first routing path is used to indicate that a next-hop node of the TLP is the fourth node.

S1205: the third node sends a TLP to the fourth node, which receives the TLP.

In one possible implementation, the TLP may include the first data, identity information of the first node and identity information of the second node.

In another possible embodiment, the TLP may include the first data, identity information of the third node and identity information of the second node. For example, the third node may determine the identity information of the third node according to the information stored in the third node, and then encapsulate the identity information of the third node into the TLP.

S1206: the fourth node determines second routing table information.

After receiving the TLP, the fourth node may determine the first routing table information according to information stored in the fourth node. The stored information is configuration information, or a variable, or a parameter, etc. For example, the fourth node may determine the second routing table information according to a reserved space of a configuration space of the fourth node. For example, the second routing table information may include identity information of the second node. The fourth node supports the first working mode, or the fourth node supports the first working mode and the second working mode.

And when the fourth node supports the second working mode, the fourth node stores the memory address allocated by the root complex. For example, the register of the fourth node stores the memory address.

S1207: and the fourth node determines a second routing path according to the second routing table information.

The fourth node may determine, according to the second routing table information and the identity information of the second node, a second routing path, where the second routing path is used to indicate a next-hop node of the TLP as the second node.

S1208: the fourth node sends a TLP to the second node, and the second node receives the TLP.

In another possible embodiment, the TLP may include the first data, identity information of the third node and identity information of the second node.

In another possible embodiment, the TLP may include the first data, the identity information of the fourth node and the identity information of the second node. For example, the fourth node may determine the identity information of the fourth node according to the information stored in the fourth node, and then encapsulate the identity information of the fourth node into the TLP.

S1209: the second node acquires the first data.

After receiving the TLP, the second node decapsulates the TLP to obtain the first data sent by the first node.

In the above embodiments of the present application, a sending node (e.g., a first node) determines identity information of a receiving node (e.g., a second node) through stored information, and then encapsulates the first data and the identity information of the receiving node together and sends the encapsulated first data to an intermediate node (e.g., a third node), so that the intermediate node forwards the first data to the receiving node. Therefore, communication among end nodes, communication among switching nodes, communication between the end nodes and the switching nodes and the like in the PCIe system do not pass through the root complex, direct communication among the nodes is achieved, and accordingly communication complexity among the nodes in the PCIe system can be reduced.

Based on the same technical concept, the embodiment of the application provides a PCIe-based communication device. The structure of the apparatus may be as shown in fig. 13, including a processing unit 1301.

The communication apparatus 1300 may be specifically used to implement the method performed by the root complex in the embodiments of fig. 10 and fig. 11, and the apparatus 1300 may be the root complex itself, or may be a chip or a chipset in the root complex or a part of a chip for performing related method functions. The processing unit 1301 is configured to write identity information of a second node into a first node, and write routing table information into a third node, where the first node is a source node of first data, the second node is a destination node of the first data, and the third node is a node where the first data passes through to the second node.

In a possible implementation, the processing unit 1301 is further configured to determine that the first node supports the first operation mode; or determining that the first node supports the first working mode and the second working mode; the first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex.

In a possible implementation manner, the processing unit 1301 is further configured to configure a first memory address for the first node when the first node is in the second operating mode, where the first memory address is a physical address in an operating memory corresponding to the CPU, and the local memory space of the first node is mapped to the local memory space of the first node.

In a possible implementation, the processing unit 1301 is further configured to determine that the third node supports the first operation mode; or determining that the third node supports the first working mode and the second working mode; the first working mode is a working mode that communication between the nodes does not pass through the root complex, and the second working mode is a working mode that communication between the nodes needs to pass through the root complex.

In a possible implementation manner, the processing unit 1301 is further configured to configure a second memory address for the third node when the third node is in the second working mode, where the second memory address is a physical address in the running memory corresponding to the CPU mapped to the local memory space of the third node.

In a possible implementation, the processing unit 1301 is further configured to write the identity information of the first node to the first node.

In a possible implementation manner, the processing unit 1301 is further configured to write identity information of a third node to the third node.

In one possible embodiment, the identity information is a BDF, or an ID number.

The division of the modules in the embodiments of the present application is schematic, and only one logical function division is provided, and in actual implementation, there may be another division manner, and in addition, each functional module in each embodiment of the present application may be integrated in one processor, may also exist alone physically, or may also be integrated in one module by two or more modules. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It is understood that the functions or implementations of the respective modules in the embodiments of the present application may further refer to the related description of the method embodiments.

In one possible implementation, the communications apparatus 1400 may be as shown in fig. 14, and the apparatus 1400 may be an end node or a chip in an end node. The apparatus 1400 may include a processor 1401. Processing unit 1301 may be, among other things, processor 1401. Optionally, the apparatus 1400 may further comprise a transceiver 1402, a memory 1403.

The processor 1401 may be a CPU, a digital processing unit, or the like. The transceiver 1402 may be a communication interface, an interface circuit such as a transceiver circuit, a transceiver chip, or the like. The apparatus 1400 further comprises: a memory 1403 for storing programs executed by the processor 1401. The memory 1403 may be a nonvolatile memory such as a Hard Disk Drive (HDD) or a solid-state drive (SSD), and may also be a volatile memory (RAM), such as a random-access memory (RAM). Memory 1403 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such.

The processor 1401 is configured to execute the program code stored in the memory 1403, and is specifically configured to execute the actions of the processing unit 1301, which is not described herein again.

The embodiment of the present application does not limit the specific connection medium among the transceiver 1402, the processor 1401, and the memory 1403. In the embodiment of the present application, the memory 1403, the processor 1401, and the transceiver 1402 are connected by the bus 1404 in fig. 14, the bus is shown by a thick line in fig. 14, and the connection manner between the other components is merely illustrative and not limited. The bus 1404 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 14, but this is not intended to represent only one bus or type of bus.

Based on the same technical concept as that of the method embodiment, the embodiment of the present application provides a PCIe-based communication device 1500. The apparatus 1500 may have a structure as shown in fig. 15, and includes a processing unit 1501 and a transceiver unit 1502.

The communication apparatus 1500 may implement the method performed by the first node in the embodiment of fig. 12, and the apparatus 1500 may be the first node itself, or may be a chip or a chip set in the first node or a part of a chip for performing related method functions. The processing unit 1501 is configured to determine, according to information stored in the first node, identity information of a second node, where the second node is a destination node of the first data; a transceiving unit 1502, configured to send a first TLP to a third node, where the first TLP includes first data and identity information of a second node.

In one possible embodiment, the first node supports a first mode of operation, which is a mode of operation in which communication between nodes does not pass through the root complex.

In a possible implementation, the first node supports a first working mode and a second working mode, the first working mode is a working mode in which communication between nodes does not pass through the root complex, and the second working mode is a working mode in which communication between nodes needs to pass through the root complex.

In a possible embodiment, the stored information further comprises identity information of the first node.

In one possible embodiment, the first TLP further includes identity information of the first node.

In one possible embodiment, the identity information is a BDF, or an ID number.

In one possible implementation, the communications apparatus 1600 may be an end node or a chip in an end node, or may be a switching node or a chip in a switching node, as shown in fig. 16. The apparatus 1600 may include a processor 1601, and may further include a transceiver 1602, a memory 1603. The processing unit 1501 may be the processor 1601, among others. The transceiving unit 1502 may be a transceiver 1602.

The processor 1601 may be a CPU, a digital processing unit, or the like. The transceiver 1602 may be a communication interface, an interface circuit such as a transceiver circuit, or a transceiver chip. The apparatus 1600 further comprises: a memory 1603 for storing programs to be executed by the processor 1601. The memory 1603 may be a non-volatile memory, such as a Hard Disk Drive (HDD) or a solid-state drive (SSD), etc., and may also be a volatile memory, such as a random-access memory (RAM). Memory 1603 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such.

The processor 1601 is configured to execute the program code stored in the memory 1603, and is specifically configured to execute the actions of the processing unit 1501, which is not described herein again. The transceiver 1602 is specifically configured to perform the operations of the transceiving unit 1502, and is not described herein again.

The embodiment of the present application does not limit the specific connection medium among the transceiver 1602, the processor 1601, and the memory 1603. In fig. 16, the memory 1603, the processor 1601 and the transceiver 1602 are connected by a bus 1604, the bus is shown by a thick line in fig. 16, and the connection manner among other components is only schematically illustrated and not limited. The bus 1604 may be divided into an address bus, a data bus, a control bus, and so on. For ease of illustration, only one thick line is shown in FIG. 16, but that does not indicate only one bus or one type of bus.

The embodiment of the present application further provides a computer-readable storage medium, which is used for storing computer software instructions required to be executed by the processor, and the computer software instructions include a program required to be executed by the processor.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A communication method for communicating PCIe over a peripheral component interconnect, performed by a root complex, the method comprising:

writing identity information of a second node into a first node, wherein the first node is a source node of first data, and the second node is a destination node of the first data;

and writing the routing table information into a third node, wherein the third node is a node which is passed by the first data to the second node.

2. The method of claim 1, further comprising:

determining that the first node supports a first mode of operation; or,

determining that the first node supports a first mode of operation and a second mode of operation;

the first working mode is a working mode that communication between nodes does not pass through a root complex, and the second working mode is a working mode that communication between nodes needs to pass through the root complex.

3. The method of claim 2, further comprising:

and when the first node is in the second working mode, configuring a first memory address for the first node, wherein the first memory address is a physical address of a running memory corresponding to a Central Processing Unit (CPU) mapped to a local memory space of the first node.

4. The method according to any one of claims 1 to 3, further comprising:

determining that the third node supports a first mode of operation; or,

determining that the third node supports a first mode of operation and a second mode of operation;

5. The method of claim 4, further comprising:

and when the third node is in the second working mode, configuring a second memory address for the third node, wherein the second memory address is a physical address of an operating memory corresponding to a Central Processing Unit (CPU) mapped to a local memory space of the third node.

6. The method according to any one of claims 1 to 3, further comprising:

and writing the identity information of the first node into the first node.

7. The method according to any one of claims 1 to 3, further comprising:

and writing the identity information of the third node into the third node.

8. A method according to any one of claims 1-3, characterized in that the identity information is a bus number, a device number and a function number BDF, or an identity ID number.

9. A communication method for transmitting PCIe (peripheral component interconnect express) based on a peripheral component interconnect, which is applied to a first node, and comprises the following steps:

the first node determines identity information of a second node according to information stored in the first node, wherein the second node is a destination node of the first data, and the identity information of the second node comes from a root complex;

the first node sends a first transaction layer packet TLP to a third node, where the first TLP includes the first data and the identity information of the second node, and the third node is a node through which the first data passes to the second node.

10. The method of claim 9, wherein the first node supports a first mode of operation, wherein the first mode of operation is a mode of operation in which communication between nodes does not pass through a root complex.

11. The method of claim 9, wherein the first node supports a first mode of operation and a second mode of operation, wherein the first mode of operation is a mode of operation in which communication between nodes does not pass through a root complex, and wherein the second mode of operation is a mode of operation in which communication between nodes passes through the root complex.

12. The method according to any of claims 9-11, characterized in that said stored information further comprises identity information of said first node.

13. The method of claim 12, wherein the first TLP further comprises identity information of the first node.

14. The method according to any one of claims 9 to 11, wherein the identity information is a bus number, a device number and a function number BDF, or an identification ID number.

15. A peripheral component interconnect express (PCIe) -based communication apparatus, wherein the communication apparatus is a root complex, comprising:

a processing unit: the system comprises a first node, a second node and a third node, wherein the first node is used for writing identity information of the second node into the first node, the first node is a source node of first data, and the second node is a destination node of the first data; and writing the routing table information into a third node, wherein the third node is a node which is passed by the first data to the second node.

16. The apparatus of claim 15, wherein the processing unit is further configured to:

determining that the first node supports a first mode of operation; or,

17. The apparatus of claim 16, wherein the processing unit is further configured to:

18. The apparatus according to any one of claims 15 to 17, wherein the processing unit is further configured to:

determining that the third node supports a first mode of operation; or,

19. The apparatus of claim 18, wherein the processing unit is further configured to:

20. The apparatus according to any one of claims 15 to 17, wherein the processing unit is further configured to:

and writing the identity information of the first node into the first node.

21. The apparatus according to any one of claims 15 to 17, wherein the processing unit is further configured to:

and writing the identity information of the third node into the third node.

22. The apparatus according to any one of claims 15 to 17, wherein the identity information is a bus number, a device number, and a function number BDF, or an identification ID number.

23. A peripheral component interconnect based PCIe communication apparatus comprising:

the processing unit is used for determining identity information of a second node according to information stored in a first node, wherein the second node is a destination node of the first data, and the identity information of the second node comes from the root complex;

a transceiving unit, configured to send a first transaction layer packet TLP to a third node, where the first TLP includes the first data and the identity information of the second node, and the third node is a node that is traversed by the first data to the second node.

24. The apparatus of claim 23, wherein the first node supports a first mode of operation, wherein the first mode of operation is a mode of operation in which communication between nodes does not pass through a root complex.

25. The apparatus of claim 23, wherein the first node supports a first operating mode and a second operating mode, wherein the first operating mode is an operating mode in which communication between nodes does not pass through a root complex, and wherein the second operating mode is an operating mode in which communication between nodes needs to pass through the root complex.

26. The arrangement according to any of the claims 23-25, characterized in that said stored information further comprises identity information of said first node.

27. The apparatus according to claim 26, wherein the first TLP further comprises identity information of the first node.

28. The apparatus according to any one of claims 23 to 25, wherein the identity information is a bus number, a device number and a function number BDF, or an identification ID number.

29. A computer readable storage medium, in which a program or instructions are stored, which when read and executed by one or more processors, may implement the method of any one of claims 1 to 8 or claims 9 to 14.

30. A chip, characterized in that the chip comprises a communication interface and at least one processor coupled with a memory for reading a computer program stored in the memory to perform the method according to any one of claims 1 to 8 or claims 9 to 14.

31. A peripheral component interconnect-based PCIe system, comprising a root complex, a first node, a second node, and a third node, wherein the root complex is configured to perform the method of any one of claims 1 to 8, the first node is configured to perform the method of any one of claims 9 to 14, the first node is a source node of first data, the second node is a destination node of the first data, and the third node is a node through which the first data passes from the first node to the second node.

32. A vehicle comprising the PCIe system of claim 31.