WO2012119369A1 - Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa - Google Patents

Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa Download PDF

Info

Publication number
WO2012119369A1
WO2012119369A1 PCT/CN2011/077898 CN2011077898W WO2012119369A1 WO 2012119369 A1 WO2012119369 A1 WO 2012119369A1 CN 2011077898 W CN2011077898 W CN 2011077898W WO 2012119369 A1 WO2012119369 A1 WO 2012119369A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
processor
packet
node controller
message
Prior art date
Application number
PCT/CN2011/077898
Other languages
English (en)
Chinese (zh)
Inventor
程永波
贺成洪
兰可嘉
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2011/077898 priority Critical patent/WO2012119369A1/fr
Priority to CN201180001573.0A priority patent/CN102318275B/zh
Publication of WO2012119369A1 publication Critical patent/WO2012119369A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
    • G06F12/0815—Cache consistency protocols

Definitions

  • the present invention relates to the field of non-uniform storage access technologies, and in particular, to a CC-NUMA-based message processing method, apparatus, and system. Background technique
  • Cache Coherence Non-uniform Memory Access is an important system structure currently used in large-scale parallel computer design.
  • each node is composed of a node controller and a plurality of central processing unit CPUs, each node is interconnected through a network, and each central processing unit CPU can access both local memory resources and other nodes in the entire system.
  • the system is called a "non-uniform" access system because each central processor accesses local memory resources faster than accessing other nodes' memory resources.
  • FIG. 1 is a basic structural diagram of a node in the prior art, it can be seen that the node controller in the node has two processor CPUs, and each processor passes through a Quick Path Interconnect (QPI) bus and the node. The controller is connected; the node controller is provided with a network interface (NI, Network Interface), and each node in the system is extended by the network interface NI interconnection, so that the memory resources in the entire system are shared.
  • NI Network Interface
  • the node controller in the node When the node controller in the node receives the packet sent by the other node controller through the network interface, the node controller performs address resolution on the received packet, and sends the packet according to the parsed packet address. The corresponding processor processes to complete access to its memory resource data. Similarly, when the processor needs to access the remote resource, the processor can send the message to the node controller through the QPI bus, and the node controller queries the routing table according to the destination node address of the packet, and passes the network interface corresponding to the destination node. The message is sent to the next hop node, and finally the message is sent to the destination node to complete the access of the resource.
  • the node controller In order to complete the sharing of resources within the system, the node controller needs to maintain the address space within the entire node. The directory, and the routing of the packets accessing the two central processors, the system resource access efficiency is low. At the same time, when the QPI bus connection between the processor and the node controller fails, or the node controller itself fails, other nodes may not be able to send a message to the node, cannot access the resources in the node, or cause the entire node to report. The processing of the file is interrupted, which in turn affects the resource access of the entire system, and the reliability of system resource access is low.
  • the present invention provides a CC-NUMA-based message processing method, apparatus, and system to improve resource access efficiency and reliability.
  • the present invention provides the following technical solutions as follows:
  • a packet processing method based on CC-NUMA in which two node controllers are configured in a node, and each node controller maintains a directory of its corresponding address space, and the method includes:
  • the destination address of the message is the node, it is determined whether the directory of the address space corresponding to the message is maintained by the node controller;
  • a packet processing method based on CC-NUMA in which two node controllers are configured in a node, and each node controller maintains a directory of its corresponding address space, and the method includes:
  • the packet and the local node are controlled. a directory maintained by the device and containing at least an address space corresponding to the message, sent to another node controller, so that the another node controller forwards the message to the processor, and maintains the The directory of the address space corresponding to the text.
  • a packet processing method based on CC-NUMA in which two node controllers are configured in a node, and each node controller maintains a directory of its corresponding address space, including:
  • the processor to which the address space corresponding to the message is located is determined, and the message is sent to the processor.
  • the present invention also provides a CC-NUMA-based message processing apparatus, in which two node controllers are configured in a node, and each node controller maintains a directory of its corresponding address space, and the message processing apparatus includes:
  • a message receiving unit configured to receive a packet sent by another node, and perform address resolution on the packet
  • an address analyzing unit configured to determine, when the destination address of the packet is a node, Whether the directory of the address space is maintained by the local node controller, and if so, the operation of the processor address determining unit is performed;
  • a processor address determining unit configured to determine a processor to which the address space corresponding to the text belongs
  • a message sending unit configured to send the message to the processor through a fast channel interconnect bus connected to the processor, so that the processor processes the “3 ⁇ 4 text”.
  • a CC-NUMA-based message processing device is configured with two node controllers in a node, and each node controller maintains a directory of its corresponding address space, and the message processing device includes: a link failure acquiring unit , for obtaining a link connection failure of the fast channel interconnection bus in the system;
  • a message receiving unit configured to receive a message sent by another node, and perform address resolution on the message
  • an address analyzing unit configured to determine, when the destination address of the text is a node, the “3 ⁇ 4 text corresponding to the message Whether the directory of the address space is maintained by the local node controller, and if so, the operation of the processor address determining unit is performed;
  • a processor address determining unit configured to determine a processing crying port to which the address space corresponding to the text belongs.
  • a fault processing unit configured to: when the link between the local node controller and the processor is faulty, the message and a directory maintained by the local node controller and including at least an address space corresponding to the packet, Sending to another node controller, so that the other node controller forwards the message to the processor and maintains a directory of the address space corresponding to the text.
  • a CC-NUMA-based message processing device is configured with two node controllers in a node, and each node controller maintains a directory of its corresponding address space, and the message processing device includes: a controller fault acquiring unit , used to obtain fault information of another node controller;
  • An address space data obtaining unit configured to acquire, by broadcast snooping, a directory of an address space maintained by the another node controller
  • a message receiving unit configured to receive a packet sent by another node, and perform an address resolution analysis unit on the packet, where the packet is determined to be a destination address of the packet, and the packet is determined to be corresponding to the packet.
  • the processor to which the address space belongs;
  • a message sending unit configured to send the message to the processor.
  • a CC-NUMA based message processing system includes two node controllers and at least two processors;
  • the node controller and the processor are connected by a fast channel interconnect bus
  • the two node controllers are connected through a network interface
  • the node controller has built-in CC-NUMA-based message processing apparatus as described above.
  • the embodiment of the present invention discloses a CC-NUMA-based message processing method, apparatus, and system, in which a node controller receives a destination address sent by another node as a node of the node.
  • a node controller receives a destination address sent by another node as a node of the node.
  • the fast channel interconnect bus sends the message to the processor; if the address space corresponding to the message does not belong to the address space maintained by the node controller, the node controller forwards the message to another node controller In order for another node controller to process the message, since two node controllers in the node respectively maintain data of a part of the address space, both node controllers can receive the message sent by other nodes, and receive the message. Address resolution of the received message, so that the message is sent to the corresponding processor to complete resource access and improve resource access. speed, This improves system performance.
  • the node controller can obtain the QPI link failure information, when the processor corresponding to the address space corresponding to the received message and the node controller
  • the node controller can forward the packet and the directory information of the address space corresponding to the packet to another node controller, and the other node controller processes the packet and sends the packet. To the corresponding processor, communication interruption due to link failure is avoided.
  • FIG. 1 is a schematic diagram of a basic structure of a node in the prior art
  • FIG. 2 is a schematic diagram showing the basic structure of a node in the present invention.
  • FIG. 3 is a flowchart of an embodiment of a method for processing a message based on CC-NUMA according to an embodiment of the present invention
  • FIG. 4 is a flowchart of another embodiment of a CC-NMUA-based packet processing method according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of another embodiment of a CC-NUMA-based message processing method according to an embodiment of the present invention.
  • FIG. 6 is a flowchart of another embodiment of a CC-NMUA-based packet processing method according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a CC-NUMA-based message processing apparatus according to an embodiment of the present invention.
  • FIG. 8 is a CC-NUMA-based message processing apparatus according to an embodiment of the present invention. Schematic;
  • FIG. 9 is a schematic structural diagram of a CC-NUMA-based message processing apparatus according to an embodiment of the present invention.
  • the technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. example. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without the creative work are all within the scope of the present invention.
  • the present invention configures two node controllers (NCs) in one node, and each node controller maintains its corresponding address.
  • the address space is also called the system address and is used to locate the specific location of the resources in the CC-NUMA system.
  • each node in the system is assigned an address space of a region, and the address space of the node is separately assigned to the processor in the node. For example, suppose that there are two processor CPUs in a node, node 1 in the system allocates 0-1TB of address space, and processor 1 and processor 2 in node 1 respectively allocate 512 GB of address space; Node 2 allocates 1TB-2TB of address space, in which processor 1 and processor 2 of node 2 respectively allocate 512 GB of address space, and the allocation of address space of other nodes in the system is similar.
  • FIG. 2 is a schematic diagram of a basic structure of a node in the present invention.
  • the node includes a node controller. NC1, node controller NC2, first processor CPU1 and second processor CPU2, wherein each processor is connected to node controller NC1 and node controller NC2 via first fast channel interconnect bus QPI0 and second fast interconnect QPI1, respectively
  • the node controller NC1 and the node controller NC2 are connected through the network interface NI.
  • Each node controller has several network interfaces NI, and the node controllers in the node can be interconnected with the node controllers in other nodes through the network interface NI.
  • the invention divides the address space of the node into two regions, and each node controller maintains a directory of the address space of one region, for example, the node of the node
  • the address space can be divided into a first address space and a second address space, wherein one node controller maintains a directory of the first address space, and another node controller maintains a directory of the second address space.
  • the first address space may include a part of the address space allocated to the address space of the first processor and a part of the address space allocated to the address space of the second processor, and the second address space is also It is possible to include both a portion of the address space in the address space allocated to the first processor and a portion of the address space in the address space allocated to the second processor.
  • the directory that maintains the address space refers to the data access situation of the node controller to the address space and the data state of the address space. For example, when a processor in another node needs to access a resource saved on a processor in the node, the processor of the other node sends a resource access request in the form of a packet, and the node controller of the node receives the packet. After that, the processing status of the resource data that the packet requests to access is recorded in the directory of the address space (for example, the data state of the address space requested to be accessed may be a modified state, an exclusive state, a shared state, etc.), and a node that sends a message. Information such as ID and processor ID.
  • each node is provided with two node controllers, and each node controller is provided with a plurality of network interfaces NI connected to the node controllers of other nodes, compared with the prior art, in the CC-NUMA system.
  • the number of NI interfaces provided by each node for connecting to other nodes is twice the number of NI interfaces provided by each node in the prior art for interconnecting between nodes. Therefore, when each CC-NUMA system Nodes are connected to each other to form a larger topology network.
  • the average number of hops of the intermediate nodes is reduced during the process of sending packets from the source node to the destination node, which improves system performance.
  • FIG. 3 is a schematic flowchart of an embodiment of a CC-NUMA-based packet processing method according to the present invention.
  • two node controllers and two nodes are configured in one node.
  • the controllers are interconnected through a network interface, and each node controller maintains a directory of its corresponding address space, and the method includes:
  • Step 301 The first node controller receives a packet sent by another node, and performs address resolution on the packet.
  • the processor of the other node may send a message to the node, where the packet includes the address of the destination node to be accessed (destination node ID) ), the address of the destination processor (processor ID), and the address space accessed Data requests, etc.
  • the node controller in the node After receiving the packet sent by other nodes through the network interface NI, the node controller in the node generally performs a cyclic redundancy check (CRC), and after receiving the CRC, the received message is received. Perform address resolution, that is, decode the address of the packet to determine whether the destination address of the packet is the local node.
  • CRC cyclic redundancy check
  • the node controller queries the routing table according to the routing table.
  • the packet is forwarded to the node controller in the other node.
  • the node controller needs to perform corresponding processing, so as to finally send the packet to the node and the packet.
  • the processor requests access to the address space data corresponding to the processor.
  • Step 302 When the destination address of the packet is the node, the first node controller determines whether the directory of the address space corresponding to the packet is maintained by the first node controller, and if yes, proceeds to step 303; Then, the process proceeds to step 304.
  • the first node controller determines that the destination address of the packet is the node according to the parsed address
  • the first node controller needs to determine the address space corresponding to the packet according to the parsed address, that is, the packet. Whether the address space to be accessed and the directory of the address space to be accessed by the message are maintained by the first node controller.
  • the parity bit of the address space may be preset, and the parity bit and the node controller are preset. Correspondence between the two. Specifically, setting the first bit in the address space to the parity bit can be set as needed. For example, the sixth bit in the address space can be set to the parity bit, and set to the parity in the address space. When the parity bit is "0", the directory of the corresponding address space is maintained by the first node controller; when the parity bit in the address space is " ⁇ , the directory of the corresponding address space is controlled by the second node controller.
  • setting the first bit in the address space to the parity bit can be set as needed.
  • one of the 6th to the 10th bits in the address space may be selected as the parity bit.
  • the numbers of the fifth and sixth digits in the address space are different, the directory of the corresponding address space is maintained by the second node controller.
  • there are other ways to distinguish the directory of the address space maintained by the first node controller and the second node controller which is not enumerated here.
  • the first node controller may determine the address space corresponding to the packet according to the correspondence between the parity bit of the address space corresponding to the packet and the node controller. Whether the parity bit corresponds to the first node controller, and if so, the directory of the address space corresponding to the message is maintained by the first node controller. For example, after receiving the packet, the first node controller performs address resolution on the packet, and determines that the parity bit (such as the fifth digit in the address) in the address of the data resource that the packet requests to access is "0". The directory of the address space that the message needs to access is maintained by the first node controller, otherwise it is maintained by the second node controller.
  • Step 303 The first node controller determines, by the processor to which the address space corresponding to the packet belongs, the packet is sent to the processor through a fast channel interconnect bus connected to the processor, so that the processor reports the packet.
  • the text is processed.
  • the first node controller may determine the required access of the packet according to the result of the address resolution of the packet.
  • the processor to which the address space belongs After determining the processor in which the address space requested to be accessed is located, the first node controller can query the routing table, and the fast channel connected to the processor by the first node controller according to the pre-configured port routing path
  • the interconnect bus QPI sends the message to the processor for the processor to process the message.
  • Step 304 The first node controller forwards the packet to the second node controller in the local node, so that the second node controller determines the processor to which the address space corresponding to the packet belongs, and the packet is sent.
  • the processor corresponding to the packet is sent to perform packet processing.
  • the first node controller passes the network interface NI between the first node controller and the second node controller.
  • the message is forwarded to the second node controller.
  • the second node controller After receiving the packet, the second node controller performs address resolution on the packet, and determines which processor in the node is required to access the address space, and the second node controller is based on the Configured port path through the second node controller
  • a fast channel interconnect bus with the processor forwards the message to the processor, and the processor processes the message.
  • a node controller in the node after receiving a packet sent by another node, a node controller in the node performs address resolution on the packet, and the directory corresponding to the address space of the packet is maintained by the node controller.
  • the node controller further determines the processor to which the address space corresponding to the message belongs, and sends the message to the processor through the fast channel interconnect bus connected to the processor; if the message corresponds to The directory of the address space is not maintained by the node controller, and the node controller forwards the message to another node controller in the node, so that the other node controller processes the message.
  • each node controller Since the directory of the address space in the node can be divided into two parts, each node controller maintains data of a part of the address space, so that other nodes in the CC-NUMA system need to access resources in the processor of the node, according to the routing path. Selecting which node controller in the node to send the message, the two node controllers in the node can receive different message requests, and respectively perform address resolution of the 3 ⁇ 4 text, and maintain the directory of the corresponding address space.
  • the first node controller sends the packet to the packet corresponding to the packet.
  • the processor depends on the address space that the message needs to access.
  • the packet is processed according to the resource, and the processed packet is sent to the first node controller, so that the first node controller returns the processed packet to the source node that sends the packet, that is, sends the resource. Accessing the requested node.
  • the processor also returns the processed message to the second node controller,
  • the two-node controller can query the routing table and select the network interface NI to return the processed message to the source node that initiated the resource access request.
  • FIG. 4 is a flowchart of another embodiment of a CC-NUMA-based message processing method according to an embodiment of the present invention.
  • the node of the present invention is configured with two node controllers, and each node controller is respectively configured.
  • the directory of the corresponding address space is maintained.
  • the method in this embodiment includes:
  • Step 401 The first node controller receives a packet sent by another node, and performs an address on the packet. Analysis.
  • Step 402 When the destination address of the packet is the node, the first node controller determines whether the directory of the address space corresponding to the packet is maintained by the first node controller, and if yes, proceeds to step 403; Then, proceed to step 404.
  • Step 403 The first node controller determines, by the processor to which the address space corresponding to the packet belongs, the packet to be sent to the processor through a fast channel interconnect bus connected to the processor.
  • Step 404 The first node controller forwards the packet to the second node controller, and the second node controller receives the packet, and performs address resolution on the packet to determine a processor to which the address space corresponding to the packet belongs.
  • the directory of the address space corresponding to the message is maintained, and the message is sent to the processor.
  • Step 405 After receiving the packet sent by the first node controller or the second node controller, the processor processes the packet, and determines whether the directory of the address space corresponding to the packet is maintained by the first node controller. If yes, the processor sends the processed message to the first node controller, so that the first node controller returns the processed message to the source node; if not, the processor processes the processed message. The text is sent to the second node controller, so that the second processor returns the processed message to the source node.
  • the processor After receiving the packet, the processor processes the packet according to the data of the address space requested by the packet. Since the node controller needs to record information such as the data state of the address space accessed by the packet, the processing is performed. The subsequent message still needs to be returned to the corresponding node controller, so that the node controller maintains the directory of the address space requested by the message. Therefore, after the processor finishes processing the packet, it needs to determine which node controller maintains the directory of the address space corresponding to the packet, and returns the processed packet to the node controller. Finally, the node controller returns the message after the processor to the source node that sent the message.
  • the node controller After the node controller receives the received message, it is required to determine whether the directory of the address space corresponding to the packet is maintained by the local node controller, and if yes, the local node controller sends the message to the node.
  • the processor corresponding to the address space of the message therefore, the processor can directly return the message to the node controller in the node to send the message to the node, that is, when the first node controller sends the message After going to a processor in the node, the processor processes the "3 ⁇ 4 text" and returns the processed message to the first node controller; when the second node controller sends the message to the node After a processor in the processor, the processor processes the packet and returns the processed packet to the second node controller.
  • Step 406 After receiving the processed text returned by the processor, the first node controller or the second node controller updates the directory of the address space corresponding to the processed message, and the processed message is processed. Returns to the source node that sent the message.
  • the node controller when the node controller receives the packet sent by the other node, it determines whether the directory of the address space corresponding to the packet is maintained by the node, and if so, the node controller needs to update the packet.
  • the directory of the address space corresponding to the packet for example, the source node that sends the request for the message, and the like.
  • the node controller needs to request the packet.
  • the data status is recorded, etc., so when the node controller determines that the directory of the address space corresponding to the message sent by the other node is maintained by the local node controller, and the node controller returns the processed message to the source. Before the node, the node controller needs to maintain the directory of the address space corresponding to the packet.
  • the above two embodiments are described by taking the packet that the other node requests the resource access of the other node as an example, and the process of the node controller and the processor after receiving the packet in the node.
  • the processor in the node may also generate a resource access request packet, query the routing table, and select an optimal routing path.
  • the optimal routing path determines whether the generated packet is sent to the first node controller or the second node controller, and after the first node controller or the second node controller receives the packet sent by a processor in the node, It also queries its own routing table to determine which externally connected NI interface in the node controller forwards the message.
  • the fast-path interconnect bus QPI between a node controller and a processor in a node may be faulty.
  • the link fails, and the message transmission processing cannot be performed, and the communication that causes the resource access is interrupted.
  • FIG. 5 is a schematic flowchart diagram of another embodiment of a CC-NUMA-based packet processing method according to the present invention.
  • the embodiment is applied to a node controller and a processor.
  • the fast channel interconnect bus fails, as in the above embodiment, two node controllers are configured in the node, and each node controller maintains its corresponding address.
  • the directory of the space includes:
  • Step 501 The first node controller receives the packet sent by the other node, and performs address resolution on the packet.
  • Step 502 When the first node controller determines that the destination address of the packet is the node and the address space corresponding to the packet When the address space maintained by the first node controller is within the address space, the processor to which the address space corresponding to the message is located is determined.
  • Step 503 If the link of the fast path interconnection bus between the first node controller and the processor fails, the first node controller corresponding to the packet and the first node controller, including at least the packet
  • the directory of the address space is sent to the second node controller, so that the second node controller maintains the directory of the address space corresponding to the message, and forwards the message to the processor to which the address space corresponding to the message belongs.
  • the node controller in the node needs to obtain the link failure of the fast channel interconnection bus QPI between the node controller and the processor.
  • the node controller in the node needs to obtain the link failure of the fast channel interconnection bus QPI between the node controller and the processor.
  • both processors in the node can obtain information about the link failure.
  • the interrupt source of the fault reports the interrupt fault information to the processor, and the processor notifies the node controller of the fast-path interconnect bus to the node controller in the node.
  • the controller receives link fault information of the fast channel interconnect bus sent by the processor. For example, when the fast channel interconnection bus between the first node controller and the certain processor in the node fails, the message data sent by the first node controller received by the processor generates an error, when the generated error data When the number of packets exceeds a preset value, the processor determines that the fast path interconnection bus between the first node controller and the processor is faulty, and the processor notifies the first node controller and the second link of the detected link failure. Node controller.
  • the node controller can also detect the QPI link failure connected thereto, and report the fault to a processor in the node, the processor links the link.
  • the failure notifies other processors and another node controller. It is also possible to notify the node controller of the fast-path interconnect bus link that the link failure of the fast-path interconnect bus link is faulty to another A node controller.
  • the processor or the first node controller invokes the basic input/output system BIOS program, performs port routing path configuration, and changes the first node controller to send the message to
  • the port routing path of the processor changes the path between the first node controller and the processor to a routing path through the network interface NI between the first node controller and the second node controller.
  • the routing path between the first node controller and the first processor in the node is reconfigured, when the first node controls After receiving the packet, the device determines that the directory of the address space corresponding to the packet is maintained by itself and the packet is sent to the first processor, because the fast channel is interconnected between the first node controller and the first processor at this time. If the bus fails, the first node controller executes an interrupt handler, and according to the reconfigured port routing path, through the network interface NI between the second node controller, the message and the first node controller maintain at least A directory containing the address space corresponding to the message is sent to the second node controller.
  • the second node controller After the second node controller receives the packet from the first node controller, the second node controller also needs to perform address resolution on the packet to determine which processor in the node belongs to the address space of the packet, and The message is sent to the processor through the fast channel interconnect bus connected to the processor, so that the processor processes the message. Of course, after the processor finishes processing the packet, the processed packet is returned to the second node controller. During the processing of the entire packet, the second node controller will catalog the address space corresponding to the packet. Perform maintenance, and finally return the processed message returned by the processor to the source node that sent the data request message.
  • the first node controller may also send the message only to the second node controller, the second node.
  • the controller performs address resolution on the packet, and sends the packet to the processor corresponding to the requested access.
  • the request for the directory modification is sent to the first node control through the NI interface with the first node controller. And the first node controller maintains the directory of the address space corresponding to the message.
  • the node controller can obtain the QPI link fault information, and when the first node controller receives the packet, determines the address corresponding to the packet.
  • the processor to which the space belongs when the QPI bus between the processor and the first node controller fails, the first node controller may forward the message and the directory information of the address space corresponding to the packet to the second node.
  • the controller performs address resolution on the packet by the second node controller, determines which processor in the node belongs to the address space corresponding to the packet, and uses the fast channel interconnection bus connected to the processor to report the packet.
  • the text is sent to the processor, so even if a QPI link in the node fails, the new routing path can be selected, and the packet is sent to the corresponding processor, thereby avoiding the communication interruption caused by the link failure. .
  • the node controller in the node fails, the node controller cannot establish a directory of the address space. For example, if the memory of the address space directory in the node controller fails, the node controller cannot reacquire. The directory of the address space. In this case, other nodes may not be able to access the data resources saved in the processor of the node, and the node cannot receive, process, or forward the packets sent by other nodes, thereby affecting the resource access of the entire CC-NUMA system.
  • FIG. 6 is a schematic flowchart diagram of another embodiment of a CC-NUMA-based packet processing method according to the present invention.
  • two node controllers are configured in a node, and each The node controller maintains a directory of its corresponding address space.
  • the method in this embodiment includes:
  • Step 601 The first node controller acquires fault information of the second node controller.
  • the second node controller's own directory is lost, or the memory of the second node controller's internal storage address space directory fails, causing the second node controller to fail to perform directory maintenance of the address space, and the failure interrupt source will be the second node controller.
  • the fault information is sent to a processor in the node, and the processor notifies the other nodes in the node and the first node controller of the fault information of the second node controller.
  • the processor also sends the fault information of the second node controller to other nodes in the system. After the other node obtains the fault information of the second node controller, the packet routing path sent to the node is re-configured, and the second node controller will not receive the packet information sent by other nodes.
  • the second node controller when the second node controller detects its own fault and cannot establish or maintain a directory of the address space, the second node controller can directly notify the first node controller of its own fault information.
  • Step 602 The first node controller acquires a directory of an address space maintained by the second node controller by using broadcast snooping.
  • the first node controller sends the interception to the other nodes in the system by means of broadcast snooping to obtain the maintenance of the first node controller.
  • the data information of the address space, the processor in the other node returns the corresponding data information to the first node controller, so that the first node controller establishes the directory information of the address space originally maintained by the second node controller.
  • Step 603 When the first node controller receives the packet sent by the other node, the address resolution is performed on the packet.
  • Step 604 When the destination address of the packet is the node, the first node controller determines the processor to which the address space corresponding to the packet belongs, and sends the packet to the address space corresponding to the packet. processor.
  • the first node controller acquires the directory of the address space maintained by the original second node controller by means of broadcast interception. At this time, the first node controller maintains the entire node.
  • the first node controller receives the packet sent by the other node, the first node controller does not need to determine whether the address space corresponding to the packet is maintained by the local node controller, and the first node controller can directly determine the packet.
  • the processor to which the corresponding address space belongs, and the message is sent to the processor through a fast channel interconnect bus connected to the processor.
  • the other node controller can obtain the node controller failure information, and obtain the data information of the address space maintained by the node controller through broadcast interception, and All the messages sent to the node are processed by another node controller, which avoids the communication interruption of the entire CC-NUMA system due to the failure of the node controller itself in the node.
  • FIG. 7 an embodiment of a CC-NUMA-based message processing apparatus according to the present invention is configured.
  • two node controllers are configured in a node, and each node controller maintains its corresponding address.
  • the message processing device may be a node controller or a part of a node controller.
  • the message processing device includes: a message receiving unit 701, an address analyzing unit 702, a processor address determining unit 703, and a message sending. Unit 704.
  • the message receiving unit 701 is configured to receive a packet sent by another node, and perform address solution on the packet.
  • the address analyzing unit 702 is configured to determine, when the destination address of the packet is the node, whether the directory of the address space corresponding to the packet is maintained by the local node controller, and if yes, perform the operation of the processor address determining unit.
  • the processor address determining unit 703 is configured to determine a processor to which the address space corresponding to the >3 ⁇ 4 text belongs.
  • the message sending unit 704 is configured to send the message to the processor through a fast channel interconnect bus connected to the processor, so that the processor processes the message.
  • the operation of the message receiving unit and the address analyzing unit in the node controller may be part of the Rbox module or the Rbox module in the node controller.
  • the packet may be sent to the address determining unit to perform the operations of the processor determining unit and the message sending unit.
  • the CCM cache Coherence Module
  • the CCM cache Coherence Module
  • the corresponding QPI bus can be selected to send the message to the corresponding processor, and the QPI link layer management module connected to the processor needs to be completed when the message is sent to the corresponding processor.
  • QPIL selects the corresponding QPI bus and sends the message to the corresponding processor.
  • the address analysis unit 702 may include:
  • An address analysis sub-unit configured to determine, according to a correspondence between a parity bit of the preset address space and a node controller, whether a parity bit of the address space corresponding to the packet corresponds to the node controller, and if The directory of the address space corresponding to the message is maintained by the local node controller.
  • the address processing device of the present embodiment further includes: an address space maintenance unit, configured to maintain a directory of an address space corresponding to the received message.
  • FIG. 8 is a schematic structural diagram of another embodiment of a CC-NUMA-based message processing apparatus according to the present invention.
  • the apparatus of this embodiment is applied to a fast channel interconnection bus between a node controller and a processor.
  • the node is configured with two node controllers in the node, and each node controller maintains its corresponding address space.
  • the packet processing device includes: a link fault obtaining unit 801 and a message receiving unit 802.
  • the link fault obtaining unit 801 is configured to acquire a link connection fault of the fast channel interconnect bus in the system.
  • the message receiving unit 802 is configured to receive a packet sent by another node, and perform address resolution on the packet.
  • the address analyzing unit 803 is configured to: when the destination address of the packet is the node, determine whether the directory of the address space corresponding to the packet is maintained by the node controller, and if yes, perform the operation of the processor address determining unit.
  • the processor address determining unit 804 is configured to determine a processor to which the address space corresponding to the text belongs.
  • the fault processing unit 805 is configured to: when the link between the local node controller and the processor is faulty, the message and the directory maintained by the local node controller and including at least the address space corresponding to the packet, Sending to another node controller, so that the other node controller forwards the message to the processor, and maintains the directory of the address space corresponding to the "3 ⁇ 4".
  • the link fault obtaining unit 801 includes: a link fault information receiving unit, configured to receive link fault information of a fast channel interconnect bus between the node controller and the processor sent by the processor, according to a manner of obtaining a link fault. .
  • the link fault detecting unit is configured to detect a link of the fast channel bus connected to the processor, and obtain a link fault of the fast channel interconnect bus.
  • FIG. 9 is a schematic structural diagram of another embodiment of a CC-NUMA-based message processing apparatus according to the present invention.
  • the message processing apparatus includes: a controller fault acquiring unit 901, an address space data acquiring unit 902, a message receiving unit 903, and an address analyzing unit 904. And a message transmitting unit 905.
  • the controller fault acquiring unit 901 is configured to acquire fault information of another node controller.
  • the address space data obtaining unit 902 is configured to obtain, by broadcast snooping, a directory of an address space maintained by the another node controller.
  • the message receiving unit 903 is configured to receive a message sent by another node, and perform an address de-address analysis unit 904 on the message, where the address address corresponding to the message is determined when the destination address of the message is the node.
  • the processor to which it belongs is configured to perform an address de-address analysis unit 904 on the message, where the address address corresponding to the message is determined when the destination address of the message is the node.
  • the message sending unit 905 is configured to send the message to the processor to which the address space corresponding to the message belongs.
  • the controller fault acquiring unit 901 includes: a first fault information receiving unit, configured to receive fault information of the another node controller sent by a processor.
  • the method includes: a second fault receiving unit, configured to receive fault information sent by the another node controller.
  • the present invention further provides a CC-NUMA-based message processing system, including: two node controllers, and at least two processors; each node controller is interconnected with each processor through a fast channel. The bus is connected. The two node controllers are connected through a network interface.
  • Each node controller has built-in CC-NUMA-based message processing apparatus described in the above embodiments of the present invention.
  • the various embodiments in the present specification are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same similar parts between the various embodiments may be referred to each other.
  • the description is relatively simple, and the relevant part can be referred to the method part.
  • the steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented directly in hardware, a software module executed by a processor, or a combination of both.
  • the software module can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or technical field. Any other form of storage medium known.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer And Data Communications (AREA)
  • Multi Processors (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne un procédé, un dispositif et un système de traitement de message basés sur une architecture CC-NUMA. Un certain contrôleur de nœud dans le nœud actif soumet un message transmis par un autre nœud à une résolution d'adresse, et, si l'adresse de destination du message correspond au nœud actif et le catalogue de l'espace adresse correspondant au message est tenu à jour par le contrôleur de nœud, il détermine un processeur auquel appartient l'espace adresse correspondant au message ; si le lien d'un bus QPI (Quick Path Interconnect) entre le contrôleur de nœud et le processeur est normal, il transmet alors le message au processeur par le bus QPI pour permettre au processeur de traiter le message ; si le lien du bus QPI entre le contrôleur de nœud et le processeur est défaillant, ou le catalogue de l'espace adresse correspondant au message n'est pas tenu à jour par le contrôleur de nœud, il réachemine le message vers un autre contrôleur de nœud dans le nœud actif et cet autre contrôleur de nœud réalise le traitement du message. Grâce au procédé selon l'invention, l'accès aux ressources du système gagne en efficacité et en fiabilité.
PCT/CN2011/077898 2011-08-02 2011-08-02 Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa WO2012119369A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2011/077898 WO2012119369A1 (fr) 2011-08-02 2011-08-02 Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa
CN201180001573.0A CN102318275B (zh) 2011-08-02 2011-08-02 基于cc-numa的报文处理方法、装置和系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2011/077898 WO2012119369A1 (fr) 2011-08-02 2011-08-02 Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa

Publications (1)

Publication Number Publication Date
WO2012119369A1 true WO2012119369A1 (fr) 2012-09-13

Family

ID=45429434

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2011/077898 WO2012119369A1 (fr) 2011-08-02 2011-08-02 Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa

Country Status (2)

Country Link
CN (1) CN102318275B (fr)
WO (1) WO2012119369A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11057302B2 (en) 2016-11-30 2021-07-06 New H3C Technologies Co., Ltd. Sending packet

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102708190B (zh) * 2012-05-15 2016-09-28 浪潮电子信息产业股份有限公司 一种CC-NUMA系统中结点控制芯片目录Cache的方法
WO2014059576A1 (fr) * 2012-10-15 2014-04-24 华为技术有限公司 Procédé de traitement d'un message de requête, procédé d'envoi, nœud et système associé
CN103092807B (zh) * 2012-12-24 2015-09-09 杭州华为数字技术有限公司 节点控制器、并行计算服务器系统以及路由方法
CN104753753B (zh) * 2013-12-31 2018-11-16 杭州华为数字技术有限公司 一种qpi报文的传输方法、设备及计算机系统
CN104935530B (zh) * 2015-04-29 2017-12-19 浪潮电子信息产业股份有限公司 一种计算机间数据交换的方法、交换机和系统
CN104899160B (zh) * 2015-05-30 2019-02-19 华为技术有限公司 一种缓存数据控制方法、节点控制器和系统
CN111427827B (zh) * 2015-12-29 2023-03-17 超聚变数字技术有限公司 一种cpu及多cpu系统管理方法
CN105808499A (zh) * 2016-04-01 2016-07-27 浪潮电子信息产业股份有限公司 一种cpu互联装置以及多路服务器cpu互联拓扑结构
WO2018032519A1 (fr) * 2016-08-19 2018-02-22 华为技术有限公司 Procédé et dispositif d'attribution de ressources, et système numa
CN107239432A (zh) * 2017-08-08 2017-10-10 郑州云海信息技术有限公司 一种具有新型拓扑结构的服务器
CN107749825B (zh) * 2017-10-24 2021-03-09 盛科网络（苏州）有限公司 一种跨芯片转发中基于源芯片id的流控方法及装置
CN114094209B (zh) * 2021-10-18 2023-10-20 华人运通(江苏)技术有限公司 电池管理系统及其通讯控制方法、装置和车辆

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6480973B1 (en) * 1999-09-30 2002-11-12 Bull Information Systems Inc. Gate close failure notification for fair gating in a nonuniform memory architecture data processing system
US6769017B1 (en) * 2000-03-13 2004-07-27 Hewlett-Packard Development Company, L.P. Apparatus for and method of memory-affinity process scheduling in CC-NUMA systems
CN1664784A (zh) * 2005-03-30 2005-09-07 中国人民解放军国防科学技术大学 大规模并行计算机系统分组并行启动方法
CN1991795A (zh) * 2005-12-28 2007-07-04 国际商业机器公司 用于信息处理的系统和方法
CN101216815A (zh) * 2008-01-07 2008-07-09 浪潮电子信息产业股份有限公司 一种双翼可扩展多处理器紧耦合共享存储器体系结构
CN101273332A (zh) * 2005-09-30 2008-09-24 英特尔公司 使用编译器的线程数据亲和性优化
US20080294832A1 (en) * 2007-04-26 2008-11-27 Hewlett-Packard Development Company, L.P. I/O Forwarding Technique For Multi-Interrupt Capable Devices

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8578130B2 (en) * 2003-03-10 2013-11-05 International Business Machines Corporation Partitioning of node into more than one partition
CN101651559B (zh) * 2009-07-13 2011-07-06 浪潮电子信息产业股份有限公司 一种存储服务在双控制器存储系统中故障切换的方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6480973B1 (en) * 1999-09-30 2002-11-12 Bull Information Systems Inc. Gate close failure notification for fair gating in a nonuniform memory architecture data processing system
US6769017B1 (en) * 2000-03-13 2004-07-27 Hewlett-Packard Development Company, L.P. Apparatus for and method of memory-affinity process scheduling in CC-NUMA systems
CN1664784A (zh) * 2005-03-30 2005-09-07 中国人民解放军国防科学技术大学 大规模并行计算机系统分组并行启动方法
CN101273332A (zh) * 2005-09-30 2008-09-24 英特尔公司 使用编译器的线程数据亲和性优化
CN1991795A (zh) * 2005-12-28 2007-07-04 国际商业机器公司 用于信息处理的系统和方法
US20080294832A1 (en) * 2007-04-26 2008-11-27 Hewlett-Packard Development Company, L.P. I/O Forwarding Technique For Multi-Interrupt Capable Devices
CN101216815A (zh) * 2008-01-07 2008-07-09 浪潮电子信息产业股份有限公司 一种双翼可扩展多处理器紧耦合共享存储器体系结构

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11057302B2 (en) 2016-11-30 2021-07-06 New H3C Technologies Co., Ltd. Sending packet

Also Published As

Publication number Publication date
CN102318275B (zh) 2015-01-07
CN102318275A (zh) 2012-01-11

Similar Documents

Publication Publication Date Title
WO2012119369A1 (fr) Procédé, dispositif et système de traitement de message basés sur une architecture cc-numa
US9256500B2 (en) Physical domain error isolation and recovery in a multi-domain system
US8346997B2 (en) Use of peripheral component interconnect input/output virtualization devices to create redundant configurations
TWI237171B (en) Tree based memory structure
US20190235777A1 (en) Redundant storage system
US11330071B2 (en) Inter-process communication fault detection and recovery system
US7962782B2 (en) Modifying connection records
WO2015035574A1 (fr) Procédé de traitement de panne, système informatique, et appareil
US10785350B2 (en) Heartbeat in failover cluster
JPH09185594A (ja) 直接バルク・データ転送
WO2015135383A1 (fr) Procédé de migration de données, dispositif, et système informatique
WO2019090997A1 (fr) Procédé et dispositif d'acquisition de données, dispositif informatique et support d'informations
WO2014206078A1 (fr) Procédé, dispositif et système d'accès à une mémoire
US9100443B2 (en) Communication protocol for virtual input/output server (VIOS) cluster communication
US20210096939A1 (en) Fault Tolerance Processing Method, Apparatus, and Server
JP2017537404A (ja) メモリアクセス方法、スイッチ、およびマルチプロセッサシステム
CN107209725A (zh) 处理写请求的方法、处理器和计算机
US8305883B2 (en) Transparent failover support through pragmatically truncated progress engine and reversed complementary connection establishment in multifabric MPI implementation
WO2019119269A1 (fr) Procédé de détection de défaillance de réseau et dispositif centre de commande
WO2022155919A1 (fr) Procédé et appareil de gestion des pannes, et système
US20220035742A1 (en) System and method for scalable hardware-coherent memory nodes
JP2006053896A (ja) コンピュータシステムの複数の処理ノードを接続するファブリック数のソフトウェアトランスペアレントな拡張
WO2023072048A1 (fr) Procédé de stockage réseau, système de stockage, unité de traitement de données et système informatique
CN113626139B (zh) 一种高可用的虚拟机存储方法及装置
TWI571077B (zh) 整合網路裝置及其服務整合方法

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201180001573.0

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11860127

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11860127

Country of ref document: EP

Kind code of ref document: A1