CN115858146A - Memory expansion system and computing node - Google Patents

Memory expansion system and computing node Download PDF

Info

Publication number
CN115858146A
CN115858146A CN202211401304.XA CN202211401304A CN115858146A CN 115858146 A CN115858146 A CN 115858146A CN 202211401304 A CN202211401304 A CN 202211401304A CN 115858146 A CN115858146 A CN 115858146A
Authority
CN
China
Prior art keywords
memory
control chip
interface
module
communication network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211401304.XA
Other languages
Chinese (zh)
Inventor
何轲
王晨赳
李晓霖
王伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202211401304.XA priority Critical patent/CN115858146A/en
Publication of CN115858146A publication Critical patent/CN115858146A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Computer And Data Communications (AREA)

Abstract

The application provides a memory expansion system and a computing node, and the system comprises: the system comprises a computing node and a remote memory node which is in communication connection through a target internet. The computing node comprises a first processor, a first control chip connected with the first processor through a first interface running a memory expansion protocol, and a first communication network card connected to the first control chip. The memory node comprises a second control chip, a second communication network card connected to the second control chip and an expanded memory; the first communication network card and the second communication network card correspond to the target internet. By the scheme, the interconnection between the computing node and the memory node at any position can be realized, so that the remote memory expansion of the computing node is realized.

Description

Memory expansion system and computing node
Technical Field
The invention relates to the technical field of internet, in particular to a memory expansion system and a computing node.
Background
A Data Center (DC for short) is a place for processing and storing mass Data, and is composed of a plurality of servers, storage devices, network devices, a power distribution system, a refrigeration system, and the like, which are deployed in different machine rooms. With the rise of graph calculation, big data analysis and deep learning, the server in the data center has an increasingly large demand on a large-capacity memory. However, since the conventional server usually uses fixed computing and memory resources, such as a fixed ratio of the number of CPU cores to the memory capacity, memory fragments are easily generated, and it is difficult to effectively utilize the memory. Moreover, when the available memory resources are limited to the server, it is difficult to improve the memory utilization, for example, the remaining memory of the server a cannot be shared with the server B, and the memory capacity of a single server has an upper limit. Therefore, aiming at the problems of the memory utilization rate and the dynamic property of the data center, a memory resource pooling technology of calculation and memory decoupling is provided at present, and by means of the calculation and the memory decoupling, flexible matching of a CPU and a memory and rapid memory configuration lifting can be achieved, memory fragments are effectively reduced, and the requirements of some applications on a large-capacity memory are met.
The term "pooling of memory resources" refers to establishing a memory resource pool by using dedicated memory nodes, so that a plurality of computing nodes (e.g., servers in a DC) can statically/dynamically use the memory nodes in the memory resource pool in a certain manner, thereby achieving the purpose of memory expansion of the computing nodes.
Technologies that can currently implement the above Memory expansion may include Remote Direct Memory Access (RDMA) technology and Compute Express Link (CXL) technology.
Although the RDMA technology can realize access to a memory node in a remote memory resource pool, native memory semantics cannot be supported by RDMA memory access, and a user is required to explicitly call a specific API in a user state to perform an access operation on a remote memory: RDMA Verbs API, which reduces development efficiency. Moreover, the virtual Memory space seen by the user at the system level is actually an address space mapped after Memory Registration (Memory Registration) is completed through an RDMA network card, and can only be operated through the above specific API provided by RDMA, and cannot be natively managed through the virtual Memory management function of the operating system. Memory registration requires operating system kernel involvement, which can introduce latency overhead and CPU utilization. In addition, RDMA requires maintaining a Queue Pair (QP) for each connection in memory, and in a system composed of multiple computing nodes, the number of QPs is large, which results in excessive consumption of memory resources. Overall, the RDMA memory access form introduces intrusive code modification and operating system call overhead compared to native memory semantics.
The CXL protocol is an interconnection protocol with cache coherency that can be used for processors, memory expansion, and accelerators, and can implement pooling of memory resources. The current CXL protocol mainly includes three protocols: cxl.io, cxl.cache and cxl.mem, wherein cxl.io is mandatory, and the cxl.mem protocol is directed to a scenario mainly for memory expansion. Mem has the greatest advantage of being capable of maintaining native memory semantics for memory expansion. The maximum limitation of the current CXL technology lies in the limitation of physical transmission distance, because the development of the current protocol only concerns the interconnection condition of the Rack-level, and the interconnection between any computing node (such as any server in a data center) and a memory node cannot be achieved.
Disclosure of Invention
The embodiment of the invention provides a memory expansion system and a computing node, which are used for realizing remote memory expansion of the computing node.
In a first aspect, an embodiment of the present invention provides a memory expansion system, where the system includes:
the system comprises a computing node and a remote memory node which is in communication connection through a target internet;
the computing node comprises: the system comprises a first processor, a first control chip and a first communication network card, wherein the first control chip is connected with the first processor through a first interface running a memory expansion protocol, and the first communication network card is connected to the first control chip;
the memory node comprises: the second control chip is connected with a second communication network card and an expansion memory on the second control chip; the first communication network card and the second communication network card correspond to the target internet.
In a second aspect, an embodiment of the present invention provides a computing node, including:
the system comprises a first processor, a first control chip and a first communication network card, wherein the first control chip is connected with the first processor through a first interface running a memory expansion protocol, and the first communication network card is connected to the first control chip and corresponds to a target internet;
the computing node is in communication connection with the memory node through the first communication network card and a second communication network card arranged in the memory node at the far end;
wherein, the memory node includes: the second processor, a second control chip connected with the second processor through a second interface, an extended memory connected to the second control chip, and the second communication network card corresponding to the target internet.
The memory expansion system provided by the embodiment of the invention comprises two types of nodes, namely a computing node and a memory node, wherein the computing node is a node which needs to perform remote memory expansion, and the memory node is a node contained in a memory resource pool. In practice, the memory nodes may be obtained by inserting extended memory on the compute nodes. In the embodiment of the invention, the computing nodes and the memory nodes are not limited by physical distance, and the interconnection between the computing nodes at any position and the memory nodes at any position can be realized. Specifically, the computing nodes and the memory nodes are connected through a target internet (such as ethernet, optical fiber, etc.). In addition, in order to implement the memory expansion function of the compute node, that is, enable the compute node to use a certain memory node as an external expansion memory, the following settings need to be performed on the compute node and the memory node: the method includes the steps that a first control chip is arranged on a computing node, the first control chip is connected with a first processor (such as a CPU) in the computing node through a first interface running a memory extension protocol (such as CXL (extensible markup language) mem protocol), and a first communication network card corresponding to a target internet is inserted into the first control chip. And a second control chip, a second communication network card connected to the second control chip and an expansion memory bank are arranged in the memory node. Based on the above configuration of the computing node and the memory node, the first control chip provides a memory expansion protocol processing function on one hand, and provides a data message transmission function of a target internet on the other hand, so that the computing node can use the memory bank inserted on the second control chip as an expansion memory of the computing node based on the memory expansion protocol, and the target internet is used for bearing a memory access request of the memory expansion protocol, so that the access request can reach a remote expansion memory, and access to any remote expansion memory is realized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic diagram of a memory expansion system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a data center and a memory resource pool according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a memory expansion system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a memory expansion system according to an embodiment of the present invention;
fig. 5 is a schematic application diagram of a memory expansion system according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a compute node according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In addition, the sequence of steps in each method embodiment described below is only an example and is not strictly limited.
Fig. 1 is a schematic diagram of a memory expansion system according to an embodiment of the present invention, and as shown in fig. 1, the memory expansion system includes: a computing node 100 and a remote memory node 200 communicatively coupled via a target internet.
Among them, the computing node 100 includes: the system comprises a first processor 101, a first control chip 103 connected with the first processor 101 through a first interface 102 running a memory extension protocol, and a first communication network card 104 connected to the first control chip 103. The memory node 200 includes: a second control chip 201, a second communication network card 202 and an expansion memory 203 connected to the second control chip 201. The first communication network card 104 and the second communication network card 202 correspond to a target internet network.
As shown in fig. 1, in practice, one or more network devices, such as switches, routers, etc., may be included in the interconnected network of computing nodes 100 and memory nodes 200.
In practical applications, the target internet may be a communication network such as an ethernet network and an optical fiber network that can implement local area network connection, or may be a mobile communication network such as a 4G network and a 5G network. The remote interconnection between the computing node at any position and the memory node at any position can be realized through the target interconnection network.
The two control chips can adopt logic programmable chips such as FPGA.
The extended memory 203 refers to an extended memory bank additionally inserted into a memory node, such as a DRAM illustrated in the figure, but not limited thereto.
In an alternative embodiment, the computing node 100 may be any server in a data center, and the memory node 200 may be any memory node (different from the computing node 100) in a memory resource pool constructed based on the data center.
For convenience of understanding, as shown in fig. 2, it is assumed that the data center includes a machine room a and a machine room B distributed in different cities, where the machine room a and the machine room B respectively include a plurality of servers, N servers (N may be all or part of the servers) are selected from the servers, the second control chip 201 is inserted into each selected server, and the extended memory 203 and the second communication network card 202 are inserted into the second control chip 201, so that the selected servers can form a memory node in the memory resource pool.
In this hypothetical scenario, compute node 100 may be any server in room a and room B that is not contained in the memory resource pool. Of course, it is understood that the memory nodes in the memory resource pool may also include devices that are not located in the data center.
In an alternative embodiment, as shown in fig. 3, the memory node 200 includes a second processor 204, and the second processor 204 is connected to the second controller chip 201 through a second interface 205, so that the second controller chip 201 is an external device of the second processor 204.
It will be appreciated that the memory node may actually be a device with computing capabilities, in which the second processor 204 (e.g., CPU) may be disposed. Only, the memory node can be used as a memory node in the memory resource pool by setting the memory node as above.
Whether the compute node 100 or the memory node 200 actually includes a local memory originally configured, such as a host memory illustrated in fig. 3. The logic of the processor to access the local memory is not affected.
The two control chips respectively disposed in the compute node 100 and the memory node 200 are connected to the respective processors in different ways. Specifically, the first control chip 103 in the computing node 100 is connected to the first processor 101 through the first interface 102 running the memory extension protocol, and the second control chip 201 in the memory node 200 is connected to the second processor 204 through the second interface 205.
In fact, optionally, the memory expansion protocol may be a cxl.mem protocol, the first interface 102 may be a PCIe interface running the cxl.mem protocol, and the second interface 205 may be a PCIe interface running the PCIe protocol.
That is, although both are PCIe interfaces from the physical layer perspective, the usage patterns of the interfaces are different, with CXL usage patterns being employed in the computing node 100 and PCIe usage patterns being employed in the memory node 200. The protocol standards that operate in different usage modes are different.
The CXL protocol was developed based on PCIe5.0, runs over the PCIe physical layer, and has the same electrical characteristics. In practical applications, a specific port is required to configure the usage mode of the PCIe interface, and when the first control chip 103 is inserted into the computing node 100 supporting the cxl.mem protocol, it is configured to determine that the first control chip 103 uses the cxl.mem protocol. When the second controller chip 201 is inserted into the memory node 200, the memory node is configured to use the PCIe protocol.
When the second controller chip 201 is connected to the second processor 204 through the PCIe interface running the PCIe protocol, actually, the second controller chip 201 exists only as a general external device to the second processor 204. Although the extended memory 203 is connected to the second control chip 201, at this time, the memory expansion protocol is not run in the memory node 200, that is, the extended memory 203 is not used as an external extended memory of the memory node by the second processor 204 through the memory expansion protocol (for example, cxl.mem protocol).
Since the first control chip 103 is inserted into the compute node 100 supporting the memory expansion protocol, the first control chip 103 may use the expanded memory 203 on the memory node 200 side as an external expanded memory of the compute node 100 based on the memory expansion protocol and a target interconnection network between the compute node 100 and the memory node 200, thereby expanding the available memory capacity of the compute node 100.
Therefore, in order to accomplish the above purpose of memory expansion, the first control chip 103 needs to run a memory expansion protocol on one hand and a communication protocol corresponding to a target internet on the other hand, that is, can process a message processing logic corresponding to the target internet.
Taking ethernet as an example, a special message related to memory access is not defined in a current data frame format of ethernet, and therefore, in the embodiment of the present invention, in order to enable the ethernet to carry an access request of the computing node 100 to the memory node 200, it is necessary to define a plurality of related messages based on a known ethernet data frame structure, which will be described below.
The memory extension protocol is referred to as cxl.mem protocol as an example and will be described below. The following will illustrate the internal functional components of the first control chip 103 and the second control chip 201 in conjunction with fig. 4.
As shown in fig. 4, the first control chip 103 includes a first module 401 corresponding to the cxl.mem protocol and a second module 402 corresponding to the target internet for performing packet processing. The second control chip 201 includes a third module 403 and a memory controller 404, which are corresponding to the target internet and used for performing message processing. Each of the above functional modules may be formed by a corresponding hardware unit.
Different modules on the first control chip 103 are connected through a third Interface, and similarly, all the functional modules on the second control chip 201 are also connected through a third Interface, and the third Interface is a bus Interface originally adopted on the control chip, for example, an Advanced eXtensible Interface (AXI) Interface.
In practical applications, the first module 401 may be a CXL IP core obtained from a third party, where the IP core refers to an Intellectual Property core, that is, an Intellectual Property core, and includes a link layer and a transaction layer processing function of the cxl.mem protocol. In an optional embodiment, since a CXL-Mem Protocol Interface (CPI) corresponding to the cxl.mem Protocol is different from a third Interface (for example, an AXI Interface) used when performing signal transmission between different modules on the first control chip 103, the first module 401 may further provide a signal conversion function, where the signal conversion function is to complete format conversion between an Interface signal format corresponding to the first Interface and an Interface signal format corresponding to the third Interface.
The CPI interface refers to the interface standard of the protocol logic, i.e. the software layer, and the above first interface: the PCIe interface refers to an interface of a physical layer. The signal conversion function is directly integrated in the first module 401, so that the signal output by the first module 401 is directly matched with the signal format supported by other modules on the first control chip 103, and the signal conversion function has better convenience.
In practical applications, the first module 401 receives a CXL access request (which refers to a memory access request triggered by a cxl.mem protocol) from the first processor 101, performs processing on a link layer and a transaction layer of the cxl.mem protocol, then performs conversion of an interface signal format (for example, converting from a CPI interface signal format to an AXI interface signal format), then sends a processing result to the second module 402, and the second module 402 performs encapsulation processing conforming to a message format transmitted on a target internet. The second module 402 and the third module 403 are actually used for processing multiple messages defined based on a preset data frame format corresponding to a target internet, and in short, complete processing such as encapsulation and decapsulation of a message suitable for transmission on the target internet.
In the embodiment of the present invention, the following messages corresponding to the memory access scenario are defined based on a preset data frame format corresponding to the target internet: a write request message, a read request message, a Negative acknowledgement (Negative acknowledgement) message, a write acknowledgement message, and a read acknowledgement message. Different memory read-write parameters (namely fields) are set in a load (payload) field of the preset data frame format to respectively form the multiple messages. The memory read-write parameters comprise: command type, memory address, data, message sequence number, and may also include a timestamp, an AXI signal field, and the like. The preset data frame format includes a normal Source Address (SA), a Destination Address (DA), a checksum (CRC), a Preamble field, a frame Type (Type) field, etc., in addition to the payload field.
The command types include command types corresponding to the five messages respectively: write request, read request, write acknowledgement, read acknowledgement, negative acknowledgement. The memory address refers to a memory address that needs to be read or written. Data refers to data to be written to or read from memory. The AXI signal field includes an AXI identification (id) signal for the request, and an AXI feedback (resp) signal when the result is returned. And the message sequence number is used for marking the sequence number of the message and is used for a reliable retransmission algorithm. Time stamps, which can be used in flow control algorithms.
Taking the case that the computing node 100 sends a memory access request to the memory node 200 as an example, at this time, the first module 401 is configured to process a memory read request received from the first processor 101, and send a processing result to the second module 402. The processing of the first module 401 is as described above, and includes the link layer, the transaction layer, and the interface signal format conversion processing of the cxl.mem protocol. The second module 402 is configured to obtain a processing result of the first module 401, perform packet encapsulation processing corresponding to the target internet on the processing result, and send an encapsulated packet to the first communication network card 104. The first communications network card 104 sends the message to the second communications network card 202. The second communications network card 202 sends the received message to the third module 403, and the third module 403 decapsulates the message and sends the decapsulated message to the memory controller 404. The memory controller 404 performs corresponding read/write operations on the extended memory 203 according to the decapsulated message.
In the embodiment of the present invention, the specific implementation of the memory controller 404 and the extended memory 203 is not limited. For example, the memory controller 404 may be a DDR (Double Data Rate) controller, such as a DDR4 or DDR5 type, although other memory types may be supported. The form of the memory controller 404 connected to the extended memory 203 is not limited, and may be some form of memory slot.
The above process may be a processing flow when the first processor 101 triggers a memory read request or a memory write request, and at this time, the second module 402 encapsulates a read request message or a write request message.
The read request message carries information such as a memory address to be read, a read request command identifier, a message serial number, a timestamp, and the like. After the third module 403 is decapsulated, a corresponding read instruction is triggered to the memory controller 404, and the memory controller 404 feeds back the read data to the third module 403, so that the third module 403 encapsulates the read data to generate a read response message, and the read data is carried in the read response message and fed back to the computing node 100. After receiving the packet, the second module 402 decapsulates the packet to obtain data contained therein, and then sends the data to the first module 401, where the first module 401 performs signal format conversion on the data, and then sends the data to the first processor 101 through a cxl. Similarly, if the second module 402 encapsulates the write request packet, the third module 403 decapsulates the write request packet to obtain the memory address and the write data contained therein, sends a corresponding write instruction to the memory controller 404, and the memory controller 404 writes the data into the corresponding memory address and feeds back acknowledgement information to the third module 403 after the data is successfully written into the corresponding memory address. The third module 403 encapsulates the write reply message for sending to the computing node 100.
The negative response packet may be a negative response packet that is sent by the third module 403 to the computing node 100 and corresponds to a sequence number of a packet when the third module 403 finds that a packet with a certain sequence number is not successfully parsed based on continuously received write request packets, so as to request retransmission of the packet.
In an optional embodiment, the second module 402 is further configured to control a sending rate of the encapsulated packet based on a set congestion control algorithm, so as to implement congestion control. For example, a Priority-Based Flow Control (PFC) algorithm may be used. The congestion control algorithm refers to the related art, and is not described herein.
Fig. 5 is a schematic application diagram of a memory expansion system according to an embodiment of the present invention. In this embodiment, a CXL over Ethernet (Ethernet) -based memory pooling architecture is proposed in combination with the features of the current CXL and RDMA architectures. The overall architecture is as shown in fig. 5, and is divided into two parts, namely a transmitting end and a receiving end.
The core is that on the basis of keeping the CXL.mem protocol externally, a sending end used as a computing node intercepts data and control content processed by the CXL.mem protocol through a Client Logic (corresponding to a second module in the above), namely intercepts an output result of the CXL IP core, combines a self-defined message structure and an Ethernet physical layer protocol, and can be extended to an Ethernet network and a switch, so that the CXL.mem protocol content is extended to a memory node at any position across positions. The memory node as the receiving end uses the corresponding Logic of the Server Logic (corresponding to the third module above) to analyze the self-defined message, and the analyzed content is operated for the local DRAM. When data is returned, the receiving end carries out self-defined message packet on the returned data through the Server Logic, after the data is transmitted through the Ethernet link, the receiving end unpacks the data through the Client Logic at the transmitting end, and returns the returned information to the CPU through the CXL IP core, thereby completing one memory access.
As shown in fig. 5, in the architecture of the memory expansion system, two FPGA boards are used: the system comprises an FPGA1 and an FPGA2, wherein the FPGA1 is connected with a computing node, mainly provides two functional modules of a CXL IP core and a Client Logic, and can be connected with a CPU through an interface for operating a CXL. Specifically, the CPU of the compute node includes a CPU core, a last-level cache (LLC for short), and a CXL RP (Root Port), and the CPU calls the CXL IP core through the CXL RP Port. And the other FPGA2 board card is arranged at the memory node and mainly provides a Server Logic function and DRAM resources to form a remote memory resource. The FPGA2 has the main functions of providing large-capacity onboard DRAM resources, constructing high-density and large-capacity DRAM resource nodes and expanding the memory capacity of a CPU (central processing unit) at the side of a computing node. The two FPGA board cards can be interconnected through network equipment such as an Ethernet (Ethernet) network cable and a Switch (Switch). In the framework, the software layer mainly relates to the setting of a self-defined message structure and the design of Client Logic and Server Logic functions.
Self-defining a message structure: mem defines at least one data packet as shown in table 1 below, taking into account the read-write semantics and link reliability transmission characteristics.
TABLE 1 CXL over Ethernet custom message structure
Figure BDA0003935120770000071
Mem includes four read-write semantics: the Master device (Master) writes data to the Slave device (Slave), the Master device (Master) reads the data from the Slave device (Slave), the Slave device returns the data to the Master device, and the Slave device feeds back a write response to the Master device. Based on the above, four Ethernet message types of the write request, the read request, the write response and the read response are defined. Wherein, the length of data that can be carried in each message is 64B.
The NAK without Data is the negative response message, and the NAK without Data means that the message does not carry Data read from the extended memory.
The SACK + NAK without Data is a message including two fields, SACK (Selective acknowledgement) and NAK. SACK + NAK + ACK with Data means that a message includes SACK, NAK and ACK fields, and also includes Data returned to the transmitting end.
For the two function modules, namely, the Client Logic and the Server Logic, referring to the description of the second module and the third module in the foregoing, the Client Logic function mainly includes: the encapsulation logic (sending time) and the decapsulation logic (receiving time) aiming at the self-defined message structure; and (3) congestion control algorithm: controlling a transmission rate based on a link congestion state; sending and receiving an execution logic of a reliability retransmission mechanism; AXI interface format conversion. The Server Logic function mainly comprises: and aiming at the unpacking logic (receiving time) and the packing logic (sending time) of the self-defined message structure, the execution logic of a sending and receiving reliability retransmission mechanism.
Fig. 6 is a schematic structural diagram of a computing node according to an embodiment of the present invention, where the computing node may be a server in a data center. As shown in fig. 6, the computing node includes: the system comprises a first processor, a first control chip and a first communication network card, wherein the first control chip is connected with the first processor through a first interface running a memory expansion protocol, and the first communication network card is connected to the first control chip and corresponds to a target internet. The computing node is in communication connection with the memory node through the first communication network card and a second communication network card arranged in the memory node at the far end.
Wherein, the memory node includes: the second processor, a second control chip connected with the second processor through a second interface, an extended memory connected to the second control chip, and the second communication network card corresponding to the target internet.
As described above, optionally, the first control chip includes a first module corresponding to the memory extension protocol and a second module corresponding to the target internet for performing message processing. Correspondingly, the second control chip comprises a third module and a memory controller, wherein the third module and the memory controller are corresponding to the target internet and are used for processing the message.
Optionally, different modules on the first control chip are connected through a third interface. When the memory extension protocol is a CXL.mem protocol, the first module provides a link layer and transaction layer processing function of the CXL.mem protocol and provides a signal conversion function; the signal conversion function is to complete format conversion between an interface signal format corresponding to the first interface and an interface signal format corresponding to the third interface.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (12)

1. A memory expansion system, comprising:
the system comprises a computing node and a remote memory node which is in communication connection through a target internet;
the computing node comprises: the system comprises a first processor, a first control chip and a first communication network card, wherein the first control chip is connected with the first processor through a first interface running a memory expansion protocol, and the first communication network card is connected to the first control chip;
the memory node comprises: the second control chip is connected with a second communication network card and an expansion memory on the second control chip; the first communication network card and the second communication network card correspond to the target internet.
2. The system according to claim 1, wherein the memory node includes a second processor, and the second processor is connected to a second control chip through a second interface, so that the second control chip serves as an external device of the second processor.
3. The system of claim 2, wherein the memory expansion protocol comprises a cxl.mem protocol, wherein the first interface is a PCIe interface running the cxl.mem protocol, and wherein the second interface is a PCIe interface running the PCIe protocol.
4. The system according to any one of claims 1 to 3, wherein the first control chip includes a first module corresponding to the memory extension protocol and a second module corresponding to the target internet for performing message processing;
the second control chip comprises a third module and a memory controller which are corresponding to the target internet and used for processing messages.
5. The system according to claim 4, wherein different modules on the first control chip are connected with each other through a third interface; the memory extension protocol comprises a CXL.mem protocol;
the first module provides link layer and transaction layer processing functions of a CXL.mem protocol and provides a signal conversion function; the signal conversion function is to complete format conversion between an interface signal format corresponding to the first interface and an interface signal format corresponding to the third interface.
6. The system according to claim 4, wherein the second module and the third module are configured to process at least one of the following packets defined based on a preset data frame format corresponding to the target internet:
a write request message, a read request message, a negative response message, a write response message and a read response message;
setting different memory read-write parameters in the load domain of the preset data frame format to respectively form the at least one message, wherein the memory read-write parameters comprise: command type, memory address, data, message sequence number.
7. The system according to claim 4, wherein the first module is configured to process the memory read request received from the first processor, and send a processing result to the second module;
the second module is used for performing message encapsulation processing corresponding to the target internet on the processing result and sending the encapsulated message to the first communication network card;
the first communication network card is used for sending the message to the second communication network card;
the third module is used for performing decapsulation processing on the message received through the second communication network card and sending the decapsulated message to the memory controller;
and the memory controller is used for performing corresponding read-write operation on the extended memory according to the decapsulated message.
8. The system according to claim 7, wherein the second module is further configured to control a sending rate of the encapsulated packet based on a set congestion control algorithm.
9. The system of claim 1, wherein the compute node comprises any server in a data center, and the memory node comprises any memory node in a memory resource pool constructed based on the data center.
10. A computing node, comprising:
the system comprises a first processor, a first control chip and a first communication network card, wherein the first control chip is connected with the first processor through a first interface running a memory expansion protocol, and the first communication network card is connected to the first control chip and corresponds to a target internet;
the computing node is in communication connection with the memory node through the first communication network card and a second communication network card arranged in the memory node at the far end;
wherein, the memory node includes: the second processor, a second control chip connected with the second processor through a second interface, an extended memory connected to the second control chip, and the second communication network card corresponding to the target internet.
11. The computing node according to claim 10, wherein the first control chip includes a first module corresponding to the memory extension protocol and a second module corresponding to the target internet for performing packet processing;
the second control chip comprises a third module and a memory controller which are corresponding to the target internet and used for processing messages.
12. The computing node of claim 11, wherein different modules on the first control chip are connected via a third interface; the memory extension protocol comprises a CXL.mem protocol;
the first module provides link layer and transaction layer processing functions of a CXL.mem protocol and provides a signal conversion function; the signal conversion function is to complete format conversion between an interface signal format corresponding to the first interface and an interface signal format corresponding to the third interface.
CN202211401304.XA 2022-11-09 2022-11-09 Memory expansion system and computing node Pending CN115858146A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211401304.XA CN115858146A (en) 2022-11-09 2022-11-09 Memory expansion system and computing node

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211401304.XA CN115858146A (en) 2022-11-09 2022-11-09 Memory expansion system and computing node

Publications (1)

Publication Number Publication Date
CN115858146A true CN115858146A (en) 2023-03-28

Family

ID=85662904

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211401304.XA Pending CN115858146A (en) 2022-11-09 2022-11-09 Memory expansion system and computing node

Country Status (1)

Country Link
CN (1) CN115858146A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116501140A (en) * 2023-06-20 2023-07-28 苏州浪潮智能科技有限公司 Memory module and memory expansion board card of server
CN116501681A (en) * 2023-06-28 2023-07-28 苏州浪潮智能科技有限公司 CXL data transmission board card and method for controlling data transmission
CN116886751A (en) * 2023-09-04 2023-10-13 浪潮(北京)电子信息产业有限公司 High-speed communication method and device of heterogeneous equipment and heterogeneous communication system
CN117009264A (en) * 2023-09-13 2023-11-07 上海云豹创芯智能科技有限公司 Method, system, chip and storage medium for realizing high-speed memory expansion in SOC

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116501140A (en) * 2023-06-20 2023-07-28 苏州浪潮智能科技有限公司 Memory module and memory expansion board card of server
CN116501140B (en) * 2023-06-20 2024-01-26 苏州浪潮智能科技有限公司 Memory module and memory expansion board card of server
CN116501681A (en) * 2023-06-28 2023-07-28 苏州浪潮智能科技有限公司 CXL data transmission board card and method for controlling data transmission
CN116501681B (en) * 2023-06-28 2023-09-29 苏州浪潮智能科技有限公司 CXL data transmission board card and method for controlling data transmission
CN116886751A (en) * 2023-09-04 2023-10-13 浪潮(北京)电子信息产业有限公司 High-speed communication method and device of heterogeneous equipment and heterogeneous communication system
CN116886751B (en) * 2023-09-04 2024-01-19 浪潮(北京)电子信息产业有限公司 High-speed communication method and device of heterogeneous equipment and heterogeneous communication system
CN117009264A (en) * 2023-09-13 2023-11-07 上海云豹创芯智能科技有限公司 Method, system, chip and storage medium for realizing high-speed memory expansion in SOC
CN117009264B (en) * 2023-09-13 2023-12-19 上海云豹创芯智能科技有限公司 Method, system, chip and storage medium for realizing high-speed memory expansion in SOC

Similar Documents

Publication Publication Date Title
CN115858146A (en) Memory expansion system and computing node
US10732879B2 (en) Technologies for processing network packets by an intelligent network interface controller
KR101686360B1 (en) Control messaging in multislot link layer flit
US7817634B2 (en) Network with a constrained usage model supporting remote direct memory access
CN111131037B (en) Data transmission method, device, medium and electronic equipment based on virtual gateway
US9037640B2 (en) Processing STREAMS messages over a system area network
CN109491809A (en) A kind of communication means reducing high-speed bus delay
US20150055649A1 (en) Cell fabric hardware acceleration
JP2004534311A (en) Configuration to create multiple virtual queue pairs from compressed queue pairs based on shared attributes
EP2824579A1 (en) Storage expansion device and server
KR20090104137A (en) Processor chip architecture having integrated high-speed packet switched serial interface
Shu et al. Direct Universal Access: Making Data Center Resources Available to {FPGA}
TW201340663A (en) Server system and management method thereof
CN113852656B (en) Data transmission method, processor system and memory access system
US10609125B2 (en) Method and system for transmitting communication data
EP4357901A1 (en) Data writing method and apparatus, data reading method and apparatus, and device, system and medium
KR101559089B1 (en) Communication protocol for sharing memory resources between components of a device
WO2022068744A1 (en) Method for obtaining message header information and generating message, device, and storage medium
CN114697387B (en) Data packet transmission method, device and storage medium
CN113285839A (en) Design method of programmable data plane software switch based on DPDK
CN114138707B (en) Data transmission system based on FPGA
CN115499889A (en) VPP-based UPF forwarding plane implementation method, device, system and related equipment
CN101312412A (en) Ethernet transmission system based on embedded technique
Krishnan Evaluation of an integrated PCI Express IO expansion and clustering fabric
CN115001627B (en) InfiniBand network subnet management message processing method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination