KR20160119050A - Disaggregated memory appliance - Google Patents

Abstract

The present invention relates to a method and system for managing one or more leaf memory modules, a plurality of leaf memory switches for managing one or more memory channels, and, via a host link, And a low-latency memory switch that connects the memory devices.

Description

Disposed memory device {DISAGGREGATED MEMORY APPLIANCE}

With large data center configurations, it is difficult to effectively provision these resources so that CPU, memory, and persistent memory resources are efficiently used by the systems. By way of example, memory is often over-provisioned, which prevents a large amount of memory from being "stranded" across multiple servers. There is a need for solutions that share and dynamically allocate these resources to multiple processors or instances so that resources of large Pools (e.g., Dynamic Memory) are efficiently utilized and no resources are stranded Do.

Furthermore, many computer applications (e.g., data center applications) require large amounts of DRAM memory. Unfortunately, it is becoming increasingly difficult to add more memory to server systems. Among other factors, increasing bus speeds causes the number of modules in the system to substantially decrease over time due to problems with signaling. On the other hand, applications using servers are increasingly ahead of the system's ability and require an increasing amount of DRAM memory. As an example, in memory databases, a Terabyte (TB) DRAM is required for efficient operation.

The two major issues that need to be addressed are: 1) how to connect a very large number of DRAMs to the memory bus without burdening the bus, 2) how to connect the DRAMs to the available volumetric space within the server, How to physically install them, or how to make ways for memory outside the server enclosure to have low latency.

New methods are needed to enable server systems that increase the amount of DRAMs in the system while maintaining low latency and high interconnect bandwidth. The methods and systems described herein may address one or more of these needs.

One embodiment provides a Disaggregated Memory Appliance. A separate memory device includes a plurality of leaf memory switches for managing one or more memory channels of one or more leaf memory modules and a plurality of leaf memory modules via a host link, And a low-latency memory switch that connects to the memory.

The features and utility of the present invention, and / or other features and utilities, will be apparent from and elucidated with reference to the embodiments described hereinafter with reference to the accompanying drawings.
1 is a diagram illustrating an exemplary data center rack configuration.
Figure 2 is a conceptual illustration of how a computation tier is connected to a shared memory device, such as a dynamic memory tier.
Figure 3 is a more detailed illustration of an embodiment of a memory device.
4 is a diagram showing at least one of the Leaf Memory Switches in more detail.
Figure 5 is a more detailed view of a low latency memory switch.

Hereinafter, the embodiments of the present invention will be described in detail. In the examples shown in the accompanying drawings, like reference numerals refer to like elements throughout. In order to explain the present invention, the following embodiments will be described with reference to the drawings.

The advantages and features of the present invention and methods for achieving this can be better understood with reference to the following detailed description of the embodiments and the accompanying drawings. However, the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the invention to those skilled in the art to which the invention pertains. The invention will only be defined by the appended claims. In the figures, the thicknesses of the layers and regions are exaggerated for clarity.

It is to be understood that, in the context of describing the invention (particularly in the context of the claims), the use of "a", "this", " It should be understood that one, singular and plural are all covered. The terms "comprises", "having", and "includes" are to be understood as open-ended terms (ie, "including, but not limited to") unless otherwise stated do.

The term " component "or " module ", as used herein, refers to software or hardware components that perform particular tasks, such as Field Programmable Gate Array (FPGA) , But is not limited to this. The component or module may advantageously be configured to reside within an addressable storage medium and configured to execute on one or more processors. Thus, a component or module may be implemented as a software component, an object-oriented software component, a class component, a task component, a process, a function, And may be implemented as a set of attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, May include components such as circuitry, data, databases, data structures, tables, arrays, and variables. The functionality and components or modules provided for these components may be combined into fewer components, or the components or modules may be combined with additional components and components or modules Can be further separated.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Each of all examples or exemplary language provided herein is for the purpose of better illustrating the present invention and is not intended to limit the scope of the present invention unless otherwise specified. Further, unless defined otherwise, all terms defined in commonly used dictionaries may be exaggeratedly interpreted.

Embodiments of the present invention provide a Disaggregated Memory Appliance. Separate memory devices enable server systems that increase the amount of DRAM in the system while maintaining low latency and high interconnect bandwidth. A separate memory device may be used in the data center and / or other environments.

The methods and systems of embodiments of the present invention may be implemented as a set of "leaf " memory systems that manage i) DIMMs in a sufficiently small number to accommodate a physical space of a Capacity-limiting Standards such as DDR4. (Ii) using a very low latency switch link (optionally linking a plurality of leaf memory systems to a plurality of hosts) (in some cases, this link is a memory architecture Iii) encapsulating semantics specific to the memory architecture in the Link Protocol, iv) managing, maintaining, configuring, and provisioning memory (which may be agnostic); Use a management processor to accept requests from the hosts for the Provisioning, v) the low latency of the memory system data and metadata (Wormhole Routing) provided during the memory provisioning process so that the endpoints use the Target Routing Data to enable routing of the target routing data . In addition, the method and system may include apparatus, buffers, switch (s), and methodologies for using the above.

By way of example, in various embodiments, the above method and system may be implemented in a system comprising: i) one or more switching layers, ii) a low latency routing protocol, iii) boot, MMU, Atomic Transactions, Simple computational complexes for computed offload, iv) Optional Fabric for linking several memory boxes, v) RAS features, vi) Dynamic memory allocation, vii) Protocol for dynamic allocation of memory Or < / RTI >

Disaggregation is one way to help dynamically allocate resources to various applications and OS instances from a shared pool. This concept is shown in Fig.

1 is a diagram illustrating an exemplary Data Center Rack configuration. The resources of the data center rack 100, typically found in a single server system, are divided into tiers and physically separated into separate enclosures (or even separate racks in the data center, Separated by rows). The three main tiers are a full tier 102, a dynamic memory tier 104 (e.g., DRAM), and a Persistent Memory tier 106 (e.g., flash). The fourth tier may include a hard disk drive tier 108.

The computing tier 102 includes a plurality of processors or CPUs (also referred to as hosts). Each of the dynamic memory tier 104 and the persistent memory tier 106 has large pools of memory resources that can be partially allocated to processors (or VMs, OS instances, threads, etc.) of the computing tier. These memory resources can be allocated at boot time and remain relatively static, or can be continuously adjusted to meet the needs of applications being executed by the processors. In some cases (such as XaaS business models), memory resources may be reassigned to each job running on a particular CPU / VM / OS instance.

Figure 2 is a conceptual illustration of how a computational tier is connected to a shared memory device, such as a dynamic memory tier. It is shown that one of the processors 200 of the computing tier 102 (e.g., CPU or SOC) is connected to one of the memory devices 202 of the dynamic memory tier 104 via the buffer 204. The buffer 204 may be attached to the processor 200 via link 206. In one embodiment, the link 206 may include a conventional high speed link such as DDRx, PCIe, SAS, SATA, QPI, or the like, or may be a new dedicated link. The buffer 204 may have a memory (e.g., DDR4) attached directly thereto such that the buffer 204 operates as a complete memory controller for both local ("near") memory as well as memory device have. The buffer 204 may not be required by itself and may be included as one or more functional blocks on the processor 200.

In addition to having (optionally) a memory directly attached locally, the buffer may be connected to the memory device 202 via a high-speed "host" Since the memory device 202 is typically a separate enclosure, this host link 208 of many embodiments may be cable-based for routing from one enclosure to another. However, the host link 208 may be crossbar based (such as in a large server system or blade-based architecture). The memory device 202 may itself be coupled to a large amount of memory 212 in order to route memory requests and data from the processor 200 to the appropriate memory resources, Switching layers.

Figure 3 is a more detailed illustration of an embodiment of a memory device. The memory device includes a large amount of memory 212. In many embodiments, a large amount of memory 212 is configured as a collection of standard memory modules 223 (such as DDR4 DIMMs) housed in an enclosure of memory device 202. The memory modules 223 may be referred to herein as leaf memory modules 223.

According to one embodiment, the memory device 202 includes a plurality of switching layers. The first switching layer may include a low latency memory switch 210 coupled to a host link 208 that may receive traffic from one or more external processors via the host link 208 ) / Requests. The second switching layer may include a plurality of leaf links 214 connecting the low latency memory switch 210 to the plurality of leaf memory switches 220. The third switching layer may include a plurality of leaf memory switches 220. The plurality of leaf memory switches 220 are connected to one or more memory channels of one or more leaf memory modules 223 (e.g., usually one to three modules in the case of DDR4) and manage these memory channels . Due to the presence of switching layers, the low latency memory switch 210 may optionally couple one or more of the external processors to the leaf memory modules 223.

In one embodiment, the low latency memory switch 210 is capable of managing traffic / requests from a large number of different CPUs or a large number of host links 208 from many different servers. The low latency memory switch 210 may examine the address associated with the incoming traffic / requests and route the traffic / request to the appropriate leaf link in the form of a traffic / request packet. The leaf link receiving the traffic / request packet from the low latency memory switch 210 examines the address field and routes the packet to the memory switch 220 corresponding to the appropriate memory channel. In one embodiment, the low latency memory switch 210 may further include a mesh interface 209 to other memory devices.

The architecture of the leaf links 214 allows for very low latency switching in and of itself. In one embodiment, by way of example, low latency switching includes wormhole switching. As is well known, wormhole switching or wormhole routing is a system for simple flow control in computer networking based on known fixed links. This is a subset of flow control methods called Flit-Buffer Flow Control. Wormhole switching breaks up large network packets into smaller pieces, called flit (flow control digits). The first flit, called the Header Flit, holds information about the path (ie, the destination address) of the packet and sets the Routing Behavior for the packet for all subsequent flits. Zero or more Body Flits lead to the back of the Head Flit, which contains the actual payload of the data. The final flit, called the tail flit, performs a culling function to prevent the connection between the two nodes. The wormhole technique does not indicate a path that directs a packet to a destination. However, the wormhole method determines the path when the packet starts from the router, and allocates buffers and channel bandwidth for the fleet level rather than the packet level.

Thus, to affect the switching of low latency of memory data fleets and metadata, one embodiment utilizes wormhole switching provided during the memory provisioning process, so that endpoints use the target routing data of the memory data fleets. More specifically, the endpoints of the fixed links between the host processors and the memory modules 223 include a concise address, a low latency memory switch 210, and a header of the fleet that allows the leaf 220 to receive the header flit. Decodes the address, re-encodes the address, and routes the payloads of the flits before the data flits reach the switch. The Routing Logic can then decode the other frit from the other Source as soon as the path through the switch is established for the original frit. In FIG. 2, buffer 204 represents the host endpoint, while in FIG. 3 memory switches 220 represent memory module endpoints.

The switching network of an embodiment of the present invention employs wormhole switching. In wormhole switching, i) packets are sent to the flits, 2) the header flit contains all routing information for the packet, and 3) the flits for a given packet are transmitted in a pipeline manner over the switching network , 4) delaying all data frites followed by blocked header flits at intermediate switching nodes, and 5) only one flit needs to be stored in any given switch.

In a further embodiment, the memory device 202 may optionally include a computing complex 216 (e.g., a processor and supporting logic and / or an MMU) to enable various functions . These may include the following:

i) Booting and initial configuration of the memory device

ii) organizing memory allocations for multiple servers or CPU "hosts"

iii) calculating "off-loading". This can reduce memory traffic between the host and the device.

(1) Simple unit operations (e.g., Read-Modify-Write)

(2) application-specific optimizations for Hadoop (eg, Map Reduce), etc.

iv) RAS features, such as:

(1) Memory Sparing

(2) Memory RAID

(3) Failover

(4) Handling Errors and Exceptions (Exception Handling)

(5) Thermal Exception Handling

(6) Throttling

(7) Hot Swap

(8) Local power mode management

The link architecture described herein may utilize wormhole switching to enable very low latency movement of memory data frit between processors and memory subsystems. The switches receive the flit and determine, based on the physical address, when the flit will move and what interconnect will be used to move the flit.

The setting of Wormhole Routing can be accomplished with the help of a computational complex 216 that is potentially included to enable various functions. These may include the following:

i) measuring the topology of interconnects from hosts and DRAM arrays;

ii) passing address information required to generate fleet headers to link end points

iii) RAS features, such as:

(1) Handling Errors and Exceptions

(2) Throttling

(3) Other

A port 218, such as an Ethernet or other network port, enables communication between the computing complex 216 and other systems (including host servers). This is an option for a communications port to configure and manage memory allocations. However, the memory allocation can also be managed through the host links 208 to the memory device 202.

In addition, the memory device 202 may include additional or specialized links to create a structure between the various memory devices. This may be important for high availability features such as alternate operations or mirroring, and may also be used to extend memory capacity to a larger size.

4 is a diagram illustrating at least one of the leaf memory switches 220 in greater detail. The leaf memory switch 220 includes a Leaf Link PHY 502. The leaf memory switch 220 optionally includes a Leaf Link Layer controller 504 for managing the leaf links 214 shown in FIG. The traffic from the low latency memory switch 210 by the leaf links 214 is passed through the leaf link PHY 502 of the Leaflet controller 504 to the Very Low Latency switch 510 Lt; / RTI > The ultra-low latency switch 510 determines which of the one or more DDR channels is the correct destination / source for the traffic. Each DDR channel includes simple / simple memory controllers 508A or 508B, and a pair of PHYs 506A or 506B (e.g., DDRx). Due to the limited memory traffic cases being addressed, the simple memory controllers 508A, 508B can be substantially simplified compared to the controllers usually found in processors.

According to embodiments of the present invention, the leaf memory switch 220 may further include a management processor 512. The management processor 512 accesses the control and data of the simple memory controllers 508A and 508B. Management processor 512 responds to requests from external processors for management, maintenance, configuration, and provisioning of leaf memory modules in memory devices. Communication with the management processor 512 may be made in the low latency memory switch 210 via a management port (not shown). The management processor 512 creates and maintains a configuration and allocation database 514 to manage the physical memory of the memory device 202.

The management processor 512 may be configured to send requests (e.g., via Ethernet) from the host processors, based on policies from the data center resource management service and authentication from the data center authentication service, And accepts and processes them.

Management processor 512 configures memory and leaf memory switches 220 to satisfy requests for memory. Management processor 512 responds to requests by accepting access and providing physical / logical access methods and memory attributes. Alternatively, the management processor 512 responds to requests by denying access based on policy, authentication, or resource constraints. The management processor 512 may supply resources by itself as needed.

The next access by the host processors to the memory device 202 may be controlled by a policy implemented through the configuration of the link, switch and memory control hardware. Management processor 512 does not participate in data movement beyond this configuration, except to access resources that are self-provisioning.

Advantages provided by use of the management processor 512 may include:

i) Enables the provision and configuration of bulk memory for multiple host processors.

ii) Allows customers to dynamically provision optimal computation, memory, and persistence combinations by avoiding stranded resources from available memory.

iii) Allows independent independent CPU, memory, and persistence replacement cycles for each individual technology roadmap

iv) Allows considerable memory capacity per server / processor / core

v) Scalable solution - Enables the addition of more memory subsystem boxes for greater capacity or greater bandwidth

While DRAM technologies are widely used and standardized, device characteristics have evolved over time and require coordination for device interfaces and controllers that manage those interfaces. As an example, a synchronous interface such as DDR may be modified to increase the clock speed to enable higher bandwidth through the interface. As a result, this requires adjustment of the number of clocks that can be required to move the DRAM from one state to the next. Further, other memory technologies may be regarded as replacing or replenishing the DRAM and may be constrained by the same or similar scaling constraints that exist in the DRAMs. These memory technologies may be transactional instead of synchronous, or they may be block-oriented rather than byte-addressable. Furthermore, large-scale deployments can have a life span that extends over the development of these technologies, or may require multiple uses of these techniques in a given deployment. Thus, a given disaggregation of memory in large scale deployments is likely to support various technologies and diverse capabilities within each of these technologies.

Further embodiments of the present invention include a low latency memory switch 10 and a leaf memory switch 220 that encapsulate the semantic specific to the memory technology using tags that uniquely identify the memory technology during provisioning, Lt; / RTI > routing protocol used by all of them. The low latency routing protocol encapsulates the nature of semantic and semantic (block / byte, synchronous / interaction, etc.) and device parameters (CAS latency, erase block size, page write latency, etc.) , And a wide range of memory technologies. The database 514 is padded by the management processor 512 and delivered to the host processors during the provisioning process. Each memory technology supported by a given memory device attaches a unique tag (device unique tag) to each technology setting in the memory device. The device unique tag identifies the semantics and parameters of each technology.

The management processor 512 may discover device semantics and parameters by querying the simple memory controllers 508A and 508B for data describing the attached memory technologies and may use such data to append the database 514 Data is used.

The host processor requesting the memory may consult the management processor 512 to obtain a unique or shared access to the memory and may specify the required technology. The management processor 512 may acknowledge and respond to a memory supply that conforms to the specifications of the hosts, or the supply may be identified as a best-effort match to the host's needs. Alternatively, the management processor 512 may expose the database 514 as a list of available technologies to the host, and the host may request the technology by a tag associated with the required technology. In some cases, as described above, the management processor 512 will provide the tag to identify the technology being delivered to the host.

Following the host's next access to the supplied memory, the description tag will be used by the host to identify the context of the given packet sent to the simple memory controllers 508A, 508B. As an example, a command for deleting a block of memory may be sent by the host to one of the simple memory controllers 508A, 508B. This command may be unique to the flash technology available in the simple memory controllers 508A, 508B, but may have a similar form to commands for other techniques. Therefore, the host can send the tag as a prefix to the command and give it a context. While this context may be implied by accessing a particular simple memory controller 508A, 508B, using tags in the command packet may be monitored, debugged, and packet verified by simple memory controllers 508A, 508B It enables the factor for.

Thus, by utilizing a low latency routing protocol, the memory device 202 may be independent of the memory architecture.

Figure 5 is a more detailed view of a low latency memory switch. As described above, the low latency memory switch 210 is able to manage traffic / requests from a large number of host links 208 that are coming from many different processors / servers. The host links 208 to the memory device may be connected to CPU processors / servers in the following ways:

a) through an existing DDR channel

i) a module-based extender using a buffer / link translator and a cable to the device,

ii) a buffer on the motherboard that uses a dedicated DDR channel (or multiple channels) that translates to a device link;

iii) Dedicated PCIe port to PCIe card or buffer

iv) Buffer-only SAS pots

v) SATA

vi) Others

b) Link signaling solutions can be any of several types

i) Optical

ii) Electrical

iii) Others

c) The link protocol may be as follows:

i) a serialized memory protocol (e.g., serialized DDR4)

ii) Packed

iii) Wormhole Routing Protocol

iv) Others

Memory switches may have varying levels of memory controller functionality (including not doing anything).

In the embodiment in which wormhole switching is used, the 0 < st > Queue to the (M-1) -th queue shown in Fig. 5 will replace the 0th flit buffer to the (M-1) flit buffer.

A separate memory device has been initiated. The invention has been described in accordance with the illustrated embodiments, and variations may be made in the embodiments, and any variations will fall within the spirit and scope of the invention. By way of example, embodiments of the invention may be implemented using hardware, software, computer-readable media including program instructions, or a combination thereof. The software recorded in accordance with the present invention may be stored in some form in a computer-readable storage medium such as a memory, a hard disk, or a CD / DVD-ROM, and may be executed by a processor. Accordingly, many modifications may be made by those of ordinary skill in the art without departing from the spirit and scope of the appended claims.

Claims (8)

A plurality of leaf memory switches for managing one or more memory channels of one or more of the leaf memory modules; And
And a low-latency memory switch that optionally connects, via a host link, one or more external processors to the plurality of leaf memory modules. The method according to claim 1,
At least some of the leaf memory switches further comprise a management processor responsive to requests from the external processors for management, maintenance, configuration, and provisioning of the leaf memory modules within the memory device Lt; / RTI > The method according to claim 1,
Latency memory switch and all of the leaf memory switches that encapsulate the specific semantics of the memory technology using tags that uniquely identify the memory technology during the supply, And a low latency routing protocol used. The method according to claim 1,
In order to enable low latency switching of memory data flits and metadata, the memory device may be configured to perform Wormhole Switching (Wormhole Switching) using target routing data provided during the memory supply process, ). A low-latency memory switch coupled to a host link; And
And a plurality of leaf links connecting the low latency memory switch to a plurality of leaf memory switches,
The low latency memory switch receives traffic / requests from one or more external processors via the host link,
Wherein the plurality of memory switches are connected to one or more of the leaf memory modules, one or more memory channels, and manage the one or more memory channels. 6. The method of claim 5,
At least some of the leaf memory switches further comprise a management processor responsive to requests from the external processors for management, maintenance, configuration, and provisioning of the leaf memory modules within the memory device Lt; / RTI > 6. The method of claim 5,
A low latency routing protocol used by all of the low latency memory switches that encapsulate the specific semantics of the memory technology using a tax uniquely identifying the memory technology during the supply, ). ≪ / RTI > 6. The method of claim 5,
In order to enable low latency switching of memory data flits and metadata, the memory device may be configured to perform Wormhole Switching (Wormhole Switching) using target routing data provided during the memory supply process, ).

Images