TW201706859A - Memory systems - Google Patents

Abstract

A memory system is described which comprises a plurality of memory modules. Each memory module comprises a non-volatile memory unit; and a controller to access information stored in the non-volatile memory unit in response to a request from a processor, to deliver accessed information to the processor, and to communicate with controllers of other memory modules. The controllers are connected so as to form a peer-to-peer network. Each given memory module is associated with a memory address range which is non-overlapping with a memory address range associated with any memory module to which the given memory module is directly connected.

Description

Memory system

The present invention is related to a memory system.

Non-electrical memory is a computer memory that retains stored information even when it is not powered. Computer systems use non-electrical memory (NVM) for long-term, sustainable storage. In some systems, the non-electrical memory system is disposed away from the processor that uses the non-electrical memory, such as in a remote data center, compared to the memory of the processor local to the dedicated memory. It is possible to achieve a larger storage capacity.

According to an embodiment of the present invention, a memory system including a plurality of memory modules is provided, wherein each memory module includes: a non-electric memory unit; and a controller, the control The device is configured to respond to a request from a processor to access information stored in the non-volatile memory unit, to deliver the accessed information to the processor, and to control other memory modules Communicating; wherein the controllers are coupled to form an inter-level memory module network, and wherein each of the given memory modules is associated with a memory address range, the memory address range and One with any record directly connected to the given memory module The memory address ranges associated with the memory module do not overlap.

1‧‧‧ memory system

2‧‧‧First memory component

3‧‧‧Second memory component

10‧‧‧ memory module

20‧‧‧ router controller

21‧‧‧Function guide

22‧‧‧Unit Guide

23, 24‧‧‧Information

25‧‧‧Information

26‧‧‧Event notification message

301~307, 401~408‧‧‧ program block

An example will now be described by way of non-limiting example, in which FIG. 1 is a simplified schematic diagram of an exemplary memory system; FIG. 2 illustrates an exemplary routing arrangement for use with an exemplary memory system. Protocol; Figure 3 is a flow diagram of one example of a method of coupling a controller to an inter-network of an exemplary memory system; and Figure 4 is an example of accessing an exemplary memory system A flow chart of one of the exemplary methods of storing information in the medium.

Many computing tasks involve the processing of large amounts of data. In order to accommodate such large-scale data sets, a processor network having multiple cores and/or connections may be used, and resources may be stored in memory resources located remotely from the processing. However, such an arrangement involves frequent data transfer between remote locations. The time involved in accessing the remote memory resource counteracts the processor speed boosting effect, making it less difficult to reduce the extra load of data processing time.

Recent advances in photonics have enabled optical connections to external memory resources. Such optical interconnects result in reduced latency and higher bandwidth, making access to remote memory resources almost as fast as native access. Therefore, a "memory group" that integrates multiple different memory resources scattered in several racks of a data center (or even scattered among several data centers) Weaving the architecture" is possible.

However, memory resources from different vendors may use a variety of different entities and virtual memory agreements. A memory organization architecture comprising memory resources from different vendors may therefore comprise a set of processor-memory communication that is likely to cause all such vendors and solutions included in the memory organization to communicate with each other. agreement.

It is conceivable that changing the composition of a given memory organization structure may have an effect on adding or removing memory resources, or may be effective when memory resources contained in the organization structure fail. Furthermore, management, utilization, and access characteristics can be dynamically changed. Depending on the number of cores used and the application at hand, the memory that is best suited for a given processor may or may not be available in the application's scheduling. The challenge of designing a memory organization architecture in terms of design is therefore to provide a highly flexible memory management system that can reflect system state changes of the memory resources in the memory organization architecture and access the memory. Changes in the execution phase of the processor of the organizational structure.

A "plug and play" device is a peripheral computer component having a specification that allows the computer system to automatically discover and install the device when it is connected to a computer system. A plug-and-play device can typically be connected to and disconnected from a computer system while a computer system is in operation.

Illustrated below is an example of a system that enables a non-electrical memory component to be dynamically added to a memory system, such as a memory organization, similar to connecting a plug-and-play device to an electrical device. Brain system. The example provides an NVM system in which the configuration of the memory system is not limited to a processor that accesses the memory, regardless of the manner. This means that a memory component can be pulled from a memory system and connected to a different memory system, which will be identical in operation and integration to the original memory system.

FIG. 1 shows an example of a memory system 1. The memory system 1 includes a plurality of memory modules 10. In this example, the memory module 10 is distributed between a first memory component 2 and a second memory component 3. The first memory component 2 and the second memory component 3 may, but need not, be remote from each other. Each memory module 10 includes a non-electrical memory (NVM) unit and a controller. Each controller is responsive to a request from a processor to access information stored in the non-volatile memory unit, to deliver the accessed information to the processor, and to other memory modules The controller communicates. The controllers are connected to form an inter-network. The controllers thereby include peers in the network between peers.

The interconnection of the memory module controllers into peers in an inter-network can enable the memory system to autonomously discover and assemble new memory modules of the system in a decentralized manner (ie, has recently been provided with a memory module for communicating with at least one controller in the network of the same level). Therefore, this memory system is scalable in size and can tolerate the failure of individual memory modules. Furthermore, the exemplary memory system does not require the use of wired interconnects or memory switches. Combinations of these features make it possible to combine memory spaces of NVM units that are independent of each other (ie, NVM units that contain different physical devices, and/or are located in different geographic locations, and/or are not connected to each other). thus The joint memory space created by the exemplary memory system appears to the external processor to be a single memory space, even if the memory modules included in the system change.

Each memory module is associated with a memory address range. In some examples, each memory module is associated with a unique memory address range. These memory address ranges are dynamically specified by the address structure enabled by the basic protocol used by this memory system. In some examples, the memory address range associated with a given memory module is determined during the connection of the controller of the memory module to the inter-network. In some examples, a controller of a given memory module is used to initially connect to a memory system in the memory module (this program may look similar to finding one of the available sub-ranges in an IP network) When determining the memory address range of one of the memory modules. In some such instances, the controller is able to determine the range of available memory addresses by contacting the proximity controller (ie, the controller directly connected to the new junction controller) and taking its memory map. The range of available memory addresses (ie, the range of memory addresses not used by any neighboring controllers).

A controller of a given memory module can propagate a range of memory addresses associated with (and thus managed by) the memory module. In some instances, a controller of a given memory module is used to promote the functionality of the memory module. In some such instances, the advertised functions include any or all of the following: size (ie, memory capacity), technology, installation date, fault address, and the like.

Each controller is associated with a role, where the role is One of the columns: routers, switches, and normal memory. A controller associated with the router role will be referred to hereinafter as a router controller, and a controller associated with the switch role will be referred to hereinafter as a switch controller, and a common memory The controller associated with the body role will be referred to below as a normal memory controller. An exemplary memory system 1 includes twelve ordinary memory controllers (indicated by the letter "M" in FIG. 1), three switch controllers (indicated by the letter "S" in FIG. 1), and two Router controllers (indicated by the letter "R" in Figure 1). Each memory component 2, 3 includes a single router controller R.

Each router controller stores a routing table, the routing table includes an identifier of each of the other router controllers in the network of the same level, and each router controller is further configured to circulate according to the stored routing table. A memory access request (eg, from a processor). Each switch controller is used to forward memory access requests to each of the other controllers in the peer network. Similar to an IP network, in some instances, some networked memory modules can forward memory access requests to all of these connected memory modules (for swapping), while some linked memories The body module can route memory access requests based on a wrap-around table (similar to a router in an IP network). The normal memory controller does not perform the bypass or exchange function.

Each router controller is associated with a unique identifier. In some instances, one of the memory system operators manually assigns such identifiers to the router controllers. In some instances, assigning an identifier to the router controller is performed by a management layer of the memory system. In some examples, the peer network stores this in a Decentralized Hash Table (DHT) Etc. In some instances, the identifiers are evenly distributed in an identifier space. For example, a uniform distribution can be compiled by setting the identifier of a given router controller to the password of a vendor-specified component ID (eg, the vendor-specified component ID of the NVM unit associated with the router controller). To reach.

In some examples, the identifiers are ordered in an identifier loop such that each controller has a leader and a successor in the identifier loop. In some examples, each controller may periodically schedule a live message to its predecessor and its successor, for example, according to one of the memory system routing protocols.

This exemplary memory system 1 can use a variety of routing protocols. An example of a suitable routing arrangement will now be described with reference to FIG.

The exemplary routing protocol includes a single-hop scheme in which all router controllers 20 of a memory system (e.g., memory system 1 of FIG. 1) form a ring, and each router controller 20 maintains a containment and A complete routing table of information about each of the other router controllers 20. For example, the information in the routing table may include any or all of the following: location in the ring, memory range, size, summary of technical and fault components, and direct connection (all roles) control ID of the device, etc.

Each router controller 20 is associated with a random 128-bit component identifier, and the identifiers are ordered in an identifier loop modulo 2^128.

The query success rate of the memory system depends on the accuracy of the information in the routing table of the router controller 20. To maintain correct local information (ie, information related to its successors and predecessors), each given router controller n periodically runs a stabilization routine in which the given router controller sends a live message To its successor s and the leading dep . Upon receipt of the one-dimensional message, the successor s (for example by comparing the identifier of n with the previous derivation identifier stored in the routing table) checks whether n is indeed its leading, if not, the successor s inform n that there is another router controller between the two. Similarly, when a one-dimensional message is received from n , p (for example, by comparing the identifier of n with the subsequent child identifier stored in the round table), it is checked whether n is indeed its successor, if not, Then notify n . If either s or p does not respond to the one-dimensional message, n continuously detects the unresponsive component until a timeout period has expired. When the timeout period expires, the n ruling does not respond to the component being unable to contact or has died.

To maintain a complete routing table that responds (eg, because a memory component is added to or removed from the memory system) to the controller that is connected or leaves the peer network, the network is subject to a change event (eg, join and leave) The notification should arrive at each router controller 20 in the network within a specified amount of time. Such notifications may originate from a controller that is directly connected to a controller that is newly added to the memory module, or that is directly connected to a new removable memory module (ie, before removal). Device. In some examples, the specified time is based on the network size (ie, the number of router controllers in the identifier ring) and can be less than one second. This exemplary routing protocol achieves this in the following manner.

The 128-bit circular identifier space is divided into k consecutive intervals, hereinafter referred to as a slice. k can be selected by one of the operators of the memory system. k, for example, may be selected based on any or all of the following: a network topology, a user served, a degree of availability sufficient to prevent overall failure, and the like. In a data center, for example, k can be selected such that no router controller is from the same area of the data center in the same tile. In the example of Figure 2, the identifier ring system (in dotted line A) is divided into two slices, so k = 2. The i- th note currently contains an identifier that falls within the scope. All router controllers 20 in the same inter-network. Since router controller 20 has a uniformly distributed random identifier, each tile will contain approximately the same number of router controllers 20 as each of the other tiles at any given time.

The routing arrangement defines a slice guide 21 for each tile. The slice guide subsystem is dynamically defined as having one of the identifiers of the router controller that is the successor of the identifier at the midpoint of the slice range. If a note contains an even number of identifiers such that no identifier corresponds to a precise midpoint, then a router controller having one of the identifiers closest to the midpoint is defined as the slice guide. For example, the i- th slice of the slice guide is an identifier. The successor of the router controller.

Similarly, each tile is divided into equal-sized intervals called cells. In some instances, each tile is divided into log (k) cells. For the sake of simplicity, in the example of Figure 2, each tile system (in dotted line B) is divided into two cells. The routing arrangement defines a unit leader 22 for each unit. The unit guiding subsystem is dynamically defined as having one of the identifiers is the unit The router controller of the successor at the midpoint of the range. If a unit contains an even number of identifiers such that no identifier corresponds to a precise midpoint, then a router controller having one of the identifiers closest to the midpoint is defined as the unit leader.

Responding to a new controller that is coupled to the peer network, the new controller receives information related to the tile leader and the unit leader from one of its neighbors (eg, the tile guide and the The identifier of the unit booter) additionally receives other information related to the data of the neighboring router controller and the routing table of the neighboring router controller.

Each of the slice guides 21 performs the following functions:

- Receive event notifications from other router controllers in the peer network.

- Aggregating received event notifications within a first predetermined time period. The first predetermined time may be based, for example, on the size of the ring, the size of the unit, the technology used by the memory system, the congestion condition, and the like. In some examples, the level of the first predetermined time period is tens of seconds.

- After the first predetermined time period has elapsed, a first aggregate message containing the aggregated event notifications is sent to each of the other tile guides in the peer network.

- Receive the first aggregated message from other slice guides.

- assembling the received first aggregated message within a second predetermined time period. The second predetermined time period can be judged empirically. In some examples, the second predetermined time period is one magnitude greater than the first predetermined time period.

- after the second predetermined time period has elapsed, transmitting a second aggregate message containing the aggregated first aggregated messages to the unit guides of the cells included in the slice.

Each unit leader 22 performs the following functions:

- Periodically send a live message to its successor.

- Periodically send a live message to its leading predecessor.

- Receive a second aggregated message from a one-leaf guide.

- The same live message sends a received second aggregate message to its successor and its predecessors.

Each router control unit that is neither a one-chip leader nor a one-unit leader performs the following functions:

- Periodically send a live message to its successor.

- Periodically send a live message to its leading predecessor.

- receiving a second collection message from one of its successors and its predecessors.

- The same live message sends a received second aggregate message to the other of its successor and its predecessor.

Thus, in this example, in response to a router controller 20 (e.g., via the stabilization routine described above) detecting the failure of its successor or having a new successor, it sends an event notification message 26 to its symbol. Slice guide 21. The slice guide 21 collects all event notifications 26 that it receives from the router controller 20 in its own package, and notifies these event collections for a predetermined period of time, after which a message 23 containing the aggregated event notifications is received ( A first aggregate message is sent to each of the other slice guides. Each of the fragment guides 21 collects all of the first aggregated messages 23 received therefrom from the signature guide 21, and collects the first aggregated messages for a predetermined period of time, and then includes a first aggregated message containing the collected Message 24 (a second aggregate message) is sent to each unit leader 22 in its slice. When receiving the second aggregation message 24, each unit guide 22 sends the information 25 contained in the second aggregation message to its subsequent operator and the preamble carried on the one-dimensional message.

Each router controller that is not a slice guide or a unit guide propagates information 25 in a single direction, that is, if the information is received from a leader, it is sent to a successor, and vice versa. Therefore, when the information 25 is received from the unit guide 22, each non-boot sub-router controller 20 The message 25 is sent to either of its successor (if the unit's successor is its predecessor) or the preamble (if the unit's leader is its successor). The router controller 20 located at the cell boundary (i.e., the router controller without a successor or the same in the same cell) does not send the information 25 to the router controller outside its cell. In this way, all router controllers in the peer network receive all event notifications for all events, but in a unit of information, it always flows from the unit leader 22 to the end of the unit.

The exemplary routing protocol described above results in the slice leader 21 having more work to do than other router controllers. This can cause problems in some memory systems, such as a poor provisioning system that allows a router controller to have a large memory space (equivalent to one of the ISP's backbone routers in an IP network). To overcome this problem, in some instances, a different routing arrangement (hereinafter referred to as a "super component routing protocol") is used. The super-component routing protocol defines a "super component" as a properly connected and properly provisioned controller, for example by providing predefined criteria for connection and provisioning. A new junction router controller is designated as a "hypercomponent" if it meets the predefined criteria during the junction procedure. In these examples, a super component ring is established parallel to the "normal" identifier ring. The blade boot subsystem is defined as the router controller in the widget loop, which is the successor of the router controller associated with the identifier at the midpoint of the patch range (this midpoint router controller) It can be in the super component ring or in any of the ordinary rings). Other aspects of the super-component routing protocol and the exemplary roads described in paragraphs 25 through 37 The same is true by the arrangement of the agreement.

These exemplary routing protocols thus facilitate the rapid propagation of notifications across the entire network with reasonable bandwidth consumption, thereby helping to maintain a complete routing table. An exemplary memory system using such exemplary routing protocols can thus achieve a high query success rate.

An example of a method for coupling a new controller to an inter-network of a memory system will now be described with reference to FIG. In a first block 301, a memory system, such as the memory system 1 of FIG. 1, is provided. The memory system includes a plurality of memory modules, each memory module includes a non-electric memory unit and a controller for responding to a request from a processor for access The information stored in the non-electrical memory unit, the accessed information is delivered to the processor, and communicated with controllers of other memory modules. The controllers are connected to form an inter-network.

In a second block 302, an additional memory module is provided for communicating with the memory system, the additional memory module including an additional controller. The communication can be wired or wireless communication. In some examples, providing an additional memory module in communication with the memory system includes establishing a communication link between the additional controller and a single controller in the peer network. In some examples, providing an additional memory module in communication with the memory system includes establishing a communication link between the additional controller and a plurality of controllers in the peer network. The controller of the peer network having a communication link to the additional controller may be any (any) controller of the peer network. Once connected After the connection is established, the additional controller accesses a controller in the peer network with a communication link. The access controller will be referred to below as the connection controller. At this point, the additional controller can be considered to have joined the DHT.

In some instances, upon coupling the DHT, the additional controller discovers the basic agreement of the memory system, which may be a fixed line in the firmware and/or hardware. The DHT is maintained in a distributed manner by all of the controllers in the network, meaning that any controller can be used as an entry point by an additional controller that wants to connect the peer network. (Either any of the controllers may be the contact controller.) In some instances, once coupled to the DHT, the additional controller receives information that may include, for example, any or all of the following: the memory system One of the agreements, a routing arrangement, a routing table, and so on.

At block 303, if the add-on controller is connected to a contact controller and connects the DHT, it receives the memory address range of each controller in the peer network. This allows the additional controller to determine where it should be in the network topology. For example, the previous controller may have left the ring (eg, because of a failure), in which case the additional controller may decide to occupy the "hole" left by the previous controller. Alternatively, in another example, the additional controller may decide to extend the current address range of the peer network beyond its current upper limit.

Then, in block 304, the additional controller is connected to a first controller by using the received address information, wherein the first controller is another controller in the peer network (ie, different from The contact controller). The The memory address ranges received by the add-on controller in block 303 are sent to the network topology to also identify the location of the sender (i.e., the controller that sent a given message). . In block 305, the additional controller uses the location information to determine if there is a direct connection to the first controller. Next, in block 306, the additional controller determines the role of the first controller if it determines that there is a direct connection with the first controller. Specifically, it determines whether the first controller is a router controller, a switch controller, or a normal memory controller. This determination is based on the information advertised by the first controller. In some examples, the additional controller performs blocks 304, 305, and 306 that are associated with each of the plurality of controllers.

In the last block 307, one of the roles of the additional controller is selected from the router, switch, and normal memory roles. The selection is based on whether there is a direct connection between the additional controller and the first controller, and if there is a direct connection, based on the role of the first controller, wherein the selected role is selected from the following Groups: routers, switches, common memory. The rules used for this selection are specified by one of the memory systems and stored by each controller in the peer network. In one example, an additional controller directly connected to a switch controller or a router controller is selected as a normal memory controller, and an add-on that is not directly connected to the router controller or a switch controller The controller is selected as a router controller. In this example, an add-on controller, if connected to a router controller, and beyond a predetermined minimum number of normal memory controllers, will be selected as a switch. The predetermined minimum The quantities may be based, for example, on any or all of the following: overall memory, memory system technology, and memory system protocols. For example, a memory system containing more than 10 memory modules with more than 1 PB of storage would contain several switches.

If an additional controller is selected as a normal memory controller, it prunes an indirect link to other controllers and negotiates with the nearest router controller or switch controller according to the rules of the memory system protocol. Its address range.

In some examples, the method additionally involves the additional controller contacting the first controller and receiving a second memory address range from the first controller. The second memory address range is associated with a second controller of the memory system, the second controller being directly coupled to the first controller. The additional controller then determines if there is a direct connection with the second controller; and if there is a direct connection, the role of the second controller is determined. In such instances, the role of the additional controller is selected based on whether there is a direct connection between the additional controller and the second controller, and if there is a direct connection, additionally based on the second control The role of the device.

A specific example of the method of Figure 3 can be performed as follows. Once an additional controller of an additional memory module is connected to a contact controller, the contact controller sends a small number of controller memory range addresses. The add-on controller therefore attempts to connect to the controller associated with the received memory range address and, in response, the controller also provides the additional controller with the address of the controller to which it is connected. This action continues, Until a predetermined maximum number of hops has been reached, wherein the maximum number is recorded by the memory system protocol.

The add-on controller then uses the memory system protocol to determine if it is directly connected to any of the controllers whose memory address range has been received. If so, it uses the information published by the directly connected controller to determine the role of the directly connected controller (ie, router, switch, or normal memory) and thereby select one of the additional controllers. . Once a character has been associated with the additional controller, the memory system is programmed (eg, if a normal memory is connected to a switch controller, the switch controller operates similar to an IP network) The agreement of a spanning tree in the road to automatically assemble the switch).

In principle, it is known that (for example, the contact controller) a single controller address is sufficient for a coupled additional controller to obtain the address of all other controllers in the inter-network, since the contact controller will be attached to the The controller shares the remaining addresses. This reduces the bandwidth used by the DHT.

An example of a method for accessing information stored in a memory system will now be described with reference to FIG. In a first block 401, a plurality of memory modules are provided. Each memory module includes a non-electrical memory unit and a controller, and the controllers are communicatively coupled to form an inter-network. Each memory module is associated with a memory address range, and the memory address range of a given memory module is unique among the adjacent memory modules in the network of the same level. In some examples, the memory modules are included in a memory system, such as included in the memory system 1 of FIG.

In a second block 402, a first controller of a first memory module receives a request from a processor for information stored in at least one of the other memory modules. In some examples, the requested information is stored only in one of the memory modules. In some examples, the requested system is stored in a plurality of memory modules.

Next, in block 403, the first controller sends the request to another controller in the peer network, or to a plurality of other controllers. The controller(s) to which the request is sent are determined by the first controller based on one of the routing protocols of the peer network. In some instances, the controller sends the request to its previous and succeeding children in an identifier ring. In some instances, the controller sends the request to its leading or succeeding child. Block 403 will be performed by each controller that receives the request until the request has arrived at all of the controllers in the peer network. Upon receiving the request, each controller determines whether any of the requested information is stored by its memory module. A controller of a memory module that stores at least a portion of the requested information in its NVM unit becomes a second controller for the purpose of the method.

In block 404, a second controller of a second memory module receives the request from another controller in the peer network. At least part of the requested information is stored in the NVM unit of the second memory module. Next, in block 405, the second controller determines that at least part of the requested information is stored in the non-electrical memory unit of the second memory module, as described above. In block 406, the second controller responds to the determination to retrieve the request from the NVM unit of the second memory module. Information is sought, and the requested information is sent to another controller in the peer network, or to multiple other controllers, according to the routing protocol. Each controller receiving the requested information forwards the requested information to at least one other controller in accordance with the routing agreement until the requested information has arrived at each of the controllers in the peer network.

In block 407, the first controller receives the requested information from another controller (ie, a second controller) in the peer network, and in block 408, the first controller pair The processor provides the requested information. The first controller can receive the requested information from a plurality of other controllers in the peer network. In such an instance, in some instances, the first controller provides the requested information to the processor in response to the first receipt of the requested information, and does not respond to subsequent receipt of the same requested information. The processor provides the requested information. The first controller may receive a first portion of the requested information from a first other controller and receive a second portion of the requested information from a second other controller, and/or may be at a different time Receiving a first portion of the requested information from a second portion of the requested information. In such cases, in some instances, the first controller waits until all portions of the requested information have been received. In response to all portions of the requested information being received by the first controller, the first controller provides the requested information to the processor.

In the case where one of the information stored in the module is received by a memory module, the above method is not required. In such a situation, the memory module receiving the request will simply retrieve the requested information from its NVM unit and deliver it to the requesting process. Device.

If a memory access request by a processor to a given memory module controller fails because the memory module is no longer in the system, the processor can access the memory by The request is sent to the successor of the given memory module to retry the retrieval of the data.

Each of the above exemplary memory systems can thus be viewed as a set of wraparound overlays built on a set of dynamic memory components such that the system has wrap-around, switch, and data access functions.

The disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatuses, and systems in accordance with the examples of the disclosure. Although the above flow chart shows a particular order of execution, a different order of execution than that shown may be employed. The blocks associated with one flowchart may be combined with the blocks of another flowchart. It will be appreciated that the processes and/or blocks of the flowcharts and/or block diagrams, and combinations of the flowcharts and/or FIG.

Such machine readable instructions may, for example, be a general purpose computer, a special purpose computer, an embedded processor, or the like formed by storing and executing such machine readable instructions. The processor of the data processing device is programmed to perform the functions described in the description and figures. In particular, a processor or processing device can execute such machine readable instructions. Thus, the functional modules of such devices and devices can be implemented by a processor executing a machine readable instruction stored in a memory or a processor operating in accordance with instructions embedded in the logic circuitry. The term "processor" is to be interpreted broadly to include a CPU, processing unit, ASIC, logic. Unit, or programmable gate array, etc. These methods and functional modules can all be performed by a single processor or divided into several processors.

The machine readable instructions can also be stored in a computer readable storage that can direct the computer or other programmable data processing device to operate in a particular mode.

The machine readable instructions can also be loaded into a computer or other programmable data processing device to cause the computer or other programmable data processing device to perform a series of operational steps to produce a computer implemented process, thereby, the computer or other The instructions executed on the planable device provide a step for implementing the functions specified by the process(s) in the flowchart and/or the block(s) in the block diagram.

While the present method, apparatus, and related aspects have been described with reference to the embodiments, various modifications, changes, omissions and substitutions may be made without departing from the spirit of the disclosure. Accordingly, the method, apparatus, and related aspects are intended to be limited only by the scope of the following claims. It is to be understood that the foregoing description is not intended to be limited,

The word "included" does not exclude the existence of the excluded components listed in a claim, "a" or its morphological changes do not exclude a plurality of, and a single processor or other unit can achieve the number of The function of the unit.

The characteristics of any subsidiary item may be combined with the characteristics of any individual item or other subsidiary item.

1‧‧‧ memory system

2‧‧‧First memory component

3‧‧‧Second memory component

10‧‧‧ memory module

Claims (15)

A memory system including a plurality of memory modules, wherein each memory module includes: a non-electric memory unit; and a controller for responding to a processor Requesting to access information stored in the non-electrical memory unit, delivering the accessed information to the processor, and communicating with controllers of other memory modules; wherein the controllers are connected Forming an inter-level memory module network, and each of the given memory modules is associated with a memory address range, the memory address range and one directly connected to the given memory The memory address ranges associated with the memory modules of the module do not overlap. The memory system of claim 1, wherein each memory module is associated with a unique memory address range. The memory system of claim 2, wherein the memory address range associated with a given memory module is determined during the connection of the controller of the memory module to the peer network. The memory system of claim 1, wherein each controller is configured to advertise the memory address range of its memory module to other controllers in the peer network. The memory system of claim 1, wherein each controller is associated with a role, the role is one of the following: a router, a switch, a general Through memory. The memory system of claim 5, wherein each router controller is configured to store a table containing an identifier of each of the other router controllers in the network of the peers, and for use in accordance with the stored The send table bypasses the memory access request; and each switch controller is used to forward the memory access request to each of the other controllers in the peer network. The memory system of claim 1, wherein each router controller is associated with a unique identifier, and wherein the peer network is for storing the identifiers in a decentralized hash table DHT. The memory system of claim 7, wherein the identifiers are evenly distributed in an identifier space. The memory system of claim 7, wherein the identifiers are ordered in an identifier ring such that each router controller has a leader and a successor in the identifier ring. The memory system of claim 9, wherein: the identifier ring comprises a plurality of equal-sized contiguous segments, each of the contiguous segments containing an identifier-like slice range, and wherein each of the slices has a slice guide Is defined as: a router controller associated with an identifier, the identifier being the successor of the identifier at the midpoint of the slice range; or, if there is no identifier at the midpoint, defined as a router controller having an identifier closest to the midpoint; and each tile includes a plurality of equal-sized connected units, each of the connected units having an identifier unit range, and wherein each unit has a single A meta-guide is defined as: a router controller associated with an identifier, the identifier being the successor of the identifier at a point in the range of the unit; or, if there is no identifier at the midpoint, Defined as a router controller with an identifier closest to the midpoint. The memory system of claim 10, wherein each of the slice guides is configured to: receive event notifications from other router controllers in the peer network; aggregate the received event notifications within a first predetermined time period; After the first predetermined time period, a first aggregate message containing the aggregated event notifications is sent to each of the other tile guides in the peer network; the first aggregate message is received from the other slice guides; Receiving the received first aggregate message within a second predetermined time period; after the second predetermined time period, sending a second aggregate message containing the aggregated first aggregated message to the unit included in the slice These unit guides. The memory system of claim 11, wherein each unit guide is used to: periodically send the live message to its successor; periodically send the live message to its preamble; receive the first from a slice guide The second collection message; and the same live message sends a received second collection message to its successor and its predecessor. The memory system of claim 12, wherein the non-unit is a guide or a Each router controller of the slice guide is used to: periodically send the live message to its successor; periodically send the live message to its predecessor; receive one from its successor and its predecessor a second aggregate message; and the same live message sends a received second aggregate message to the other of its successor and its predecessor. The memory system of claim 1, wherein the memory system comprises non-electrical memory cells, the non-electrical memory cells being different from each other in at least one of the following: capacity; manufacturer; storage technology; Body agreement, memory system technology. A method includes: providing a plurality of memory modules, each memory module including a non-electric memory unit and a controller; wherein the controllers are communicatively coupled to form an inter-network And the memory module is associated with a memory address range, and the memory address range of a given memory module is unique among the adjacent memory modules in the network of the same level The first controller of the first memory module receives a request from a processor for information stored in at least one of the other memory modules; the first controller is based on the network between the peers a routing protocol that sends the request to at least one other controller in the peer network; The second controller of the second memory module receives the request from another controller in the network of the same level, wherein the requested information is at least partially stored in the second memory module. In the electrical memory unit, the second controller determines that at least part of the requested information is stored in the non-electrical memory unit of the second memory module; the second controller is from the second memory The non-electrical memory unit of the body module retrieves the requested information, and according to the routing agreement, sends the requested information to at least one other controller in the peer network; the first controller Receiving the requested information from another controller in the peer network; and the first controller provides the requested information to the processor.