Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
  • G06F13/14—Handling requests for interconnection or transfer
  • G06F13/16—Handling requests for interconnection or transfer for access to memory bus
  • G06F13/1668—Details of memory controller
  • G06F13/1673—Details of memory controller using buffers
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
  • G06F13/14—Handling requests for interconnection or transfer
  • G06F13/16—Handling requests for interconnection or transfer for access to memory bus
  • G06F13/1668—Details of memory controller
  • G06F13/1684—Details of memory controller using multiple buses

Definitions

• Embodiments of the present invention relate to the field of memory components. More particularly, embodiments of the present invention relate to a method for setting parameters and determining latency in a chained device system.
• Electronic systems and circuits have made a significant contribution toward the advancement of modern society and are utilized in a number of applications to achieve advantageous results.
• Electronic technologies such as digital computers, calculators, audio devices, video equipment, and telephone systems have facilitated increased productivity and reduced costs in analyzing and communicating data, ideas and trends in most areas of business, science, education and entertainment.
• the electronic systems designed to provide these results include memories.
• accessing memory resources in a fast and efficient manner can involve complicated protocols.
• Numerous electronic devices include processors that operate by executing software comprising a series of instructions for manipulating data in the performance of useful tasks.
• the instructions and associated data are typically stored in a memory.
• Memories usually consist of a location for storing information and a unique indicator or address.
• the ability to access a memory and transfer information quickly and conveniently usually has a significant impact on information processing latency.
• the configuration of a memory usually affects the speed at which memory locations are accessed.
• Traditional attempts at memory control are often very convoluted and complex.
• Conventional daisy chain systems often involve latencies that can become very large due to large buffer sizes and scheduling amongst the various nodes within the daisy chain. It is often problematic for controllers and/or central processors to determine latency for read requests without complex scheduling algorithms and artificial limitations of the system.
• Traditional approaches often lead to long average latencies, very complex control mechanisms, larger buffer sizes, or a combination of such problematic concerns.
• a need has arisen to determine the exact latency of a request without complex scheduling algorithm while avoiding artificial limitations of the system. Moreover, a need has arisen to determine the exact latency without using long average latencies and large buffer sizes. Furthermore, a need has arisen to determine the dynamic minimal and maximal latency by using the available bandwidth, the number of outstanding requests, priority and minimal roundtrip latency, to name a few. Thus, a need has arisen to determine the minimal, maximal and exact latency for a request in a daisy chained device. It will become apparent to those skilled in the art after reading the detailed description of the present invention that the embodiments of the present invention satisfy the above mentioned needs.
• a storage system includes a plurality of storage nodes for storing information.
• the storage system may also include a master controller, e.g., a CPU, for controlling the flow of information to the plurality of storage nodes.
• a master controller e.g., a CPU
• the plurality of storage nodes include at least more than one node that may be organized in a chained network, e.g., a daisy chain network.
• at least one storage node includes an upstream communication buffer, e.g., internal and external buffer.
• the internal buffer of a first storage node is used for storing information pertaining to the first storage node whereas the external buffer is used for storing information pertaining to a storage node other than the first storage node.
• other storage nodes may similarly have external and internal buffers for storing information accordingly.
• the upstream communication buffer temporarily stores responses from an internal memory core while waiting for responses from other storage nodes further down the daisy chain to be forwarded upstream.
• the master controller controls the flow of information to the storage nodes based on one or more constraints of the upstream communication buffer, e.g., internal and external buffers, of the storage nodes.
• the master controller controls the flow of information such that communication between the master controller and the storage nodes have a determined maximum latency.
• the constraint of the upstream communication buffer is based on providing priority to responses to a storage node, e.g., memory node, that are further down the daisy chain stream.
• the master controller determines the individual latency for each storage node based on a vector of outstanding responses in the system, minimum latencies for each of the storage nodes, a burst length of the request, a clock frequency, a width of the bus coupling the master controller to the storage nodes and a memory node count of the number of the storage nodes in the system, to name a few. It is appreciated that the minimum latency may be determined as the roundtrip time of a request to be sent and a response to be received from a particular storage node when there are no other pending requests.
• the maximum latency of any request may be defined using the same equation and the minimum latency of a given storage node in the system.
• the master controller e.g., CPU, may build requests to be sent to storage nodes based on the apriori knowledge of latency for each node in order to reduce response time in the daisy chain system.
• Figure 1 shows an exemplary system in accordance with one embodiment of the present invention.
• Figure 2 shows an exemplary storage flow diagram in accordance with one embodiment of the present invention.
• Figures 3 shows an exemplary flow diagram for rearranging requests prior to forwarding the request in accordance with one embodiment of the present invention.
• these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.
• the system 100 in one embodiment, comprises a master controller 110 and a plurality of memory components commonly referred to as storage nodes in the instant application.
• the master controller 110 may be a CPU that may control the flow of information.
• a first storage node 120 e.g., memory A
• a second storage node 130 e.g., memory B
• the storage nodes are coupled in a daisy chain structure or configuration.
• non-daisy structures or configurations may be coupled to the daisy structure.
• the master controller 1 10 may control the flow of information and may send a plurality of requests to storage nodes. Requests may include a read request, a write request, etc.
• the request from the master controller 110 is sent down the daisy chain.
• a request M 112 message destined for the third storage node 140 may be sent from the master controller 110 to the first storage node 120 where the first storage node 120 passes the request M 112 to the second storage node 130, which in turn passes that request to the destined storage node 140.
• a request from the master controller 110 travels down the daisy chain structure.
• the master controller 110 may send a request A 114 message destined for the first storage node 120 down the daisy chain structure.
• the request message A 114 is received by the first storage node 120.
• each request/response is labeled based on the destined storage node.
• the request/response for the first storage node 120 e.g., memory A
• the request/response for the third storage node 140 e.g., memory M
• M the request/response for the third storage node 140.
• the corresponding storage node responds back to the master controller 110 via the daisy chain structure.
• the third storage node 140 may send the response M 112 to the second storage node 130 which in turn passes the response to the first storage node 120 and ultimately back to the master controller 110.
• the first storage node 120 may respond by sending the response back to the master controller 110 (not shown).
• priority is given to a response packet from a storage node furthest from the master controller 1 10.
• storage nodes further down the stream in the daisy chain structure have priority over storage nodes closer to the master microcontroller 110.
• the response packet from the third storage node 140 e.g., memory M
• the response packet from the second storage node 130 is given priority over the response packet from the first storage node 120, e.g., Memory A.
• each storage node is equipped with at least one upstream buffer.
• the upstream buffer may store data responses from the storage node itself or store data responses from another storage node that is further from the master controller 1 10.
• the upstream buffer in the first storage node 120 e.g., Memory A, may comprise an internal buffer 122 and an external buffer 124.
• the internal buffer 122 is operable to store responses from the first storage node 120 and the external buffer 124 is operable to store response packets from storage nodes further from the master controller 1 10, e.g., the second storage node 130 and the third storage node 140.
• the upstream buffer in the second storage node 130 may comprise an internal buffer 132 and an external buffer 134.
• the internal buffer 132 is operable to store responses from the second storage node 130 and the external buffer 134 is operable to store response packets from storage nodes further from the master controller 110, e.g., the third storage node 140.
• other storage nodes e.g., the third storage node 140, may similarly comprise an upstream buffer that comprises an internal buffer 142 and an external buffer 144 that operate in similar manner described above.
• the following example illustrates the priority concept in case of a collision between two responses from different storage nodes.
• the request A 1 14 is received by the first storage node 120.
• the first storage node 120 similarly receives the response M 112 from the third storage node 140.
• the first storage node 120 has to choose which response first gets sent to the master controller 110. Since the third storage node 140 is further from the master controller 110 in comparison to the first storage node 120, the second storage node 120 sends the response M 112 from the third storage node 140 to the master controller 110.
• the first storage node 120 While the first storage node 120 sends the response M 112 message to the master controller 1 10, the first storage node 120 temporarily stores the response to the message A 114 in the internal buffer 122 such that the response can be sent subsequent to the response M 112 message. In contrast, if the first storage node 120 receives the response M 112 message while transmitting the response A 114 to the master controller 110, the response M 112 message is temporarily stored in the external buffer 124 for later transmission to the master controller 110. It is appreciated that other storage nodes behave substantially similar to the first storage node 120.
• the amount of time that a response waits in an upstream buffer may be represented in the following equation:
• N is the length of the robust request in bytes.
• the clock period is represented as toe and the size of the bus is represented as W bytes.
• NAV is also known as the burst length (BL) of the response.
• equation (1) may be represented as: BL/2 * 1/tc ⁇ (2).
• the maximum latency time for a request from the furthest node, e.g., the third storage node 140 can be represented in the following equation:
• Max Lat (Max Node) Min Lat (Max Node) + (Max Node -I) * BL/2 * 1/tc ⁇ (3)
• Min Lat is the minimum latency for a request for a given storage node.
• Min Lat (A) is the minimum roundtrip latency in the system when there are no requests pending.
• Min Lat (B) is the minimum roundtrip latency in the system when there are no requests pending.
• Max Node is the node furthest from the master controller 1 10. In this exemplary embodiment, the Max Node is the third storage node 140.
• the requests to an inner storage node waits until the request to the further nodes are completed.
• the maximum latency of a node closer to the master controller 110 is also related to the maximum latency of the furthest node, e.g., Max Node which is the third storage node 140.
• the maximum latency of a node closer to the master controller 110 is also related to Max Lat (Max Node).
• the maximum latency for a given storage node is the sum of minimum latency of the given node, e.g., the latency for the second storage node 130, and the maximum latency between the given node, e.g., the second storage node 130, and the maximum latency storage node, e.g., the third storage node 140, in addition to any additional buffer delays between the given node, e.g., the second storage node 130, and the master controller 110.
• the maximum latency for a given storage node e.g., x
• Max Lat (x) Min Lat (x) + Min Lat (Max Node) + (Max Node -x)*(BL/2 * 1/tc ⁇ ) - Min Lat (x) + (N - l)(BL/2 * 1/ too (4)
• Max Lat (x) Min Lat (Max Node) + (Max Node -l)*(BL/2 * 1/tc ⁇ ) (5).
• the maximum latency for any request may be defined using the same equation (5).
• the maximum latency for any given storage node may be defined using the minimum latency of any request for a given node in the system.
• the master controller 110 may control the flow of information to storage nodes based on one or more constraints, e.g., latency constraint, of the upstream communication buffer, e.g., internal and external buffers, of the storage nodes.
• the master controller controls the flow of information such that communication between the master controller and the storage nodes have a determined maximum latency.
• the constraint of the upstream communication buffer is based on providing priority to responses to a storage node, e.g., memory node, that are further down the daisy chain stream.
• the exact latency may be determined by the master controller 110.
• the master controller 110 may build a vector of response data of request storage node names and use the vector to calculate the individual latency of a given request. It is appreciated that the determination of an individual latency is based on the apriori knowledge of the daisy chain structure.
• the built vector may be represented as follows:
• the master controller 110 expects the oldest response to request C to arrive at the master controller 110 next, which is the C to the right side of the vector. It is appreciated that "0" represents when there are no data to be returned on a given cycle. It is further appreciated that the response A in the middle may be younger than the requests to its left due to differing latencies between storage nodes C and A.
• the vector when the master controller 1 10 inserts a request, e.g., an A request, the vector may be represented as follows:
• a request since A request has a shorter latency in comparison to a B request, the A request may be inserted at a minimum latency slot in the vector to reduce the latency in the system. It is further appreciated that insertion of A request in this exemplary embodiment depends on the speed of the link, the minimum latency of A and the minimum latency for a B request.
• the vector when the master controller 1 10 inserts a B request the vector may be represented as OBOBOBBBBCCCACCCCCCCCCCCCCCCC.
• the master controller 110 inserts a C request the vector may be represented as COOBOBBBBCCCACCCCCCCCCCCCCC.
• the bus may be idle for extra time as represented by "0" shown.
• the vector when a B request is to be inserted after a C request, the vector may become OCBOOBOBBBBCCCACCCCCCCCCCCCCC. It is appreciated that B is inserted ahead of C because the latency of B request is shorter in comparison to the latency of C request. By inserting the request with a shorter latency ahead of the request with longer latency, the latency of the system is reduced.
• the vector may be represented as OOCBAOBOBBBBCCCACCCCCCCCCCCC. It is appreciated that A is inserted ahead of B and C request because the latency of an A request is shorter in comparison to the latency of a B or a C request. As a result, the exact latency may be determined in the daisy chain structure.
• controlling the flow information may reduce the latency in the system and speed up the process for a request based on latency of the buffers and other constraints.
• the information regarding the distribution of and traffic flow between memory nodes and a master controller may be obtained.
• the information may include the number of memory components in a daisy chain structure.
• the information may include the length of the robust request in bytes, the period of the clock and the size of the bus in bytes, to name a few.
• the information on the distribution may indicate that the storage nodes are arranged in a daisy chain structure and/or that a storage nodes further from the master controller 110 are given priority over storage nodes closer to the master controller 110 when there is a collision between the responses in various storage nodes.
• the latency for a responding data request may be determined. For example, it may be determined whether the request is for the first storage node 120, the second storage node 130 or the third storage node 140. Once the destined storage node or memory component is determined, the latency for the response to the data request may be determined. In one example, the latency may be determined using equations (1) through (5) described and presented above. Accordingly, the latency may be based upon a maximal latency and/or minimal latency. As described and presented above, minimal latency may be determined by the roundtrip time of a request to reach a storage node and for the storage node to respond and send the respond to the master controller 110 when there are no other requests pending.
• a vector of requests may be formed in order to determine the exact latency as described and presented above. It is further appreciated that in one embodiment, the latency may be based on latency for a request under constraints. It is appreciated that constraints may include latency of storage nodes, priority of various storage nodes and maximum latency based on the priority of various storage nodes, to name a few.
• FIG. 3 an exemplary flow diagram 300 for rearranging requests prior to forwarding the request in accordance with one embodiment of the present invention is shown.
• a new request is inserted to a vector that comprises requests to be forwarded to storage nodes.
• the vector of requests is built by the master controller 110 as discussed and presented above. Using the built vector, the exact latency may be determined.
• the requests within the vector that now includes the new inserted request are rearranged. It is appreciated that in one embodiment, the rearrangement is based on the corresponding latency of the new request and corresponding latencies of the requests within the vector. For example, as presented and discussed above when an A request is to be inserted, it may be inserted ahead of a B request within the vector because the latency of an A request is shorter in comparison to the latency of a B request. Similarly, other new requests may be inserted and the request entries of the vector may be rearranged based on the latency of the new request and the entries within the vector. Rearranging the requests may therefore reduce the latency within the system, thereby improving the speed of the system.
• each storage node is known apriori by the master controller. Moreover, the latency of storage nodes are governed by the same relationship, thereby eliminating the need for complex scheduling algorithm while avoiding artificial limitations on the system. Furthermore, the latency is determined using given resources, e.g., buffer size, bandwidth, the number of outstanding requests, priority and minimal roundtrip latency, to name a few. Thus, the need for large buffer size is eliminated since the requests may be rearranged to reduce latency and reduce the need to store large information in buffer.
• resources e.g., buffer size, bandwidth, the number of outstanding requests, priority and minimal roundtrip latency

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Data Exchanges In Wide-Area Networks (AREA)
• Information Transfer Systems (AREA)
• Small-Scale Networks (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Memory System (AREA)
• Multi Processors (AREA)

Priority Applications (1)

Applications Claiming Priority (16)

Publications (1)

Family

ID=40670682

Family Applications (4)

Family Applications After (3)

Country Status (3)

Families Citing this family (28)

Citations (4)

Family Cites Families (23)

• 2008
  • 2008-11-21 US US12/276,010 patent/US8732360B2/en active Active
  • 2008-11-21 US US12/276,116 patent/US8601181B2/en active Active
  • 2008-11-21 US US12/276,061 patent/US8930593B2/en active Active
  • 2008-11-21 US US12/276,143 patent/US8874810B2/en active Active
  • 2008-11-25 JP JP2010534981A patent/JP5429572B2/ja active Active
  • 2008-11-25 WO PCT/US2008/013188 patent/WO2009070324A1/en not_active Ceased
  • 2008-11-25 WO PCT/US2008/013186 patent/WO2009070322A1/en not_active Ceased
  • 2008-11-25 JP JP2010534982A patent/JP5566899B2/ja active Active
  • 2008-11-25 JP JP2010534980A patent/JP2011505037A/ja active Pending
  • 2008-11-25 WO PCT/US2008/013194 patent/WO2009070326A1/en not_active Ceased
  • 2008-11-25 WO PCT/US2008/013185 patent/WO2009070321A1/en not_active Ceased
  • 2008-11-25 JP JP2010534979A patent/JP5948628B2/ja active Active
• 2016
  • 2016-02-12 JP JP2016025072A patent/JP2016095881A/ja active Pending

Patent Citations (4)

Also Published As

Similar Documents

Legal Events