WO2002073619A2

WO2002073619A2 - System latency levelization for read data

Info

Publication number: WO2002073619A2
Application number: PCT/US2002/007226
Authority: WO
Inventors: Jeffery W. Janzen; Keeth Brent; Kevin J. Ryan; Troy A. Manning; Brian Johnson
Original assignee: Micron Technology, Inc.
Priority date: 2001-03-13
Filing date: 2002-03-12
Publication date: 2002-09-19
Also published as: JP2007272929A; WO2002073619A3; CN1507629A; EP1374245A2; US6851016B2; AU2002255686A1; CN101159163A; US20020133666A1; WO2002073619A9; KR20040005888A; US6658523B2; US20040107326A1; KR100607740B1; JP2004524641A

Abstract

In a high speed memory subsystem differences in each memory device's minimum device read latency and differences in signal propagation time between the memory device and the memory controller can result in widely varying system read latencies. The present invention equalizes the system read latencies of every memory device in a high speed memory system by comparing the differences in system read latencies of each device and then operating each memory device with a device system read latency which causes every device to exhibit the same system read latency.

Description

SYSTEM LATENCY LEVELIZATION FOR READ DATA

FIELD OF THE INVENTION

The present invention relates generally to high speed synchronous

memory systems, and more particularly to setting read latencies of memory

devices so that read data from any memory device arrives at the memory

controller at the same time.

BACKGROUND OF THE INVENTION

An exemplary computer system is illustrated in Fig. 1. The

computer system includes a processor 500, a memory subsystem 100, and an

expansion bus controller 510. The memory subsystem 100 and the expansion

bus controller 510 are coupled to the processor 500 via a local bus 520. The

expansion bus controller 510 is also coupled to at least one expansion bus 530,

to which various peripheral devices 540-542 such as mass storage devices,

keyboard, mouse, graphic adapters, and multimedia adapters may be attached. The memory subsystem 100 includes a memory controller 400

which is coupled to a plurality of memory modules 301-302 via a plurality of

signal lines 401a-401d, 402, 403, 404, 405a-405d. The plurality of data

signal lines 401a-401d are used by the memory controller 400 and the

memory modules 301-302 to exchange data DATA. Addresses ADDR are

signaled over an plurality of address signal lines 403, while commands CMD

are signaled over a plurality of command signal lines 402. The memory

modules 301-302 include a plurality of memory devices 101-108 and a register

201-202. Each memory device 101-108 is a high speed synchronous memory

device. Although only two memory modules 301, 302 and associated signal

lines 401a-401d, 402, 403, 404, 405a-405d are shown in Fig. 1, it should be

noted that any number of memory modules can be used.

The plurality of signal lines 401a-401d, 402, 403, 404, 405a-405d,

which couple the memory modules 301, 302 to the memory controller 400 are

known as the memory bus 150. The memory bus 150 may have additional

signal lines which are well known in the art, for example chip select lines,

which are not illustrated for simplicity. Each row of memory devices 101-104,

105-108 which span the memory bus 150 is known as a rank of memory.

Generally, single side memory modules, such as the ones illustrated in Fig. 1, contain a single rank of memory. However, double sided memory modules

containing two ranks of memory may also be employed.

A plurality of data signal lines 401a-401d couple the memory

devices 101-108 to the memory controller 400. Read data is output serially

synchronized to the read clock signal RCLIC, which is driven across a plurality

of read clock signal lines 405a-405d. The read clock signal RCLKis generated

by the read clock generator 401 and driven across the memory devices 101-

108 of the memory modules 302, 301, to the memory controller 400.

Commands and addresses are clocked using a command clock signal CCLK

which is driven by the memory controller across the registers 201, 202 of the

memory modules 301, 302, to a terminator 402. The command, address, and

command clock signal lin s 402-404 are directly coupled to the registers 201,

202 of the memory modules 301, 302. The registers 201, 202 buffer these

signals before they are distributed to the memory devices 101-108 of the

memory modules 301, 302. The memory subsystem 100 tlierefore operates

under at least a read clock domain governed by the read clock RCLK and a

command clock domain governed by the command clock CCLK. The

memory subsystem 100 may also have additional clock domains, such as one

governed by a write clock (not shown). When a memory device 101-108 accepts a read command, a data

associated with that read command is not output on the memory bus 150 until

a certain amount of time has elapsed. This time is known as device read

latency. A memory device 101-108 can be programmed to operate at any one

of a plurahty of device read latencies, ranging from a minimum device read

latency (which varies from device to device) to a maximum latency period.

However, device read latency is only one portion of the read latency

seen by the memory controller 400. This read latency seen by the memory

controller, known as system read latency, is the sum of the device read latency

and the latency caused by the effect of signal propagation time between the

memory devices 101-108 and the memory controller 400. If the signal

propagation between each memory device 101-108 and the memory controller

400 were identical, then the latency induced by the signal propagation time

would be a constant and equally affect each memory device 101-108.

However, as Fig. 1 illustrates, commands CMD, addresses ADDR, and the

command clock CCLK are initially routed to registers 201, 202 before they are

distributed to the memory devices 101-108. Each memory device 101-104,

105-108 on a memory module 301, 302 is located at a different distance from

the register 201, 202. Thus each memory device 101-104 will receive a read command issued by the memory controller 400 at different times.

Additionally, there are also differences in distance between the memory

controller 400 and the registers 201, 202 of the two memory modules 301,

302. .Register 201 (on memory module 301) is closer to the memory

controller 400 and will therefore receive commands, addresses, and the

command clock before register 202 (on memory module 302). Thus, every

memory device 101-108 of the memory subsystem 100 has a different signal

path length to the memory controller for its command CMD, address ADDR,

and command clock CCLK signals and will receive a read command issued by

the memory controller at varying times. At the high clock frequencies (e.g.,

300 MHz to at least 533 MHz), these timing differences become significant

because they may overlap clock cycle boundaries.

Due to differences in each memory device's 101-108 minimum

device read latency and differences in their command CMD, address ADDR,

and command clock CCLK signal propagation, each memory device 101-108

may have a different system read latency. Since each memory device stores

only a portion of a memory word, the memory controller normally reads a

plurahty of memory devices in parallel. The differences in system read latencies

among the memory devices 101-108 of the memory subsystem 100 makes this task difficult. Accordingly, there is a need for an apparatus and method to

equahze the system read latencies of each memory device so that the memory

controller can efficiently process a read transaction across multiple memory

devices.

SUMMARY OF THE INVENTION

The present invention is directed at a method and apparatus for

equalizing the system read latencies of each memory device in a high speed

memory system. The equahzation process ensures that each memory device

responds to the memory controller with the same system read latency,

regardless of each device's minimum device read latency and differences in

signal propagation time due to differences h the memory device's physical

location on the memory bus. Each memory device has a plurahty of

configuration lines which can be used by the memory controller to set the

memory device to operate at any one of a plurahty of device read latencies

longer than the device's minimum device read latency. During the

equahzation process, each memory device is initially operated its minimum

device read latency. The memory controller reads a calibration pattern to determine each memory device's system read latency. The memory controller

calculates an offset which may be added to each memory device's device read

latency to cause each memory device to operate at a system read latency equal

to the slowest observed system read latency when each memory device is

operated at its minimum device read latency. Each memory device is thereafter

operated at an increased device latency, with the amount of increase equal to

the offset associated with the memory device. In this manner, all memory

devices in the memory system are equalized to operate with the same system

read latency.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other advantages and features of the invention

will become more apparent from the detailed description of the preferred

embodiments of the invention given below with reference to the

accompanying drawings in which:

FIG. 1 is a block diagram illustrating a computer system with an

high speed memory system; FIG. 2 is a timing diagram showing the read latencies of the

plurahty of memory devices wliich comprise the high speed memory system of

Fig. 1 prior to equahzation;

FIG. 3 A is a more detailed diagram showing a memory module 301

in accordance with the present invention;

FIG 3B is a more detailed diagram showing one of the memory

devices of the memory module illustrated in FIG. 3 A;

FIG. 4 is a diagram showing the relationship between a memory

device's device read latency and the states of the configuratio lines;

FIG. 5 is a flow chart showing how the memory controller equalizes

system read latencies across the memory devices of the memory system; and

FIG. 6 is a is a timing diagram showing the read latencies of the

plurahty of memory devices which comprise the high speed memory system

after equahzation.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Now referring to the drawings, where like reference numerals

designate like elements, there is shown in Fig. 2 a timing diagram of a read

operation issued by the memory controller 400 to each memory device 101-

108, with each memory device set to operate at its minimum device read

latency. A memory device's minimum device read latency is based upon its

construction and can vary from device to device. In the example illustrated in

Fig. 2, the memory devices DRAM-1 101, DRAM-2 102, DRAM-3 103, and

DRAM-4 104 of the memory module 301 closest to the memory controller

400 have minimum device read latencies of 7, 8, 5, and 6 clock cycles,

respectively. The memory devices DRAM-5 105, DRAM-6 106, DRAM-7

107, and DRAM-8 108 of the memory module 302 furthest from the memory

controller 400 have minimum device read latencies of 8, 6, 8, and 7 clock

cycles respectively. Minimum device latency is measured as the number of

clock cycles following the initiation of a read command RD before read data is

available on the memory bus 150. Due to differences in the length of the signal propagation path for

the command CMD and command clock CCLK signals, each of the memory

devices 101-108 in d e memory subsystem 100 receives a read command RD

issued by the memory controller 400 at varying times. Fig. 2 shows the

memory controUer issuing a read command centered on clock cycle TO. The

memory devices 101-104 on the memory module 301 located closest to the

memory controller 400 receive the read command between clock cycles Tl

and T2, while the memory devices 105-108 on the memory module 302

located furthest from the memory controller receive the read command

between clock cycles Tl and T3. The system read latency to each of the

memory devices 101-108 is a function of both the device read latency and the

signal propagation time between the memory controUer 400 and the memory

devices. For example, the memory devices 101-104 in the memory module

301 located closest to the memory controUer 400 have system read latencies of

9, 10, 6, and 7 clock cycles, respectively. The memory devices 105-108 in the

memory module 302 located furthest from the memory controUer 400 have

system read latencies of 10, 8, 9, and 8 clock cycles, respectively. Note that

the difference in system read latencies is large enough that memory module

103 completes its data output before memory module 102 begins data output. Now referring to Fig. 3A, there is shown a more detaUed diagram of

one of the memory modules 301 h accordance with the present invention. In

addition to the read clock signal lines 405a-405d, data signal lines 401a-401d,

command clock signal line 404, plurahty of command signal lines 402, and

plurahty of address signal lines 403, each memory device 101-104 is also

coupled to the register 201 via a plurahty of configuration lines 410. (These

pluralities of configuration lines 410 were not illustrated in Fig. 1 in order to

avoid cluttering that diagram.) In the exemplary embodiment each plurahty of

configuration lines 410 each include at least 3 configuration signal lines 411-

413 carrying configuration signals CFGO, CFGl, and CFG2, respectively. For

each memory device, the memory controUer 400 can set the states of the

configuration lines 411-413 by sending commands CMD and addresses

ADDR into register 201.

Fig. 3B is a more detailed diagram of one of the memory devices

101 shown in Fig. 3 A. Suitable memory devices include any type of high

speed DRAM. Thus, the principles of the present invention may be

incorporated into any type of single or double data rate synchronous memory

device, or Advance DRAM Technology (ADT) memory devices. The memory

device 101 includes a control circuit (including address decoders) 2000 coupled to a plurahty of signal lines, h cluding the command clock signal line

404, a plurahty of command signal lines 402, a plurahty of address signal lines

403, and the plurahty of configuration lines 410. The memory device 101 also

includes a write data path 2002 and a read data path 2003 both of which are

coupled to the data signal hue 401a and d e plurahty of memory arrays 2001

(via I/O Gating circuit 2006). The read data path is coupled to the read clock

signal line 405a via a read clock delay lock loop (DLL), which is used to

synchronize read data output with the read clock. The read data path also

includes a serializer 2004, which converts d e parallel data read from the

plurahty of memory arrays 2001 into the serial data output on the data signal

line 401a in synchronism with the read clock signal RCLK.

The memory devices DRAM-1 101- DRAM-4 104 are wired to

respond to the different states of the configuration lines 411-413 to thereby

operate at different selectable device read latencies. Fig. 4 shows how a

memory device 101-104 can be made to operate across an 8 -cycle variation in

device read latency, ranging from the minimum device read latency to the

nύnirnum device read latency plus 7 clock cycles. In alternate embodiments

there may be more or less configuration lines with a corresponding change in

the number of permitted device latencies. Alternatively, tiiere may be additional configuration lines directed towards memory functions not related

to device read latency. For example, an additional configuration line can be

used to enable or disable the read clock DLL 2005.

The states of each of the plurahty of configuration lines 410 can be

set by the memory controUer 400. For example, the memory controUer may

include a command which causes the register 201, 202 of the memory module

301, 302 to assert a state on the plurality of configuration lines 410

corresponding to an address asserted on the plurahty of address signal lines

403. Thus the memory controUer 400 is capable of changing a memory

device's 101-108 device read latency, and therefore also the memory device's

system read latency by varying the states of the configuration lines 411-413.

The memory controUer 400 uses the plurahty of configuration lines

410 to equahze the system read latencies across aU memory devices 101-108 of

the memory subsystem 100. Referring to Fig. 5, the process begins at step

1001 with the memory controUer 400 mstructing aU memory devices 101-108

to operate at their minimum device read latencies. The memory controUer 400

can instruct the memory devices to operate at minimum device read latency by

asserting the appropriate command CMD and address ADDR signals on the

plurahty of command signal lines 402 and die plurahty of address signal lines 403, respectively, thereby causing a specific state of the configuration lines

CFGO, CFGl, CFG2 to be set. As shown in Fig. 4, tiie state of tiie

configuration lines CFGO, CFGl, CFG2 cause the memory devices 101-108

to operate a specific latencies. Thus, one aspect of d e invention is that the

device read latency of each memory device is specified using relative numbers.

This is in contrast to prior art memory systems, which specific latencies as

actual clock cycles, thereby requiring a memory controUer to be aware of t e

minimum device read latency for each memory device. For example, if a device

has a minimun device read latency of 2 clock cycles, a prior art memory

controUer would need to know that 2 clock cycles corresponded to the

minimuni device read latency because in order to program the device to

operate at its minimum device read latency, the memory controUer would need

to program the latency value by using the actual number of clock cycles, which

in this case would be 2 clock cycles. In d e present invention, however, the

memory controUer 400 does not need to know the minimum device read

latency for each memory device 101-108 because read latencies are specified as

offsets from the minimum read latency.

At step 1002, the memory controUer reads a calibration pattern

from each memory device 101-108, noting the minimum operational system read latency for each memory device 101-108. The calibration pattern is

formatted to permit the memory controUer to easUy identify when data first

arrives at the memory controUer. In the exemplary embodiment each memory

device 101-108 returns 8-bits of data per read command, the data being

seriaUy driven across d e data signal lines 401a-401d to the memory controUer

400. A good calibration pattern would permit the memory controUer to easUy

recognize when the first bit of data arrives at the memory controUer. In the

exemplary embodiment, the preferred calibration pattern is a byte in which the

first bit which arrives at the memory controUer is set to one state the remaining

bits are set to a different state. Thus (binary) 01111111 or (binary) 10000000

would be preferred calibration patterns.

At step 1003, the memory controUer 400 determines the largest

value of the set of minimum operational system read latency. At step 1004,

for each memory device 101-108, the memory controUer 400 computes an

offset equal to the difference between that memory device's system read

latency and the largest value of the set of minimum operational system read

latencies. At step 1005, the memory controUer 400 instructs that memory

device to operate with an increased device read latency. The amount of increased latency is equal to the offset and is controUed by the state of the

signals asserted on the memory device's plurahty of configuration lines 410.

For example, Fig. 2 showed a memory system having 8 memory

devices DRAM-1 101- DRAM-8 108 with system read latencies of 9, 10, 6, 7,

10, 8, 9, and 8 clock cycles respectively. The largest observed system read

latency is 10 clock cycles. The offsets for the memory devices 101-108 is equal

to the difference between the largest observed system read latency, which in

this example is 10 clock cycles, and the system read latency of each memory

device. In this example, the offsets for memory devices 101-108 are equal to

1, 0, 4, 3, 0, 2, 1, and 2, respectively. Thus the memory controUer 400 would

operate memory device 101 at an increased device read latency of one 1 cycle,

wh le memory device 102 would be operated at an increased device read

latency of 0 clock cycle (i.e., equal to the minimum device read latency). Fig.

3 illustrates that the end result of this process is a memory system in which

each memory device 101-108 has an equal system read latency. As a

consequence, when read commands are issued to memory devices DRAM-1

101 - DRAM-8 108, the memory controUer wiU see the read data from aU

memory device of aU memory modules at substantially the same time. WhUe certain embodiments of the invention have been described

and illustrated above, the invention is not Umited to these specific

embodiments as numerous modifications, changes and substitutions of

equivalent elements can be made without departing from the spirit and scope

of d e invention. Accordingly, the scope of the present mvention is not to be

considered as Hmited by the specifics of the particular structures which have

been described and Ulustrated, but is only limited by the scope of the appended

claims.

Claims

What is claimed as new and desired to be protected by LettersPatent of the United States is:

1. A memory device comprising:

a memory array;

a control circuit coupled to the memory array;

at least one of configuration line coupled to said control circuit;

wherein said control circuit operates the memory device at a selected

device read latency based upon a state of a signal asserted on said at least one

configuration line.

2. The memory device of claim 1, wherein said set of device

read latencies includes the memory device's minimum, device read latency.

3. The memory device of claim 1, wherein said control circuit

interprets the state of signals asserted on said first plurahty of configuration

lines as a number of clock cycles and operates the memory device at a device

read latency equal to the minimum device read latency plus the number of

clock cycles.

4. The memory device of claim 1, wherein the control circuit,

responsive to a command issued by an external memory controUer, outputs to

said memory controUer a calibration pattern as read data.

5. The memory device of claim 4, wherein said calibration

pattern includes at least two successive bits which have a different logic state.

6. The memory device of claim 5, wherein said calibration

pattern has its first bit set to a binary 0 and aU subsequent bits set to a binary 1.

7. The memory device of claim 5, wherein said cahbration

pattern has its first bit set to a binary 1 and aU subsequent bits set to a binary 0.

8. The memory device of claim 1, wherein said at least one

configuration line includes a plurahty of configuration lines.

9. The memory device of claim 1, wherein the set of device read

latencies mcludes N device latencies ranging from the device minimum read

latency to a number of clock cycles equal to the device minimum read latency

plus N-l clock cycles.

10. The memory device of claim 9, wherein N equals 8.

11. The memory device of claim 1, further comprising:

an additional configuration line, wherein said additional

configuration line has a signal state which enables or disable a read clock delay

lock loop of said memory device.

12. A memory module comprising:

a plurahty of memory devices; and

a register for providing configuration information to said

plurahty of memory devices;

wherein each of said memory device further comprises,

a memory array;

a control circuit coupled to d e memory array;

at least one of configuration line coupled to said register and

said control circuit; wherein said control circuit operates the memory device at a

selected device read latency based upon a state of a signal asserted on said

at least one configuration line.

13. The memory module of claim 12, wherein said set of device

read latencies includes the memory device's minimum device read latency.

14. The memory module of claim 12, wherein said control circuit

interprets the state of signals asserted on said at least one configuration line as a

number of clock cycles and operates the memory device at a device read latency

equal to the minimum device read latency plus d e number of clock cycles.

15. The memory module of claim 12, wherein the control

circuit, responsive to a command issued by an external memory controUer,

outputs to said memory controUer a calibration pattern as read data.

16. The memory module of claim 15, wherein said calibration

pattern mcludes at least two successive bits which have a different logic state.

17. The memory module of claim 16, wherein said calibration

18. The memory module of claim 16, wherein said calibration

19. The memory module of claim 12, wherein said at least one

configuration line includes a plurality of configuration lines.

20. The memory module of claim 12, wherein the set of device

read latencies includes N device latencies ranging from the device minimum,

read latency to a number of clock cycles equal to the device minimum read

latency plus N-l clock cycles.

21. The memory module of claim 20, wherein N equals 8.

22. The memory module of claim 12, further comprising:

an additional configuration line, wherein said additional

lock loop of said memory device.

23. A mediod of operating a memory device, the memory device

having at least one configuration line, comprising: operating the memory device at a selected device read latency based

upon a state of a signal asserted on said at least one configuration line.

24. The method of claim 23, wherein said set of device read

latencies includes the memory device's mhiimum device read latency.

25. The method of claim 23, wherein said control circuit

interprets the state of the signal asserted on said at least one configuration line

as a number of clock cycles and operates the memory device at a device read

latency equal to the minimum device read latency plus d e number of clock

cycles.

26. The method of claim 23, further comprising the step of:

responsive to a command from a memory controUer, outputting a

calibration pattern.

27. The method of claim 26, wherein said calibration pattern

includes at least two successive bits which have a different logical state.

28. The method of claim 27, wherein said cahbration pattern has

its first bit set to a binary 0 and aU subsequent bits set to a binary 1.

29. The metiiod of claim 27, wherem said cahbration pattern has

its first bit set to a bmary 1 and aU subsequent bits set to a binary 0.

30. The method of claim 23, wherein said at least one

configuration line includes a plurahty of configuration lines.

31. A computer system comprising:

a processor;

a memory controUer coupled to the processor;

at least one memory module coupled to the memory controUer,

each of said memoiy modules comprising a plurahty of memory devices;

wherein each of said memory devices further comprises,

a memory array;

a control circuit coupled to the memory array;

at least one configuration line coupled to said control circuit; wherein said control circuit operates the memory device at a

selected device read latency based upon a state of a signal asserted on said

at least one configuration line.

32. The computer system of claim 31, wherem said set of device

read latencies includes the memory device's minimum device read latency.

33. The computer system of claim 31, wherein said control

circuit interprets the state of a signal asserted on said at least one configuration

line as a number of clock cycles and operates the memory device at a device

read latency equal to the minimum device read latency plus die number of

clock cycles.

34. The computer system of claim 31, wherein the control

circuit, responsive to a command issued by an external memory controUer,

outputs a cahbration pattern.

35. The computer system of claim 34, wherein said cahbration

pattern includes at least two successive bits which have a different logical state.

36. The computer system of claim 35, wherein said cahbration

37. The computer system of claim 35, wherem. said cahbration

38. The computer system of claim 31, wherein said at least one

configuration line mcludes a plurahty of configuration lines.

39. The computer system of claim 31, wherein the set of device

read latencies includes N device latencies ranging from the device minimum

read latency to a number of clock cycles equal to the device minimum read

latency plus N-l clock cycles.

40. The computer system of claim 39, wherein N equals 8.

41. A metiiod of operating a memory system, the memory system

having at a plurahty of memory devices and a memory controUer, comprising

the steps of:

responsive to a command from the memory controUer, setting each

of the plurahty of memory devices to operate at its minimum device read

latency;

measuring the system read latency for each of the plurahty of

memory devices at said memory controUer; determining a maximum system read latency at said memory

controUer, said maximum system read latency being equal to the maximum of

the plurahty of system read latencies;

calculating a plurahty of offsets at said memory controUer, each of

said plurahty of offsets being associated with a corresponding one of the

plurahty of memory devices and being equal to the difference between the

maximum system read latency and the system read latency of the

corresponding one of the plurahty of memory devices; and

setting each of the plurahty ofmemory devices to operate at an

increased device read latency by the memory controUer, wherein the amount of

increased device read latency is equal to the offset associated with that one of

the plurality ofmemory devices.

42. The method of claim 41, wherein the step of measuring

further comprises:

sending a cahbration pattern from each memory device in response

to a command from said memory controUer.

43. The metiiod of claim 42, wherein said cahbration pattern

includes at least two successive bits wliich have a different logical state.

44. The method of claim 43, wherein said cahbration pattern has

its first bit set to a binary 0 and aU subsequent bits set to a binary 1.

45. The method of claim 44, wherem said cahbration pattern has

its first bit set to a binary 1 and aU subsequent bits set to a binary 0.