US20080189383A1 - Distributed cache between servers of a network - Google Patents
Distributed cache between servers of a network Download PDFInfo
- Publication number
- US20080189383A1 US20080189383A1 US11/829,886 US82988607A US2008189383A1 US 20080189383 A1 US20080189383 A1 US 20080189383A1 US 82988607 A US82988607 A US 82988607A US 2008189383 A1 US2008189383 A1 US 2008189383A1
- Authority
- US
- United States
- Prior art keywords
- server
- message
- cache
- file
- distributed cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
Definitions
- the subject matter disclosed generally relates to the field of network servers.
- Computer networks typically include a plurality of client computers that are linked to a number of servers through a communication network.
- the servers may be coupled to one or more mass storage systems, sometimes referred to as a network attached storage (NAS).
- NAS network attached storage
- the network attached storage is capable of storing a large number of files.
- the client computer sends a file request to the server. If the server cannot satisfy the file request it retrieves the file from the NAS.
- Mass storage systems typically contain a large number of hard disk or optical drives. Accessing data from a drive is relatively slow. To minimize access time the retrieved file is typically stored in a cache memory of the server. A subsequent request to the server for the file can be satisfied by the server cache.
- the size of the cache in each server is relatively limited. Consequently, many file requests result in the accessing of the NAS, which is slow and decreases the useful bandwidth of the network. It is therefore desirable to increase the effective cache of the servers in a network.
- cooperative caching In an article entitled “Efficient Cooperative Caching using Hints”, Prasenjit Sarkar and John Hartman, Department of Computer Science, University of Arizona, Tuscon, the authors discuss a concept referred to as cooperative caching.
- a server that receives a file request that it cannot fulfill seeks to have the request satisfied by the cache of another server.
- the server caches are effectively combined to create one large cache that can be accessed by each server.
- the cooperative cache system includes a manager that controls access to the cache.
- the manager locates files within the cache of a server(s) and controls the transfer of the located files.
- the manager also insures cache coherency between each server cache.
- the cache manager resides on a separate server connected to the network. Having a separate server increases the cost and complexity of implementing a cooperative or distributive cache system. Additionally, the manager server may become inoperative thereby eliminating the cooperation between caches. It would be desirable to have a cooperative cache system that did not require a central manager(s). Such a system must also insure cache coherency and a transmission protocol that assures proper receipt of transferred data.
- a distributed cache system includes a distributed cache module which resides in a first server that has a cache.
- the distributed cache module requests and retrieves a file block located in a cache of a second server.
- FIG. 1 is a hardware schematic of a network
- FIG. 2 is a hardware schematic of a server in the network
- FIG. 3 is a schematic showing a pair of servers having a distributed cache
- FIG. 4 is a schematic showing a file block being transferred from a cache of one server to another server
- FIG. 5 is a software schematic of a server in the network
- FIG. 6 is a software schematic of a coordinated communications manager of the server
- FIG. 7 is an illustration of messages being transmitted between servers of the network
- FIG. 8 is a software schematic of a cache manager of the server
- FIG. 9 is a diagram showing a prediction transaction between servers of the network.
- FIG. 10 is a diagram showing a data transaction between servers of the network
- FIG. 11 is a diagram showing a forward data transaction between servers of the network.
- a distributed cache module that allows for a distributed cache between multiple servers of a network without using a cache manager.
- the distributed cache module transmits each message with a logical timestamp.
- the distributed cache module of a server that receives the message will delay forwarding of the message to, for example, a client computer, if preceding timestamps are not received. This insures a correct order of timestamped messages without requiring a central manager to allocate and control the transmission of the messages within the network.
- Each distributed cache module will request and possibly retrieve data from the cache of another server in response to a file request for the data.
- the data of a file may be accessed by a plurality of servers joined in a file context.
- FIG. 1 shows a communication network 10 .
- the network 10 may include a plurality of servers 12 that are coupled to a plurality of client computers 14 .
- the servers 12 may also be coupled to a network attached storage 16 (NAS).
- the NAS 16 may include one or more mass storage systems 18 coupled to the network by a NAS controller 20 .
- each mass storage system 18 may include a plurality of mass storage devices such as hard disk drives (not shown).
- the NAS controller 20 controls the flow of information between the servers 12 and the mass storage systems 18 .
- the network 10 may include routers, hubs, etc. (not shown) that allow information to be transferred between the client computers 14 , servers 12 and NAS 16 .
- the network may be, or include, a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN).
- LAN local area network
- MAN metropolitan area network
- WAN wide area network
- the network 10 may be the Internet.
- FIG. 2 shows an embodiment of a server 12 .
- the server 12 includes a network input/output (I/O) interface 30 that is coupled to a bus 32 .
- the network I/O interface 30 is coupled to the network 10 .
- the I/O interface 30 may include electrical circuits and software that operate Ethernet communication protocol.
- the I/O interface 30 may include buffer memory 34 that temporarily stores information transceived with the network.
- the server 12 includes a processor 36 , random access memory (RAM) 38 and a mass storage device 40 that are all coupled to the bus 32 . Access to memory 38 may be controlled by a direct memory access (DMA) controller 42 .
- the server 12 may also include cache memory 44 .
- the cache may be a portion of memory 38 allocated to caching.
- the cache 44 may include control logic, maps, etc. (not shown) required to access the contents of this block(s) of memory.
- the server 12 may also have a monitor 46 with an accompanying interface port 48 and a keyboard/mouse 50 with a corresponding interface port 52 .
- the network servers 12 A and 12 B operate to combine the caches 44 of each server 12 into a collective distributed cache 44 ′.
- the distributed cache 44 ′ allows any of the client computers 14 to quickly access information within the cache 44 ′ without having to retrieve such information from the NAS 16 .
- server 12 A may retrieve file blocks A, C and D from the NAS 16 .
- Server 12 B may retrieve file blocks B and E.
- File blocks A, C and D are stored within the cache 44 A of server 12 A.
- File blocks B and E are stored within the cache 44 B of server 12 B.
- a client may request file block E.
- the server 12 A can retrieve the file from server 12 B.
- the server 12 A then provides the data block to the requesting client.
- the server 12 B can provide file block E in less time than the NAS 16 , which must access mass storage devices such as hard disk drives.
- FIG. 5 shows the various software modules operated by each server 12 .
- Each server 12 typically includes an operating system 100 that is linked to a plurality of applications 102 .
- the operating system 100 that typically issues and accepts two types of commands, administration 104 and file I/O 106 .
- Administration request can mount, unmount file systems, collect statistics, etc.
- File I/O requests include access to specific file data. The requests may be retrieved from memory by a virtual file system (VFS) 108 .
- VFS virtual file system
- the server 12 may operate a distributed cache module 112 that is linked to a network file system (NSF) module 114 .
- the NSF 114 provides an interface to the network 10 .
- the distributed cache module 112 processes requests from the virtual file system 108 to determine if the requests can be satisfied by the server cache, or the cache of another server within a distributed cache.
- the distributed cache module 112 may include a virtual/node (Vnode) layer 116 that provides an interface with the virtual file system 108 .
- Vnode virtual/node
- the Vnode layer 116 can be constructed to interface with either an operating system sold by Sun Microsystems, Inc. under the trademark SOLARIS, or Microsoft, Inc. under the trademarks WINDOWS NT and WINDOWS 2000.
- Table I provides a list of requests that can be made through the Vnode layer 116 adapted to run with the SOLARIS platform.
- Vop_access( ) Checks access to a given Vnode Vop_addmap( ) Increments the map count Vop_close( ) Close a file Vop_cmp( ) Compare two Vnodes Vop_create( ) Create a file Vop_delmap( ) Decrement the map count Vop_dispose( ) Frees a page from a Vnode Vop_dump( ) Dump kernel debug data Vop_dumpctl( ) Prepare for a kernel dump Vop_frlock( ) Handle file and record locks Vop_fsync( ) Write dirty pages for a Vnode Vop_getattr( ) Get the attributes for a Vnode Vop_getpage( ) Read file data into a page Vop_getsecattr( ) Get access control attributes Vop_inactive( ) Free resources for a Vnode Vop_ioctl(
- the Vnode layer 116 may pass a request from the virtual file system 108 directly to a file system driver (FSD) layer 118 , or to a cache manager (CM) layer 120 through an operating system (OS) top layer 122 .
- the CM layer 120 may be linked to a cache layer 124 used to retrieve and store data within the server cache 44 .
- the CM layer 120 is also linked to a coordinated communications manager (CCM) layer 126 that can control the transfer of information with the other servers 12 in the distributed cache.
- the CM layer 120 may be linked to the FSD layer 118 by a FSD connect layer 128 .
- the distributed cache module 112 may further include a uniform datagram protocol (UDP) connect layer 130 that is directly linked to a TCP/IP layer 132 .
- UDP uniform datagram protocol
- NT/2000 based systems typically generate two types of request mechanisms. The first is a fast I/O request, the second is an I/O request packet (IRP).
- IRP requests are synchronous call-return mechanisms intended to access cached file blocks. IRP requests are asynchronous, send request and go to sleep, request mechanisms. Fast I/O requests are optional for NT/2000 based systems.
- IRP requests are formatted by the Vnode layer 116 into module cache request packets (MCRPs) that are sent to the CM layer 120 . Information located in cache, either directly in the server cache, or within the distributed cache, is transferred to a kernel buffer and a TRUE indication is provided back through the Vnode layer to the virtual file system 108 . If the information is not within cache, the Vnode layer reformats the IRP requests into module file request packets (MFRPs) that are linked to the FSD layer 118 .
- MFRPs module file request packets
- Fast I/O requests are formatted by the Vnode layer 116 into module cache request packets (MCRPs) that are sent to the CM layer 120 .
- MCRPs module cache request packets
- Information located in cache is transferred to a user buffer and a TRUE indication is provided back to the virtual file manager 108 . If the information is not within cache the virtual file manager receives a FALSE indication and then generates an IRP request.
- Tables II and III summarize the structure of various I/O requests from the Vnode layer 116 formatted as MCRP and MFRP, respectively.
- I/O Request Type Defines type of request- Process ID The process initiating the request Request ID Unique ID for this request (response must have same ID). Request sub ID Unique sub ID for request (response must have same sub ID). (Out) Response To be filled in by the Cache Manager (0 indicates a Code no-error response). (Out) New lock ID Returned lock ID (Out) Cache data* Pointer to the cached data block (Out) Cache data len The number of bytes in the cache data block, or the number of bytes transferred I/O Volume ID Volume identifier I/O Filename Fully qualified path name for file I/O File Type Indicates whether a normal file or a directory.
- I/O Location Byte start location in file I/O Length Byte length of block
- I/O Lock ID The ID for the request.
- I/O Wait TRUE if the caller can wait for the data. If FALSE, and the data is not in the distributed cache, an error must be returned. If TRUE, the Cache Manager may request data blocks from the FSD interface. I/O Buffer* Data buffer address
- I/O Volume ID Volume identifier I/O Filename Fully qualified path name for file I/O File Type Indicates whether a normal file or a directory. I/O Location Byte start location in file I/O Length Byte length of block I/O Lock ID The ID for the request. I/O Process ID The process ID making the request I/O Buffer* Data buffer address
- FIG. 6 shows an embodiment of a CCM layer 126 .
- the CCM layer 126 includes a context layer 140 that is coupled to a connect layer 142 and a UDP/TLI/TDI layer 144 by a point to point (P2P) layer 146 , or a broadcast (Bcast) layer 148 .
- the UDP/TLI/TDI layer 144 provides an interface to the network communications stack.
- the context layer 140 maintains groups of servers within a “context”. Servers are typically grouped into a coalition that can share distributed cache. Each server can join and leave a context within the coalition.
- a context may be access to a particular data file. Each server in a context has requested or will request access to the file.
- the P2P layer 146 allows dedicated communication between two servers.
- Table IV lists the point to point transactions that can be performed by the CM 120 and CCM 126 layers.
- the source is attempting to discover information relating to a file from the destination Data Request The source is attempting to retrieve a copy of the file's data from the destination. Data Demand The source is attempting to retrieve the original of the file's data from the destination. Forward Request The source is attempting to send the original of the file's data to the destination.
- the Bcast layer 148 allows for a more distributed communication between the servers.
- the Bcast transactions are issued when the Vnode layer 116 issues a MRCP packet with a file open complete, file open create or file close Opcode.
- the Bcast layer 148 may generate two types of messages; a causal broadcast (CBCAST), or an absolute broadcast (ABCAST).
- the CBCAST adds a logical timestamp to each message transmitted in a given process to insure that the messages are delivered in a correct order.
- the logical timestamp is a means of insuring orderly processing of messages without a central manager.
- the logical timestamp is a vector timestamp not a true stamp of time.
- the vector time stamp includes an “n” vector array, where n is the number of servers in a context. When a context is first created the time stamp is cleared to all zeroes.
- the first server that sends a message in the process increments its variable stamp.
- the incremented timestamp is sent with the message.
- the server Upon receipt of the message the server will retrieve the timestamp and determine whether each preceding timestamp has been received. If not, the message is maintained in buffer and delayed.
- FIG. 7 provides an example of the use of the logical timestamp.
- the first server M 1 sends a message with the timestamp (1.0.0).
- the second server M 2 receives the message and sends a second message with timestamp (1.1.0).
- the third server M 3 receives the first and second messages and sends a message with the timestamp (1.1.1).
- the first server M 1 does not receive the message from the second server M 2 until after receiving the message from the third server M 3 .
- the first server M 1 will delay the third message M(1.1.1) until receiving the second message M(1.1.0) because the server M 1 did not receive all preceding timestamps.
- the logical timestamp insures an orderly processing of messages without the need for a network manager.
- the ABCAST protocol insures that all destination servers receive messages in the same order when joining or leaving a context.
- the ABCAST resides on top of the CBCAST protocol.
- the ABCAST protocol includes a cookie that is passed between servers.
- the cookie may be assigned to the server with the lowest ID in a context. That server may maintain the cookie until it leaves the context.
- a server with a cookie can issue an ABCAST message by adding a context member ID to the message.
- a server without the cookie must request the cookie or wait for the present owner to pass the cookie. Only the current owner of the cookie can broadcast an ABCAST message.
- the context modules of the servers allow a server to join a context, without the need for a network manager.
- the joining server sends an ABCAST message to join a context.
- the joining server may send a join message after receiving a file request from a client computer for a file.
- the server Before joining a coalition the server attempts to either allocate or obtain a member ID.
- the server sends a ConnectAcquireID message that contains an allocated member ID.
- the message includes a common header which is the ID of the root context or coalition.
- the message may also include a member ID field and a character string.
- the server also initiates a timer.
- the timer insures that the server does not wait indefinitely for a response. If there is no response the server retains the self selected member ID.
- the member ID selected by the server may be used by another server but in another context.
- the other server will return a NACK message but without a suggested ID for the transmitting server.
- the transmitting server will then try a new member ID. If the other server is in the same context it will respond with a NACK message and a suggested member ID.
- the server with the lowest member ID in the context sends an ACK return message with a view.
- the view contains the member IDs for all other members in the context. Since each server in a context knows all the other servers in the context, only one server need respond.
- the joining server will then transmit a join ABCAST message to join the context.
- the server takes a member ID that is one number higher than the highest member ID in use. Table V shows the various messages for context messages.
- JoinRequest This is used to indicate a request to join a context.
- JoinRequestAck This is sent in response to the join request. If sent, it also contains a list of member nodes already members of the context.
- JoinRequestNack If a node has a reason to prevent the requesting node from joining.
- Join This is sent as a notification by a node that it has joined the context.
- Leave This is used to indicate that a node is leaving the context.
- User data This is used to denote that the message contents are for an upper layer application.
- ContextIAcquireID Used to acquire a context ID.
- ContextACcquireID NACK Used to indicate the requested ID is already in use, and if possible, to give the actual ID to use.
- Update request Sent to a node to request an update to its list of known contexts i.e. contexts for which it is a member. This is sent when a node has been detected as having “bounced”. Update ACK Response sent with list of contexts.
- the CBCAST and ABCAST broadcast may be transmitted and received in accordance with a first in first out broadcast (FBCAST) protocol.
- FBCAST creates an order for transmitting messages between servers.
- the FBCAST protocol includes the following messages:
- Message Type Description DATA Sends the actual user data message to all members REQUEST NACK (Broadcast) Sent to all members to request a NACK if the message has not been received.
- NACK Direct Sent by a member if a message number has not been received.
- FLAG_NEWSEQUENCE Start of a new FBCAST sequence
- FLAG_RETRANS This message is a retransmission of another message
- FBCAST is performed by a sublayer within the BCAST LAYER 148 .
- FBCAST holds a transmitted message within a queue until a timer expires. When the timer expires FBCAST will remove the message if another message has been sent or a REQUEST NACK has been previously transmitted. If another message has not been received then FBCAST sends a REQUEST NACK.
- the FBCAST When a message is received the FBCAST removes the header and determines whether the message is in order or the start of a new sequence. If the message is in order, FBCAST forwards the message to the proper protocol engine. If the received message is not in order the message is stored in a queue and a timer is set. If a subsequent message is received that creates a proper sequence, the messages are forwarded. If the timer expires, FBCAST sends a NACK message to the transmitting server and resets the timer. FBCAST continues this process until it receives a message(s) that creates a proper sequence or the transmitting server discontinues transmission.
- the connect layer 142 creates connections to each server in a context and detects when servers do not respond.
- the connect layer 142 can locate servers within a context with various addressing schemes and tables. The assignment of member IDs can be performed by the connect layer 142 .
- the connect layer 142 may prepend a header to each message. Table VI is an example of fields that may be in the header.
- Protocol version (always 0x0001).
- Message Type The type of message being sent. Message Flags Individual flags to control the protocol Coalition ID Identifies coalition to which member belongs. Destination The Connect ID the message is destined for Member ID Source Member ID The Connect ID the message was sent from Message ID The ID number for this message (0 means “no ID” and is used for ACK and NACK messages only) Protocol This defines the protocol layer the message is from and to. Member TB Value The “tie breaker” value for the sending computer Ack Along with Message type is used to refine actual message. Coalition name Textual name for the coalition. Not in all messages. TimeStamp Time when message was sent. This only appears in Query messages.
- the connect layer 142 may also send various messages.
- Table VII is an example of the message that can be sent by the connect layer.
- FIG. 8 shows an embodiment of a CM layer 120 .
- the CM layer 120 may include an upper layer interface layer 160 and a request handler layer 162 that provide an interface with the OS interface layer 122 of the distributed cache module 112 .
- the CM layer 120 may further have a lower interface layer 164 that provides an interface to the P2P 146 and Bcast 148 layers of the CCM layer 126 .
- the CM layer may include a prediction manager 166 .
- the prediction manager 166 contains predictions on the possible location of information requested from the virtual file system 108 .
- the predictions may include the location of a file's range of blocks within the distributed cache.
- server 12 A would contain a prediction that file block E is located at server 12 B.
- Server 12 A would then generate a point to point transaction to retrieve file block E from server 12 B.
- the CM layer 120 may include a data manager 168 that moves data between the caches of the distributed cache. To provide a more efficient means of locating and retrieving cached information the CM layer 120 may include a name manager 170 , a range system 172 and a block system 174 .
- Each file is represented by a name structure entry.
- the entry may be indexed by a key that is produced by compressing the filename.
- compression may be utilized with a CRC32 technique.
- the keys are stored in a hash key table that is managed by the name manager 170 .
- Each name entry has pointers to a number of range entries that are managed by the range system 172 .
- Each range entry includes pointers to a list of data blocks that are managed by the block system 174 .
- the blocks contain the actual data stored in the cache 44 .
- FIG. 9 shows a transaction diagram of a prediction request from one server to another server.
- the prediction request can be sent if one server has information that the other server was the last to contain the desired file.
- the server may sequentially query one or more servers for predictions.
- FIG. 10 shows the transfer of data between servers.
- FIG. 11 shows the forwarding of data between servers. No ACK messages are used to send the data in either transaction. Forwarding data is typically performed when the cache of one server is full. Forwarding the data empties some of the server's cache to allow further use of the local cache.
- each server may include a coalition management console (CMC) agent 180 that can receive messages and commands from a CMC station 182 connected to the network 10 .
- the CMC station 182 may transmit configuration parameters to the CMC agents 180 .
- the configuration parameters may define which servers 12 are located in various coalitions.
- the CMC agent 180 may provide a hostname for mapping functions by the connect layer 142 of the CCM layer 126 .
- the user may request a file through an application.
- the request filters to the Vnode layer 116 which formats the request into a MRCP packet that is forwarded to the CM manager 120 .
- the CM layer 120 determines if the requested data is in cache. If the requested data is not in cache then the prediction manager layer 166 determines whether there is a prediction that the requested data is located in the cache of another server. If the data is located in another cache of the distributed cache then the servers enter the transaction shown in FIG. 10 to obtain the data from the remote server.
- the CM layer 120 may conduct the transaction shown in FIG. 9 to obtain a prediction from another server. If the request cannot be satisfied by any cache within the coalition of servers then the distributed cache module provides a FALSE indication to the virtual file system 108 . The virtual file manager then generates a IRP request which is reformatted by the Vnode layer 116 into a MFRP packet. The MFRP packet is processed by the NFS 114 and TCP/IP 132 (if applicable) layers to retrieve the requested file, or portions thereof, from the NAS 16 .
- the network may contain a rogue computer that does not contain a distributed cache module. Consequently, the rogue computer may have dirty data within its cache that is different than the data within the NAS.
- the distributed cache module 112 may verify a file within the NAS before reading it. If the date and time of a file within the NAS 16 matches the date and time of the cached data, the distributed cache knows that the cache data is valid. If the dates and/or times don't match, the cached data is flushed from the distributed cache.
- the CM layer includes a write-through cache process.
- a server that writes new data into its cache sends a notification of the new data to the other servers in a context. For example, referring to FIG. 3 , if the user of server 12 A changes file block A, server 12 A sends a notification to server 12 B that block A contains new data.
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A distributed cache module that allows for a distributed cache between multiple servers of a network without using a central cache manager. The distributed cache module transmits each message with a logical timestamp. The distributed cache module of a server that receives the message will delay forwarding of the message to, for example, a client computer, if preceding timestamps are not received. This insures a correct order of timestamped messages without requiring a central manager to allocate and control the transmission of the messages within the network. Each distributed cache module will request and possibly retrieve data from the cache of another server in response to a file request for the data. The data of a file may be accessed by a plurality of servers joined in a file context.
Description
- 1. Field of the Invention
- The subject matter disclosed generally relates to the field of network servers.
- 2. Background Information
- Computer networks typically include a plurality of client computers that are linked to a number of servers through a communication network. The servers may be coupled to one or more mass storage systems, sometimes referred to as a network attached storage (NAS). The network attached storage is capable of storing a large number of files.
- To open a file the client computer sends a file request to the server. If the server cannot satisfy the file request it retrieves the file from the NAS. Mass storage systems typically contain a large number of hard disk or optical drives. Accessing data from a drive is relatively slow. To minimize access time the retrieved file is typically stored in a cache memory of the server. A subsequent request to the server for the file can be satisfied by the server cache.
- The size of the cache in each server is relatively limited. Consequently, many file requests result in the accessing of the NAS, which is slow and decreases the useful bandwidth of the network. It is therefore desirable to increase the effective cache of the servers in a network.
- In an article entitled “Efficient Cooperative Caching using Hints”, Prasenjit Sarkar and John Hartman, Department of Computer Science, University of Arizona, Tuscon, the authors discuss a concept referred to as cooperative caching. In a cooperative caching scheme a server that receives a file request that it cannot fulfill seeks to have the request satisfied by the cache of another server. The server caches are effectively combined to create one large cache that can be accessed by each server.
- The cooperative cache system includes a manager that controls access to the cache. The manager locates files within the cache of a server(s) and controls the transfer of the located files. The manager also insures cache coherency between each server cache. The cache manager resides on a separate server connected to the network. Having a separate server increases the cost and complexity of implementing a cooperative or distributive cache system. Additionally, the manager server may become inoperative thereby eliminating the cooperation between caches. It would be desirable to have a cooperative cache system that did not require a central manager(s). Such a system must also insure cache coherency and a transmission protocol that assures proper receipt of transferred data.
- A distributed cache system. The system includes a distributed cache module which resides in a first server that has a cache. The distributed cache module requests and retrieves a file block located in a cache of a second server.
-
FIG. 1 is a hardware schematic of a network; -
FIG. 2 is a hardware schematic of a server in the network; -
FIG. 3 is a schematic showing a pair of servers having a distributed cache; -
FIG. 4 is a schematic showing a file block being transferred from a cache of one server to another server; -
FIG. 5 is a software schematic of a server in the network; -
FIG. 6 is a software schematic of a coordinated communications manager of the server; -
FIG. 7 is an illustration of messages being transmitted between servers of the network; -
FIG. 8 is a software schematic of a cache manager of the server; -
FIG. 9 is a diagram showing a prediction transaction between servers of the network; -
FIG. 10 is a diagram showing a data transaction between servers of the network; -
FIG. 11 is a diagram showing a forward data transaction between servers of the network. - Disclosed is a distributed cache module that allows for a distributed cache between multiple servers of a network without using a cache manager. The distributed cache module transmits each message with a logical timestamp. The distributed cache module of a server that receives the message will delay forwarding of the message to, for example, a client computer, if preceding timestamps are not received. This insures a correct order of timestamped messages without requiring a central manager to allocate and control the transmission of the messages within the network. Each distributed cache module will request and possibly retrieve data from the cache of another server in response to a file request for the data. The data of a file may be accessed by a plurality of servers joined in a file context.
- Referring to the drawings more particularly by reference numbers,
FIG. 1 shows acommunication network 10. Thenetwork 10 may include a plurality ofservers 12 that are coupled to a plurality ofclient computers 14. Theservers 12 may also be coupled to a network attached storage 16 (NAS). The NAS 16 may include one or moremass storage systems 18 coupled to the network by aNAS controller 20. By way of example, eachmass storage system 18 may include a plurality of mass storage devices such as hard disk drives (not shown). The NAScontroller 20 controls the flow of information between theservers 12 and themass storage systems 18. - The
network 10 may include routers, hubs, etc. (not shown) that allow information to be transferred between theclient computers 14,servers 12 and NAS 16. The network may be, or include, a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN). By way of example, thenetwork 10 may be the Internet. -
FIG. 2 shows an embodiment of aserver 12. Theserver 12 includes a network input/output (I/O) interface 30 that is coupled to abus 32. The network I/O interface 30 is coupled to thenetwork 10. By way of example, the I/O interface 30 may include electrical circuits and software that operate Ethernet communication protocol. The I/O interface 30 may include buffer memory 34 that temporarily stores information transceived with the network. - The
server 12 includes aprocessor 36, random access memory (RAM) 38 and amass storage device 40 that are all coupled to thebus 32. Access tomemory 38 may be controlled by a direct memory access (DMA)controller 42. Theserver 12 may also includecache memory 44. The cache may be a portion ofmemory 38 allocated to caching. Thecache 44 may include control logic, maps, etc. (not shown) required to access the contents of this block(s) of memory. - The
server 12 may also have amonitor 46 with an accompanyinginterface port 48 and a keyboard/mouse 50 with a corresponding interface port 52. - As shown in
FIG. 3 , thenetwork servers 12A and 12B operate to combine thecaches 44 of eachserver 12 into a collective distributedcache 44′. The distributedcache 44′ allows any of theclient computers 14 to quickly access information within thecache 44′ without having to retrieve such information from theNAS 16. - By way of example,
server 12A may retrieve file blocks A, C and D from theNAS 16. Server 12B may retrieve file blocks B and E. File blocks A, C and D are stored within thecache 44A ofserver 12A. File blocks B and E are stored within the cache 44B of server 12B. A client may request file block E. As shown inFIG. 4 , instead of having to retrieve file block E from theNAS 16, theserver 12A can retrieve the file from server 12B. Theserver 12A then provides the data block to the requesting client. The server 12B can provide file block E in less time than theNAS 16, which must access mass storage devices such as hard disk drives. -
FIG. 5 shows the various software modules operated by eachserver 12. Eachserver 12 typically includes anoperating system 100 that is linked to a plurality ofapplications 102. Theoperating system 100 that typically issues and accepts two types of commands,administration 104 and file I/O 106. Administration request can mount, unmount file systems, collect statistics, etc. File I/O requests include access to specific file data. The requests may be retrieved from memory by a virtual file system (VFS) 108. - The
server 12 may operate a distributedcache module 112 that is linked to a network file system (NSF)module 114. TheNSF 114 provides an interface to thenetwork 10. The distributedcache module 112 processes requests from thevirtual file system 108 to determine if the requests can be satisfied by the server cache, or the cache of another server within a distributed cache. - The distributed
cache module 112 may include a virtual/node (Vnode)layer 116 that provides an interface with thevirtual file system 108. By way of example, theVnode layer 116 can be constructed to interface with either an operating system sold by Sun Microsystems, Inc. under the trademark SOLARIS, or Microsoft, Inc. under the trademarks WINDOWS NT and WINDOWS 2000. By way of example, Table I provides a list of requests that can be made through theVnode layer 116 adapted to run with the SOLARIS platform. -
TABLE I Request Description Vop_access( ) Checks access to a given Vnode Vop_addmap( ) Increments the map count Vop_close( ) Close a file Vop_cmp( ) Compare two Vnodes Vop_create( ) Create a file Vop_delmap( ) Decrement the map count Vop_dispose( ) Frees a page from a Vnode Vop_dump( ) Dump kernel debug data Vop_dumpctl( ) Prepare for a kernel dump Vop_frlock( ) Handle file and record locks Vop_fsync( ) Write dirty pages for a Vnode Vop_getattr( ) Get the attributes for a Vnode Vop_getpage( ) Read file data into a page Vop_getsecattr( ) Get access control attributes Vop_inactive( ) Free resources for a Vnode Vop_ioctl( ) Handle I/O control requests for a Vnode Vop_fid( ) Get unique file ID for a Vnode Vop_link( ) Create a link to the Vnode Vop_lookup( ) Translate a given path/filename to a Vnode Vop_map( ) Map file ranges into memory pages Vop_mkdir( ) Create a directory Vop_open( ) Open a file Vop_pageio( ) Handle swap file access requests Vop_pathconf( ) Establishes file system parameters Vop_poll( ) Handle requests for the poll( ) system function Vop_putpage( ) Write a file page Vop_read( ) Map a file range to memory pages Vop_readdir( ) Read a directory Vop_readlink( ) Follow a link Vop_realvp( ) Gets a real Vnode from a supplied Vnode Vop_remove( ) Delete a file Vop_rename( ) Rename a file Vop_rmdir( ) Remove a directory Vop_rwlock( ) Sets write lock for a write operation Vop_rwunlock( ) Clears a write lock Vop_seek( ) Seek within a file Vop_setattr( ) Sets attributes for a Vnode Vop_setfl( ) Sets file locks on the Vnode Vop_setsevattr( ) Set access control list attributes Vop_shrlock( ) Shared lock support Vop_space( ) Frees space for a Vnode Vop_symlink( ) Create a symbolic link between two path/files - The
Vnode layer 116 may pass a request from thevirtual file system 108 directly to a file system driver (FSD)layer 118, or to a cache manager (CM)layer 120 through an operating system (OS)top layer 122. TheCM layer 120 may be linked to acache layer 124 used to retrieve and store data within theserver cache 44. TheCM layer 120 is also linked to a coordinated communications manager (CCM)layer 126 that can control the transfer of information with theother servers 12 in the distributed cache. TheCM layer 120 may be linked to theFSD layer 118 by aFSD connect layer 128. The distributedcache module 112 may further include a uniform datagram protocol (UDP) connectlayer 130 that is directly linked to a TCP/IP layer 132. - NT/2000 based systems typically generate two types of request mechanisms. The first is a fast I/O request, the second is an I/O request packet (IRP). Fast I/O requests are synchronous call-return mechanisms intended to access cached file blocks. IRP requests are asynchronous, send request and go to sleep, request mechanisms. Fast I/O requests are optional for NT/2000 based systems. IRP requests are formatted by the
Vnode layer 116 into module cache request packets (MCRPs) that are sent to theCM layer 120. Information located in cache, either directly in the server cache, or within the distributed cache, is transferred to a kernel buffer and a TRUE indication is provided back through the Vnode layer to thevirtual file system 108. If the information is not within cache, the Vnode layer reformats the IRP requests into module file request packets (MFRPs) that are linked to theFSD layer 118. - Fast I/O requests are formatted by the
Vnode layer 116 into module cache request packets (MCRPs) that are sent to theCM layer 120. Information located in cache, either directly in the server cache, or within the distributed cache, is transferred to a user buffer and a TRUE indication is provided back to thevirtual file manager 108. If the information is not within cache the virtual file manager receives a FALSE indication and then generates an IRP request. - Tables II and III summarize the structure of various I/O requests from the
Vnode layer 116 formatted as MCRP and MFRP, respectively. -
TABLE II Field Description Type Always set to MCRP Size Byte size of structure I/O Request Type Defines type of request- Process ID The process initiating the request Request ID Unique ID for this request (response must have same ID). Request sub ID Unique sub ID for request (response must have same sub ID). (Out) Response To be filled in by the Cache Manager (0 indicates a Code no-error response). (Out) New lock ID Returned lock ID (Out) Cache data* Pointer to the cached data block (Out) Cache data len The number of bytes in the cache data block, or the number of bytes transferred I/O Volume ID Volume identifier I/O Filename Fully qualified path name for file I/O File Type Indicates whether a normal file or a directory. I/O Location Byte start location in file I/O Length Byte length of block I/O Lock ID The ID for the request. I/O Wait TRUE if the caller can wait for the data. If FALSE, and the data is not in the distributed cache, an error must be returned. If TRUE, the Cache Manager may request data blocks from the FSD interface. I/O Buffer* Data buffer address -
TABLE III Field Description Type Always set to MFRP Size Byte size of structure Request ID Unique ID for this request (response must have same ID). Requestor ID Set to either VNODE or CACHE (Out) Response To be filled in by the FSD Interface layer (0 Code indicates a no-error response). FS valid TRUE if the system specific info is valid in the FS Union (i.e. Windows IRP). FS Union Pointer to the Windows IRP block. Note *** for non-Windows implantations, this will point to the system specific structure which is applicable. I/O valid TRUE if the I/O fields are valid (only used if a system specific structure wasn't attached). I/O Request Type See MCRP I/O Request types (above). I/O Volume ID Volume identifier I/O Filename Fully qualified path name for file I/O File Type Indicates whether a normal file or a directory. I/O Location Byte start location in file I/O Length Byte length of block I/O Lock ID The ID for the request. I/O Process ID The process ID making the request I/O Buffer* Data buffer address -
FIG. 6 shows an embodiment of aCCM layer 126. TheCCM layer 126 includes acontext layer 140 that is coupled to aconnect layer 142 and a UDP/TLI/TDI layer 144 by a point to point (P2P)layer 146, or a broadcast (Bcast)layer 148. The UDP/TLI/TDI layer 144 provides an interface to the network communications stack. Thecontext layer 140 maintains groups of servers within a “context”. Servers are typically grouped into a coalition that can share distributed cache. Each server can join and leave a context within the coalition. By way of example, a context may be access to a particular data file. Each server in a context has requested or will request access to the file. - The
P2P layer 146 allows dedicated communication between two servers. The following Table IV lists the point to point transactions that can be performed by theCM 120 andCCM 126 layers. -
TABLE IV Prediction Request The source is attempting to discover information relating to a file from the destination Data Request The source is attempting to retrieve a copy of the file's data from the destination. Data Demand The source is attempting to retrieve the original of the file's data from the destination. Forward Request The source is attempting to send the original of the file's data to the destination. - The
Bcast layer 148 allows for a more distributed communication between the servers. The Bcast transactions are issued when theVnode layer 116 issues a MRCP packet with a file open complete, file open create or file close Opcode. TheBcast layer 148 may generate two types of messages; a causal broadcast (CBCAST), or an absolute broadcast (ABCAST). - CBCAST adds a logical timestamp to each message transmitted in a given process to insure that the messages are delivered in a correct order. The logical timestamp is a means of insuring orderly processing of messages without a central manager. The logical timestamp is a vector timestamp not a true stamp of time. The vector time stamp includes an “n” vector array, where n is the number of servers in a context. When a context is first created the time stamp is cleared to all zeroes. The first server that sends a message in the process increments its variable stamp. The incremented timestamp is sent with the message. Upon receipt of the message the server will retrieve the timestamp and determine whether each preceding timestamp has been received. If not, the message is maintained in buffer and delayed.
-
FIG. 7 provides an example of the use of the logical timestamp. There are three servers in the group, the vector array is therefore (x, y, z). The first server M1 sends a message with the timestamp (1.0.0). The second server M2 receives the message and sends a second message with timestamp (1.1.0). The third server M3 receives the first and second messages and sends a message with the timestamp (1.1.1). The first server M1 does not receive the message from the second server M2 until after receiving the message from the third server M3. The first server M1 will delay the third message M(1.1.1) until receiving the second message M(1.1.0) because the server M1 did not receive all preceding timestamps. The logical timestamp insures an orderly processing of messages without the need for a network manager. - The ABCAST protocol insures that all destination servers receive messages in the same order when joining or leaving a context. The ABCAST resides on top of the CBCAST protocol. The ABCAST protocol includes a cookie that is passed between servers. By way of example, the cookie may be assigned to the server with the lowest ID in a context. That server may maintain the cookie until it leaves the context.
- A server with a cookie can issue an ABCAST message by adding a context member ID to the message. A server without the cookie must request the cookie or wait for the present owner to pass the cookie. Only the current owner of the cookie can broadcast an ABCAST message.
- The context modules of the servers allow a server to join a context, without the need for a network manager. The joining server sends an ABCAST message to join a context. By way of example, the joining server may send a join message after receiving a file request from a client computer for a file.
- Before joining a coalition the server attempts to either allocate or obtain a member ID. The server sends a ConnectAcquireID message that contains an allocated member ID. The message includes a common header which is the ID of the root context or coalition. The message may also include a member ID field and a character string.
- The server also initiates a timer. The timer insures that the server does not wait indefinitely for a response. If there is no response the server retains the self selected member ID. Alternatively, the member ID selected by the server may be used by another server but in another context. The other server will return a NACK message but without a suggested ID for the transmitting server. The transmitting server will then try a new member ID. If the other server is in the same context it will respond with a NACK message and a suggested member ID.
- As yet another alternative, the server with the lowest member ID in the context sends an ACK return message with a view. The view contains the member IDs for all other members in the context. Since each server in a context knows all the other servers in the context, only one server need respond. The joining server will then transmit a join ABCAST message to join the context. The server takes a member ID that is one number higher than the highest member ID in use. Table V shows the various messages for context messages.
-
TABLE V Message type Description JoinRequest This is used to indicate a request to join a context. JoinRequestAck This is sent in response to the join request. If sent, it also contains a list of member nodes already members of the context. JoinRequestNack If a node has a reason to prevent the requesting node from joining. Join This is sent as a notification by a node that it has joined the context. Leave This is used to indicate that a node is leaving the context. User data This is used to denote that the message contents are for an upper layer application. ContextIAcquireID Used to acquire a context ID. ContextACcquireID NACK Used to indicate the requested ID is already in use, and if possible, to give the actual ID to use. Update request Sent to a node to request an update to its list of known contexts (i.e. contexts for which it is a member). This is sent when a node has been detected as having “bounced”. Update ACK Response sent with list of contexts. - The CBCAST and ABCAST broadcast may be transmitted and received in accordance with a first in first out broadcast (FBCAST) protocol. FBCAST creates an order for transmitting messages between servers. The FBCAST protocol includes the following messages:
-
Message Type Description DATA (Broadcast) Sends the actual user data message to all members REQUEST NACK (Broadcast) Sent to all members to request a NACK if the message has not been received. NACK (Direct) Sent by a member if a message number has not been received. - The FBCAST protocol includes a header with the following fields:
-
Header Value Description Source Member Member ID sending the message Message number The message number for this message Service type The service type (ABCAST, CBCAST, etc) for this message. Flags Various flags attached to this message - The flag field includes the following flags:
- FLAG_NORMAL—A normal message
- FLAG_NEWSEQUENCE—Start of a new FBCAST sequence
FLAG_RETRANS—This message is a retransmission of another message - FBCAST is performed by a sublayer within the
BCAST LAYER 148. FBCAST holds a transmitted message within a queue until a timer expires. When the timer expires FBCAST will remove the message if another message has been sent or a REQUEST NACK has been previously transmitted. If another message has not been received then FBCAST sends a REQUEST NACK. - When a message is received the FBCAST removes the header and determines whether the message is in order or the start of a new sequence. If the message is in order, FBCAST forwards the message to the proper protocol engine. If the received message is not in order the message is stored in a queue and a timer is set. If a subsequent message is received that creates a proper sequence, the messages are forwarded. If the timer expires, FBCAST sends a NACK message to the transmitting server and resets the timer. FBCAST continues this process until it receives a message(s) that creates a proper sequence or the transmitting server discontinues transmission.
- Referring back to
FIG. 6 , theconnect layer 142 creates connections to each server in a context and detects when servers do not respond. Theconnect layer 142 can locate servers within a context with various addressing schemes and tables. The assignment of member IDs can be performed by theconnect layer 142. Theconnect layer 142 may prepend a header to each message. Table VI is an example of fields that may be in the header. -
TABLE VI Header Field Description Version Protocol version (always 0x0001). Message Type The type of message being sent. Message Flags Individual flags to control the protocol Coalition ID Identifies coalition to which member belongs. Destination The Connect ID the message is destined for Member ID Source Member ID The Connect ID the message was sent from Message ID The ID number for this message (0 means “no ID” and is used for ACK and NACK messages only) Protocol This defines the protocol layer the message is from and to. Member TB Value The “tie breaker” value for the sending computer Ack Along with Message type is used to refine actual message. Coalition name Textual name for the coalition. Not in all messages. TimeStamp Time when message was sent. This only appears in Query messages. - The
connect layer 142 may also send various messages. Table VII is an example of the message that can be sent by the connect layer. -
TABLE VII Message Type Description Normal Data A normal DATA message, sent by upper layer protocols/apps Discovery Sent at startup to test a member ID. Machines will respond with Discovery ACK or NACK messages. Discovery ACK An acknowledgement that the ID test was successful. This is ONLY sent at initial startup, to allow the starting Connect entity to build its member list. This message also contains a complete list of known members. Discovery NACK An ack of the Discovery message which indicates the ID was in use. This message also contains a complete list of known members. Test ID Test if an ID number is in use. ID NACK The tested ID number was in use. Query A message sent to query the condition of the site/station. Query ACK An acknowledgement for the Query message. ID Claim Used to claim a specific ID ID Claim NACK If ID already in use after claim process initiated. Shutdown A node sends this when it is shutting down. -
FIG. 8 shows an embodiment of aCM layer 120. TheCM layer 120 may include an upperlayer interface layer 160 and arequest handler layer 162 that provide an interface with theOS interface layer 122 of the distributedcache module 112. TheCM layer 120 may further have alower interface layer 164 that provides an interface to theP2P 146 andBcast 148 layers of theCCM layer 126. - The CM layer may include a
prediction manager 166. Theprediction manager 166 contains predictions on the possible location of information requested from thevirtual file system 108. The predictions may include the location of a file's range of blocks within the distributed cache. In the example shown inFIGS. 3 and 4 ,server 12A would contain a prediction that file block E is located at server 12B.Server 12A would then generate a point to point transaction to retrieve file block E from server 12B. - The
CM layer 120 may include adata manager 168 that moves data between the caches of the distributed cache. To provide a more efficient means of locating and retrieving cached information theCM layer 120 may include aname manager 170, arange system 172 and ablock system 174. - Each file is represented by a name structure entry. The entry may be indexed by a key that is produced by compressing the filename. By way of example, compression may be utilized with a CRC32 technique. The keys are stored in a hash key table that is managed by the
name manager 170. Each name entry has pointers to a number of range entries that are managed by therange system 172. Each range entry includes pointers to a list of data blocks that are managed by theblock system 174. The blocks contain the actual data stored in thecache 44. -
FIG. 9 shows a transaction diagram of a prediction request from one server to another server. The prediction request can be sent if one server has information that the other server was the last to contain the desired file. Alternatively, the server may sequentially query one or more servers for predictions. -
FIG. 10 shows the transfer of data between servers.FIG. 11 shows the forwarding of data between servers. No ACK messages are used to send the data in either transaction. Forwarding data is typically performed when the cache of one server is full. Forwarding the data empties some of the server's cache to allow further use of the local cache. - Referring to
FIGS. 1 and 5 , each server may include a coalition management console (CMC)agent 180 that can receive messages and commands from aCMC station 182 connected to thenetwork 10. TheCMC station 182 may transmit configuration parameters to theCMC agents 180. The configuration parameters may define whichservers 12 are located in various coalitions. TheCMC agent 180 may provide a hostname for mapping functions by theconnect layer 142 of theCCM layer 126. - In operation, the user may request a file through an application. The request filters to the
Vnode layer 116 which formats the request into a MRCP packet that is forwarded to theCM manager 120. Using thename manager 170 and therange system 172 theCM layer 120 determines if the requested data is in cache. If the requested data is not in cache then theprediction manager layer 166 determines whether there is a prediction that the requested data is located in the cache of another server. If the data is located in another cache of the distributed cache then the servers enter the transaction shown inFIG. 10 to obtain the data from the remote server. - If the prediction manager does not contain a prediction regarding the request, the
CM layer 120 may conduct the transaction shown inFIG. 9 to obtain a prediction from another server. If the request cannot be satisfied by any cache within the coalition of servers then the distributed cache module provides a FALSE indication to thevirtual file system 108. The virtual file manager then generates a IRP request which is reformatted by theVnode layer 116 into a MFRP packet. The MFRP packet is processed by theNFS 114 and TCP/IP 132 (if applicable) layers to retrieve the requested file, or portions thereof, from theNAS 16. - The network may contain a rogue computer that does not contain a distributed cache module. Consequently, the rogue computer may have dirty data within its cache that is different than the data within the NAS. To prevent the reading of invalid data, the distributed
cache module 112 may verify a file within the NAS before reading it. If the date and time of a file within theNAS 16 matches the date and time of the cached data, the distributed cache knows that the cache data is valid. If the dates and/or times don't match, the cached data is flushed from the distributed cache. - The CM layer includes a write-through cache process. A server that writes new data into its cache sends a notification of the new data to the other servers in a context. For example, referring to
FIG. 3 , if the user ofserver 12A changes file block A,server 12A sends a notification to server 12B that block A contains new data. - While certain exemplary embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad invention, and that this invention not be limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those ordinarily skilled in the art.
Claims (21)
1. A distributed cache system that includes a first server that has a first cache and a second server that has a second cache, comprising:
a first distributed cache module that resides in the first server and transfers a message and a logical timestamp; and,
a second distributed cache module that resides in the second server and receives the message, said second distributed cache module delays a transfer of the message if said second server has not received all preceding logical timestamps.
2. The system of claim 1 , wherein said first distributed cache module has a cookie.
3. The system of claim 1 , wherein said first and second distributed cache modules each have a unique member ID in a context.
4. The system of claim 1 , wherein said first distributed cache module assigns its own unique member ID.
5. The system of claim 1 , wherein said first distributed cache module contains a prediction of a location of data in said second cache of said second server.
6. The system of claim 1 , wherein said logical timestamp includes an n array where n is equal to a number of servers in a file context.
7. A network, comprising:
a first server that includes a first cache and a first distributed cache module, said first distributed cache module transfers a message and a logical timestamp; and,
a second server that includes a second cache and a second distributed cache module, said second distributed cache module receives the message and delays the transfer of the message if said second server has not received all preceding logical timestamps.
8. The network of claim 7 , wherein said first distributed cache module has a cookie.
9. The network of claim 7 , wherein said first and second distributed cache modules each have a unique member ID for a context.
10. The network of claim 7 , wherein said first distributed cache module assigned its own unique member ID.
11. The network of claim 7 , wherein said first distributed cache module contains a prediction of a location of data in said second cache of said second server.
12. The network of claim 7 , wherein said logical timestamp includes an n array where n is equal to a number of servers in a context.
13. The network of claim 7 , further comprising a rogue server and a network attached storage, said first and second distributed cache modules verify a file before transferring a file from said network attached storage.
14. The network of claim 7 , wherein said first cache includes a first file block and said second cache includes a second file block, said first distributed cache module requests and retrieves said second file block from said second cache.
15. A method for transferring a message between a first server that has a first cache and a second server that has a second cache, comprising:
transmitting a message and a logical timestamp from the first server;
receiving the message at the second server;
determining whether the second server has received all preceding logical timestamps; and,
delaying a transfer of the message if the second server has not received all preceding logical timestamps.
16. The method of claim 15 , wherein the first server has a cookie.
17. The method of claim 15 , further comprising transmitting from the first server a message with a self assigned member ID to enter a context.
18. The method of claim 17 , further comprising transmitting from the second server to the first server a NACK message with a suggested member ID.
19. The method of claim 17 , further comprising transmitting from the second server an ACK message and member IDs for every server in a context.
20. The method of claim 15 , wherein the second server requests and retrieves a data file block located in the first cache of the first server.
21-52. (canceled)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/829,886 US20080189383A1 (en) | 2002-12-06 | 2007-07-28 | Distributed cache between servers of a network |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/313,861 US7254617B2 (en) | 2002-12-06 | 2002-12-06 | Distributed cache between servers of a network |
US11/829,886 US20080189383A1 (en) | 2002-12-06 | 2007-07-28 | Distributed cache between servers of a network |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/313,861 Continuation US7254617B2 (en) | 2002-12-06 | 2002-12-06 | Distributed cache between servers of a network |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080189383A1 true US20080189383A1 (en) | 2008-08-07 |
Family
ID=32468362
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/313,861 Expired - Fee Related US7254617B2 (en) | 2002-12-06 | 2002-12-06 | Distributed cache between servers of a network |
US11/829,886 Abandoned US20080189383A1 (en) | 2002-12-06 | 2007-07-28 | Distributed cache between servers of a network |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/313,861 Expired - Fee Related US7254617B2 (en) | 2002-12-06 | 2002-12-06 | Distributed cache between servers of a network |
Country Status (1)
Country | Link |
---|---|
US (2) | US7254617B2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102169494A (en) * | 2011-04-06 | 2011-08-31 | 北京师范大学 | Distributed remotely-sensed data managing system and method |
US20120297010A1 (en) * | 2011-05-18 | 2012-11-22 | Alibaba Group Holding Limited | Distributed Caching and Cache Analysis |
CN104156475A (en) * | 2014-08-25 | 2014-11-19 | 中国联合网络通信集团有限公司 | Geographic information reading method and device |
CN109117422A (en) * | 2017-06-23 | 2019-01-01 | 中国软件与技术服务股份有限公司 | Globally unique number rapid generation and system in a kind of extensive high concurrent system |
Families Citing this family (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8225002B2 (en) * | 1999-01-22 | 2012-07-17 | Network Disk, Inc. | Data storage and data sharing in a network of heterogeneous computers |
US7062515B1 (en) * | 2001-12-28 | 2006-06-13 | Vignette Corporation | System and method for the synchronization of a file in a cache |
US6959361B2 (en) * | 2002-04-25 | 2005-10-25 | Sun Microsystems, Inc. | Distributed caching mechanism for pending memory operations within a memory controller |
US7254617B2 (en) * | 2002-12-06 | 2007-08-07 | Scott Ruple | Distributed cache between servers of a network |
JP4131514B2 (en) * | 2003-04-21 | 2008-08-13 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Network system, server, data processing method and program |
US20050188055A1 (en) * | 2003-12-31 | 2005-08-25 | Saletore Vikram A. | Distributed and dynamic content replication for server cluster acceleration |
US7403945B2 (en) * | 2004-11-01 | 2008-07-22 | Sybase, Inc. | Distributed database system providing data and space management methodology |
US7647424B2 (en) * | 2005-06-15 | 2010-01-12 | Hostway Corporation | Multi-level redirection system |
US7657602B2 (en) * | 2006-01-30 | 2010-02-02 | International Business Machines Corporation | Preventing human error in instant messaging |
US8719216B2 (en) * | 2007-10-24 | 2014-05-06 | The Boeing Company | Caching of web form post-query requests |
US8131872B2 (en) * | 2008-05-30 | 2012-03-06 | International Business Machines Corporation | Affinity-based transaction processing |
US8108623B2 (en) * | 2008-10-26 | 2012-01-31 | Microsoft Corporation | Poll based cache event notifications in a distributed cache |
US8161244B2 (en) * | 2009-05-13 | 2012-04-17 | Microsoft Corporation | Multiple cache directories |
US9952968B2 (en) * | 2010-01-29 | 2018-04-24 | Micro Focus Software, Inc. | Methods and system for maintaining data coherency in distributed data cache network |
US9229793B2 (en) * | 2010-04-22 | 2016-01-05 | Salesforce.Com, Inc. | System, method and computer program product for routing messages to a server |
US8625635B2 (en) * | 2010-04-26 | 2014-01-07 | Cleversafe, Inc. | Dispersed storage network frame protocol header |
US8396936B2 (en) * | 2010-10-28 | 2013-03-12 | Hewlett-Packard Development Company, L.P. | Computer system with cooperative cache |
US8965879B2 (en) * | 2011-06-03 | 2015-02-24 | Microsoft Technology Licensing, Llc | Unique join data caching method |
US8990243B2 (en) * | 2011-11-23 | 2015-03-24 | Red Hat, Inc. | Determining data location in a distributed data store |
US10031933B2 (en) | 2014-03-02 | 2018-07-24 | Netapp, Inc. | Peer to peer ownership negotiation |
US9898520B2 (en) | 2014-03-25 | 2018-02-20 | Open Text Sa Ulc | Systems and methods for seamless access to remotely managed documents using synchronization of locally stored documents |
US20160162218A1 (en) * | 2014-12-03 | 2016-06-09 | International Business Machines Corporation | Distributed data deduplication in enterprise networks |
US11003632B2 (en) | 2016-11-28 | 2021-05-11 | Open Text Sa Ulc | System and method for content synchronization |
US10594771B2 (en) * | 2017-02-09 | 2020-03-17 | International Business Machines Corporation | Distributed file transfer with high performance |
US11301431B2 (en) | 2017-06-02 | 2022-04-12 | Open Text Sa Ulc | System and method for selective synchronization |
CN108449270A (en) * | 2018-03-21 | 2018-08-24 | 中南大学 | Buffer memory management method priority-based in opportunistic network |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5544345A (en) * | 1993-11-08 | 1996-08-06 | International Business Machines Corporation | Coherence controls for store-multiple shared data coordinated by cache directory entries in a shared electronic storage |
US6327630B1 (en) * | 1996-07-24 | 2001-12-04 | Hewlett-Packard Company | Ordered message reception in a distributed data processing system |
US20030182512A1 (en) * | 2002-03-22 | 2003-09-25 | Per Hammarlund | Use of a context identifier in a cache memory |
US20030212866A1 (en) * | 1995-02-20 | 2003-11-13 | Misako Takahashi | Memory control apparatus and method for assigning a cache memory to store clean and dirty data based on a memory in the cache memories |
US20040064650A1 (en) * | 2002-09-27 | 2004-04-01 | International Business Machines Corporation | Method, system, and program for maintaining data in distributed caches |
US7254617B2 (en) * | 2002-12-06 | 2007-08-07 | Scott Ruple | Distributed cache between servers of a network |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7085843B2 (en) * | 2000-07-13 | 2006-08-01 | Lucent Technologies Inc. | Method and system for data layout and replacement in distributed streaming caches on a network |
US7035911B2 (en) * | 2001-01-12 | 2006-04-25 | Epicrealm, Licensing Llc | Method and system for community data caching |
US7213071B2 (en) * | 2001-04-03 | 2007-05-01 | International Business Machines Corporation | Quality of service improvements for network transactions |
US20020184403A1 (en) * | 2001-04-06 | 2002-12-05 | Dahlin Michael D. | Methods for near-optimal bandwidth-constrained placement in a wide-area network |
-
2002
- 2002-12-06 US US10/313,861 patent/US7254617B2/en not_active Expired - Fee Related
-
2007
- 2007-07-28 US US11/829,886 patent/US20080189383A1/en not_active Abandoned
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5544345A (en) * | 1993-11-08 | 1996-08-06 | International Business Machines Corporation | Coherence controls for store-multiple shared data coordinated by cache directory entries in a shared electronic storage |
US20030212866A1 (en) * | 1995-02-20 | 2003-11-13 | Misako Takahashi | Memory control apparatus and method for assigning a cache memory to store clean and dirty data based on a memory in the cache memories |
US6327630B1 (en) * | 1996-07-24 | 2001-12-04 | Hewlett-Packard Company | Ordered message reception in a distributed data processing system |
US20030182512A1 (en) * | 2002-03-22 | 2003-09-25 | Per Hammarlund | Use of a context identifier in a cache memory |
US20040064650A1 (en) * | 2002-09-27 | 2004-04-01 | International Business Machines Corporation | Method, system, and program for maintaining data in distributed caches |
US7254617B2 (en) * | 2002-12-06 | 2007-08-07 | Scott Ruple | Distributed cache between servers of a network |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102169494A (en) * | 2011-04-06 | 2011-08-31 | 北京师范大学 | Distributed remotely-sensed data managing system and method |
US20120297010A1 (en) * | 2011-05-18 | 2012-11-22 | Alibaba Group Holding Limited | Distributed Caching and Cache Analysis |
US9380127B2 (en) * | 2011-05-18 | 2016-06-28 | Alibaba Group Holding Limited | Distributed caching and cache analysis |
CN104156475A (en) * | 2014-08-25 | 2014-11-19 | 中国联合网络通信集团有限公司 | Geographic information reading method and device |
CN109117422A (en) * | 2017-06-23 | 2019-01-01 | 中国软件与技术服务股份有限公司 | Globally unique number rapid generation and system in a kind of extensive high concurrent system |
Also Published As
Publication number | Publication date |
---|---|
US20040111486A1 (en) | 2004-06-10 |
US7254617B2 (en) | 2007-08-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7254617B2 (en) | Distributed cache between servers of a network | |
US7617216B2 (en) | Metadata offload for a file server cluster | |
US7653722B1 (en) | Server monitoring framework | |
US7840618B2 (en) | Wide area networked file system | |
US7266555B1 (en) | Methods and apparatus for accessing remote storage through use of a local device | |
US7620991B2 (en) | Optimized network cache for virus scanning by examining the magic bytes of a file | |
US7590747B2 (en) | Distributed storage cluster architecture | |
EP1410247B1 (en) | A network storage system | |
US9213720B2 (en) | Peer-to-peer redundant file server system and methods | |
US7203731B1 (en) | Dynamic replication of files in a network storage system | |
US7266556B1 (en) | Failover architecture for a distributed storage system | |
US7734720B2 (en) | Apparatus and system for distributing block data on a private network without using TCP/IP | |
US7426576B1 (en) | Highly available DNS resolver and method for use of the same | |
US7428540B1 (en) | Network storage system | |
US10783121B2 (en) | Techniques for optimizing data flows in hybrid cloud storage systems | |
US20060167969A1 (en) | Data caching based on data contents | |
US7281168B1 (en) | Failover architecture for local devices that access remote storage | |
WO2021007752A1 (en) | Return-to-source method and related device in content delivery network | |
US20130318229A1 (en) | Method and system for name space propagation and file caching to remote nodes in a storage system | |
CN108540510B (en) | Cloud host creation method and device and cloud service system | |
WO2023185454A1 (en) | Data access method and related apparatus | |
JP2005063374A (en) | Data management method, data management device, program for the same, and recording medium | |
CN110109871B (en) | Cross-site high-energy physical data access method and system | |
KR100751075B1 (en) | The Remote Mirroring System using the Parallel Transfer Technique on NAS | |
Lu et al. | A high performance cluster file system with standard network file system interface |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |