US9697227B2 - Concurrent access and transactions in a distributed file system - Google Patents

Concurrent access and transactions in a distributed file system Download PDF

Info

Publication number
US9697227B2
US9697227B2 US14/524,979 US201414524979A US9697227B2 US 9697227 B2 US9697227 B2 US 9697227B2 US 201414524979 A US201414524979 A US 201414524979A US 9697227 B2 US9697227 B2 US 9697227B2
Authority
US
United States
Prior art keywords
file system
operation
inode
update intent
inodes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/524,979
Other versions
US20160117336A1 (en
Inventor
Mohit Aron
Ganesha SHANMUGANATHAN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Cohesity Inc
Original Assignee
Cohesity Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cohesity Inc filed Critical Cohesity Inc
Priority to US14/524,979 priority Critical patent/US9697227B2/en
Assigned to Cohesity, Inc. reassignment Cohesity, Inc. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ARON, MOHIT, SHANMUGANATHAN, GANESHA
Publication of US20160117336A1 publication Critical patent/US20160117336A1/en
Application granted granted Critical
Publication of US9697227B2 publication Critical patent/US9697227B2/en
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G06F17/30194
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/176Support for shared access to files; File sharing support
    • G06F16/1767Concurrency control, e.g. optimistic or pessimistic approaches
    • G06F17/30168

Abstract

Embodiments described herein provide techniques for maintaining consistency in a distributed system (e.g., a distributed secondary storage system). According to one embodiment of the present disclosure, a first set of file system objects included in performing the requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into an inode associated with each identified file system object. Each file system object corresponding to the inode is modified as specified by the update intent in that inode. After modifying the file system object corresponding to the inode, the update intent is removed from that inode.

Description

BACKGROUND

Field

Embodiments presented herein generally relate to distributed computing. More specifically, embodiments presented herein provide a high-performance distributed file system that provides concurrent access and transaction safety.

Description of the Related Art

A distributed computing network system includes multiple computer systems which interact with one another to achieve a common goal. The computer systems are networked to form a cluster. Computer systems in the cluster may share different resources, such as data, storage, processing power, and the like.

An example of a distributed computing network system is a secondary storage environment. A cluster of secondary storage systems may provide services for primary storage systems. For instance, secondary storage systems may provide backup, copy, and test and development services for data residing in primary storage. The secondary storage cluster can expose data backed up from the primary storage system to clients systems, which can read or write data to the file system.

A distributed file system needs to support concurrent access to file system objects, e.g., files and directories, while also maintaining a consistent state. When different nodes in the cluster may access the file system concurrently (e.g., in response to read and write requests sent by clients), it is important that the file system remain consistent. That is, updates to the file system performed by one node are visible to other nodes of the cluster. Further, consistency requires tolerance for node failures, such that if a node fails while performing an update to the file system, the incomplete file system operations transactions need to be either completed or aborted.

SUMMARY

One embodiment presented herein provides a method for performing a transaction in a distributed file system. The method generally includes identifying a first set of file system objects modified in performing a file system operation in response to a request to perform the file system operation. The method also includes inserting an update intent corresponding to the requested file system operation into an inode associated with each identified file system object. The update intent specifies an ordered sequence of inodes associated with the requested file system operation. For each inode, (i) either the inode or the file system object corresponding to the inode is modified according to the update intent in that inode, and (ii) the update intent is removed from that inode after the modification.

Another embodiment presented herein provides a computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform an operation for performing a transaction in a distributed file system. The operation itself generally includes identifying a first set of file system objects modified in performing a file system operation in response to a request to perform the file system operation. The operation also includes inserting an update intent corresponding to the requested file system operation into an inode associated with each identified file system object. The update intent specifies an ordered sequence of inodes associated with the requested file system operation. For each inode, (i) either the inode or the file system object corresponding to the inode is modified according to the update intent in that inode, and (ii) the update intent is removed from that inode after the modification.

Another embodiment provides a system having a processor and a memory storing program code. The program code, when executed on the processor, performs an operation for performing a transaction in a distributed file system. The operation itself generally includes identifying a first set of file system objects modified in performing a file system operation in response to a request to perform the file system operation. The operation also includes inserting an update intent corresponding to the requested file system operation into an inode associated with each identified file system object. The update intent specifies an ordered sequence of inodes associated with the requested file system operation. For each inode, (i) either the inode or the file system object corresponding to the inode is modified according to the update intent in that inode, and (ii) the update intent is removed from that inode after the modification.

Another embodiment provides a method for locking entities to be modified in an operation in a distributed file system. The method generally includes receiving a request to perform a file system operation. The request specifies one or more file system objects to be modified in a distributed file system managed by a cluster of nodes. A node-specific lock is requested on each of the file system objects. The node-specific lock prevents other nodes in the cluster from accessing the locked file system objects during the file system operation. Upon receiving the node-specific locks on the file system objects, a process-specific lock is requested from a process-specific lock manager on each of the file system objects. The process-specific lock prevents other local processes from accessing the file system objects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example computing environment, according to one embodiment.

FIG. 2 illustrates an example secondary storage server, according to one embodiment.

FIG. 3 illustrates an inode for an object in a distributed file system, according to one embodiment.

FIG. 4 illustrates a method for obtaining locks on file system objects, according to one embodiment.

FIG. 5 illustrates example of objects modified by a file system operation, according to one embodiment.

FIG. 6 illustrates a method for atomically updating file system objects, according to one embodiment.

FIG. 7 illustrates a method for committing or aborting a file system transaction left in an intermediate state due to a node failure, according to one embodiment.

FIG. 8 illustrates a node in a distributed file system, according to one embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments presented herein provide a high-performance distributed file system. The distributed file system is stored on a cluster of computing nodes. Each node can read and write to any file system object (e.g., a file or directory) and perform common file system operations, such as copy, rename, delete, etc., in response to requests sent by clients. For example, in one embodiment, the distributed file system disclosed herein may be exposed to a client as an NFS mount point. In such case, the distributed file system manages concurrency and consistency issues transparently to an NFS client. Specifically, nodes of the cluster can perform file system operations requested by multiple clients concurrently and each node can process multiple requests concurrently. Further, the nodes in the cluster ensure that each file system transaction requested by a client that updates or modifies multiple file system objects is either completed or aborted.

In one embodiment, a two-level locking scheme allows file system entities (e.g., files and directories of the file system) to be accessed concurrently. Specifically, when a node in the distributed cluster receives a request to access the file system (e.g., to modify data, read data, etc.), the node requests a node-level lock, referred to as a “ticket,” from a ticket service, for each file system object associated with a requested file system operation (e.g., a set of inodes). The ticket service is a distributed lock management service that issues tickets for file system entities to nodes of the cluster. A ticket provides a node with shared or exclusive access to a given file system object. Once a ticket is obtained, the node can access files associated with that ticket. In addition to the cluster-level lock, i.e., the ticket, a process operating on the file also requests a process-wide lock for each file system object associated with the requested file system operation. A process-wide lock provides a process with shared or exclusive access to a given entity on that node. Once tickets and process-wide locks are obtained for each associated file system object, the node may perform a requested file system operation on the locked objects. Examples of operations include create operations (where the node creates a file or directory) or delete operations (where the node deletes a file or directory from the file system). The tickets and locks allow the nodes to perform file system operations safely and concurrently. For example, when acting as an NFS mount point, the process on the nodes can perform any file system operations requested by an NFS client.

In addition to addressing concurrency issues via the two-tiered locking mechanism discussed above, the distributed file system also supports transactions such that if a file system operation modifies multiple entities, the file system does not become inconsistent should a node performing that operation crash before the changes can be completed. In one embodiment, a process operating on the file stores an update intent in a structure used to describe attributes of that entity, such as an inode. As known, an inode is a data structure that describes a file system object, e.g., a file or directory. The update intent may include a unique identifier associated with the file system operation. Further, the update intent in an inode may specify each file system object participating in the transaction. In addition, the list of file system objects is ordered according to the order the operations should be performed to complete the file system operation.

After writing an update intent for a given transaction to the inodes, the process performs the transaction on each file system object affected by the operation. After completing the operation specified in each inode, the process removes the update intent from the inode.

Inserting an update intent prior to performing a transaction and removing the update intent when finalizing the transaction ensures file system consistency, particularly in cases where a node fails during any stage of the transaction. In one embodiment, a node may complete or abort a transaction left in an intermediate state due to failure of another node performing the transaction.

For example, consider a case where a node performs a rename operation that involves file system entities A, B, and C, where A is a source directory file handle, B is a destination directory file handle, and C is a file handle to the file being renamed. Assume that the process executing in the node has inserted an update intent in inodes corresponding to entities A, B, and C describing the rename operation to be performed. Further, assume that the node completes updating entity A but crashes before (or while) finalizing entity B. At the point of failure, the inode for entity A no longer stores the update intent associated with the rename operation, but file handles B and C still have an update intent because the node did not complete the rename operation. When another node accesses entities B or C, the node detects that the file system handles have an update intent from the incomplete transaction.

In one embodiment, when a node detects an update intent in an inode, the node can either complete or abort the incomplete transaction associated with that update intent. To so do, the node identifies the sequence of inodes referenced by the update intent. Doing so allows the node to determine which inodes were part of an incomplete transaction. Once identified (and after obtaining a ticket and lock on that set of inodes), the node can determine whether the incomplete transaction should be completed or abandoned.

To do so, the node evaluates which (if any) of the inodes store an update intent associated with the incomplete transaction (as identified using a transaction-specific identifier in the update intent). If the operation was not fully prepared at the time it was interrupted, then one (or more) nodes at the end of the sequence will not store an update intent. In such a case, the operation is abandoned and the update intent is simply removed from all the inodes associated with the transaction. If all of the inodes have an update intent, then the operation was interrupted after being fully prepared, but before being committed. In this case, the failed operation is rolled forward. If one (or more) nodes at the beginning of the sequence do not have an update intent, then the operation was interrupted after being prepared and after being partially performed by the nodes at the beginning of the sequence which do not have an update intent. In this case, the operation can be completed by performing the operation specified by the update intent in the relevant inodes at the end of the sequence.

Advantageously, the techniques described herein provide consistency in a distributed file system. The two-level locking scheme allows nodes to access and modify file system entities concurrently. Further, writing an update intent to inodes participating in a transaction provides a commit mechanism that does not rely on any centralized database or journaling techniques to ensure transaction consistency. As a result, the distributed file system disclosed herein may scale more efficiently than distributed systems that maintain transaction databases and journals.

Note, embodiments are described below using a distributed secondary storage system as a reference example of a cluster which exports a distributed file system to clients. However, one of ordinary skill in the art will recognize that the distributed file system described herein may be adapted for use with a variety of distributed computing clusters.

FIG. 1 illustrates an example computing environment 100, according to one embodiment. As shown, the computing environment includes a primary storage server 105 and a cluster 122. As shown, the cluster 122 includes multiple computing servers 110 that host and manage a distributed service (e.g., secondary storage services). The primary storage server 105 and secondary storage servers 110 are each connected via a network 120. The primary storage server 105 and secondary storage servers 110 may provide storage for a data center.

In particular, the cluster 122 of secondary storage servers 110 form a distributed file system which provides backup storage and services to the primary storage server 105. Illustratively, each secondary storage server 110 (or “node”) of the distributed file system includes bridge components 112 and data gathering components 114. The bridge components 112 may provide software applications that allow external clients to access each node. The data gathering components 114 are generally configured to retrieve data from primary storage server 105 (e.g., by communicating with the primary storage server 105 using a backup protocol). Further, each node includes one or more hard disk drives (HDDs) 116 and solid-state drives (SSDs) 118 for storing data from primary storage server 105. The SSDs 118 increase the performance running secondary storage transactions, such as serial-write transactions. In one embodiment, the distributed file system is the SnapFS® developed by Cohesity, Inc.

In one embodiment, distributed cluster 122 may expose backup data to applications 127 on clients 125. For example, assume the distributed cluster 122 provides a backup of a file system on the primary storage server 105. In such a case, the cluster 122 could create a snapshot of a file system on the primary storage server 105 every few minutes. Further, the cluster 122 could expose a clone of one of the snapshots to applications 127. For example, the cluster 122 could export a clone as an NFS mount point (or SMB, CIFS, etc.), allowing the clients 125 to mount the snapshot over a directory in the local file system of the client 125. In such a case, the bridge components 112 may respond to requests from clients 125 to perform NFS operations (or operations for SMB, CIFS, etc.) on the mounted directory (e.g., lookup, create, read, write, rename, remove, etc.). As described below, nodes 110 of the cluster 122 manage concurrent access to the clone made by NFS clients using both node level locks (tickets) and local process locks on inodes touched by a given file system operation. That is, a request from a client 125 may be directed to any of the nodes 110. Additionally, the nodes 110 maintain a consistent distributed file system by writing an update intent to each inode participating in a given file system operation. Doing so allows the cluster 122 to maintain a consistent file system state, even when a node fails in the middle of a file system operation.

FIG. 2 further illustrates a secondary storage server 110, according to one embodiment. As shown, the secondary storage server 110 includes bridge components 112, data gathering components 114, and a node identifier 205. The node identifier 205 is a distinct value that may be based on the MAC address of the secondary storage server 110.

As stated, the data gathering components 114 communicate with primary storage servers in the computing environment 100. Illustratively, the data gathering components 114 may retrieve data from primary storage servers 201 and 202, where each primary storage server 201 and 202 execute backup protocols such as recovery manager (RMAN) 203 and vStorage APIs for Data Protection (VADP).

The secondary storage server includes a file system 210. The file system 210, in conjunction with the file systems of other secondary storage servers in the cluster, forms a distributed system used to store backup data from the primary storage services. For example, assume the bridge components 112 receive a virtual machine disk (VMDK) file from the backup data gathering components 114. In such a case, the bridge components 112 writes the VMDK file to disk storage 160. In addition, file system metadata is stored in SSD 118. However, as a VMDK file can be quite large (e.g., terabytes), bridge components 112 may store the VMDK file as set of smaller blobs (binary large object) files managed by a blob manager, each having a set of chunks (e.g., approximately 8 KB each). Backup data below a given size threshold (e.g., files less than 256K bytes) could be stored in the SSD 118 directly along with the file system metadata.

Once stored, the bridge components 112 may expose file system 210 to clients through file system protocols, such as NFS mount point 212. Further, the bridge components 112 may include processes for performing NFS operations requested by clients.

Further, the secondary storage server 110 may include a ticket service 222. The ticket service 222 is an in-memory distributed lock manager that issues locks (“tickets”) to requesting nodes. In one embodiment, nodes in the cluster elect a “leader” ticket service 222 that issues tickets for entities of the file system 210. Further, the ticket services 222 executing on the remaining nodes are “slave” ticket services 222. The leader ticket service 222 issues a ticket to a requesting node (or queues the request). A ticket provides a node-level lock on a file system object such as a file or directory. A node that holds a ticket on a file has access to that entity. When the node acquires a ticket from the leader ticket service 222, the leader ticket service 222 pins the ticket. While a ticket is pinned, other processes cannot obtain the ticket. When the node finishes performing a given operation, the node unpins the ticket. The corresponding slave ticket service 222 caches the ticket until requested by the leader ticket service 222 (e.g., when another node requests the ticket).

Further, the ticket may provide either shared or exclusive access. For instance, a node 110 may request an exclusive ticket to files involved in a write operation. In contrast, a shared ticket may be issued to multiple nodes, allowing multiple read operations to occur concurrently.

The bridge components 112 may include a process-wide (or “local”) lock service 224. The process-wide lock service 224 issues a lock to file system objects touched by an operation performed by the node. That is, if a node has a process-wide lock on a set of inodes participating in a file system operation, then other processes on the node 110 are prevented from accessing that set of inodes until the operation is complete, and the local process locks are released. Like the tickets (or node-level) locks, locks issued by the process-wide lock service 224 can be shared or exclusive.

Files and directories in the file system 210 are each associated with an inode 228. As known, an inode is a data structure that stores file system attributes and metadata describing a file or directory. For example, an inode 228 for a file may include information such as a file name, file permissions, a date the file was last modified, etc. In one embodiment, the bridge components 112 may add an update intent in inodes 228 touched by a given file system operation.

An update intent provides information about an operation to be performed on the file system object corresponding to a given inode 228. Prior to initiating a file system operation, the bridge components 112 insert an update intent into the inode 228 of each file system object that will be modified during the operation. After the bridge process modifies a file system object, the bridge components 112 remove the update intent from the corresponding inodes 228. Doing so allows other nodes that access the entity to detect whether a node had failed while performing an operation. More specifically, if a node fails (or a bridge component 112 fails) while performing a file system operation that modifies multiple file system objects (e.g., an NFS rename), some of the inodes 228 participating in the operation will have an update intent related to the interrupted operation. When bridge encounters an inode with a “stale” update intent (typically as part of performing a current file system operation that touches that inode), the bridge process either completes or aborts the interrupted operation, depending on the progress of that operation when the previous node failed. Further, in one embodiment, a background process may identify inodes 228 that have an update intent from incomplete operations.

FIG. 3 illustrates an inode 300 for objects in a distributed file system, according to one embodiment. As shown, the inode 300 includes an inode number 305, file attributes 310, and an update intent 315. The inode number 305 provides a distinct identifier for each object in the file system. File attributes 310 contain details about the file, including file permissions, ownership, size, location of data blocks on disk, etc.

In one embodiment, update intent 315 stores information about a pending operation to be performed on the file system object corresponding to inode 300. A node adds an update intent 315 in an inode 300 of a file system entity that the node intends to modify. For example, assume a client sends a NFS delete operation request to delete a file “yyy” located in a directory “c”. In response, the bridge process obtains a node-level ticket and a process lock on the inodes of these entities. Once obtained, the bridge process inserts an update intent 315 in the inode 300 of the of the directory “c” and in the inode of the file “yyy”.

In one embodiment, the update intent 315 includes an update ID 316, an intended action 317, and an entity list 319. The update ID 316 provides a distinct identifier for each file system transaction that touches multiple inodes. Further, the bridge process uses the update ID 316 to distinguish one file system transaction from another.

The intended action 317 specifies an operation performed as part of a common transaction. Continuing the previous example, the intended action 317 in the update intent 315 for file “yyy” and directory “c” is a delete operation on file “yyy”.

Further, the entity list 319 includes each inode participating in the transaction identified by the update ID 316. Continuing the previous example, the entity list 319 for the delete operation may include the inode number 305 of the inodes for directory “c” and file “yyy.” In addition, the entity list 319 may specify an order in which the intended action should be performed. For example, assume A is an inode for a directory where a file is being created and B is an inode for that file. If an operation occurs by creating B and inserting an update intent in B and then storing an update intent in A. In this situation, if the node crashes prior to storing the update intent in A, nothing in the entire file system would refer to B, thus orphaning B. Therefore a transaction is ordered by inserting the update intent in A first and then into B. Inserting the update intent in A first and B next ensures that if the node crashes prior to inserting the update intent in B, no such orphaning would occur. Conversely, assume that B were being deleted from A. In such a case, the sequence would be to first store the update intent in B and then store the update intent in A. Therefore, if the node crashes before inserting the update intent in A, B would not be orphaned (because A would still refer to B).

FIG. 4 illustrates a method 400 for obtaining locks on file system objects modified during a transaction, according to one embodiment. As stated, prior to performing a file system operation, a bridge process executing on a node acquires a node-level ticket and a process-wide lock on each file system object modified by the file system operation.

As shown, method 400 begins at step 405, where a bridge process receives a request from a client to perform an operation. The request may reference a NFS file handle, such as a file handle for a directory. In turn, the bridge process is able to identify the inode of the file corresponding to the file handle. For example, assume that the node receives a request to rename a file named “yyy” to “zzz”. The request may specify file handles for a source directory (i.e., the handle of the directory containing the file “yyy”), a file handle of a destination (i.e., a handle to where the renamed file “zzz” will be located), and the file “yyy”. At step 410, the bridge process identifies file system objects modified by the operation.

At step 415, the bridge process requests, from the distributed ticket service an exclusive ticket for the file system objects identified at step 405. Continuing the previous example, the node requests tickets for exclusive access for the inode of the source directory, the inode of the destination directory, and the inode of the file being renamed. In one embodiment, the ticket services queues each request by the node for the respective entity. If the request of the node is at the front of the queue, the ticket service issues the respective ticket to the node.

At step 420, the ticket service determines whether requested tickets are currently being held by other nodes. If so, then at step 425, the slave ticket service 222 corresponding to the requesting node waits until the held tickets are released by the nodes holding the tickets. Otherwise, if the requested tickets are not currently being held by other nodes, then at step 430, the node receives the tickets from the ticket service. The bridge process pins the tickets. While a given ticket is pinned to a process, the leader ticket service cannot take away the ticket, which allows the bridge process to assume that it owns the tickets and has complete access to the inodes. In this case, the node receives tickets for exclusive access to the inodes.

At step 435, the bridge process requests a process-wide lock for each of the file system objects from the process-wide lock service executing on the node performing the requested file system operation. As stated, process-wide locks ensure that processes running on one node do not have access to the locked objects while the bridge process is performing the requested operation. For example, in this case, the node requests process-wide locks for the source directory, the destination directory, and the file “yyy”. At step 440, the bridge process performs the operation. FIGS. 5 and 6 further illustrate examples of file system operations being performed. Once the operation is complete, the bridge process unpins the tickets. The slave ticket service running on the node caches the tickets, which allows the elected leader ticket service to retrieve the tickets when requested by a remote node.

FIG. 5 illustrates an example system operation performed by a node in a distributed file system, according to one embodiment. Specifically, FIG. 5 illustrates an example of a rename operation. As shown, inodes 501 1-3 correspond to a source directory, destination directory, and the filename “yyy,” respectively. In this example, inode numbers 505 1-3 of each respectively entity are 10, 11, and 12. The source directory and destination directory correspond to different directories and therefore are associated with separate inodes. Further, the file metadata 510 1-3 each describe attributes of the entities (e.g., permissions, whether the entity is a file or directory, etc.).

Prior to performing the rename operation, the node inserts an update intent 515 1-3 corresponding to the operation to each of the inodes 501 1-3. The update ID 516 for each of the inodes is 555. The entity list 519 1-3 specifies that the entities with inode numbers 505 1-3 will be modified by the intended rename operation.

The bridge process modifies each entity according to the rename operation. After modifying a given entity or inode, the bridge process removes the update intent 515 from the corresponding inode 501. Therefore, if the bridge (or node) fails while performing this transaction, another node may be able to complete or abort the intended operation after detecting that the update intent 515 still present in any of the inodes 501 1-3.

Further, to ensure a consistent file system, the node may modify entities in a particular order. For example, in a create operation where a file is created in a particular directory, the node may update the directory prior to associating the new file with the directory. As another example, in a delete operation where a file is deleted from a particular directory, the inode for the file is deleted first.

FIG. 6 illustrates a method 600 for atomically performing file system operations in a distributed file system, according to one embodiment. As stated, a bridge process executing on a node may perform NFS operations in response to requests from clients. Of course, the distributed file system may use a variety of other file system protocols. Assume that the bridge process performing a given operation obtained a node-level ticket and a process-wide lock on each of the file system entities being modified by the operation.

Method 600 begins at step 605, where the bridge process generates an update intent for each inode participating in the transaction. Further, at step 610, the node inserts the update intent in each inode. Steps 605 and 610 act as a prepare phase for the transaction.

At step 615, the bridge process modifies the entity based on the update intent of the associated inode. After the bridge process modifies the entity, at step 620, the node removes the update intent from the inode. Steps 615 and 620 act as a commit phase for the transaction.

FIG. 7 illustrates a method 700 for committing or aborting an incomplete transaction, according to one embodiment. In some instances, the bridge process (or node) may fail while performing a transaction that modifies multiple file system objects. For example, assume that the node performing the rename operation discussed above fails after completing the update on the inode of the source directory. That is, the bridge process has not yet modified the entities corresponding to the destination directory and file “yyy”. In such a case, the inodes of the destination directory and the file “yyy” still store an update intent corresponding to the rename operation. During normal execution in the distributed secondary storage cluster, another node may access either of the inodes for the destination directory or the file, e.g., to perform a file system operation that touches these inodes. If the bridge process of that node detects that one of the inodes for a current operation and stores an update intent, then the current operation is aborted. The bridge process aborts and releases the locks on the file system objects to be modified in the operation. The bridge process may send the inodes to a “fixer” process executing on the node that either completes or aborts the prior operation depending on the state of the inodes involved in that operation.

Method 700 begins at step 705, where the node evaluates the update intent in the inode). Assume, for this discussion, that when the bridge components 112 initiate an operation and identifies a “stale” update intent in an inode participating in that transaction, the bridge component 112 sends the metadata to the fixer process. The fixer process determines the update ID, the operation that was to be performed, and a sequence of inodes included in the operation. Continuing the previous example, assume that the current entity corresponds to the entity with inode 501 2 with inode number 11 from FIG. 5. The fixer process identifies that the other file system objects have inode numbers of 10 and 12.

At step 710, the fixer process accesses the other entities associated with the update. The fixer process requests a node-level ticket from the ticket service for each entity in the list of entities in the node with a “stale” update intent. After acquiring the tickets, the fixer process also requests process-wide locks for the same entities.

At step 715, the fixer process determines which inodes do not have an update intent relative to the inode with the stale update operation. In this case, the entity with inode number 10 (i.e., the source directory file handler) does not have an update intent. At step 720, the fixer process determines whether the entities without an update intent come before or after the inode with the stale update intent. The absence of an update intent in a preceding inode indicates that the previous node had already finalized the update to the entity referenced by that inode. In contrast, the absence of an update intent in a succeeding entity implies that the node had failed while preparing for the operation.

If the inode or inodes without an update intent precede the current entity in sequence, then at step 725, the node completes the operation specified in the update intent. That is, because the previous node had already completed modifying the entities, the fixer process should roll forward with the rest of the operation specified in the update metadata. However, if the inodes without an update intent come after the current entity in sequence, then at step 730, the fixer process aborts the intended operation by removing the update intent from the inodes following the inode with the “stale” update intent. Continuing the previous example, the inode for the source directory no longer has an update intent. Because the source directory precedes the current entity (i.e., the destination directory) in sequence, the fixer process completes the rename operation.

FIG. 8 illustrates an example secondary storage server 800 configured to allocate VIP addresses among other secondary storage servers in a distributed system, according to one embodiment. As shown, computing system 800 includes, without limitation, a central processing unit (CPU) 805, a network interface 815, a memory 820, and storage 830, each connected to a bus 817. Server 800 may also include an I/O device interface 810 connecting I/O devices 812 (e.g., keyboard, display and mouse devices) to the computing system 800. Further, in context of this disclosure, the computing elements shown in computing system 800 may correspond to a physical computing system (e.g., a system in a data center) or may be a virtual computing instance executing within a computing cloud.

CPU 805 retrieves and executes programming instructions stored in memory 820 as well as stores and retrieves application data residing in the storage 830. The bus 817 is used to transmit programming instructions and application data between CPU 805, I/O devices interface 810, storage 830, network interface 815, and memory 820. Note, CPU 805 is included to be representative of a single CPU, multiple CPUs, a single CPU having multiple processing cores, and the like. Memory 820 is generally included to be representative of a random access memory. Storage 830 may be a disk drive storage device. Although shown as a single unit, storage 830 may be a combination of fixed and/or removable storage devices, such as fixed disc drives, removable memory cards, or optical storage, network attached storage (NAS), or a storage area-network (SAN).

Illustratively, memory 820 includes bridge components 824, and data gathering components 826. The bridge components 824 provide a file system architecture for the distributed secondary storage system. The bridge components 824 may perform operations on the file system in response to NFS requests sent by clients. The data gathering components 826 include modules that allow the secondary storage server 800 to communicate with primary storage servers and retrieve copies of data from the primary storage servers.

The memory 820 further includes, without limitation, a file system 825, a ticket service 826, and a process-wide lock service 827. The file system 825 collaborates with the file systems of other secondary storage servers in the cluster to form a distributed system used to store backup data from the primary storage services. The bridge components 824 expose the file system 825 to clients through file system protocols, such as NFS, SMB, CIFS, and iSCSI. The ticket service 826 is an in-memory distributed lock manager that issues tickets to requesting nodes. The tickets provide a node-level lock to entities of the file system 825. The process-wide lock service 827 issues process-wide locks for an entity that is specific to an operation performed by the secondary storage system 800. That is, process wide locks are local in scope to the secondary storage system 800. Both the tickets and the process-wide locks may grant either shared or exclusive access to a particular entity.

The storage 830 includes a node identifier 832. The node identifier 220 is a distinct value that may be based on the MAC address of the secondary storage server 110. The inodes 834 are data structures for the file and directory entities that store metadata describing file system attributes for each entity. Further, when performing a given operation on an entity, an inode 834 may store an update intent describing the update to be performed.

Aspects of the present disclosure may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples a computer readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the current context, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus or device.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. Each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the disclosure and its practical applications, to thereby enable others skilled in the art to best utilize the disclosure and various embodiments with various modifications as may be suited to the particular use contemplated.

While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.

Claims (15)

What is claimed is:
1. A method for performing a transaction in a distributed file system, the method comprising:
in response to a request to perform a file system operation, identifying a first set of file system objects modified in performing the requested file system operation, wherein each file system object has an associated inode;
upon determining that a first one of the inodes in the first set of file system objects stores an update intent from an incomplete file system operation:
identifying, in the update intent from the incomplete file system operation, a first ordered sequence of inodes, the first ordered sequence associated with the incomplete file system operation,
retrieving, as available, the inodes specified by the first ordered sequence, and
upon determining (i) that a first set of one or more inodes at the beginning of the first ordered sequence includes the update intent related to the incomplete file system operation and (ii) that a second set of one or more inodes following the first set of indoes does not include the update intent related to the incomplete file system operation, removing the update intent from each inode specified by the first ordered sequence that includes the update intent;
inserting an update intent corresponding to the requested file system operation into the inode associated with each identified file system object, wherein the inserted update intent specifies a second ordered sequence of inodes, the second ordered sequence being associated with the requested file system operation; and
for each inode in the second ordered sequence:
modifying either the inode or the file system object corresponding to the inode as specified by the update intent in that inode, and
after modifying the inode or the file system object corresponding to the inode, removing the update intent from that inode.
2. The method of claim 1, wherein completing the incomplete file system operation comprises:
upon determining that (i) the first set of one or more inodes at the beginning of the first ordered sequence does not include the update intent related to the incomplete file system operation and that (ii) the second set of one or more inodes following the first set of inodes each includes the update intent related to the incomplete file system operation, completing the file system as specified by the update intent in the second set of nodes according to the first ordered sequence.
3. The method of claim 2, further comprising, removing the update intent from each of the second set of inodes.
4. The method of claim 1, wherein the update intent specifies an identifier corresponding to the requested file system operation, the operation to be performed, and the second ordered sequence of the inodes modified in performing the requested operation.
5. The method of claim 1, further comprising, prior to inserting the update intent corresponding to the requested operation, obtaining a node-specific lock and a process-specific lock on each inode to be modified.
6. The method of claim 1, wherein the distributed file system is exposed to clients as a Network File System (NFS), Server Message Block (SMB), or Common Internet File System (CIFS) mount point, and wherein the requested file system operation is an NFS, SMB, or CIFS operation from a client sent to a node of the distributed file system.
7. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform an operation for performing a transaction in a distributed file system, the operation comprising:
in response to a request to perform a file system operation, identifying a first set of file system objects modified in performing the requested file system operation, wherein each file system object has an associated inode;
upon determining that a first one of the inodes in the first set of file system objects stores an update intent from an incomplete file system operation:
identifying, in the update intent from the incomplete file system operation, a first ordered sequence of inodes, the first ordered sequence associated with the incomplete file system operation,
retrieving, as available, the inodes specified by the first ordered sequence, and
upon determining (i) that a first set of one or more inodes at the beginning of the first ordered sequence includes the update intent related to the incomplete file system operation and (ii) that a second set of one or more inodes following the first set of inodes does not include the update intent related to the incomplete file system operation, removing the update intent from each inode specified by the first ordered sequence that includes the update intent;
inserting an update intent corresponding to the requested file system operation into the inode associated with each identified file system object, wherein the inserted update intent specifies a second an ordered sequence of inodes, the second ordered sequence being associated with the requested file system operation; and
for each inode in the second ordered sequence:
modifying either the inode or the file system object corresponding to the inode as specified by the update intent in that inode, and
after modifying the inode or the file system object corresponding to the inode, removing the update intent from that inode.
8. The computer-readable medium of claim 7, wherein the update intent specifies an identifier corresponding to the requested file system operation, the operation to be performed, and the ordered sequence of the inodes modified in performing the requested operation.
9. The computer-readable medium of claim 7, wherein the operation further comprises, prior to inserting the update intent corresponding to the requested operation, obtaining a node-specific lock and a process-specific lock on each inode to be modified.
10. The computer-readable medium of claim 7, wherein the distributed file system is exposed to clients as a Network File System (NFS), Server Message Block (SMB), or Common Internet File System (CIFS) mount point, and wherein the requested file system operation is an NFS, SMB, CIFS operation from a client sent to a node of the distributed file system.
11. A system comprising:
a processor; and
a memory storing program code, which, when executed on the processor performs an operation for performing a transaction in a distributed file system, the operation comprising:
in response to a request to perform a file system operation, identifying a first set of file system objects modified in performing the requested file system operation, wherein each file system object has an associated inode;
upon determining that a first one of the inodes in the first set of file system objects stores an update intent from an incomplete file system operation:
identifying, in the update intent from the incomplete file system operation, a first ordered sequence of inodes, the first ordered sequence associated with the incomplete file system operation;
retrieving, as available, the inodes specified by the first ordered sequence; and
upon determining (i) that a first set of one or more inodes at the beginning of the first ordered sequence includes the update intent related to the incomplete file system operation and (ii) that a second set of one or more inodes following the first set of inodes does not include the update intent related to the incomplete file system operation, removing the update intent from each inode specified by the first ordered sequence that includes the update intent,
inserting an update intent corresponding to the requested file system operation into the inode associated with each identified file system object, wherein the inserted update intent specifies a second ordered sequence of inodes, the second ordered sequence being associated with the requested file system operation, and
for each inode:
modifying either the inode or the file system object corresponding to the inode as specified by the update intent in that inode, and
after modifying the inode or the file system object corresponding to the inode, removing the update intent from that inode.
12. The system of claim 11, wherein completing the incomplete file system operation comprises:
upon determining that (i) the first set of one or more inodes at the beginning of the first ordered sequence does not include the update intent related to the incomplete file system operation and that (ii) the second set of one or more inodes following the first set of inodes each includes the update intent related to the incomplete file system operation, completing the file system as specified by the update intent in the second set of nodes according to the first ordered sequence.
13. The system of claim 12, wherein the operation further comprises, removing the update intent from each of the second set of inodes.
14. The system of claim 11, wherein the operation further comprises, prior to inserting the update intent corresponding to the requested operation, obtaining a node-specific lock and a process-specific lock on each inode to be modified.
15. The system of claim 11, wherein the distributed file system is exposed to clients as a Network File System (NFS), Server Message Block (SMB), or Common Internet File System (CIFS) mount point, and wherein the requested file system operation is an NFS, SMB, or CIFS operation from a client sent to a node of the distributed file system.
US14/524,979 2014-10-27 2014-10-27 Concurrent access and transactions in a distributed file system Active 2035-07-23 US9697227B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/524,979 US9697227B2 (en) 2014-10-27 2014-10-27 Concurrent access and transactions in a distributed file system

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US14/524,979 US9697227B2 (en) 2014-10-27 2014-10-27 Concurrent access and transactions in a distributed file system
EP15854376.9A EP3295330A4 (en) 2014-10-27 2015-10-23 Concurrent access and transactions in a distributed file system
PCT/US2015/057230 WO2016069413A1 (en) 2014-10-27 2015-10-23 Concurrent access and transactions in a distributed file system
US15/628,484 US10275469B2 (en) 2014-10-27 2017-06-20 Concurrent access and transactions in a distributed file system

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/628,484 Continuation US10275469B2 (en) 2014-10-27 2017-06-20 Concurrent access and transactions in a distributed file system

Publications (2)

Publication Number Publication Date
US20160117336A1 US20160117336A1 (en) 2016-04-28
US9697227B2 true US9697227B2 (en) 2017-07-04

Family

ID=55792144

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/524,979 Active 2035-07-23 US9697227B2 (en) 2014-10-27 2014-10-27 Concurrent access and transactions in a distributed file system
US15/628,484 Active US10275469B2 (en) 2014-10-27 2017-06-20 Concurrent access and transactions in a distributed file system

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/628,484 Active US10275469B2 (en) 2014-10-27 2017-06-20 Concurrent access and transactions in a distributed file system

Country Status (3)

Country Link
US (2) US9697227B2 (en)
EP (1) EP3295330A4 (en)
WO (1) WO2016069413A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10338917B2 (en) * 2015-08-14 2019-07-02 Alibaba Group Holding Limited Method, apparatus, and system for reading and writing files

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10108624B1 (en) * 2015-02-04 2018-10-23 Amazon Technologies, Inc. Concurrent directory move operations using ranking rules
US10339003B2 (en) 2017-06-01 2019-07-02 International Business Machines Corporation Processing data access transactions in a dispersed storage network using source revision indicators
US20190205554A1 (en) 2017-12-28 2019-07-04 Dropbox, Inc. File system authentication

Citations (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5454108A (en) * 1994-01-26 1995-09-26 International Business Machines Corporation Distributed lock manager using a passive, state-full control-server
US5537645A (en) 1989-05-15 1996-07-16 International Business Machines Corporation File lock management in a distributed data processing system
US5940841A (en) * 1997-07-11 1999-08-17 International Business Machines Corporation Parallel file system with extended file attributes
US6058400A (en) * 1998-04-28 2000-05-02 Sun Microsystems, Inc. Highly available cluster coherent filesystem
US6144999A (en) * 1998-05-29 2000-11-07 Sun Microsystems, Incorporated Method and apparatus for file system disaster recovery
US6163856A (en) * 1998-05-29 2000-12-19 Sun Microsystems, Inc. Method and apparatus for file system disaster recovery
US20020188697A1 (en) * 2001-06-08 2002-12-12 O'connor Michael A. A method of allocating storage in a storage area network
US20020196744A1 (en) * 2001-06-26 2002-12-26 O'connor Michael A. Path discovery and mapping in a storage area network
US20030158834A1 (en) * 2002-02-15 2003-08-21 International Business Machines Corporation Writable file system snapshot with ditto address feature
US20040133573A1 (en) * 2001-01-11 2004-07-08 Z-Force Communications, Inc. Aggregated lock management for locking aggregated files in a switched file system
US6772155B1 (en) * 2001-04-04 2004-08-03 Ncr Corporation Looking data in a database system
US20050010620A1 (en) * 2003-05-30 2005-01-13 Veritas Software Corporation Multi-volume file support
US20050065986A1 (en) * 2003-09-23 2005-03-24 Peter Bixby Maintenance of a file version set including read-only and read-write snapshot copies of a production file
US20060136509A1 (en) * 2004-12-16 2006-06-22 Syam Pannala Techniques for transaction semantics for a database server performing file operations
US20070276878A1 (en) 2006-04-28 2007-11-29 Ling Zheng System and method for providing continuous data protection
US20080091680A1 (en) 2001-07-17 2008-04-17 International Business Machines Corporation Distributed locking protocol with asynchronous token prefetch and relinquish
US7487228B1 (en) * 2003-01-30 2009-02-03 Red Hat, Inc. Metadata structures and related locking techniques to improve performance and scalability in a cluster file system
US20100114849A1 (en) * 2008-10-30 2010-05-06 Kingsbury Brent A Allocating Priorities to Prevent Deadlocks in a Storage System
US7730258B1 (en) * 2005-11-01 2010-06-01 Netapp, Inc. System and method for managing hard and soft lock state information in a distributed storage system environment
WO2011023134A1 (en) 2009-08-28 2011-03-03 Beijing Innovation Works Technology Company Limited Method and system for managing distributed storage system through virtual file system
US20120036161A1 (en) * 2002-11-01 2012-02-09 Bluearc Uk Limited Distributed File System and Method
US20120101991A1 (en) * 2010-06-19 2012-04-26 Srivas Mandayam C Map-Reduce Ready Distributed File System
US8255590B2 (en) 2008-07-03 2012-08-28 Panasonic Corporation Image file transfer apparatus that detect whether transferred image files to an external device has been aborted or not
US20120259820A1 (en) * 2011-04-08 2012-10-11 Symantec Corporation Method and system for restarting file lock services at an adoptive node during a network filesystem server migration or failover
US20130097136A1 (en) * 2011-10-17 2013-04-18 Pie Digital, Inc. Method and system for acessing domain specific in-memory database management system
US20130179480A1 (en) * 2012-01-05 2013-07-11 Stec, Inc. System and method for operating a clustered file system using a standalone operation log
US8620640B2 (en) 2003-08-05 2013-12-31 Sepaton, Inc. Emulated storage system
US20140025877A1 (en) * 2010-12-13 2014-01-23 Fusion-Io, Inc. Auto-commit memory metadata
US20140082145A1 (en) * 2012-09-14 2014-03-20 Peaxy, Inc. Software-Defined Network Attachable Storage System and Method
US20140173426A1 (en) 2012-12-19 2014-06-19 Microsoft Corporation Seamlessly incorporating online content into documents
US8863297B2 (en) 2012-01-06 2014-10-14 Mobile Iron, Inc. Secure virtual file management system
US8972345B1 (en) * 2006-09-27 2015-03-03 Hewlett-Packard Development Company, L.P. Modifying data structures in distributed file systems
US20150278397A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Namespace management in distributed storage systems
US20150278243A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Scalable file storage service
US20150277969A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Atomic writes for multiple-extent operations
US20150280959A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Session management in distributed storage systems
US20150356116A1 (en) * 2014-06-06 2015-12-10 Panzura, Inc. Distributed change notifications for a distributed filesystem
US20150356110A1 (en) * 2014-06-06 2015-12-10 Panzura, Inc. Managing opportunistic locks in a distributed filesystem
US20150379009A1 (en) * 2014-06-25 2015-12-31 Cohesity, Inc. Distributed key-value store
US9274710B1 (en) * 2014-03-31 2016-03-01 Amazon Technologies, Inc. Offset-based congestion control in storage systems

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5218695A (en) * 1990-02-05 1993-06-08 Epoch Systems, Inc. File server system having high-speed write execution
US5613105A (en) * 1993-06-30 1997-03-18 Microsoft Corporation Efficient storage of objects in a file system
US6973455B1 (en) * 1999-03-03 2005-12-06 Emc Corporation File server system providing direct data sharing between clients with a server acting as an arbiter and coordinator
US6654772B1 (en) * 1999-04-28 2003-11-25 Emc Corporation Multi-volume extent based file system
US6895413B2 (en) * 2002-03-22 2005-05-17 Network Appliance, Inc. System and method for performing an on-line check of a file system
US8195624B2 (en) * 2005-06-30 2012-06-05 Phoenix Technologies Ltd. Shared file system management between independent operating systems
US8364638B2 (en) * 2005-09-15 2013-01-29 Ca, Inc. Automated filer technique for use in virtualized appliances and applications
US7752173B1 (en) * 2005-12-16 2010-07-06 Network Appliance, Inc. Method and apparatus for improving data processing system performance by reducing wasted disk writes
WO2009134772A2 (en) * 2008-04-29 2009-11-05 Maxiscale, Inc Peer-to-peer redundant file server system and methods
US8266114B2 (en) * 2008-09-22 2012-09-11 Riverbed Technology, Inc. Log structured content addressable deduplicating storage
US20100241673A1 (en) * 2009-03-23 2010-09-23 Riverbed Technology, Inc. Virtualized data storage system architecture
US8655848B1 (en) * 2009-04-30 2014-02-18 Netapp, Inc. Unordered idempotent logical replication operations
US8671072B1 (en) * 2009-09-14 2014-03-11 Netapp, Inc. System and method for hijacking inodes based on replication operations received in an arbitrary order
US8868748B2 (en) * 2010-10-11 2014-10-21 International Business Machines Corporation Two-level management of locks on shared resources
EP2840481B1 (en) * 2013-07-02 2017-09-06 Hitachi Data Systems Engineering UK Limited Apparatus and computer program product for virtualization of a file system

Patent Citations (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5537645A (en) 1989-05-15 1996-07-16 International Business Machines Corporation File lock management in a distributed data processing system
US5454108A (en) * 1994-01-26 1995-09-26 International Business Machines Corporation Distributed lock manager using a passive, state-full control-server
US5940841A (en) * 1997-07-11 1999-08-17 International Business Machines Corporation Parallel file system with extended file attributes
US6058400A (en) * 1998-04-28 2000-05-02 Sun Microsystems, Inc. Highly available cluster coherent filesystem
US6144999A (en) * 1998-05-29 2000-11-07 Sun Microsystems, Incorporated Method and apparatus for file system disaster recovery
US6163856A (en) * 1998-05-29 2000-12-19 Sun Microsystems, Inc. Method and apparatus for file system disaster recovery
US20040133573A1 (en) * 2001-01-11 2004-07-08 Z-Force Communications, Inc. Aggregated lock management for locking aggregated files in a switched file system
US6772155B1 (en) * 2001-04-04 2004-08-03 Ncr Corporation Looking data in a database system
US20020188697A1 (en) * 2001-06-08 2002-12-12 O'connor Michael A. A method of allocating storage in a storage area network
US20020196744A1 (en) * 2001-06-26 2002-12-26 O'connor Michael A. Path discovery and mapping in a storage area network
US20080091680A1 (en) 2001-07-17 2008-04-17 International Business Machines Corporation Distributed locking protocol with asynchronous token prefetch and relinquish
US20030158834A1 (en) * 2002-02-15 2003-08-21 International Business Machines Corporation Writable file system snapshot with ditto address feature
US20120036161A1 (en) * 2002-11-01 2012-02-09 Bluearc Uk Limited Distributed File System and Method
US7487228B1 (en) * 2003-01-30 2009-02-03 Red Hat, Inc. Metadata structures and related locking techniques to improve performance and scalability in a cluster file system
US20050010620A1 (en) * 2003-05-30 2005-01-13 Veritas Software Corporation Multi-volume file support
US8620640B2 (en) 2003-08-05 2013-12-31 Sepaton, Inc. Emulated storage system
US20050065986A1 (en) * 2003-09-23 2005-03-24 Peter Bixby Maintenance of a file version set including read-only and read-write snapshot copies of a production file
US20060136509A1 (en) * 2004-12-16 2006-06-22 Syam Pannala Techniques for transaction semantics for a database server performing file operations
US7730258B1 (en) * 2005-11-01 2010-06-01 Netapp, Inc. System and method for managing hard and soft lock state information in a distributed storage system environment
US20070276878A1 (en) 2006-04-28 2007-11-29 Ling Zheng System and method for providing continuous data protection
US8972345B1 (en) * 2006-09-27 2015-03-03 Hewlett-Packard Development Company, L.P. Modifying data structures in distributed file systems
US8255590B2 (en) 2008-07-03 2012-08-28 Panasonic Corporation Image file transfer apparatus that detect whether transferred image files to an external device has been aborted or not
US20100114849A1 (en) * 2008-10-30 2010-05-06 Kingsbury Brent A Allocating Priorities to Prevent Deadlocks in a Storage System
WO2011023134A1 (en) 2009-08-28 2011-03-03 Beijing Innovation Works Technology Company Limited Method and system for managing distributed storage system through virtual file system
US20120101991A1 (en) * 2010-06-19 2012-04-26 Srivas Mandayam C Map-Reduce Ready Distributed File System
US20160078052A1 (en) * 2010-06-19 2016-03-17 Mapr Technologies, Inc. Map-Reduce Ready Distributed File System
US20140025877A1 (en) * 2010-12-13 2014-01-23 Fusion-Io, Inc. Auto-commit memory metadata
US20120259820A1 (en) * 2011-04-08 2012-10-11 Symantec Corporation Method and system for restarting file lock services at an adoptive node during a network filesystem server migration or failover
US20130097136A1 (en) * 2011-10-17 2013-04-18 Pie Digital, Inc. Method and system for acessing domain specific in-memory database management system
US20130179480A1 (en) * 2012-01-05 2013-07-11 Stec, Inc. System and method for operating a clustered file system using a standalone operation log
US8863297B2 (en) 2012-01-06 2014-10-14 Mobile Iron, Inc. Secure virtual file management system
US20140082145A1 (en) * 2012-09-14 2014-03-20 Peaxy, Inc. Software-Defined Network Attachable Storage System and Method
US20140173426A1 (en) 2012-12-19 2014-06-19 Microsoft Corporation Seamlessly incorporating online content into documents
US20150280959A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Session management in distributed storage systems
US20150278397A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Namespace management in distributed storage systems
US20150278243A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Scalable file storage service
US9274710B1 (en) * 2014-03-31 2016-03-01 Amazon Technologies, Inc. Offset-based congestion control in storage systems
US20150277969A1 (en) * 2014-03-31 2015-10-01 Amazon Technologies, Inc. Atomic writes for multiple-extent operations
US20150356116A1 (en) * 2014-06-06 2015-12-10 Panzura, Inc. Distributed change notifications for a distributed filesystem
US20150356110A1 (en) * 2014-06-06 2015-12-10 Panzura, Inc. Managing opportunistic locks in a distributed filesystem
US20150379009A1 (en) * 2014-06-25 2015-12-31 Cohesity, Inc. Distributed key-value store

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Christopher K. Hess Roy H. Campbell-"An application of a context-aware file system"-Published in: Journal Personal and Ubiquitous Computing archive-vol. 7 Issue 6, Dec. 2003-pp. 339-352-Springer-Verlag London, UK. *
Christopher K. Hess Roy H. Campbell—"An application of a context-aware file system"—Published in: Journal Personal and Ubiquitous Computing archive—vol. 7 Issue 6, Dec. 2003—pp. 339-352—Springer-Verlag London, UK. *
International Search Report and Written Opinion for corresponding International Application No. PCT/US15/57230; mailed Feb. 10, 2016 (18 pages).
International Search Report and Written Opinion for corresponding International Application No. PCT/US15/57320; mailed Mar. 7, 2016 (16 pages).
Jaegeuk Kim, Heeseung Jo, Hyotaek Shim, Jin-Soo Kim, and Seungryoul Maeng-"Efficient Metadata Management for Flash File Systems" Published in: Object Oriented Real-Time Distributed Computing (ISORC), 2008 11th IEEE International Symposium on-May 5-7, 2008-pp. 535-540. *
Jaegeuk Kim, Heeseung Jo, Hyotaek Shim, Jin-Soo Kim, and Seungryoul Maeng—"Efficient Metadata Management for Flash File Systems" Published in: Object Oriented Real-Time Distributed Computing (ISORC), 2008 11th IEEE International Symposium on—May 5-7, 2008—pp. 535-540. *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10338917B2 (en) * 2015-08-14 2019-07-02 Alibaba Group Holding Limited Method, apparatus, and system for reading and writing files

Also Published As

Publication number Publication date
EP3295330A1 (en) 2018-03-21
EP3295330A4 (en) 2018-11-07
US20160117336A1 (en) 2016-04-28
WO2016069413A1 (en) 2016-05-06
US10275469B2 (en) 2019-04-30
US20170351701A1 (en) 2017-12-07

Similar Documents

Publication Publication Date Title
US8930947B1 (en) System and method for live migration of a virtual machine with dedicated cache
US8694469B2 (en) Cloud synthetic backups
US8898388B1 (en) NVRAM caching and logging in a storage system
JP5671615B2 (en) MapReduce immediately distributed file system
US7299378B2 (en) Geographically distributed clusters
US9235524B1 (en) System and method for improving cache performance
US7472129B2 (en) Lossless recovery for computer systems with map assisted state transfer
US8997097B1 (en) System for implementing a virtual disk in a virtualization environment
US8850130B1 (en) Metadata for managing I/O and storage for a virtualization
US8627012B1 (en) System and method for improving cache performance
US8898112B1 (en) Write signature command
US8677055B2 (en) Flexible way of specifying storage attributes in a flash memory-based object store
US9639299B2 (en) Managing virtual hard drives as blobs
US9298392B2 (en) System and method for full virtual machine backup using storage system functionality
EP3215940B1 (en) Data management system
US7877357B1 (en) Providing a simulated dynamic image of a file system
CN103827865B (en) Use asynchronous log-based replication to improve the database cache
KR101833114B1 (en) Fast crash recovery for distributed database systems
US9104331B2 (en) System and method for incremental virtual machine backup using storage system functionality
US6850969B2 (en) Lock-free file system
US8868487B2 (en) Event processing in a flash memory-based object store
KR20150021117A (en) Enhanced data management virtualization system
US8972405B1 (en) Storage resource management information modeling in a cloud processing environment
CA2921039A1 (en) System and method for virtual machine conversion
US8661068B1 (en) Managing global metadata caches in data storage systems

Legal Events

Date Code Title Description
AS Assignment

Owner name: COHESITY, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ARON, MOHIT;SHANMUGANATHAN, GANESHA;REEL/FRAME:034044/0518

Effective date: 20141027

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY