EP2100235A2

EP2100235A2 - Method and apparatus to automatically commit files to worm status

Info

Publication number: EP2100235A2
Application number: EP06733911A
Authority: EP
Inventors: Jeffrey Merrick; William Mcgovern
Original assignee: Network Appliance Inc
Current assignee: NetApp Inc
Priority date: 2006-01-25
Filing date: 2006-01-25
Publication date: 2009-09-16
Also published as: WO2007086844A3; WO2007086844A2

Abstract

A system is provided to commit data to persistent storage. The system comprises a configuration component to set an autocommit period, a scanner to detect that the autocommit period has expired for a file and a commit component to commit the file to write once read many (WORM) status In one embodiment, when an autocommit period is set, the system may be configured to automatically commit a file to WORM status if the file has not been changed for the autocommit period.

Description

METHOD AND APPARATUS TO AUTOMATICALLY COMMIT FILES TO

WORM STATUS

FIELD OF THE INVENTION

[0001] At least one embodiment of the present invention pertains to networked storage systems and, more particularly, to a method and apparatus to automatically commit files to WORM status. BACKGROUND

[0002] Many businesses rely on some form of WORM (write once, read many) data storage to store critical data. In many industries, such as the financial services and healthcare industries, businesses are required by strict records-retention regulations to archive important data, such as emails, transaction information, patient records, audit information, as well as other types of documents and data. Such records-retention regulations include, for example, Securities Exchange Commission (SEC) Rule 17a-4 (17 C.F.R. § 240.17a-4(f)), which regulates broker-dealers; Health Insurance Portability and Accountability Act (HIPAA), which regulates companies in the healthcare industry; Sarbanes-Oxley (SOX), which regulates publicly traded companies; 21 C.F.R. Part 11 , which regulates certain companies in the life sciences industry; and, DOD 5015.2-STD, which regulates certain government organizations; etc. Regulatory agencies generally recognize the ability of WORM data storage to ensure the permanence of archived data and therefore often require that only nonerasable, non-rewritable WORM storage be used for meeting their regulations. Software applications that manipulate data that may be subject to records-retention regulations are sometimes referred to as compliance applications.

[0003] Some existing systems (e.g., networked storage systems) provide a conventional (hierarchical) file system interface in a storage server to allow users to access stored data and to designate at least a subset of the data as WORM data through the file system interface. A networked storage system may include one or more storage servers, which may be storage appliances. A storage server may provide services related to the organization of data on mass storage devices, such as disks. Some of these storage servers are commonly referred to as filers or file servers. An example of such a storage server is any of the Filer products made by Network Appliance, Inc. in Sunnyvale, California. The storage appliance may be implemented with a special-purpose computer or a general-purpose computer. Depending on the application, various networked storage systems may include different numbers of storage servers. Various applications, including compliance applications, may be permitted to create and modify data on a storage appliance.

[0004] Some compliance applications do not have a built-in capability to assign a retention time to files or to commit files to WORM status. The files therefore may need to be committed to WORM status manually (e.g., by an administrator copying the files to a WORM storage device). On the other hand, some compliance applications may not have a capability to notify a storage system of when the application has completed the modifying operations on a file so that the file may be treated as closed. Furthermore, when a compliance application communicates with a storage server via open communications protocols, such as NFS or CIFS, the network traffic may not be indicative of the status of a file with respect to the status of the file as being open or closed for further modifications. For example, NFS does not have a mechanism to indicate when a file is closed. CIFS does have a mechanism of indicating that a file has been closed, but there are many applications that will close a file and then reopen it for writing again. SUMMARY

[0005] A system and method are provided to commit files to WORM status. The system comprises a configuration component to set an autocommit period; a scanner to detect that the autocommit period has expired for a file; and a commit component to commit the file to write once read many (WORM) status.

[0006] Other aspects of the invention will be apparent from the accompanying figures and from the detailed description that follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] One or more embodiments of the present invention are illustrated by way of example and not limited to the figures of the accompanying drawings, in which like references indicate similar elements and in which:

[0008] Figure 1 is a schematic block diagram of an environment including a storage system that may be used with one embodiment of the present invention;

[0009] Figure 2 is a schematic block diagram of a storage server that may be used with one embodiment of the present invention;

[0010] Figure 3 is a schematic block diagram of a storage operating system that may be used with one embodiment of the present invention;

[0011] Figure 4 is a schematic block diagram of a WORM component, according to one embodiment of the present invention; and

[0012] Figure 5 is a flow chart illustrating a method to automatically commit files to WORM status, according to embodiments of the invention.

DETAILED DESCRIPTION

[0013] As described in detail below, the technique introduced herein addresses issues associated with committing files to WORM status even where an application that modified the file does not provide information of whether the file will not be modified any further. It will be noted that, for the purposes of this description, a file may be considered to have a WORM status if the file cannot be deleted or modified until a predetermined end of retention period. In one embodiment an administrator may be permitted to configure a storage system to automatically commit files to WORM status after the files have not been modified for a predetermined period of time. This predetermined period of time may be referred to as an autocommit period. The autocommit period may be dependent upon a particular application, as for some applications the files may need to remain opened for writing longer than for other applications. For example, the autocommit period may be applicable to files storing financial records. [0014] As described in further detail below, in one embodiment of the present invention, a user (e.g., an administrator) may designate a volume on a storage server as a WORM volume and set an autocommit period associated with that volume (e.g., at least two hours for some systems). When an autocommit period is set, the storage server may automatically commit a file to WORM status if the file has not been changed for the autocommit period. In one embodiment, a file that has been committed to WORM status remains unmodifiable for a predetermined retention period, which may be a default value. In one embodiment, the storage server may send out an Enhanced Messaging Service (EMS) message to a designated recipient (e.g., an administrator) every time the system automatically commits a file to WORM status.

[0015] In one embodiment, the present invention may be implemented in the context of a storage-oriented network, e.g., a network that includes one or more storage servers that store and retrieve data on behalf of one or more clients. Such a network may be used, for example, to provide multiple users with access to shared data or to backup mission critical data. An example of such a network is illustrated in Figure 1.

[0016] Figure 1 is a schematic block diagram of an environment 100 including a storage system 110 that may be advantageously used with one embodiment of the present invention. The storage system 110, in one embodiment, may be configured to access information requested by clients such as a client 140 via a network 150. For example, the storage system 110 may store files created or modified by an application 142 running on the client 140. The application 142 may be a compliance application. [0017] The storage system 110 may store data on any type of attached array of writable storage device media such as video tape, optical, DVD, magnetic tape, bubble memory, electronic random access memory, micro- electro mechanical and any other similar media adapted to store information, including data and parity information. However, as illustratively described herein, the data is preferably stored on disks 120, such as hard disk drives (HDD) and/or direct access storage devices (DASD), of an array 130.

[0018] In one embodiment, storage of information on array 130 may be implemented as one or more storage "volumes," such as a volume 132 and a volume 134, that comprise a collection of physical storage disks 120 cooperating to define an overall logical arrangement of virtual block number (vbn) space on the volumes. Each logical volume is generally, although not necessarily, associated with its own file system. The disks within a logical volume are typically organized as one or more groups, wherein each group may be operated as a redundant array of independent disks (RAID). Most RAID implementations, such as a RAID-4 level implementation, enhance the reliability/integrity of data storage through the redundant writing of data "stripes" across a given number of physical disks in the RAID group, and through the appropriate storing of parity information with respect to the striped data. An illustrative example of a RAID implementation is a RAID-4 level implementation, although it will be understood that other types and levels of RAID implementations may be used in accordance with the inventive principles described herein. A volume may be designated, e.g., at the time of volume creation, as a WORM volume, such that at least some files on a WORM volume may be committed to a WORM status to remain unmodifiable for a predetermined retention period. [0019] As noted above, the storage system 110 may service client requests over the computer network 150. The computer network 150 may comprise a point-to-point connection or a shared medium, such as a local area network. Illustratively, the computer network 150 may be embodied as an Ethernet network or a Fibre Channel (FC) network. The client 140 may communicate with the storage system over network 150 by exchanging discrete frames or packets of data according to pre-defined protocols, such as the Transmission Control Protocol/Internet Protocol (TCP/IP). [0020] The client 140 may be a general-purpose computer configured to execute applications 142. Moreover, the client 140 may interact with the storage system 110 in accordance with a client/server model of information delivery. That is, the client may request the services of the storage system, and the system may return the results of the services requested by the client by exchanging packets over the network 150. The clients may issue packets including file-based access protocols, such as the Common Internet File System (CIFS) protocol or Network File System (NFS) protocol, over TCP/IP when accessing information in the form of files and directories. Alternatively, the client may issue packets including block-based access protocols, such as the Small Computer Systems Interface (SCSI) protocol encapsulated over TCP (iSCSI) and SCSI encapsulated over Fibre Channel (FCP), when accessing information in the form of blocks. [0021] It will be noted that any or all of the components of system 100 and associated hardware may be used in various embodiments of the present invention. However, other configurations of the networked data storage system may include more or fewer devices discussed above. [0022] Additional details of one embodiment of a storage server are illustrated in Figure 2. A storage system 200 comprises a processor 222, a memory 224, a network adaptor 226, and a storage adaptor 228, interconnected by a system bus 250.

[0023] In the illustrative embodiment, the memory 224 comprises storage locations that are addressable by the processor and adaptors for storing software program code. A storage operating system 300, portions of which are typically resident in memory and executed by the processing elements, functionally organizes the system 200 by, inter alia, invoking storage operations executed by the storage system. It will be apparent to those skilled in the art that other processing and memory means, including various computer readable media, may be used for storing and executing program instructions pertaining to the inventive technique described herein. [0024] The network adaptor 226 comprises the mechanical, electrical and signaling circuitry needed to connect the storage system 200 to clients (e.g., the clients 140 of Figure 10) over a computer network. The storage adaptor 228 cooperates with the storage operating system 300 executing on the system 200 to access information requested by a user (or client). [0025] Figure 3 illustrates the operating system 300 in greater details according to one embodiment of the invention. As used herein, the term "storage operating system" generally refers to the computer-executable code operable on a computer that manages data access and may implement file system semantics, such as the Data ONTAP® storage operating system, implemented as a microkernel, and available from Network Appliance, Inc. of Sunnyvale, California, which implements a Write Anywhere File Layout (WAFL™) file system. The storage operating system can also be implemented as an application program operating over a general-purpose operating system, such as UNIX® or Windows NT®, or as a general-purpose operating system with configurable functionality, which is configured for storage applications.

[0026] The storage operating system 300 comprises a series of software layers organized to form an integrated network protocol stack or, more generally, a multi-protocol engine that provides data paths for clients to access information stored on the storage system using block and file access protocols. The protocol stack includes a media access layer 310 of network drivers (e.g., gigabit Ethernet drivers) that interfaces to network protocol layers, such as the IP layer 312 and its supporting transport mechanisms, the TCP layer 314 and the User Datagram Protocol (UDP) layer 316. A file system protocol layer provides multi-protocol file access and, to that end, includes support for the Direct Access File System (DAFS) protocol 318, the NFS protocol 320, the CIFS protocol 322 and the Hypertext Transfer Protocol (HTTP) protocol 324. A virtual interface (Vl) layer 326 implements the Vl architecture to provide direct access transport (DAT) capabilities, such as remote direct memory access (RDMA), as required by the DAFS protocol 318.

[0027] An iSCSI driver layer 328 provides block protocol access over the TCP/IP network protocol layers, while a FC driver layer 330 receives and transmits block access requests and responses to and from the storage system. The FC and iSCSI drivers provide FC-specific and iSCSI-specific access control to the blocks and, thus, manage exports of LUNs to either iSCSI or FCP or, alternatively, to both iSCSI and FCP when accessing the blocks on the storage system. In addition, the storage operating system includes a storage module embodied as a RAID system 340 that manages the storage and retrieval of information to and from the volumes/disks in accordance with I/O operations, and a disk driver system 350 that implements a disk access protocol such as, e.g., the SCSI protocol. [0028] Bridging the disk software layers with the integrated network protocol stack layers is a virtualization system that is implemented by a file system 380 interacting with virtualization modules illustratively embodied as, e.g., vdisk module 390 and SCSI target module 370. The vdisk module 390 is layered on the file system 380 to enable access by administrative interfaces, such as a user interface (Ul) 375, in response to a user (system administrator) issuing commands to the storage system. The SCSI target module 370 is disposed to provide a translation layer of the virtualization system between the block (LUN) space and the file system space, where LUNs are represented as blocks. The Ul 375 is disposed over the storage operating system in a manner that enables administrative or user access to the various layers and systems.

[0029] The file system 380 is illustratively a message-based system that provides logical volume management capabilities for use in access to the information stored on the storage devices, such as disks. That is, in addition to providing file system semantics, the file system 380 provides functions normally associated with a volume manager. These functions include (i) aggregation of the disks, (ii) aggregation of storage bandwidth of the disks, and (iii) reliability guarantees, such as mirroring and/or parity (RAID). The file system 380 illustratively implements a write anywhere file system having an on-disk format representation that is block-based using, e.g., 4 kilobyte (kB) blocks and using index nodes ("inodes") to identify files and file attributes (such as creation time, access permissions, size and block location).

[0030] The file system 380, in one embodiment, may include a WORM component 382. The WORM component 382 may be configured to identify files that, according to a preset criteria, may be considered to be closed (i.e., when an application completed modification operations on the file) and commit files that have not been modified for a predetermined period of time to WORM status.

[0031] Figure 4 is a schematic block diagram of a WORM component 400, according to one embodiment of the present invention. The WORM component 400 may include a compliance clock 410, a configuration component 420, a scanner 430, and a commit component 440. [0032] The compliance clock 410, in one embodiment, is different from a system clock in that that the compliance clock 410 has certain security features that restrict any user from modifying the time on it. In one embodiment, the commands that could be used to modify the clock are disabled for the compliance clock 410. The time the compliance clock has (compliance clock time) is periodically written to all of the volumes on the storage system, both the WORM volumes and the non-WORM volumes. In one embodiment, when a volume is mounted or brought on line, the compliance clock time that was last written to that volume is compared to the current compliance clock time for the system and, if the compliance clock time for the volume is earlier than the compliance clock time for the system, the system's compliance clock time is moved back to match the compliance clock time that was last written for the volume. Thus, from the point of view of any volume, the compliance clock 410 never jumps forward. [0033] Returning to Figure 4, the configuration component 420 may be utilized to define various settings associated with a WORM volume. In one embodiment, the configuration component 420 permits an administrator to specify an autocommit period (e.g., utilizing a CLI interface or a GUI interface) that indicates when a file may be committed to WORM status after the file has been closed for writing. Other settings may include retention period for files that have been committed to WORM status, as well as a definition of what constitutes a modification operation performed on a file. For example, the configuration component 420 may define a modification operation to include a change to a file's contents, but to exclude any change to a file's attributes. Every time a file in the storage system is created or modified (e.g., by a compliance application), a modification time stamp (here, referred to as "mtime") is updated. [0034] Another component illustrated in Figure 4 is the scanner 430. The scanner 430 may be configured to scan the files on a WORM volume and to identify those files that are closed and hence are ready to be committed to WORM status. This determination may be made, in one embodiment, by comparing the modification time stamp for a file (the mtime) with the current compliance clock value (compliance time). In one embodiment, the modification time stamp for a file (the mtime) may be stored in the file's inode along with other metadata related to the file. If the difference between the mtime and the compliance time for a file exceeds the autocommit period, the scanner 430 may identify the file as ready to be committed to WORM status. The scanner 430 then communicates information regarding the files that are ready to be committed to WORM status to the commit component 440, and the commit component 440 commits such files to WORM status.

[0035] It will be noted, that there may be a delay between the time the scanner 430 identifies a file as being ready to be committed to WORM status and the time the commit component 440 actually commits the file to WORM status. The operations involved in the commit process, in one embodiment, include setting the file's read-only attribute and associating an end of retention time for the file. The commit component 440 may determine retention time for the file utilizing a default retention time value for the volume and the modification time for a file associated with the file. Specifically, retention time may be calculated by increasing mtime by the retention period. In one embodiment, the system will protect a file from deletion and modification until retention time has been reached. [0036] Figure 5 is a flowchart illustrating a method 500 to automatically commit files to WORM status, according to one embodiment of the present invention. The method may be performed by processing logic of the storage system that may comprise hardware (e.g., dedicated logic, programmable logic, microcode, etc.), software (such as run on a general purpose computer system or a dedicated machine), or a combination of both. [0037] Referring to Figure 5, the method 500 begins at operation 502 with processing logic (e.g. the configuration component 420) setting the autocommit. The autocommit period setting may be, for example, a system- wide parameter. In some embodiments, the autocommit period may be set on a per volume basis. The processing logic sets the retention time on a file to a default or a user-defined value (operation 504) and defines criteria to determine whether a file is closed by checking if the file has been modified (operation 506). At operation 508, the scanner 430 starts scanning the files on a WORM volume (e.g., by accessing inodes associated with the files). For each file, the scanner 430 determines whether the difference between the current compliance time and the file's mtime is greater than the autocommit period (operation 510). If the difference between the current compliance time and the file's mtime is greater than or equal to the autocommit period, the file is committed to WORM status at operation 512. If the difference between the current compliance time and the file's mtime is less than the the autocommit period, the scanner 430 continues to scan the files on the WORM volume at operation 514. [0038] Thus, a method and apparatus to automatically commit files to WORM data storage have been described. Although the present invention has been described with reference to specific exemplary embodiments, it will be recognized that the invention is not limited to the embodiments described, but can be practiced with modification and alteration within the spirit and scope of the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense.

[0039] Some portions of the foregoing detailed descriptions are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self- consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. [0040] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. [0041 ] The present invention also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs)₁ EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. [0042] The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear from the description above. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein. [0043] A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, a machine-readable medium includes read only memory ("ROM"); random access memory ("RAM"); magnetic disk storage media; optical storage media; FLASH memory devices; electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.); etc.

[0044] Whereas many alterations and modifications of the present invention will no doubt become apparent to a person of ordinary skill in the art after having read the foregoing description, it is to be understood that any particular embodiment shown and described by way of illustration is in no way intended to be considered limiting. Therefore, references to details of various embodiments are not intended to limit the scope of the claims which in themselves recite only those features regarded as essential to the invention.

Claims

CLAIMSWhat is claimed is:

1. A system comprising: a configuration component to set an autocommit period; a scanner to detect that the autocommit period has expired for a file; and a commit component to commit the file to write once read many (WORM) status.

2. The system of claim 1 , wherein the configuration component is further to set a retention period.

3. The system of claim 2, wherein the commit component is to set a retention for the file based on the retention period.

4. The system of claim 2, wherein the retention period is a default retention period.

5. The system of claim 1 , wherein the scanner comprises: logic to scan a plurality of files in a volume; and logic to determine that a file from the plurality of files has remained unmodified for at least the autocommit period.

6. The system of claim 5, wherein metadata associated with the volume identifies the volume as a WORM volume.

7. The system of claim 1 , wherein the commit component is to commit the file to WORM status by designating the file as read only.

8. The system of claim 1 , wherein the configuration component is to set a period within which a file is to be committed to WORM after an expiration of the autocommit period.

9. A method comprising: setting an autocommit period; detecting that the autocommit period has expired for a file; and committing the file to write once read many (WORM) status.

10. The method of claim 9, further comprising setting a retention period.

11. The method of claim 10, wherein committing the file to WORM status comprises setting a retention for the file based on the retention period.

12. The method of claim 9, wherein committing the file to WORM status comprises designating the file as read only.

13. The method of claim 9, detecting that the autocommit period is expired for the file comprises: scanning a plurality of files in a volume; and determining that the file has remained unmodified for at least the autocommit period.

14. The method of claim 13, wherein determining that the file has remained unmodified is based on a predetermined definition of a modifying operation.

15. A storage system to migrate data to persistent storage, the system comprising: a compliance clock, wherein commands that can be used to modify the compliance clock are disabled; a scanner to detect that a file has not been modified for an predetermined period of time with respect to a current compliance clock reading; and a commit component to change the file status to read only and to set a retention for the file.

16. The system of claim 15, further comprising a configuration component to set the predetermined period of time and a retention period.

17. The system of claim 16, wherein the commit component is further to determine the retention for the file based on current compliance clock reading and the retention period.

18. A machine-readable medium having stored thereon data representing sets of instructions which, when executed by a machine, cause the machine to: set an autocommit period; detect that the autocommit period has expired for a file; and commit the file to write once read many (WORM) status.