JP5574774B2 - File server, continuous snapshot management method, and continuous snapshot management program - Google Patents

Description

The present invention relates to a file server, a continuous snapshot management method, and a continuous snapshot management program, and more particularly, to a file server, a continuous snapshot management method, and a continuous snapshot management program for creating a continuous snapshot .

  A file server is a computer or device having a function of sharing a storage device of itself with other computers on a network and making them available from other computers. As the storage device, a non-volatile and random accessible device such as a hard disk or a flash memory is used. The file server abstracts and handles data as a resource called a file through a function of a basic software (OS) called a file system, and provides access means such as generation / deletion and reading / writing of data.

  In order to make data management easier and more efficient, file systems usually manage files hierarchically with persistent resources, which are abstractions of files or directories themselves, called folders or directories, and path names. However, the file server also has a function of making such a name space transparently accessible from other computers. Access and operation functions for these files and name spaces are transmitted between the client and the server through a predetermined network protocol (remote file management protocol). As a typical remote file management protocol, NFS (Network File System), CIFS (Common Internet File system), or WebDAV (Web-based Distributed Authoring and Versioning) extending HTTP is known.

  Since the file server plays a role of storing user data, high reliability and a data protection function are required. In order to protect data from hardware failure, a file server is often made redundant by combining a plurality of storage devices and applying a technique such as mirroring or RAID (Redundant Arrays of Inexpensive Disks).

  These redundancy methods are effective in protecting data from storage device failures, but they can be used to prevent accidental user operations, data corruption due to software bugs, or data tampering with viruses. This is not effective because all redundant data is affected at the same time.

  One way to avoid data loss due to problems not caused by these hardware failures is to regularly back up the data in the storage device to tape or other storage devices. Alternatively, in the file server, the state of the storage device (block device) or file system at a certain point in time is left frozen, and the frozen state (snapshot) can be referred to later, so that the above-mentioned failure There is something to provide to prevent data loss due to. File system snapshots copy file data and file system internal data called metadata that is stored on the same storage device to manage the file, and copy data and metadata. This is realized by adopting a writing method called “Copy-on-Write” that efficiently leaves the past state by writing to a new location without overwriting.

  However, regular backups and snapshots cannot be increased frequently due to the load on the operating system and the time taken to create backups and snapshots themselves, resulting in incorrect operations and bugs after the last acquisition time. It is difficult to avoid data loss due to data destruction and alteration.

  As a technology to overcome this, NILFS (New Implementation of the file system) that can continuously acquire snapshots based on CDP (Continuous Data Protection) that continuously transfers backups to other machines and log structured file system method Log-Structured Filesystem) (see Non-Patent Document 1, for example). NILFS applies a log structured file system in which changed data and metadata blocks that manage it are added to the disk, and checkpoints generated periodically by the log structured file system (successful with a file system). This is a technology that makes a consistent state) accessible as a snapshot. NILFS automatically and implicitly generates a large number of snapshots (or checkpoints) in the local file system, and makes detailed snapshots at the granularity of writing modified data back to disk and writing with guaranteed synchronization. Can be generated.

  FIG. 18 shows the concept of the continuous snapshot file system.

  In a normal file system, files edited and saved by the application are overwritten on the disk, and no past version data remains. When backups are performed regularly, the state at the time of backup can be restored, but changes between backups, in particular, data that has changed since the last backup was acquired are not protected. On the other hand, in NILFS, the previous data is not overwritten and remains on the storage device as a snapshot, and data and directories lost due to the above-mentioned artificial or software failure can be restored.

  NILFS's continuous snapshot method does not require dedicated hardware as such a data protection solution, the overhead on the operating system is almost negligible, and guarantees the integrity of the entire file system, not the file unit. It has the advantage that it can be recovered easily.

Koji Sato, Ryusuke Konishi, Seiji Kihara, Ryoji Amami, Satoshi Moriai “Design and Implementation of Log Structured File System NILFS”, IPSJ Transactions on Computing Systems (ACS), Vol.2, No.1, pp.110 -122, 2009.

  However, if a large number of snapshots are additionally generated by a continuous snapshot creation technique such as NILFS, the past data held by the snapshots is maintained, so that the capacity of the storage device becomes tight. For this reason, operations such as deleting snapshots as necessary to free disk space are necessary. However, with the above-mentioned file server, such operations cannot be performed autonomously, improving user convenience and maintainability. It will be seriously damaged.

  Since deletion of a large number of snapshots generated here places an I / O load on the file server, it is desirable to adjust so that the responsiveness of access from the user does not deteriorate. One method is to shift the snapshot deletion process outside business hours such as at night, but in a system where snapshots are generated at a high frequency, such as NILFS, during the scheduled deletion process. A large number of snapshots are accumulated in the storage device, and the usage amount of the storage device varies greatly. Since a capacity more than necessary is consumed transiently, a capacity shortage is likely to occur.

  On the other hand, when the viewpoint is moved to the client side of the file server, a large number of snapshots are provided, so that the user who views them is confused, and the snapshot processing of the client system cannot handle a large number of snapshots. The harmful effects of Some of today's commodity PC OSs use the file server's snapshot function to extract file change history and allow the user to refer to the history of each file. Since the snapshot is referred to as a background process, the processing time and memory consumption increase according to the number of snapshots, and the responsiveness and stability of the client PC (personal computer) are reduced.

  Furthermore, as one method of using the shared file server, it may be used for backing up data on the user's PC. In such applications, the user's data on the file server is updated only at the frequency of backup. Update data between backups is not protected, and the detailed snapshot creation function of continuous snapshots cannot be used.

Another problem of providing a large number of snapshots on a file server by continuous snapshots is that it becomes difficult to find a desired file. For this reason, file search is important as an auxiliary means.
In particular, a file search function based on an index search method is effective as a means for efficiently searching for a file from a large number of snapshots. NILFS continuous snapshots provide time spread in addition to the expansion of the file system name space, but by assigning each of the past snapshots as subdirectories, the files on the snapshot are also included in the search target. be able to.

  However, when searching by simply assigning each snapshot to the name space in this way, the history of the file is divided in units of snapshots, which is wasteful in terms of user convenience and search index efficiency. is there. In particular, it is required to maintain a search index automatically and efficiently for situations where snapshots are continuously generated and deleted.

The present invention has been made in view of the above points. In a situation where snapshots are continuously generated and deleted, data lost due to the latest erroneous operation is preserved, and past snapshots after a certain period of time are preserved. It is an object of the present invention to provide a file server, a continuous snapshot management method, and a continuous snapshot management program that can automatically and efficiently maintain snapshots that remain as time passes.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 1) detects a change in the state of the file contents and attributes, the structure of the directory tree, etc., and automatically and additionally creates a snapshot in the same volume of the storage means 110. A file server that manages snapshots for
Snapshots are acquired from the file system 151, and the snapshots are grouped based on information on the generation time of the snapshots. All snapshots generated after a past time point that is a predetermined time later than the current time are the files. Snapshot acquisition means 1611 to be stored in the system;
For snapshots prior to the past time point, snapshot deletion means for deleting snapshots in a step-by-step manner from the table of the file system 151 so that the interval between generation times of snapshots is equal to or greater than the time determined for each group. 1612,
File server, a snapshot, NFS (Network File System), through a remote file management protocols such as CIFS (Common Internet File S ystem) , when a server that shares a continuous snapshots exposed via a network from a client terminal ,
Grouping means for dividing the snapshots into a plurality of groups according to the duration from the generation time to the current time, and storing them in the file system;
A disclosure unit that selects and publishes a predetermined number of snapshots from each group of the file system, thereby limiting the number of snapshots to be accessed by the client terminal within a predetermined number;

The present invention (Claim 2) is the file server of Claim 1,
Even when a snapshot is deleted from the table of the file system 151, the data and metadata areas on the disk used by the snapshot are not released immediately, but operate according to a predetermined schedule, or the CPU usage rate And a release means for releasing the area later than when the snapshot is deleted by a background program that operates during a period when the system load such as the disk input / output amount is lower than a certain level.

  FIG. 2 is a diagram for explaining the principle of the present invention.

According to the present invention (Claim 3), a snap to a file system that automatically and additionally creates a snapshot in the same volume of the storage means by detecting a change in the state of the file contents and attributes, the structure of the directory tree, etc. A continuous snapshot management method in a file server for managing shots,
The file server snapshot acquisition means
Snapshots are acquired from the file system (step 1), snapshots are grouped based on the snapshot generation time information (step 2), and are generated after a past time point that is a predetermined time after the current time. Save all snapshots in the file system (step 3)
File server snapshot deletion means
For snapshots prior to the past time point, delete snapshots step by step from the file system table so that the interval between snapshot generation times is equal to or greater than the time determined for each group (step 4),
File server, a snapshot, NFS (Network File System), CIFS through (Common Internet File S ystem) such remote file management protocol, when a server that shares a continuous snapshots exposed via a network from a client terminal ,
File server grouping means
The snapshots are divided into multiple groups according to the duration from the generation time to the current time, stored in the file system,
File server disclosure means
By selecting and releasing a certain number of snapshots from each group of the file system, the number of snapshots accessed by the client terminal is kept within a certain number.

Further, the present invention (Claim 4 ) is the continuous snapshot management method according to Claim 3 ,
File server release means
Even when a snapshot is deleted from the file system table, the data and metadata areas on the disk used by the snapshot are not released immediately, but operate according to a predetermined schedule, or the CPU usage rate The area is released later than when the snapshot is deleted by a background program that operates during a period when the system load such as the disk I / O amount is lower than a certain level.

The present invention (Claim 5) is a continuous snapshot management program for causing a computer to function as each means constituting the file server according to Claim 1 or 2 .

  As described above, the present invention has the following effects.

Claim 1, according to the 3,5 invention (first invention), in a file server to automatically create a continuous number of snapshots, the preserving data loss by the last erroneous operation, also has passed a predetermined time With regard to past snapshots, continuous snapshot management that achieves both data protection and history management can be performed mechanically, such that a user needs a snapshot that remains necessary as time passes. In addition, a file server that can be operated for a long period of time can be realized so that a volume in which continuous snapshots are accumulated does not become full due to an increase in history (snapshot). In addition, when a continuous snapshot is published via a remote file management protocol from a client machine via a network, and when a client machine performs a process of accessing a snapshot that extracts a file history from a snapshot, By limiting the number of reference snapshots to a certain number, it is possible to reduce the load on both the client machine and the file server, and improve responsiveness and stability. In addition, when the user browses the snapshot through the GUI on the client PC, usability when searching for and selecting a snapshot of a desired time and version is improved.

According to the second, fourth, and fifth aspects of the invention (second invention), the file server according to the first aspect of the invention can be used even when the file server does not have a file access from the user or outside the business hours. Snapshot deletion will be performed in the low access time, so that many continuously generated snapshots can be deleted without reducing the responsiveness when the user accesses the file on the file server as much as possible. become.

It is a principle block diagram of this invention. It is a figure for demonstrating the principle of this invention. It is a block diagram of the continuous snapshot file server in the 1st Embodiment of this invention. It is a flowchart of operation | movement of the snapshot management daemon in the 1st Embodiment of this invention. It is a figure for demonstrating the stepwise deletion of the snapshot in the 1st Embodiment of this invention. It is a block diagram of the continuous snapshot file server in the 2nd Embodiment of this invention. It is a flowchart of operation | movement of the release daemon in the 2nd Embodiment of this invention. It is a figure which shows the sharing method of a snapshot. It is a flowchart of the selection process of the snapshot in the 3rd Embodiment of this invention. It is a block diagram of the backup system in the 4th Embodiment of this invention. It is a flowchart of operation | movement of the synchronous daemon in the 4th Embodiment of this invention. It is a figure which shows the relationship between the lifetime of a file in the continuous snapshot file server by 1st, 2nd embodiment, and a snapshot number. It is a structural example of the search index in the 5th Embodiment of this invention. It is a block diagram of the search index production | generation apparatus in the 6th-8th embodiment of this invention. It is a flowchart of the search index production | generation process in the 6th Embodiment of this invention. It is a flowchart of the update process of the search index in the 7th Embodiment of this invention. It is a flowchart of the optimization process of the search index in the 8th Embodiment of this invention. It is a figure which shows the concept of a continuous snapshot file system.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 3 shows a configuration example of the continuous snapshot file server in the first embodiment of the present invention.

  The continuous snapshot file server 100 shown in FIG. 1 includes a storage device (disk) 110 and a network interface 120, and its basic software OS (Operating System) 150 includes a continuous snapshot file system 151 and devices of the network interface 120. Assume that the network communication unit 152 includes a driver, a network protocol stack, and the like. In addition, a file sharing service 165 and a snapshot management daemon 161 are provided as programs for the client PC. Here, the storage device 110 can improve availability and reliability by applying a redundancy technique such as mirroring or RAID without losing generality so as to avoid data loss due to hardware failure.

  The continuous snapshot file system 151 performs block I / O (a1) on the volume on the storage device 110, and provides system resources such as files and directories to applications in the user space in the same manner as a normal file system. The function of storing data in 110 and referring to or managing data in the storage device 110 is provided. Here, “block I / O” means that the file system of the present invention targets block devices, and therefore inputs and outputs in block units.

  The file sharing service 165 makes these resources of the continuous snapshot file system 151 accessible from the client PC through the network communication unit 152 and the network interface 120 (a2 to a5).

  The snapshot management daemon 161 periodically checks the state of the continuous snapshot file system 151 (a6), and deletes the snapshot as time elapses (a7).

  FIG. 4 shows a flowchart of the operation of the snapshot management daemon in the first embodiment of the present invention.

Step 101) First, the snapshot management daemon 161 is configured to store the “recent” snapshot retention period T _recent and “every hour” snapshot from the configuration file (pointing to the storage area of the system that cannot be accessed from the outside). The retention period T _hourly , the “ _daily ” snapshot retention period T _daily , and the “weekly” snapshot retention period T _weekly are read and set on a memory (not shown).

  Step 102) The snapshot number and time stamp of the existing snapshot are acquired from the continuous snapshot file system 151.

  Step 103) Also, the current time is acquired from the basic software 150 and set to the reference time T.

Step 104) Then, from the acquired snapshot, a snapshot whose generation time t of the snapshot is t ≧ T−T _recent is excluded.

Step 105) Next, a snapshot having a generation time t of t ≧ T−T _hourly is selected, and the generation time intervals of the snapshots are examined in order from the oldest or newest one, and the interval is less than 1 hour. Delete the snapshot and exclude the remaining snapshots within the selected period from being processed.

Step 106) Similarly, select a snapshot whose generation time t is t ≧ T−T _daily , delete the intermediate snapshot so that the interval is one day or longer, and exclude the remaining snapshots from the processing target. .

Step 107) Select a snapshot whose generation time t is t ≧ T−T _weekly , delete the intermediate snapshot so that the interval is one week or longer, and exclude the remaining snapshots from the processing target.

  Step 108) Delete the intermediate snapshot so that the interval between the remaining snapshots is one month or longer.

  Step 109) Then, it is checked whether or not there is a stop request.

  Step 110) After sleeping for a predetermined time, the process proceeds to Step 102, and the processing after acquisition of the snapshot list is repeated.

  The snapshots thinned out in this manner are those in which the number of snapshots is suppressed as time passes as shown in FIG.

  In this embodiment, the snapshot management daemon 161 performs only snapshot deletion on the assumption that the snapshot creation is automatically created by the continuous snapshot file system 151. By causing the snapshot management daemon 161 to issue an instruction, snapshot creation is supported, but a continuous snapshot file server can be configured for a file system that is not automatically created.

  In addition, when the continuous snapshot file system 151 does not have a function for managing snapshot information in an integrated manner, the performance of the file server is deteriorated due to cross access of a large number of snapshots. In that case, the list information including the time stamp information of the snapshot is held in the storage unit in the snapshot management daemon 161, and only the information of the newly created snapshot is acquired from the continuous snapshot file system 151. Therefore, it can be implemented so as to suppress the performance degradation.

[Second Embodiment]
In the present embodiment, a process for deleting a snapshot and releasing the storage device 110 outside a business time period when there is no access from the user or when there is little access from the user will be described.

  FIG. 6 shows the configuration of the continuous snapshot file server in the second embodiment of the present invention.

  In the figure, the same components as those in FIG. The configuration in FIG. 6 is a configuration in which a release daemon 162 is added to the configuration in FIG.

  The continuous snapshot file server 100 according to the present embodiment is configured based on the continuous snapshot file server according to the first embodiment, but the snapshot management daemon 161 does not actually perform the snapshot when deleting the snapshot. The data area on the storage device 110 used by the shot is not released, and only the deletion request is marked on the designated snapshot (a6 ′).

  In this embodiment, it is the release daemon 162 that releases the data area of the storage device 110 used by the snapshot, which is implemented as a background program in the user space or the basic software 150. It can be implemented as a program that is included or not included as part of the file system.

  FIG. 7 is a flowchart of processing of the release daemon according to the second embodiment of this invention.

Step 201) First, the release daemon 162 allocates a CPU usage rate threshold value R _cpu , an I / O load threshold value R _{I / O} , and one release processing allocation from a setting file (not shown) in the server 100. Read time T _slice and set to memory (not shown).

Step 202) Next, the CPU usage rate L _cpu is acquired from the system.

Step 203) The CPU usage rate L _cpu and the CPU usage rate threshold value R _cpu are compared. If L _cpu <R _{cpu, the procedure} proceeds to step 211, and if L _cpu <R _cpu , the _{procedure proceeds} to step 204.

Step 204) If L _cpu <R _cpu , measure or obtain I / O load L _{I / O} from the system.

Step 205) If L _{I / O} <R _{I / O} , the process proceeds to step 206, and if L _{I / O} <R _{I / O, the process} proceeds to step 211.

Step 206) If L _{I / O} <R _{I / O} , a list of snapshots marked for deletion from the continuous snapshot file system 151 is acquired.

  Step 207) Next, the release processing start time is acquired from the basic software 150.

  Step 208) Then, from the acquired snapshots, one snapshot whose deletion request is marked by the snapshot management daemon 161 is released.

Step 209) It is confirmed whether the elapsed time t from the start time is t ≦ T _slice . If the elapsed time has not yet passed, the process proceeds to Step 207 and the processing of Steps 207 and 208 is repeated. If the elapsed time has passed, the process proceeds to step 210.

  Step 210) When the elapsed time has passed, the presence or absence of a stop request is checked.

  Step 211) After sleeping for a predetermined time, the processing from step 202 onward is repeated.

  In the above explanation, the disk area used by the snapshot is released in units of snapshots. However, multiple snapshots can be deleted in step 208 without loss of generality, or the data constituting the snapshot can be deleted. It can be a predetermined size unit.

  In addition, as a method of releasing disk space, the disk space is divided into multiple sections (segments), and the disk blocks included in each segment are used depending on the current state of the snapshot or file system not subject to deletion request. There is a method of determining whether or not, copying all blocks in use to an empty area, and releasing a segment. Even when such a release method is adopted, the processing unit of step 208 can be a single or a plurality of segment units of the disk area, so that the delay process of releasing the disk area can be realized without impairing generality.

[Third Embodiment]
In the present embodiment, for the sake of explanation, the continuous snapshot file server 100 according to the first or second embodiment shown in FIG. 8 can browse and reference a snapshot from a client PC connected via a network. Shows an example of how to publish them as shared folders.

  FIG. 8 is a diagram for explaining an example of a snapshot sharing method according to the third embodiment of the present invention. In the example of the figure, the current file of the file system and its name space are allocated to the shared folder “share” 10 and published, while a series of snapshots are given names based on the generation time of the snapshot, and the shared folder “share” ”In the snapshot folder“ snapshots ”20 corresponding to“ 10 ”. The snapshot folder 20 may be published so that it can be directly accessed by the user in the same way as a normal shared folder, or may be published by being assigned to a specific subdirectory (for example, “.snapshots”) on the current file system. is there. Alternatively, the directory may be disclosed to the client PC as a directory that is not directly visible to the user for reference by the snapshot function of the client PC. In addition, the name of the subdirectory to which the snapshot is allocated may be given based on the snapshot identification number instead of the generation time.

  The generation of the namespace of the snapshot folder 20 and the allocation to the individual snapshots are performed through the file sharing service unit 165 of the continuous snapshot file server 100.

  FIG. 9 is a flowchart of snapshot selection processing according to the third embodiment of the present invention, and shows an example of snapshot selection processing for publishing a snapshot folder to a client PC.

  In the example shown in the figure, the snapshots are divided into three groups (in order from the newest one, the “hot” group, the “cold” group, and the “remaining” group).

Step 301) First, when the file sharing service unit 165 receives an acquisition request for the subdirectory name in the snapshot folder 20 from the client PC, the file sharing service unit 165 acquires the number of displayed snapshots N _hot , N _cold , and N _rest for each group. To do.

Step 302) Acquire retroactive times T _hot and T _cold for grouping snapshots based on the current time.

  Step 303) The selected snapshot storage memory (not shown) table H is initialized.

  Step 304) A list of snapshots is acquired from the continuous snapshot file system 151.

  Step 305) The current time is acquired and set as the reference time T.

Step 306) After that, from the acquired snapshots, those having a generation time t of t ≧ T−T _hot are selected as belonging to the “hot” group, and among them, a maximum of N _hot items in order from the new one And register it in Table H.

Step 307) Similarly, select snapshots whose generation time t is T−T _hot > t ≧ T−T _cold as belonging to the “cold” group, and select a maximum of N _{cold in} order from the newest one And add it to table H.

Step 308) Further, the snapshots whose generation time t is t <T−T _cold are selected as belonging to the “remaining” group, and a maximum of N _rest are selected in order from the newest one and added to the table H.

  Step 309) Finally, the snapshot registered in the table H is output.

  For example, by outputting from the table H corresponding to the subdirectory for each snapshot shown in FIG.

  The above-described snapshot selection processing is not performed every time a snapshot folder browsing request is made from the client PC, but can be cached in the file sharing service 165 for a short period of time in order to improve the response speed. The snapshot grouping method (period) need not be the same as the dividing method (period) according to the first embodiment. Since the first embodiment deletes the snapshot, the snapshot acquired in step 304 reflects the result. Further, when applying the second embodiment, the snapshot being deleted is excluded from the snapshot acquired in step 304.

[Fourth Embodiment]
FIG. 10 shows the configuration of the backup system in the fourth embodiment of the present invention.

  The backup system shown in the figure is configured by connecting a continuous snapshot file server 100 and a client PC 200 according to the first or second embodiment via a network.

  A client PC (PC) 200 includes a storage device 210 and a network interface 220, and its basic software (Operating System) 250 includes a local file system 251, a device driver for the network interface 220, a network protocol stack, and the like. 252, a file sharing client unit 255, and a notification function unit 254 that notifies the userland application of a change operation on the local file system. A synchronization daemon 261 is provided as a userland program.

  The file system 251 is a normal file system, performs block I / O (b1) on the volume on the storage device 210, provides system resources such as files and directories, and provides data storage / reference or management functions. On the other hand, the file sharing client unit 255 is connected to the continuous snapshot file server 100 through the network communication unit 252 and the network interface 220, and accesses the browsing, reference, and change of the shared folder of the server 100 according to the access right. (B2 to b4).

  The synchronization daemon 261 checks the change of the directory registered in advance in the local file system 251 through the notification function unit 254 (b5, b6), and the change content is stored in the shared folder of the continuous snapshot file server 100 through the file sharing client unit 250. To be reflected in the previously registered directory.

  FIG. 11 is a flowchart of the operation of the synchronization daemon according to the fourth embodiment of this invention.

Step 401) The synchronization daemon 261 first reads and sets a pair of the synchronization source directory D _src on the local file system 251 and the synchronization destination directory D _dest on the shared folder of the continuous snapshot file server 100 from a setting file or the like.

Step 402) Further, the notification function unit 254 is set to monitor file operations and directory operations in the snapshot D _src of the snapshot existing from the continuous snapshot file system 151.

Step 403) Next, it is confirmed whether or not communication with the continuous snapshot file server 100 having the shared folder specified in the synchronization destination directory D _dest is possible, and otherwise waits for a certain time until communication is possible.

  Step 404) If the confirmation process has timed out, the process proceeds to step 405, and if not, the process proceeds to step 410.

  Step 405) Next, it is checked whether or not there is a stop request. If there is a stop request, the process proceeds to Step 406, and if not, the process proceeds to Step 403.

Step 406) If there is a stop request, the notification function unit 254 is set to cancel the monitoring of the synchronization source directory D _src and the process ends.

Step 410) If communication with the file server 100 is possible, first, in order to make the states of the synchronization source directory D _src and the synchronization destination directory D _dest coincide, synchronization processing is recursively performed from the top of D _src and D _dest . If it is a directory, check the directory structure, create or delete the directory for D _dest so that the structure matches, and if it is a file, check whether the contents match. Copy the file to the target directory on _dest .

Step 411) After the synchronization processing is completed, the synchronization daemon 261 receives the event e = <d, f, o> of the file operation / directory operation from the notification function unit 254, and if not, waits for a certain time until the event arrives. Here, d, f, and o are a target file from the top directory of D _src , a relative path of the directory, an operation type, and an operation optional parameter, respectively. The types of operations include creation / deletion / movement / renaming of files and directories, change of file contents, change of various attributes such as file / directory access rights, time stamp, owner information, and group information. The option parameter is incidental information necessary for performing each operation.

  Step 412) If timed out, the process proceeds to step 415, and if before time-out, the process proceeds to step 413.

Step 413) When the event is accepted in Step 411, the change operation f (o) is executed with the directory D _dest / d in the shared folder (d is the relative path of the target file / directory in D _src ).

  Step 414) If the change operation in step 413 fails, the process proceeds to step 405, and if successful, the process proceeds to step 415.

  Step 415) If the change operation is successful, or if the above-described event acceptance times out, the presence or absence of a stop request is confirmed. If there is, the process proceeds to Step 406, and the subsequent processing is performed. If not, the process proceeds to Step 411. Transition.

  In the above description, operations for monitoring and reproducing changes are limited to files and directories, but can be extended to symbolic links and other special files without loss of generality. It is also possible to create a procedure so that the program does not become unresponsive for a long time by generating a worker thread and executing the processing of steps 410 to 415. Further, in the description of the present embodiment, a case is described in which a pair of synchronization source and synchronization destination directories is described, but the case where there are a plurality of linked directory pairs can be expanded without impairing generality.

[Fifth Embodiment]
FIG. 12 shows an example of the relationship between the file lifetime and the snapshot number in the continuous snapshot file server according to the first or second embodiment. FIG. 13 shows an example of a search index in the fifth embodiment of the present invention. The example of the search index of FIG. 13 is a configuration example of the search index according to the present embodiment corresponding to the state of FIG.

  In the example of FIG. 12, it is assumed that “file A” was created at the time of snapshot number 2500, the contents were changed at the time of snapshot number 2520, and deleted at snapshot number 2540.

In addition, it is assumed that “file B” is created at the time of snapshot number 2510 and continues to the present. Since the content of “file A” does not change from the snapshot number 2500 to immediately before the snapshot number 2520, the search keyword is also the same. This is expressed as K _{A, 2500 based on the} starting point. Similarly, the search keyword set of “file A” with the snapshot number [ ₂₅₂₀ , 2540] is represented as K _{A, 2520} , and the keyword set of “file B” is represented as K _{B, 2510} .

As shown in the example of FIG. 13, the search index format according to the fifth embodiment corresponding to this example includes entries corresponding to each keyword set starting with “FilePath”, “StartNum”, “EndNum”. Identified by two fields.
The "FilePath" field stores the path name of the file, the "StartNum" field indicates the first number of the snapshot in which the file exists, and the "EndNum" field indicates the status of the file when it is deleted or changed Stores the number of the snapshot for which has been resolved. Since “File B” continues until now, a null value is stored in the “EndNum” field to express the state. Of course, the value indicating that it has continued until now can be any value as long as it is identified from a valid snapshot number. In the "EndNum" field, the snapshot number in which a certain state has been resolved is used, but the last snapshot number in which a certain state continues without sacrificing generality, that is, used here. A number obtained by subtracting 1 from the value can also be used. Here, the period during which the same state of the file continues is given by the snapshot number, but it is also possible to use the snapshot number and the time stamp (creation time) value of the snapshot. The expression of the present embodiment shows the semantic structure of the search index. Of course, various structures and names can be used for the actual index file data structure and field identification names.
[Sixth Embodiment]
In the following sixth to eighth embodiments, a snapshot corresponding to a keyword input by a user is transmitted to a Web server on NAS (Network Attached Storage) through a Web interface such as Google (registered trademark) search. Can be searched for, including past files.

  In the following embodiment, a search index generation device for generating a search index used for efficiently searching a file from a large number of snapshots of the continuous snapshot file server 100 will be described.

  FIG. 14 shows the configuration of the index generation apparatus in the sixth to eighth embodiments of the present invention. The index generation apparatus 300 includes a search index creation unit 310, a search index update unit 330, a search index optimization unit 340, and a search index reference unit 350.

  The search target file 1100 is a file that exists in the storage device 110 of the continuous snapshot file server 100 and is managed by the continuous snapshot file system 151.

  The search index reference unit 350 of the search index generation device 300 accesses the Web server 600.

  In the present embodiment, the search index creation unit 310 will be described. The search index update unit 330 will be described later in the seventh embodiment, and the search index optimization unit 340 will be described later in an eighth embodiment.

  FIG. 15 is a flowchart of search index creation processing according to the sixth embodiment of the present invention.

  Step 601) The search index creation unit 310 first initializes an existing search index or creates a new empty search index.

Step 602) Then, the snapshot list S = {s ₁ , s ₂ ,..., S _N } is obtained from the continuous snapshot file server 100. However, where N is the number of snapshots, it is assumed that there is order relation _{s 1 <s 2 <... <} s N.

Step 603) Next, a directory tree on the storage device 110 of the continuous snapshot file system 151 is searched for the snapshot s _1, and F ₁ = {f ₁ , f ₂ ,. _m1 } is extracted as a processing target.

Step 604) selects a file f unprocessed from F ₁ of the search target files 1100.

  Step 605) Then, search keywords are extracted from all files to be searched.

To step 606) the file, the extracted search keyword and the entry to the path name of the FilePath field search file, the StartNum field snapshot number s _1, search index storage a new entry that EndNum field is null value Add to 500 search indexes.

Step 607) When all the files of F _{1 to be} searched are processed, the process proceeds to step 608, and when all the files are not processed, the process proceeds to step 604.

  Step 608) Subsequently, index i is initialized to 1 to continuously track changes for a series of snapshots.

  Step 609) If i = N, the process is immediately terminated. Otherwise, the process proceeds to step 610.

Step 610) to extract the search target file F _{i + 1} on the snapshot s _{i + 1.}

Step 611) The file f included in F _i is selected.

Step 612) It is determined whether or not the file f exists on the snapshot s _{i +1. If} it exists, the process proceeds to Step 613. If it does not exist, the process proceeds to Step 616.

Step 613) snapshots s _i on file f and snapshot s _{i + 1} on the data of f determines whether the same, if the same processing proceeds to step 617, if not identical to step 614 Transition.

Step 614) The search keyword (K _{i + 1, f} ) of _f is extracted.

Step 615) The keyword set k _{1 + 1, f} is stored in the search index of the search index storage device 500, and the entry is registered in the following fields.

・ File Path: f
・ StartNum: s _{i + 1}
・ EndNum: null
Step 616) From the search index of the search index storage device 500: FilePath = f,
・ StartNum <s _{i + 1} ,
・ EndNum = null
Search for the entry and change EndNum to s _{i + 1} .

Step 617) to determine whether it has processed all of the files F _i, if treated proceeds to step 618, if not treated proceeds to step 611.

Step 618) The file f ′ included in F _{i + 1} and not included in F _i is selected.

Step 619) The search keyword (K _{i + 1, f ′} ) of the file f _′ is extracted.

Step 620) The keyword set (K _{1 + 1, f ′} ) is stored in the search index storage device 500. Register the entry in the following fields:

・ File Path: f´
・ StartNum: s _{i + 1}
・ EndNum: null
Step 621) It is determined whether all the files of F _{i + 1} have been processed. If processed, the process proceeds to step 622, and if not processed, the process proceeds to step 618.

In steps 610 to 621 described above, the search target file 1100 is examined for the snapshot s _i processed earlier and the next snapshot s _{i + 1} , and the contents are compared. For a file that has been changed or deleted with s _{i + 1} , an entry having the file of the search index storage device 500 in FilePath, the StartNum field being smaller than s _{i + 1} , and the EndNum field having a null value Find and set s _{i + 1} in its EndNum field. In addition, search keywords are extracted for files that have been changed or newly created with s _{i + 1} , and for each file, the extracted search keywords and their entries are displayed in the FilePath field as the path name of the search file, and in the StartNum field. A new entry having a null value in the EndNum field is added to the search index at the snapshot number s _{i + 1} . Then, the index i is incremented, and the process is repeated from step 609 until i = N.

[Seventh Embodiment]
In the present embodiment, search index update processing by the search index update unit 330 will be described.

  FIG. 16 is a flowchart of search index update processing according to the seventh embodiment of the present invention.

  The search index update processing of the present embodiment is performed by a resident program of the search index update unit 330.

Step 701) First, the last snapshot number used for the search index of the search index storage device 500 is acquired and set as S _prev . S _prev is determined by exhaustively examining the entries in the search index storage device 500, or the value is used by storing S _N in the search index generation process described above.

Step 702) Then, it is confirmed whether there is a snapshot s whose number is s> s _prev . If it does not exist, the process proceeds to step 703, and if it exists, the process proceeds to step 705.

  Step 703) If it does not exist, the presence or absence of the end request is confirmed. If there is, the process ends immediately. If not, the process proceeds to Step 704.

  Step 704) After sleeping for a predetermined time, the process proceeds to Step 702.

Step 705) If there is a snapshot s with the number s> s _prev in step 702, a snapshot with s> s _prev is selected and set as s _next .

Step 706) The search target file 1100 is examined for the snapshot s _prev and the snapshot s _next , the contents are compared, and the changed or deleted file is extracted.

Step 707) The unprocessed file f is selected from the updated file F _cm .

Step 708) The search keyword (K _{next, f} ) of _f is extracted.

Step 709) The keyword set (K _{next, f} ) is stored in the search index storage device 500. Register the entry in the following fields:

・ File Path: f
・ StartNum: s _next
・ EndNum: null
Step 710) If all the files of F _cm have been processed, the process proceeds to step 711. If not, the process proceeds to step 707.

Step 711) The file F _md changed or deleted between the snapshots s _prev and s _next is extracted.

Step 712) The unprocessed file f ′ is selected from the file F _md .

Step 713) From the search index of the search index storage device 500,
・ File Path: f´
・ StartNum <s _{i + 1}
・ EndNum: null
And change EndNum to s _next .

Step 714) It is determined whether all the files in the file F _md have been processed. If processed, the process proceeds to Step 715. If not processed, the process proceeds to Step 712.

In steps 705 to 715 described above, if a new snapshot s exists, a snapshot satisfying s> s _prev is selected and set as s _next . Then, the search target file 1100 is examined for the snapshot s _prev and the snapshot s _next , and the contents are compared (step 705). As a result, for a file that has been changed or deleted with s _next , find an entry that has the corresponding file in the search index in FilePath, the StartNum field is smaller than s _next , and the EndNum field is null. Set s _next in the field. Also, search keywords are extracted for files that have been changed or newly created with s _next , and for each file, the extracted search keywords and their entries are the path name of the search file in the FilePath field, and the StartNum field is a snapshot. A new entry having a null value in the EndNum field is added to the search index at the number s _next (steps 706 to 715). Then, s _next is set as the final snapshot number, and the processing from step 701 is repeated.

[Eighth embodiment]
In the present embodiment, search index optimization processing by the search index optimization unit 340 will be described.

  FIG. 17 is a flowchart of search index optimization processing according to the eighth embodiment of the present invention.

  Step 801) First, the search index optimization unit 340 acquires a list of snapshots (numbers) (S = {s1, s2,..., Sn}) from the continuous snapshot file server 100.

  Step 802) Next, each entry of the search index in the search index storage device 500 is examined, and one unprocessed entry e is extracted.

  Step 803) Whether EndNum = null is checked. If it is null, the process proceeds to Step 806. If it is not null, the process proceeds to Step 804.

  Step 804) It is determined whether the snapshot number s satisfying StartNum field ≦ s <EndNum field exists in S. If it exists, the process proceeds to Step 806, and if it does not exist, the process proceeds to Step 805.

  Step 805) Delete entry e from the search index.

  Step 806) When all the entries of the search index have been processed, the process is terminated, and when there is an unprocessed entry, the process proceeds to Step 802.

  In steps 802 to 806 described above, an entry whose non-null value is set in the EndNum field and the snapshot number s satisfying StartNum field ≦ s <EndNum field is not included in the acquired snapshot list is deleted. By applying this process to all entries in the search index, unnecessary entries are deleted and the search index is reduced. This search index optimization processing may be executed by a resident program or may be executed according to a predetermined schedule.

  The operation of the constituent elements of the continuous snapshot file server in the above first to third embodiments is constructed as a program and installed in a computer used as the continuous snapshot file server to be executed, or a network It is possible to circulate through.

  Further, the operation of the components of the client terminal in the fourth embodiment can be constructed as a program, installed in a computer used as a client terminal and executed, or distributed via a network.

  In addition, the operation of the index generation device according to the fifth to eighth embodiments can be constructed as a program, installed in a computer used as the index generation device and executed, or distributed via a network. is there.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

100 continuous snapshot file server 110 storage means, storage device 120 network interface 150 basic software (OS)
151 File System, Continuous Snapshot File System 152 Network Communication Unit 161 Snapshot Management Daemon 162 Release Daemon 165 File Sharing Service Unit 200 Client PC
210 Storage device 220 Network interface 250 Basic software (OS)
251 Local file system 252 Network communication unit 254 Notification function unit 255 File sharing client unit 261 Synchronization daemon 300 Search index generation device 310 Search index creation unit 330 Search index update unit 340 Search index optimization unit 350 Search index reference unit 500 Search index storage Device 600 Web server 1100 Search target file 1611 Snapshot acquisition means 1612 Snapshot deletion means

Claims (5)

A file server that detects changes in the status of file contents and attributes, directory tree configuration, etc., and manages snapshots for file systems that automatically and additionally create snapshots in the same volume of storage means. And
Snapshots are acquired from the file system, snapshots are grouped based on the generation time information of the snapshots, and all snapshots generated after a past point in time that is set back from the current time are the files. Snapshot acquisition means to be stored in the system;
For snapshots prior to the past time point, snapshot deletion is performed in such a manner that snapshots are deleted step by step from the file system table so that the interval between the generation times of snapshots is equal to or longer than the time determined for each group. Means,
When said file server, a snapshot, NFS (Network File System), through a remote file management protocols such as CIFS (Common Internet File S ystem) , is a server that shares a continuous snapshots exposed via a network from a client terminal In addition,
Grouping means for dividing the snapshots into a plurality of groups according to the duration from the generation time to the current time, and storing them in the file system;
An publishing means for keeping the number of snapshots to be accessed by the client terminal within a certain number by selecting and publishing a certain number of snapshots from each group of the file system;
A file server comprising: Even when a snapshot is deleted from the file system table, the data and metadata areas on the disk used by the snapshot are not released immediately, but operate according to a predetermined schedule, or the CPU usage rate 2. The file server according to claim 1, further comprising release means for releasing the area after a snapshot deletion by a background program that operates during a period when the system load such as the disk input / output amount is lower than a certain level. Continuous on a file server that manages snapshot management for file systems that detect state changes such as file contents and attributes, directory tree configuration, etc., and create snapshots automatically and additionally in the same volume of storage means A snapshot management method,
The file server snapshot acquisition means comprises:
Snapshots are acquired from the file system, snapshots are grouped based on the generation time information of the snapshots, and all snapshots generated after a past point in time that is set back from the current time are the files. Save it in the system,
The snapshot deletion means of the file server is
For snapshots prior to the past time point, snapshots are deleted step by step from the file system table so that the interval between snapshot generation times is equal to or greater than the time determined for each group,
When said file server, a snapshot, NFS (Network File System), through a remote file management protocols such as CIFS (Common Internet File S ystem) , is a server that shares a continuous snapshots exposed via a network from a client terminal Is
The file server grouping means comprises:
The snapshots are divided into a plurality of groups according to the duration from the generation time to the current time, stored in the file system,
The file server publishing means comprises:
A continuous snapshot management method, wherein a predetermined number of snapshots are selected from each group of the file system and released, thereby suppressing the number of snapshots to be accessed by the client terminal within a predetermined number. The file server release means comprises:
Even when a snapshot is deleted from the file system table, the data and metadata areas on the disk used by the snapshot are not released immediately, but operate according to a predetermined schedule, or the CPU usage rate 4. The continuous snapshot management method according to claim 3, wherein the area is released later than when the snapshot is deleted by a background program that operates during a period when the system load such as the disk input / output amount is lower than a certain level.   The continuous snapshot management program for functioning a computer as each means which comprises the file server of Claim 1 or 2.

Images