JP3117003B2 - Wide-area distributed file system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed file system in which a file server for storing files and a client for referring to the files are connected via a wide area network, and more particularly to a distributed file system for caching frequently used files. It is.

[0002]

2. Description of the Related Art In many distributed file systems, file access is speeded up by caching a file stored in a file server to a client. That is, by holding a copy of the file stored in the file server on the client side, when an access request to the file is generated next time, the file can be processed in the client without accessing the file server. . In recent years, a distributed file system may be built using a wide area network. In this case, a server specialized in caching called a cache server is installed between the local area network and the wide area network, and connected to the local area network. Some clients get files from the file server through the cache server. Factors affecting file access performance include file server performance, network file transfer speed, and cache hit rate.

As a conventional technique for increasing the cache hit rate, there is a technique disclosed in Japanese Patent Application Laid-Open No. 7-93205 and a Coda distributed file system. In these techniques, file priorities are set for each user or client. The hit rate of the cache is improved by selecting frequently accessed files based on file access statistics, setting them to high priority, and leaving them in the cache area with a high probability.

[0004]

However, in the above prior art, the transfer rate of the wide area network is lower than that of the local area network, and since there are many users, the congestion situation greatly changes depending on the time zone and timing. In some cases, it may not be possible to connect to the file server. For this reason, if the file is not in the cache area or updated on the file server side, the file is acquired from the file server. At this time, the file is transferred via the wide area network, It takes time to transfer data to the server, and in some cases, the data cannot be transferred.

Further, since the priority of the file is set for each user or client, it is considered that users of the same department frequently refer to the same file. However, the current cache method increases the cache hit rate of the entire department. It is not possible.

An object of the present invention is to provide a wide-area distributed file system that improves the cache hit rate and speeds up file access.

[0007]

According to the present invention, a plurality of file servers for storing a plurality of files, a client for accessing the files stored on the file server, and a cache server for storing the files are provided over a wide area. when connected to the network, the cache server, prefetch area for caches file been cache management unit, the read-ahead, the client
Caches target files requested to be accessed by
The cache management unit is configured to include a cache area used for access, and an access history database that records the history of access requests from clients.
More often than the access history database
Selected the most frequently accessed files from
The file is updated from the access history database.
Identify and analyze new history to estimate file update interval
The estimated update interval is set as the prefetch interval, and the
By periodically prefetching frequently accessed files to the cache server at the specified prefetch interval ,
A wide-area distributed file system characterized by accelerating file access is obtained.

Further, according to the present invention, the cache management unit periodically analyzes the access history database, determines a file to be prefetched, and determines a read interval, and a prefetch selection module that records the determined file name and interval. A wide-area distributed file system is provided, comprising a management table and a cache management table for managing the cache area.

Further, according to the present invention, the wide area network manages network congestion and fault information ,
The prefetch selection module is the most efficient file transfer
A wide-area distributed file system is provided in which a network monitoring device that determines a time at which the operation can be performed is installed.

Further, according to the present invention, there is provided a wide-area distributed file system, wherein the plurality of file servers include a file area for storing files and a file management table for managing the file areas.

[0011]

The cache server records a record of the client's file access in the access history database. By analyzing the access history database by the look-ahead selection module, it is possible to analyze the files frequently accessed by the department using the cache server and the update interval.

The cache server reads the prefetch file determined by the prefetch selection module into the prefetch area at a time designated by the wide area network monitoring device. If the file requested by the client is in either the prefetch area or the cache area, the cache server acquires the file from there. By pre-reading a frequently referenced file in advance in addition to the normal cache, the hit rate is improved and the file access speed is increased.

Further, by setting up a cache server for each business unit called a section, files related to the business are pre-read and cached in the cache server, and the hit rate is improved.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings. As shown in FIG. 1, a wide area distributed file system according to the present invention includes a plurality of file servers 5-1 to 5-5 for storing a plurality of files.
n, a cache server 3 connected to the local network 2 for caching files, and a cache server 3
, And clients 1-1 to 1-n accessing the files stored in the file server via the cache server 3.

The cache server 3 includes a cache management unit 31, a cache area 32 used for a normal file cache, a prefetch area 33 for caching a prefetched file, and an access history database for recording a history of access requests from clients. 34.

The cache management unit 31 periodically analyzes the access history database and determines a file to be prefetched and a prefetch selection module 311 for determining a read interval, a prefetch management table 312 for recording the determined file name and interval, and a cache area. It comprises a cache management table 313 to manage.

The wide area network 4 is provided with a network monitoring device 4-1 for managing network congestion and fault information. The file server has a file area 52-n for storing files and a file management table 51-n for managing the file area 52-n.

Next, the operation of this embodiment will be described with reference to FIGS.
This will be described in detail with reference to FIG. When there is a file access request from the client 1-1 to the cache server 3 (step S6), the cache management unit 31 first determines whether or not the target file exists in the cache server by using the look-ahead management table 312 and then the cache management table. 3
The confirmation is made in step 13 (step S61, step S62).

If the file exists, it connects to the file server (steps S610 and S62).
1) Inquire of the file server whether or not the file in the prefetch area and the file in the cache area are the latest one (steps S611 and S622).
As a result, if the file in the prefetch area or the cache area is the latest, the file is passed to the client (step S618, step S67).

When the connection to the file server fails, a warning is notified to the client (step S6).
17) The file in the prefetch area or cache area is unconditionally set as a file to be passed to the client (steps S618 and S67). As a result, even if the connection to the file server cannot be established due to the state of the wide area network, the file can be transferred to the client.

If the file is not the latest one, the latest one is obtained from the file server and replaced with the current file. If the target file is a pre-read area, and if the size of the new file is larger than the existing file (step S613), the lower priority file is evicted from the pre-read area to secure the size, and the size is reserved. The file is replaced, the prefetch management table 312 is updated (step S614, step S615), and the flushed file is moved to the cache area (step S616). If not, the current file is replaced with a new file (step S619, step S619).
Step S620). If the target file is a cache area, or if there is a file that has been evicted from the prefetch area to the cache area, the cache management unit 31
Performs a cache update process according to the size of the cache area (steps S65 and S66). Since the processing procedure is the same as the normal cache update processing, it will not be described here.

When the file does not exist in the prefetch area or the cache area, the cache server connects to the file server (step S63) and acquires the file (step S64). The acquired file is cached in the cache area (step S65, step S6
6). If the connection with the file server fails, the file to be passed to the client cannot be obtained, and an error is notified to the client (step S68). All file acquisition requests from the client are recorded in the access history database 34 (step S69).

The cache management unit 31 periodically reads the prefetch selection module 311 to select a file to be prefetched, and creates a prefetch management table (step S).
7). The file to be pre-read is selected from those having a large number of references from the access history database (step S7).
1). With respect to the selected file, an update history is determined from the access history database 34 and analyzed to estimate a file update interval. The estimated update interval is set as a prefetch interval (step S72).

With respect to the selected file, the time during which the network is open from the wide area network monitoring device 4-1 is determined as update timing (step S73).
The prefetch management table 312 is updated based on the above result (step S74). The cache management unit 31 periodically refers to the prefetch management table 312 to determine a file that requires prefetch from the prefetch interval and the last read date and time.
A prefetch process is performed (step S8). The processing procedure is the same as the processing performed when the client reads a file in the pre-read area (similar to the processing from step S610).

[0025]

Next, an embodiment of the present invention will be described with reference to FIGS. 5 and 6 in addition to FIGS. 5 and 6
FIG. 5 is a diagram showing an embodiment of items of each mounted management table and their states.

File management tables 51-1 to 51-n
Then, the file name and size of the file to be managed and the update date and time are managed. The access history database 34 records the file server name where the cached file exists, the file name, the file size, the update date and time of the file recorded on the file server, and the access date and time of the client. Each time a client access request occurs (the file is cached, prefetched,
It is recorded in the access history database 34 (whether at any of the file servers).

The prefetch management table 312 includes a file name in the prefetch area, a file server name where the file exists, a file size, an update date and time when prefetching,
The last look-ahead date and time and the look-ahead interval determined by the look-ahead module are managed. The prefetch management table 312 is created by the prefetch election module referring to the access history database 34 and determining a file to be prefetched and its prefetch interval. The update date and time are read from the file server upon request from the client (see FIG. 3).
Are updated also in steps S615 and S620).

The cache management table 313 manages a file name in the cache area, a file server name where the file exists, a file size, and an update date and time. For explanation of the implementation example, provisional values are set for the sizes of the cache area and the prefetch area, and the items of each table.

Next, a file request from a client will be described with reference to FIG. When there is a read request for the file f1-1 from the client, the cache management unit 31 checks from the prefetch management table 312 whether the file f1-1 is in the prefetch area (Step S).
61). File f1-1 is a pre-read management table 312
Therefore, whether the file is the latest is confirmed from the file management table 51-1 of the file server 51 (steps S610 and S611). At this time, if the connection to the file server 51 cannot be established (failure in step S610), a warning is issued to the client (step S617), and the file f1-1 in the prefetch area is passed to the client (step S67). If the update date and time are the same as the file, the file f1-1 in the pre-read area is passed to the client, but the file f1-1 is acquired from the file server 51 (step S612). Since the read new file f1-1 has increased in size, it cannot fit in the prefetch area. Therefore, the file f2-1 having the lower priority is expelled from the prefetch management table 312 and moved to the cache area (step S616). The cache area processes the file f2-1 according to a normal cache management procedure (step S65). If the management is of the FIFO type, the file f1-3 is flushed and the file f2-1 is inserted. The access history database 34 contains the file f1-1
And the access date and time are recorded (step S6).
9). Each management table is changed as shown in FIG.

Next, the pre-read file selection process will be described with reference to FIG.
This will be described with reference to FIG. The prefetch selection module 311 is a file f that has more references than the access history database 34.
1-1, f2-2 and f1-2 are selected (step S7).
1). Next, comparing the past update date and time of each file and estimating the file update interval,
Day, 7 days (step S72). Also, from the data of the wide area network monitoring device 4-1, it was found that the wide area network can transfer files most efficiently at 1:00 am (step S73). Therefore, the prefetch management table is updated based on these pieces of information (step S7).
4). Updates are only made where they have changed. In this example, the file f1-1 is updated with a pre-reading interval of 6 days and the time of the last read date and time at 1:00 am, and the file f1-2 is updated.
Item is newly added. For the newly added item (here, file f1-2), the date is left blank in the last read date and time, so that it is always read ahead at 1:00 am. Each management table is updated as shown in FIG.

The cache management unit 31 periodically checks the look-ahead management table 312. If the date and time obtained by adding the last read date and time and the prefetch interval have passed, or if the date of the last read date and time is blank and the specified time has been reached, the prefetch process is performed. Here, for the file f1-2, prefetching is performed at 1:00 am The procedure is the same as the processing performed when the client reads a file in the pre-read area (similar to the processing from step S610).

[0032]

According to the present invention, since a frequently referenced file is pre-read in the cache server in advance, even if the transfer speed of the wide area network is low and the file server cannot be connected in some cases, the file can be promptly transmitted to the client. Can be transferred.

According to the present invention, since a cache server is provided for each department, the cache hit rate can be increased for each department.

[Brief description of the drawings]

FIG. 1 is a configuration diagram illustrating an embodiment of a wide-area distributed file system according to the present invention.

FIG. 2 is a flowchart of an access processing request from a client.

FIG. 3 is a flowchart showing a continuation of the flowchart of the access processing request from the client shown in FIG. 2;

FIG. 4 is a flowchart of a prefetch file selection process and a prefetch process.

FIG. 5 shows management table values used for explaining the embodiment.

FIG. 6 shows management table values used for explaining the embodiment.

FIG. 7 shows management table values used for explaining the embodiment.

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-10-320337 (JP, A) JP-A-11-15714 (JP, A) JP-A-10-214217 (JP, A) JP-A-10- 260889 (JP, A) 19960712, IPSJ Research Report Vol. 96 No. 63 IPSJ SIG Notes 96-DPS-77 96 63, Look-ahead target decision strategy in WWW look-ahead proxy server, Kenichi Chinen, Ei Yamaguchi, Nara Institute of Science and Technology (58) Fields surveyed (Int. Cl. ⁷ , DB name) G06F 12/00 545 G06F 12/00 533 G06F 12/00 514

Claims (4)

(57) [Claims] When a plurality of file servers storing a plurality of files, a client accessing a file stored in the file server, and a cache server storing the file are connected to a wide area network, The cache server includes a cache management unit, a read-ahead area for caching read-ahead files, and a target file requested to be accessed by a client.
Cache area used for caching Le, and is configured by including an access history database that records the history of the access request from the client, the cache
The cache management unit periodically executes the access history database.
Files that are accessed more frequently
File and select the file
The update history is determined from the history database, analyzed, and
Guess the update interval of the file, and
Interval, and set the read-ahead interval to
A wide-area distributed file system characterized by speeding up file access by periodically prefetching a high-degree file to the cache server. 2. A cache server according to claim 1, wherein said cache management unit periodically analyzes an access history database to determine a file to be read ahead and a read-ahead interval, and a look-ahead management table to record the determined file name and interval.
2. The wide-area distributed file system according to claim 1, further comprising a cache management table for managing the cache area. 3. The wide area network manages network congestion and fault information, and stores the prefetch selection module.
3. The wide area distributed file system according to claim 2, further comprising a network monitoring device for determining a time at which the file transfer can be performed most efficiently . 4. The wide area according to claim 1, wherein the plurality of file servers have a file area for storing files and a file management table for managing the file areas. Distributed file system.