JP3386964B2 - Gateway device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed file system in which a plurality of server computers distributed on a network and a plurality of client computers are interconnected by a communication line via a gateway device (computer), and more particularly, to a distributed file system. The present invention relates to a gateway device that relays files in a distributed file system when a plurality of client computers access file objects in a storage device inside a server computer via the gateway computer.

[0002]

2. Description of the Related Art In the following explanation, "server computer" means
A computer that provides some kind of service on the network, and a "client computer" is a computer that requests and receives the service of such a server computer. The "file object" is the network protocol used by the file system,
A set of network address (name of server computer), file name, and file entity.
Here, the "file object name" refers to a set of a network protocol, a network address, and a file name.

Conventionally, in the distributed file system as described above, a read request for a file object on a server computer from a plurality of client computers is
It was once being relayed by a gateway computer on the way. The file object read from the server computer by the gateway computer is relayed to the client computer. At the same time, those file objects are stored in a cache file (fixed disk, semiconductor memory, etc.) inside the gateway computer. An example of such a gateway computer is disclosed in Japanese Patent Application Laid-Open No. 4-313126 (the title of the invention "file input / output method of distributed file system").

With reference to FIG. 12, the system disclosed in this prior art has a gateway computer 110, a server computer 100 and a plurality of client computers 120 interconnected via the gateway computer 110.
including. The server computer 100 is shared by a plurality of client computers 120.

The server computer 100 stores a disk device 136 for storing files, a file input / output response means 134 for inputting / outputting the contents of files of the disk device 136 via a network, and block information in the disk device 136. Block information response means 132 for responding to the client computer 120 via the network
Including and

The client computer 120 sends a block information request to the block information responding means 132 of the server computer 100, receives block information from the block information requesting means 152, and a file of the server computer 100 via the gateway computer 110. Input / output response means 1
And a file input / output requesting means 154 for sending a file input / output request to and receiving the file.

The gateway computer 110 is interposed between the client computers 120 (plurality) and the server computer 100, and a disk device 136 in the server computer 100.
A disk cache 144 for storing the cached contents, a file input / output requesting means 154, and a file input / output responding means 1.
And a cache management unit 142 interposed between 34.

With reference to FIG. 12, this system operates as follows. First, the operation when the client computer 120 accesses a file in the disk device 136 in the server computer 100 and reads one block will be described. The block information requesting unit 150 issues a block information request message to the server computer 100 via the gateway computer 110 to request acquisition of block information regarding the block to be read.

In response to this message, the block information response means 132 in the server computer 100 retrieves the corresponding block information from the disk device 136, and sends a block information response message containing the block information to the client via the gateway computer 110. Calculator 120
It is returned to the block information requesting means 152 inside.

The file input / output requesting means 150 of the client computer 120 sends a file access request for requesting the reading of the block to be read to the cache managing means 142 in the gateway computer 110 based on the returned block information. Issue.

Upon receiving the file access request, the cache management means 142 compares the block information related to the file access request with the block information related to the read target block stored in the disk cache 144. And the disk cache 144
If the corresponding block information does not exist, or if the contents (update time) of both block information are different (when the cache of the block to be read is not valid), the cache management unit 142 causes the file access request described above. Is issued to the file input / output response means 134 of the server computer 100.

The file input / output response means 134 of the server computer 100 responds to this file access request by
The file in the disk device 136 is accessed, the corresponding block is read, and the block is transferred to the gateway computer 110.

The cache management means 142 stores this block in the disk cache 144, and sets or updates the block information related to the block in the disk cache 144. At the same time, the cache management means 142 transfers the block to the file input / output request means 154 of the client computer 120.

The operation of this system when there is no valid file in the disk cache 144 has been described above.

When there is an access request for the same file next time, the cache management means 142 of the gateway computer 110 fetches the block from the disk cache 144 and the file input / output request means of the client computer 120 which issued the access request. The block is transferred to 154. So in this case,
No transfer request is issued from the gateway computer 110 to the server computer 100. Therefore, the speed of relay access can be increased.

A similar technique is disclosed in Japanese Patent Laid-Open No. Sho 63-200244.
JP-A-63-20184 (Title of Invention: "Remote File Access System"). In any of the technologies, when an access request for a file object is issued, the modification time of the file object held by the server computer is stored in the cache file in order to check whether the content of the file object has been updated. Compare with the change time. When the contents of the cache file are old, the client computer reads the updated contents of the server computer. However, in the technologies described in these two publications, the cache file exists in the client computer, and therefore, JP-A-4-3131
Unlike the system disclosed in Japanese Patent Publication No. 26, the gateway computer is not included in the system.

However, these three prior art documents have a common feature in that the contents of the file object of the server computer and the cached file object do not conflict with each other.

In this way, the prevention of the discrepancy between the files that should originally be the same is called maintaining the integrity. Also, the data of the file in which the discrepancy occurs is called stale data.

The basis of the above-mentioned conventional technique uses an algorithm for checking the modification time of the corresponding file object of the server computer via the network and comparing it with the modification time of the content of the corresponding cache file.
Therefore, it is necessary to make an inquiry to the server computer via the network. If the effective transfer rate of the network is low, or if the load on the server computer having the corresponding file object is too high to respond immediately, such a query itself may take a considerable amount of time. Therefore, depending on the network file system, such a file cache system may not be adopted. That is, in such a system, although the file object held by the server computer is updated, the corresponding file object in the cache of the gateway computer is not necessarily updated. A typical example of such a network file system system is the H in the so-called Internet.
There is a wide-area distributed multimedia information system using TTP (HyperText Transfer Protocol).

The Internet is a global network using the TCP / IP protocol as its basic protocol. A globally distributed multimedia information provision system built on the Internet is World Wi
Called de Web (WWW). WWW can handle file objects distributed on the network.
These file objects include various types such as image data, audio data, video image data, etc., as well as simple text data. Since the WWW is attractive to both the information provider and the information user (user), the traffic related to WWW on the network is explosively increasing. In the WWW system, the user of the client computer only needs to execute browser software having a graphical user interface on the client computer, and the information composed of various file objects held by the server computers distributed on the network is successively displayed. Can be accessed. The WWW has recently been remarkably popular due to such ease of operation.

This WWW system uses TCP / I
A file object is transferred by the HTTP protocol built on the P protocol.

In implementing the HTTP protocol, it is widely practiced to relay and transfer a file object by a gateway computer as described in JP-A-4-313126. The data is cached in the gateway device. Also, in the WWW system, it is accepted that stale data is generated in relay transfer of a file object. That is, the case where the contents cached in the gateway computer do not match the file object of the server computer is allowed.

The ratio of the stale data in the cache file will be called the stale data ratio. Unless you update the contents of the cache file in any way, the stale data rate will increase. Generally, when the tail data ratio of the cache file increases, the user operating the client computer voluntarily judges that the contents are old,
Try to reload the file object from the server machine. In this case, a protocol that does not use the contents of the cache file is used. As a result, the cache file system in the gateway computer tends to be meaningless.

On the other hand, there is an advantage of the protocol that allows the generation of the stale data as described above. When the client computer relays access to the file object of the server computer to the gateway computer for access, and if the cache file of the gateway computer is hit, the file object is read from the cache file and transferred to the client. . It is not necessary to inquire the server device of the modification time of the file object. Therefore, the file object is fetched from the cache and returned to the client in a short time.

A network file protocol that accepts tail data, such as the HTTP protocol, has advantages for use in global wide area networks. That is, if the file object is cached in the cache of the gateway device, access to the server device does not occur. As a result, there is an advantage that the response time can be shortened.

A gateway computer that speeds up access to a file object on the network and reduces internet traffic as described above is also called a "proxy server device".

“Proxy” means “agent”. As its name implies, the proxy server device fulfills the function of accepting the access request on behalf of the server computer and relaying the transfer of the file object when the client computer accesses the file object of the server computer on the network. The concept of the proxy server device will be described with reference to FIG.

Referring to FIG. 13, the proxy server device 13
Is provided, for example, by intervening between an internal network 23 provided in a certain company and an external network 25 (Internet or the like) outside the company.
The internal network 23 includes a plurality of client computers 2
4, the external network 25 also includes a plurality of server computers 11. Each server computer 11
It holds the file object 12.

The proxy server device 13 receives the cache file 16 and the read request 19 from the client computer, and returns the file object as data 20 when the file object exists in the cache file 16, and the cache file 16 is returned. If there is no valid file object in the server computer 11 via the external network 25
A proxy process 14 that issues a read request 17 to the client computer, receives the corresponding file object as data 18, and returns it to the client computer 24;
And an access log 15 for recording transfer records of file objects by the process 14. proxy process 1
4 has a function of newly writing the file object in the cache file 16 when the file object is obtained from the server computer 11.

The proxy server device 13 operates as follows. First, the client computer 24 in the internal network 23 makes a read request 1 to the file object 12 of the server computer 11 on the external network 25.
9 is issued to the proxy process 14 in the proxy server device 13. The proxy process 14 uses this read request 1
In response to 9, the cache file 16 is accessed,
It is determined whether or not there is cache data of the file object in the cache file. If the file object is cached, the cache file expiration date unique to the proxy server device 13 is compared with the last modification time of the cached file object. If the cache is within the expiration date, the file object is read from the cache file (22),
The data 20 is returned to the client computer 24 of the internal network 23 that issued the read request 19.

If the file object does not exist in the cache file, or if it exists, but the expiration date has passed, it is determined that there is no valid file object in the cache file 16. In that case, the proxy process 14 issues a read request 17 for the file object 12 to the server computer 11 of the external network 25. In response to this, the server computer 11 sends the file object 12
Is returned to the proxy process 14 as data 18. pr
The oxy process 14 stores the file object name (that is, a combination of the network protocol name, the network address name of the server computer, and the file name) as the access log in the log file 15, and the substance of the data of the file object (that is, the file object Itself) and the writing time (26).

The proxy process 14 further transfers the data 20 to the client computer 24 in the internal network 23, and writes the file object in the cache file 16 (21).

Therefore, in the cache file 16, the file object name, the substance of the file object, and the time when the file object is acquired from the server device (the time of the last change) are recorded.

The proxy server is HTTPD or DeleGat
Software such as e is connected to the network-connected UN
It is generally realized by executing on a computer running IX.

FIG. 14 shows the physical configuration of the proxy server device. Referring to FIG. 14, the proxy server device is composed of a workstation 200 operating UNIX. The workstation 200 has a CPU (central processing unit) 20.
2, a memory 206 connected to the CPU 202 by an internal bus 204, and a file I / O (input / output)
A device 208 and a first network I / O interface 2 connected to an external network via a router 210
12 and a second network I / O interface 214 connected to the internal network. I / O device 2
In 08, cache file accumulation, access log,
Also, a file unit 216 used as a storage location for various work files is connected.

This workstation 200 has a CPU
The 202 implements the proxy server device 13 by executing the software such as HTTPD and DeleGate described above.

FIG. 15 shows another configuration example of the proxy server device. This proxy server device comprises a workstation 300. Workstation 300
The configuration is similar to that of the workstation 200 in FIG. 14, except that the second network I / O interface 214 in FIG. 14 is replaced with a serial port 302 connected to a modem device 304. . The workstation 300 is provided with a modem device 304.
And is connected to an internal network via a public telephone line network 306.

15, the same parts as those of FIG. 14 are designated by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated here.

FIG. 16 shows still another configuration example of the proxy server device. The proxy server device shown in FIG. 16 is also realized by the workstation 400. This workstation 400 has the same configuration as the workstation 200 shown in FIG. 14, but is connected to the internal network 23 instead of the first network I / O interface 212 and the second network I / O interface 214. The difference is that it has only one network I / O interface 402. The internal network 23 is connected to the external network 25 via the router 210, and is also connected to a plurality of client computers 24. Workstation 400
Can communicate with the server computer 11 of the external network 25 via the internal network 23 and the router 210, and can communicate with the user 31 via the internal network 23 and the client computer 24.

16 and 14, the same parts are given the same reference numerals and names. Their functions are also the same. Therefore, detailed description thereof will not be repeated here.

The proxy server device has a relay data cache mechanism as described above. I.e. proxy
The server device caches the data of the file object to be relayed in the cache file device existing therein.

An expiration date specific to the proxy server device is set in the cache file of the proxy server device. If an access request to the file object on the same server computer is received from the client computer within the valid period after the file object was written to the cache, the file object previously written to the cache file is fetched. Transfer to the client computer. This prevents access to the server computer.

The effect of the cache by such a gateway device will be specifically described below. A proxy server device is installed in a company organization and the external network is 64
The connection shall be made at a speed of kbps. Also this proxy
The server, on the other hand, shall be connected to a local area network inside the corporate organization at a speed of 10 Mbps.
The speed difference between the external and internal networks is more than one digit. Such an example is often seen, and a typical example thereof is the configuration shown in FIG.

At this time, the proxy server caches the relay file object, and the hit rate of the cache is Hitrate. Hitrate has 1 cache for access requests to file objects.
1 when a 00% hit, 0 when a 0% hit
Is an index.

It is assumed that the data reading speed from the cache file and the data writing speed to the cache file in the proxy server device are sufficiently higher than the network speed. That is, the time required to input / output the file object to / from the cache file can be ignored.

At this time, the average access speed Vave when the file object on the external network is accessed from the inside is expressed by the following equation 1.

[0047]

[Equation 1]

If the cache file does not exist, H
itrate is 0. Therefore average speed (kbps)
Is the external network access speed of 64 kb
is equal to ps. If Hitrate is 1, that is, if a 100% cache file is hit, the average speed (kbps) is equal to the speed of the internal network and is 10,000 kbps (= 10 Mbps). In other words, the higher the hit rate, the higher the average access speed seen from the client computer. Figure 1 shows a graph of Equation 1 showing the relationship between hit rate and average access speed.
7 shows.

The cache hit rate depends on the amount of cache files. That is, if a large number of file objects are stored in the cache file, the probability that the accessed file object will be included in the cache file is high and the hit rate is high. In order to show such a relationship, the relationship between the cache hit rate and the accumulated amount of cache files is shown as a graph in FIG. As shown in FIG.
If the cache file size is small, the cache hit rate is close to 0, and if the cache file size is increased, the cache hit rate gradually rises.

In the proxy process, as a method of maintaining the contents of the cache file, a control method of invalidating a file object after a certain period has elapsed since the writing by the first file access occurred. This fixed period is called the expiration date. For this expiration date, the person who manages the proxy server device sets an appropriate value according to the actual situation. The longer the validity period, the larger the amount of data stored in the cache file. As a result, the cache hit rate increases. That is, in order to increase the hit rate of the cache, it is sufficient to set the expiration date of the cache longer.

Since the storage capacity of the file device for storing the cache file is finite, the expiration date cannot be extended indefinitely. The cache file whose expiration date has passed is deleted at some timing.
In this specification, the time until the cache file is erased is called the erase time.

According to the experiments by the inventor of the present application, the cache expiration date and the file amount accumulated in the cache file are substantially proportional to each other. This relationship is shown in FIG. This is because the amount of access from the internal network to the external network is almost constant regardless of the day. Therefore, the following relationship holds.

[0053]

[Equation 2]

From this relationship, as shown in FIG. 17, it can be seen that the cache hit rate increases as the cache expiration date increases.

The relationship between the cache expiration date and the amount of file objects differs depending on the frequency at which the internal user who uses the client computer accesses the file object of the server computer on the external network. Although the cache expiration date and the cache file storage amount are almost proportional, the proportional coefficient is the network speed,
Internal client Varies depending on the usage environment such as the number of computer users

In the system related to the inventor of the present application, the cache hit rate is about 12% when the cache file is valid for about one day. Expiration date is 14
It has been experienced that the cache hit rate will be about 41% daily. It has been observed that when the expiration date is 14 days, the file object is accumulated in a cache file of about 700 MB.

[0057]

In the cache file in the proxy server device, in order to increase the cache hit rate as much as possible, it is effective to set the expiration date as long as possible and store the data as long as the capacity of the cache file allows. . However, on the other hand, the contents of the file object of the server computer on the external network may be modified or changed. If the expiration date of the cache file is long, the user of the client computer on the internal network will read the old contents accumulated in the cache file even though the file object of the server computer has been changed during that period. You will end up with something.

Consider a case where a file object such as a weather map or an electronic newspaper is provided on a server computer. Many of these file objects have the same name, but information is updated every day due to the nature of the data. For example, considering that the cache expiration date is 7 days, the information such as weather maps and electronic newspapers in the cache becomes old with a probability of 6/7 or more, and eventually becomes useless information.

The ratio of the contents of the cache file in which the original data existing in the external network is changed and does not match the contents of the cache is called a stale data ratio. That is, if the cache expiry date is increased, the cache storage size is increased and the cache hit rate is increased, but as a result of the extended expiry date, the stale data rate of the cache content also increases.

Therefore, in the prior art, keeping the freshness of the contents of the file object in the cache file and keeping the cache hit rate high are conflicting goals.

As described above, generally, when the stale data ratio of the cache file increases, the user of the client computer voluntarily determines that the contents should be old and reloads the file object data from the server computer. Become. For this purpose, a protocol is specified that actually loads from the server computer without referring to the cache file. As a result, the cache tends to be meaningless.

Such an increase in the stale data rate is not desirable, and therefore the validity period of the cache cannot be set too long. Usually 24 hours or less. However, if the expiration date is set to this extent, the file objects are not sufficiently accumulated in the cache file and the cache hit rate does not increase. As a result, there is a problem that the average file object transfer rate cannot be sufficiently increased. In addition, it is preferable to minimize the adverse effect on other normal processes in increasing the cache hit rate. further,
Even when there are many other normal processes, it is preferable that the work for improving the cache hit rate can always be performed, and it is desirable that the work for that purpose itself be performed as efficiently as possible.

Therefore, an object of the present invention as set forth in claim 1 is to keep the contents of the cache file fresh and
It is also to improve the cache hit rate.

An object of the present invention is to keep the contents of the cache file fresh, improve the cache hit rate, and shorten the time required for prefetching.

An object of the present invention is to keep the contents of the cache file fresh and improve the cache hit rate without adversely affecting the normal use of the user.

An object of the present invention is to constantly keep the contents of the cache file fresh and improve the cache hit rate.

An object of the present invention is to keep the contents of the cache file fresh and improve the cache hit rate without generating unnecessary traffic.

The object of the invention as defined in claim 11 is to steadily carry out the work for keeping the contents of the cache file fresh and improving the cache hit rate while minimizing the adverse effect on the normal use of the user. It is to provide a gateway device that can be performed.

It is an object of the invention as set forth in claims 12 to 16 to keep the contents of the cache file fresh and to improve the cache hit rate, both efficiently and efficiently. Is to provide.

[0070]

According to a first aspect of the present invention, a gateway device is provided so as to be interposed between a first network in which a client computer exists and a second network in which a server computer exists. In response to the access request for the file object from the client computer of the first network,
It includes first network file relay means for making an access request to the file object to the server computer designated by the file object. The first network file relay means temporarily stores the acquired file object until a predetermined expiration date elapses, cache file means for recording the last modification time in file object units, and File transfer recording means for accumulating transfer records of file objects made within a fixed period, and referring to the last modification time in response to an access request for a file object, a file object within the expiration date exists in the cache file means. Transfer means for determining whether or not to perform access, selectively accessing the cache file means or the server computer designated by the file object based on the determination result to acquire the file object and transfer it to the client computer Including and The gateway device further prefetches the file object in the transfer record from the server computer according to a predetermined schedule,
Second network file relay means for storing in the cache file means.

The file object transferred within a fixed period is prefetched by the second network file relay means according to a predetermined schedule even if there is no access request. Not only the valid one but also the expired one are prefetched and newly stored in the cache file means and become valid. When an access request is actually made, there is a high possibility that the file object that is within the expiration date will exist in the cache file. Further, even if the expiration date is lengthened, the file object in the cache file means can be matched with the file object of the server computer with a high probability because it is updated to the latest one at the time of prefetch. Therefore, it is possible to keep the contents of the cache file fresh and improve the cache hit rate.

A gateway device according to a second aspect is the gateway device according to the first aspect, wherein the second network file relay means simultaneously transmits a plurality of file objects from the server computer via the second network. Prefetch in parallel.

Since prefetching is performed simultaneously in parallel, the time required for prefetching can be shortened as compared with the case where sequential prefetching is performed.

A gateway device according to a third aspect of the present invention is the gateway device according to the second aspect, wherein the second network file relay means avoids simultaneous occurrence of prefetches from the same server computer. Determines the prefetch order for.

In transfer recording, access requests to the same server computer are often concentrated. If access requests for prefetching are concentrated on the same server computer at one time, the load on the server computer and the load on the communication path take a long time to transfer the file object. Therefore, by determining the prefetch order so as to avoid simultaneous occurrence of prefetches from the same server computer, the load is not concentrated on such a specific server computer, and the prefetch speed can be improved.

A gateway device according to a fourth aspect is the gateway device according to the second aspect, wherein the second network file relay means extracts a list of file objects from the transfer record according to the appearance frequency in the transfer record. , The file objects are prefetched simultaneously in parallel according to the order of the file objects in the list.

When prefetching a file object, a file object having a high appearance frequency in transfer recording is preferentially prefetched. Access requests to such file objects are expected to show a high rate in the future, so an improvement in cache hit rate can be expected.

According to a fifth aspect of the present invention, in the gateway apparatus according to the fourth aspect, the second network file relay means changes the order of the file objects in the list, and the file objects according to the changed order. Prefetches are performed in parallel at the same time.

During transfer recording, access requests to the same server computer are often concentrated. If the access for prefetching is concentrated on the same server computer at one time, the load on the server computer and the load on the communication path take a long time to transfer the file object. By changing the order of the file objects in the list of file objects extracted from the transfer record and prefetching in that order, it becomes possible to avoid simultaneous prefetching from the same server computer. Therefore, the load is not concentrated on a specific server computer, and the prefetch speed can be improved.

A gateway device according to a sixth aspect is the gateway device according to the fifth aspect, wherein the second network file relay means divides the list into a plurality of blocks, and a file object order for each block. , And prefetch file objects simultaneously in parallel according to the order on the list in which the order is changed.

By performing such a replacement, it is possible to avoid the intensive generation of access requests to the same server computer, and the order of frequency of occurrence in the list of file objects is maintained at the granularity of the block size. Be done. Therefore,
Prefetching of file objects that appear frequently in the list can be performed without inviting access requests to a specific server computer.

The gateway device according to claim 7 is the gateway device according to any one of claims 2 to 6, wherein the first network file relay means is a file from a client computer of the first network. It is possible to activate a plurality of first transfer processes for processing an access request of an object in parallel at the same time. The second network file relay means is capable of simultaneously activating a plurality of second transfer processes for prefetching one file object in parallel, and the number of the first transfer processes. Is detected and the new activation of the second transfer process is controlled so that the sum of the number of the first transfer processes and the number of the second transfer processes is equal to or less than a predetermined upper limit.

Since the activation of the second process can be suppressed when a large number of the first processes are occurring, and the large number of the second processes can be activated when a small number of the first processes are occurring. The access request of the file object by a normal user is not adversely affected by the prefetch. In addition, since the prefetch can be performed at any time when there are few access requests from normal users, the freshness of the cache can be maintained.

The gateway device according to claim 8 is the gateway device according to any one of claims 1 to 7, wherein the second network file relay means prefetches the file object only in a predetermined time zone. Do.

Prefetching can be performed during a time period when the access request by the user is very small to keep the cache file fresh, and prefetching can be prevented during the time period when the access request by the user occurs. File users' access requests for file objects are not adversely affected by prefetching.

The gateway device according to claim 9 is the gateway device according to any one of claims 1 to 8, wherein the second network file relay means validates the cache file of the first network file relay means. File objects are prefetched periodically with a period less than the deadline.

Since the file object of the cache file is prefetched periodically in a cycle less than the expiration date of the cache file, the file object in the cache file is always kept within the expiration date. As a result, the effect of prefetching can be maintained for the entire period between prefetches.

The gateway device according to claim 10 is the gateway device according to claim 1, wherein the cache file means of the first network file relay means is provided until a predetermined erasing time longer than the expiration date has passed. ,
Do not delete the file object stored in the cache file means. The transfer means accesses the corresponding server computer in response to the access request for the file object whose expiration date has passed but has not been deleted, and the file object in the server computer is updated after the previous acquisition. It is determined whether or not it has been done. When it has not been updated, the file object in the cache file means is transferred to the client computer as a newly acquired file object. When the file object is updated, a transfer request for the file object is issued to the server computer specified by the file object, the updated file object is acquired, and the acquired file object is transferred to the client computer.

Even a file object whose expiration date has passed is not deleted before the deletion time. When an access request for such a file object occurs, the file object that has not been updated on the server computer even if it has expired,
It can be obtained from the cache file without directly obtaining from the server computer. Therefore, generation of useless traffic can be suppressed. In addition, if there is such an access request and the file object has not been updated in the server computer, the same effect as prefetching can be obtained without actually reacquiring the file object from the server computer.

The gateway device according to the invention of claim 11 is the gateway device according to any one of claims 2 to 6, wherein the first network file relay means is a client computer of the first network. It is possible to simultaneously activate a plurality of first transfer processes for processing the access request of the file object from the. The second network file relay means can simultaneously activate a plurality of second transfer processes for prefetching one file object in parallel. The second network file relay means detects the number of transfer processes already in operation at the time of starting the second transfer process, and detects the number of transfer processes in operation and the second transfer process to be started. The new activation of the second transfer process is controlled so that the sum of the number of transfer processes is equal to or less than the sum of the number of transfer processes in operation and a predetermined constant.

Since the activation of the second process can be suppressed when a large number of the first processes are occurring, and a large number of the second processes can be activated when a slight amount of the first process does not occur. File users' access requests for file objects are not adversely affected by prefetching. In addition, since the prefetch can be performed at any time when the demand from the normal user is small, the freshness of the cache can be maintained. Further, even if the number of transfer processes in operation reaches the upper limit, the second transfer processes up to a predetermined constant number can be activated, so that prefetching can be steadily performed. Therefore, the freshness of the cache can be maintained. Moreover, since it is not necessary to reduce the capacity of the cache file means, the cache hit rate is also improved.

A gateway device according to the invention described in claim 12 is the gateway device according to claim 1,
The second network file relay means stores the transfer record of the file object made within a certain period in the past together with the information indicating whether the change of the file object is detected or not. Further includes. The second network file relay means extracts from the transfer records recorded at the time of prefetching the file objects by the second file transfer recording means, the file objects for which a change has been detected at the time of the immediately preceding prefetch, and extracts these file objects. Is prefetched from the server computer and stored in the cache file means with priority over the file object being transferred and recorded by the file transfer recording means.

The file object whose change has been detected at the previous fetch is prefetched with priority over the other file objects. It is considered that such a file object is frequently changed after that. Therefore, in addition to the effect of the invention described in claim 1, in order to keep the contents of the cache file fresh, a file object that is likely to have changed since the last fetch is preferentially prefetched. , Prefetch can be performed efficiently.

The gateway device according to the invention described in claim 13 is the gateway device according to claim 12, wherein the second network file relay means stores a character string pattern designated in advance, A file object having a character string that matches the predetermined character string pattern is excluded and extracted from the transfer records recorded by the file transfer recording means.

Among the file objects, there is also a file object in which prefetching is meaningless or the efficiency of the entire system is rather lowered because of prefetching. Some of such file objects can be known in advance as containing a specific character string. Therefore, such a specific character string is designated in advance and stored as a character string pattern in the second network file relay means, and a file object having a character string that matches this character string pattern is excluded and extracted, and the rest is left. Prefetch only the file object of.

By doing so, more useful data can be prefetched to the cache file means by prefetching. Therefore, in addition to the effect of the invention described in claim 12, the contents of the cache file are kept fresh. And improving the cache hit rate can be efficiently achieved.

In the gateway device according to the invention described in claim 14, in addition to the configuration of the invention described in claim 12, the transfer record accumulated by the second file transfer recording means is a file of the acquired file object. Including size. The second network file relay means stores in advance information specifying the maximum value of the size of the file object to be transferred, and selects the maximum value stored from the transfer records recorded by the second file transfer recording means. Exclude and extract file objects with large file sizes.

Prefetching a file object having a large data size takes time. Usually, the prefetch process is often performed in a limited time zone when there are few users. If it is attempted to complete the prefetch of as many file objects as possible within this limited time period, it is desirable to avoid prefetching large file objects. In some cases, within a finite prefetch time, prefetching of a large size file object may not complete, and as a result prefetching may not even occur at all. Therefore, by avoiding prefetching a file object with a large data size, a large number of file objects can be prefetched into the file cache means within a finite prefetch time, and the effective amount of data stored in the cache file means can be saved. You can increase. As a result, keeping the contents of the cache file fresh and improving the cache hit rate are
It is done efficiently.

The gateway device according to the invention described in claim 15 is the gateway device according to claim 12, wherein the transfer record accumulated by the second file transfer recording means relates to the file object for which the acquisition is attempted. It also contains the status code of the server computer. The second network file relay means pre-stores a specific status code of the server computer, and a file having a status code that matches the specific status code from the transfer records recorded by the second file transfer recording means. Exclude and extract objects.

When the server computer is in an abnormal state in which file transmission cannot be performed, it is useless to request the server computer to transmit a file object. If the file object is prefetched to another valid server computer without performing such wasteful file access, more valid data can be stored in the file cache means within a finite time. . As a result, the contents of the cache file can be kept fresh and the cache hit rate can be improved efficiently.

A gateway device according to the invention described in claim 16 is the gateway device according to any one of claims 1 to 15, for realizing a function as a client computer for the first network file relay means. It further includes the means of.

By giving the gateway device a function as a client computer, an access request for a file to be prefetched can be issued to the gateway device through the function as a client computer. Therefore, the amount of file objects to be prefetched can be further increased, and the contents of the cache file can be kept fresh and the cache hit rate can be improved together and efficiently.

[0103]

DETAILED DESCRIPTION OF THE INVENTION

[First Embodiment] A gateway device (referred to as a proxy server device) to which the present invention is applied can be easily realized by a computer running a UNIX OS equipped with a network interface, with reference to the following description and drawings. is there. An example of the configuration of such a proxy server device is as shown in FIG. Since the proxy server device of FIG. 16 has already been described, detailed description of its configuration will not be repeated here.

Normally, the user 31 uses the client computer 24 to execute the proxy server device (prox in FIG. 16).
For the server device 400), the network 2 is used to acquire the file object 12 of the server computer 11.
Request via 3 File object 12
Is composed of a set of the protocol name, the network address name of the server computer, the name of the file object, and the substance of the file data, as described above.

FIG. 1 shows the system configuration of the device 53 according to the first embodiment of the proxy server device according to the present invention. With reference to FIG. 1, the proxy server device 53 includes a first proxy process 14 connected to the internal network 23 and the external network 25, a cache file 16 used by the first proxy process 14, and a first proxy process.
The access log 15 in which the y process 14 records the name of the file object as an access log (transfer record)
And the past access log 1 of the first proxy process 14
The prefetch process 36 that generates a request for prefetching the file object listed in the access list 30 by reading 0 from the server computer 11 of the external network 25 and the prefetch process 36 In response to the prefetch request, obtain the corresponding file object from the server 11 of the external network 25,
A second pr to keep the contents of the cache file up to date by re-storing it in the cache file 16.
and an oxy process 34.

The first proxy process 14 and the second prox
Both the y process 34 and the prefetch process 36 are stored as programs in the memory 206 shown in FIG. 16, for example, and are realized by the CPU 200 executing them.

The first proxy process 14 corresponds to the transfer means described in the claims, and the cache file 1
6 corresponds to cache file means, access log 1
Reference numeral 5 corresponds to the file transfer recording means. Also, the prefetch process 36, the access list 30, and the second prox
The y process 34 constitutes a second network file relay means.

The second proxy process 30, which is unique to the gateway device according to the present invention, is activated under the following conditions.

(1) The second proxy process 30 has the same cache file 1 as the first proxy process 14.
Use 6

(2) The cache expiration date of the second proxy process 34 is set to 0. Ie the second prox
Access to the server computer via the y process will result in a cache miss-hit state and must be external network 2
5, the file object is acquired and stored in the cache file 16.

The operation of the proxy server device 53 shown in FIG. 1 will be described below. It is assumed that the client computer 24 in the internal network 23 issues a network access request 19 to this proxy server device 53.
The first proxy process 14 receives this network access request 19, and first, the I / O interface 202
The cache file 16 existing in the file device 216 is accessed via (see FIGS. 15, 16 and 17) (21). Also, the first proxy process 14 uses this network access request and cache file 16
It is compared with the file object name in (22) to determine whether or not they match. If there is a match, the first proxy process 14 further extracts the last modification time in the cache file 16 of the file object (22) and compares it with the current time. If the current time is within the valid period starting from the last modification time of the file object, the file object is extracted from the cache file 16 (22) and the network I / O interface 402 (see FIG. 16) is set. The file object is returned to the client computer 24 via (20).

If a copy of the corresponding file object does not exist in the cache file 16, or if it exists, but the cache expiration date has expired, the first proxy process 14 uses the network I
The file object transfer request 17 is sent to the server computer 11 of the external network 25 via the / O interface 402 (see FIG. 16). Server 11 at this time
Is specified by the network address name in the file object.

The relevant server 11 returns the requested file object to the first proxy process 14 (18). That is, this file object is the first
Obtained by the proxy process 14.

The first proxy process 14 writes this file object in the cache file 16 together with its last modification time (21), and records the name of this file object in the access log 15 (26).

On the other hand, the proxy server device 53 of the present invention
Differs from the conventional proxy server device in that the prefetch process 36 and the second proxy process 30 are run by the CPU to prefetch the file object.

Prefetch process 36 and second pr
The oxy process 34 operates as follows. The prefetch process 36 reads the past access log 15 of the first proxy process 14 (27) and creates the file access list 30 in the file device (28). The prefetch process 36 further includes this access list 30.
Read the file object posted in (29),
Second access request for the file object
(31) to the proxy process 34 of. Second
The proxy process 34 gives this access request to the server 11 of the external network 25 (32) and acquires the file object (33). The acquired file object is written in the cache file 16 together with its last modification time. That is, this causes the file object in the cache file 16 to be rewritten to the latest version even if there is no access request from the client computer 24. As mentioned above, the second
Since the cache expiration date of the proxy process 34 is set to 0, when the access request 31 is passed from the prefetch process 36 to the second proxy process 34, the access request for the file object is always issued by the second proxy process 34. External network 20
Given to.

When the second proxy process 34 is activated in this way, the following two effects can be obtained.

(1) In the first proxy process 14,
The expiration date of the cache file 16 is set to a certain value. This expiration date shall be M hours. If the prefetch process 36 is the second proxy process 34,
If an access request for prefetching the file object is given to the first proxy process 14 instead, the probability of hitting the cache file 16 is high. In this case, the original file object in the external network 25 is not accessed and prefetching is not effectively performed.

On the other hand, as described above, the second proxy process 3
No. 4 has its cache expiration date set to zero.
Therefore, when an access request is given to the second proxy process, a cache miss always occurs and the file object is accessed to the external network 25 (32). In this way, the file object is acquired from the server (33), and the acquired file object is written in the cache file 16 (35), so that the file object in the cache file 16 is maintained in the latest state.

(2) The first proxy process 14 leaves the access log 15. The prefetch process 36 creates an access list 30 in which the file object name to be prefetched is extracted from this access log 15.

If the prefetch process 36 is the first pr
If an access request for prefetching a file object in the server computer is given to the oxy process 14, the resulting access log will be recorded in the access log 15. Prefetch process 36
Creates a list 30 of file objects to be accessed based on this access log 15 in the next prefetch operation, so that the same file object will appear multiple times in the access list 30 created as a result. It will be. Such access list 3
If the prefetch is performed by 0, it may not be said that the tendency of the file object access request from the client computer 24 is reflected.
By performing the prefetch using the second proxy process 34, the access log obtained by the prefetch is not written in the access log 15. Therefore, the access list 30 accurately reflects the tendency of access requests for file objects from the client computer 24.

When creating the access list 30, the prefetch process 36 extracts from the access log 15 the file object name of a certain past period (this is n days). However, n × 24 (time) is the first
The cache expiration time of the proxy process 14 is greater than M hours.

As already described with reference to FIG. 14, normally, the shorter the cache expiration date, the smaller the capacity accumulated in the cache file, and the larger the capacity, the larger the capacity.
The hit rate increases as the number of file objects accumulated in the cache file increases.
It explained with reference to. However, I have already explained that increasing the cache file size also increases the stale data rate.

In the gateway device of the present invention, the first
The cache expiration date of the proxy process 14 is M hours. However, the file object targeted by the prefetch process 36 is n days (n × 24>
It is extracted from the access log of M). Therefore, the prefetch process 36 and the second proxy process 3
The number of file objects stored in the cache file 16 by 4 is n × 24 hours. That is, since the amount of data of the stored file object is sufficiently large, the cache hit rate becomes high.
On the other hand, since the cache expiration date of the first proxy process 14 is kept at M hours, there is no possibility that the stale data rate will increase if prefetching is performed at an appropriate time.

As an example to which the present invention can be applied very effectively, there is a WWW system on the Internet. WWW
The procedure for performing the prefetch process of the present invention in the proxy server device on the system will be described below.

In the WWW system, the file object names distributed on the network are Uniform Resource L
It is expressed and specified in a format called ocator (URL). An example of the URL is shown below.

[0127]

[Equation 3]

In Expression 3, "http" indicates the protocol to be used. "Www.xxx.co.jp" is on the network
It indicates the address of the TTP server computer and is selected so that no two are the same on the network. "/Test/index.html" shows the file name in the server computer.

When using the DeleGate generally used as the first proxy process, an example of the access log is as follows: internal network client computer name, time, "HTTP protocol (file acquisition request)"", And so on.

[0130]

[Equation 4]

The access log shown in Expression 4 corresponds to a log when the applicant accesses an information page published on the Internet as shown in FIG. Since the page shown in FIG. 2 is composed of text and 6 pieces of graphic data, if this page is accessed once, 7 access logs including access to 6 pieces of graphic data are formed. To be done. That is, in the above log, the first line is an access request for the text of this information page, and the remaining six lines are the graphic data (sharpcolor.gif) of the logo of the applicant's company pasted on the cover, the applicant company. The logo of the division called the Information Systems Business Division (isg.gif) and the graphic data of the four products of the applicant company (Zauru
s.gif, Shoin.gif, Prostation.gif, S2.gif).

The HTTP protocol is composed of a command character string (GET, etc.), a URL, and a protocol version ("HTTP / 1.0", etc.). The following is an example.

[0133]

[Equation 5]

The HTTP protocol is described in References 3 and 4 of the references listed in the last table. The implementation procedure of the prefetch process will be described below, and the preconditions therefor are (1) and (2) below.

(1) The first proxy process sets the cache expiration date M (time) shorter than the cache log collection period n days. For example M = 24 hours, n = 7
The day

(2) The second proxy process is activated under the two conditions that the cache file is shared with that of the first proxy process and the cache expiration date is set to 0.

Under these conditions, the prefetch process of the procedure described below is activated at a fixed time every day.
To automatically start the process at a certain time every day, if the gateway device operates under UNIX OS, run UNIX
Use the timer function (cron) included in the OS.
This is a well-known feature associated with the UNIX OS, so its details are not discussed here. Similar functions are often provided by OSs other than UNIX OS.

FIG. 3 shows the control flow of the prefetch process according to the present invention. A program for realizing this process is arranged in the memory 206 of FIG. 16, for example, and is executed by the CPU 202. The network I / O interface 4 is used for input / output of this process and the network and file device 216, respectively.
02 and the I / O interface 202 are used.

Referring to FIG. 3, first in step 1, the first
Collect the access log files of the proxy process for n days. The specific procedure for this collection is as follows.

The access log file name of the first proxy process can be set by the first proxy process so that the name changes every day. For example, the date can be added at the end of the log file name. This will produce the following log file:

[0141]

[Table 1]

In order to collect the latest n days of log files, it is convenient to sort the log file name list in descending order of date. Under the UNIX OS, you can run the / bin / ls command with the -t option to sort the list of names for such files, from newest to oldest. An example of such a command and its result is shown below.

[0143]

[Table 2]

From these file lists, the log file names for the latest seven days, for example, will be extracted. In this case, / usr / ucb / h, which is standard on UNIX
The top 7 lines can be extracted using the ead command. hea
The d command outputs the specified number of lines from the standard input to the standard output. So, the standard output of the ls -t command is head
If the standard input of the command is used, the log file names for the last 7 days can be extracted. To do this, connect the ls -t command and the head command using "|"
You can pass it to. In this way, connecting standard input / output using "|" is commonly used in UNIX. If you want to continuously process such data under an OS that cannot be connected to such standard input / output, you need to use another method. Will be equipped with.

The command after the above connection and the result thereof are shown below.

[0146]

[Table 3]

Next, in step 2 of FIG. 3, a prefetch access list is created. This access list shall be put together in one file "workfile.txt". This file is a record of past access file objects line by line. Therefore, in order to create this file, the access log including the file object name to be prefetched is selected under the following conditions.

(1) The HTTP command character string is a file object read request (GET), and (2) the protocol part of the file object name is
It is http, and (3) the file object name is suitable for prefetching (exclude those not suitable for prefetching).

The access log is extracted and a list is created using the above conditions. Further, only the field of the file object name is extracted from this list to create a prefetch access list consisting of the file object names to be prefetched.

The above-mentioned condition (2) is for extracting only the file object name that is meaningful for prefetching. As described above, DeleGate software is used in the present embodiment as the first and second proxy processes. The http software is the only protocol used by this proxy software to generate cache files. Therefore, it is only the access log that complies with such http protocol that is meaningful for prefetching.
Other protocols are meaningless because they are not subject to cache processing even if they are prefetched.

As shown in the condition (3), file object names that are not suitable for prefetching include those that include "?" In the URL. ? URL notation including
It is used when a client user manually inputs a character string to the server computer. Therefore, prefetching a file object having a URL notation including this character may lead to unexpected results. To perform the process of step 2 under UNIX OS,
The following commands provided with S may be used.

[0152]

[Table 4]

For example, by inputting the following command, the file that has undergone the processing of step 2 is the file "work".
It is output as a list in "file.txt".

[0154]

[Table 5]

The progress of this period will be described below. “Workfi
The format of the access log in "le.txt" is as described above.

[0156]

[Table 6]

Therefore, by outputting "workfile.txt" as standard output and inputting it into grep -v "\?" ,? Extract lines that do not include and output to standard output. It is further used as an input to grep "\" GET ", whereby only the GET command is extracted and output to the standard output.

Further, considering that a space is a delimiter between fields, the seventh field in each line of “workfile.txt” corresponds to the file object name, so aw
Only this file object name part is extracted using the k command. Furthermore, in order to extract only the line in which the protocol part of this file object name part is http, it is filtered by grep "^ http: //". Note that among the command parameters, ^ indicates the beginning of the line.

As described above, the prefetch access list output to workfile2.txt in step 2 is an access record from a plurality of client computers in the internal network to the server computer. Therefore, the same file object name may appear in this list many times. Therefore, in step 3 of FIG. 3, the URL list is sorted in the order of appearance frequency of file object names,
In addition, the same URL is grouped together so as not to overlap.

In UNIX, such sort work is performed by the OS.
You can do this with the following commands provided with.

[0161]

[Table 7]

Specifically, the command is as follows.

[0163]

[Table 8]

That is, the file which is only URL in workfile2.txt is output to the standard output, and this is used as the standard input of sort, and the file object names are arranged in alphabetical order. The following is an example of workfile2.txt.

[0165]

[Table 9]

Workfile2.txt | The result of sort is in alphabetical order (ASCII code order) as follows.

[0167]

[Table 10]

Further, when this output is input to the standard input of uniq -c, lines having the same file object name are aggregated, and the occurrence frequency is added as the first field to output each record. Therefore, the output is as follows. Like

[0169]

[Table 11]

Considering these first fields as numerical values, the fields are sorted in descending order. For that, sort -n -r
The results in Table 11 are given to the standard input of. As a result, each record is arranged in the order of appearance frequency, and the result is as follows.

[0171]

[Table 12]

That is, the result of executing the command in Table 8
The file workfile3.txt is a prefetch access list with one file object name per line in the order of frequency of occurrence.

Further, referring to FIG. 3, in step 4, it is checked whether or not one row of prefetch access list 25 can be read. If there is no row to be read, this is the end of the prefetch access list, and this processing ends. If there is a row to read, go to step 5.

In step 5, the second proxy process 34 is used based on the read file object name in one line to start a child process for network access to the server computer. This child process is called a prefetch child process. Waiting for the end of this child process, the process is returned to step 4, and a prefetch process loop for repeating the processes of steps 4 and 5 is formed.

Although not shown in FIG. 3, the prefetch process loop generated by the processing of steps 1 to 5 is forced to end at the designated time.
Set the S timer. The OS timer can be performed by designating a designated time operation using the at command that is provided as standard in UNIX OS. More specifically,
The next command is given to the OS using the process ID number of the prefetch process in steps 1 to 5 described above.

[0176]

[Table 13]

With this command, it is possible to make a reservation with respect to the OS so as to issue a kill command to terminate the process at 8:00.

As a method of implementing the network access unit, the HTTP protocol may be directly generated from the prefetch process. However, if a WWW client program such as lynx (see Reference 6) that is generally used in UNIX is used, prefetch child process operation according to the HTTP protocol can be realized without newly creating a program that generates the HTTP protocol. The method will be described below.

Since lynx is a program that runs on the UNIX OS, the UNIX notation will be used in the following description.

In lynx, if the following command is specified,
The specified URL can be read and written to a file called temp_file.

[0181]

[Table 14]

Furthermore, access via the proxy process is also
It can be used from lynx by setting the environment variable http_proxy of UNIX OS. If the proxy process is proxy software such as DeleGate, U
By specifying proxy in RL notation, access via proxy is possible. See reference 12 for this.

The case where the latter method is adopted will be described below. proxy process has network address pr
If it is a process using port 10001 of TCP / IP of gateway computer which is oxyserver, file object of server computer via this proxy process http://www.xxx.co.jp/test/index .html
The following command in Table 15 may be specified to access the.

[0184]

[Table 15]

Therefore, when carrying out the present invention, the proxy process may be similarly set to the port of the second proxy process.

The object of the present invention is to increase the number of cache files managed by the first proxy process by n × 24 / M times and keep the latest state by prefetching such a file object (URL).
Since temp_file itself is not used, this file can be discarded.

In step 1 of FIG. 3, n days worth of access log files were collected. This number n may be determined from the maximum capacity of the cache file and the file transfer record. For example, if 100 MB of cache data is accumulated in a 1 GB cache file device per day, then n may be set to about 5. As a result, the prefetch capacity is about 500 MB, and the cache file device will not overflow. In this way, the number n that does not cause the cache file to overflow can be easily calculated by considering the accumulation record of the cache file.

The prefetch process as described above can be activated from the internal network to the external network during a time period when the client computer user does not access the network frequently. By doing so, it is possible to operate so that the file object access by the prefetch process does not prevent the internal network user from accessing the external network.

In the above-described first embodiment, the prefetch process is started from 21:00 when the use of the client computer is low and finished at the latest by 8:00 the next day. In this case, 8:00 the next day
If the prefetch process is still running for a minute, it will now be killed. By starting the prefetch process from one time and forcibly ending it at another time in this way, even if the prefetch list is large and prefetch does not end by the specified time, the usage of the client computer will increase. The prefetch process is killed during the start time. This does not prevent the client computer from using the proxy server device.

In the first embodiment, the cache expiration date M of the first proxy process is set to 24 hours. Therefore, the contents of the cache data updated by the prefetch process at night are 21
Until time, it is at least effective.

The following effects can be obtained by using the proxy server device described above. The contents of the cache file are information that has been updated to the user within the last 24 hours. On the other hand, such information is held for the file object for which access has been requested in the past n days. That is, cache data for n days is stored in the cache file. It has already been described with reference to FIG. 13 that the cache hit rate increases as the amount of cached data increases. It is empirically known that the cache hit rate becomes three times or more when the cache file accumulation period is set to 24 hours and prefetch is performed using the access logs for the past n days.

If the above-mentioned prefetch is performed at a time interval smaller than or equal to the cache expiration date M time of the first proxy process, it is clear that the cache file is always kept within the expiration date. Is. Therefore, the effect of prefetch is kept effective from one prefetch to the next prefetch.

[Second Embodiment] Next, a second embodiment of the gateway device of the present invention will be described. The second embodiment is similar to the first embodiment, but is an improvement of steps 3 and 4 of the flowchart (see FIG. 3) corresponding to the first embodiment.

In the first embodiment, in step 4,
File object names are extracted one by one from the prefetch access list, and the external network is accessed through the second proxy process to acquire the file object. However, if the number of rows in the prefetch access list becomes large, this method may require a long time from the start to the end of all accesses.

By the way, the TC used by the WWW system
In a network using the P / IP protocol, communication is performed in packet units. So a computer
It is apparently possible to perform simultaneous communication with a plurality of server computers on the external network. Therefore, the prefetch child process started in step 5 shown in FIG.
A plurality of devices can be activated at the same time and operated in parallel.

File object access is performed through various routes on the external network. When there is a route with a low transfer rate in the middle of the route and it may become a bottleneck, or when it is not possible to respond immediately to an access request due to a heavy work load on the server computer itself on the external network. And so on. Therefore, the actual transfer speed of the file object is usually smaller than the maximum transfer speed of the first communication path connecting the proxy server device and the external network.

For example, even if the maximum transfer speed of the communication path (communication path 27 in FIG. 16) connecting the proxy server device and the external network is 64 kbps, the actual transfer speed from the server computer located overseas is 1/10 of that. Often less than Therefore, it is possible to make a plurality of network connections simultaneously within the maximum transfer capacity of the first communication path connecting the proxy server device and the external network. That is, a plurality of prefetch child processes are activated to obtain a plurality of file objects at the same time. As a result, the transfer capacity of the communication path is
It can be used near 0%.

Therefore, the maximum number of parallel executions of prefetch child processes that simultaneously execute a plurality of prefetch network connections is set in advance. This number is a MAXPROCESS number and is a constant placed in the memory (memory 206 in FIG. 16) in each step described below.

The specific procedure will be described below with reference to FIG. In the flowchart shown in FIG. 4, steps 1 to 4 are the same as steps 1 to 4 shown in FIG. 3, respectively. Therefore, detailed description of these steps will not be repeated. In FIG. 4, step 5.1.2 replaces step 5 of FIG.
~ 5.2.4 are provided. Before performing the operation shown in FIG. 4, the processes variable is defined in the memory. The processes variable indicates the number of prefetch child processes currently running in the background.

First, in step 5.2.1, one line of the file object name shown in the URL prefetch access list is fetched. Then, one prefetch child process that accesses this file object using the second proxy process is started as a background process. It should be noted that in UNIX OS, a process can usually be executed in the background. To run a process in the background, start it with & on the command line. A specific command example is shown below.

[0201]

[Table 16]

Subsequently, in step 5.2.2, the number of prefetch child processes running in the background is counted and substituted in the processes variable.

In step 5.2.3, the processes variable is the maximum prefetch child process number MAXP specified in advance.
A determination is made as to whether it is less than ROCESS. proc
If the esses variable is less than MAXPROCESS, control is step 4
From step 5.2.1, the prefetch child process is started in the background.

If the processes variable is greater than or equal to the specified maximum number of prefetch child processes MAXPROCESS, then step 5.
At 2.4, the operation is put to sleep for a certain time (for example, 10 seconds), and the control is returned to step 5.2.2. After that, the processes of steps 5.2.2 to 5.2.4 are repeated, and when the preceding prefetch child process is completed, a new prefetch child process is activated, and the total number of prefetch child processes is increased. Keep it below MAX PROCESS.

By advancing the simultaneous processing according to the above procedure, the time from the start to the end of the prefetch can be shortened. Therefore, it becomes possible to prefetch more file objects by the designated end time already described. Since the amount of file objects that can be preloaded into the cache file can be increased as compared with the first embodiment, the cache hit rate can be increased as a result. According to the experiment, it is confirmed that the prefetch time can be shortened sufficiently by taking the value of MAXPROCESS as 10 to 20.

[Third Embodiment] Next, a proxy server device according to a third embodiment will be described. The device of the third embodiment is a further improvement of the device of the second embodiment. More specifically, step 3 is further improved so that the processing performed in steps 5.2.1 to 52.4 shown in FIG. 4 is performed at a higher speed. The process flow is shown in FIG.

In the first and second embodiments, the prefetch child processes are simultaneously activated within the range of MAXPROCESS,
The prefetch time was shortened by advancing the processing in parallel.

However, as the nature of the prefetch access list created by the procedure of steps 1 to 3 of the first and second embodiments, the same server address on the external network often appears consecutively. As described above, in the WWW system, one information page of the server computer displayed on the client computer side is often composed of a plurality of file objects such as text and a plurality of graphic objects. This is because. For example, the information page as shown in FIG. 2 is WWW data in which text and graphics are mixed. The following seven file objects make up this data.

[0209]

[Table 17]

When a certain information page is composed of a plurality of file objects, they are accessed at substantially the same time and consecutively. Therefore, these file object names appear consecutively in the prefetch list. Ah Then, when a prefetch child process that fetches these file object names in order and obtains a plurality of them at the same time is activated, access to only a specific server occurs at the same time. Generally, a server computer is configured to be able to handle a plurality of accesses at the same time, but if a large number of accesses are concentrated at one time, the response will naturally deteriorate. Therefore, if you use this kind of access method,
It can be a bottleneck in the prefetch process.

Therefore, in the third embodiment, the prefetch access list is rearranged not in the original order itself but in an order in which access is not concentrated on a specific server. That is, the file names in the prefetch access list are interleaved. in this case,
If you sort the file object names randomly,
The possibility that access will be concentrated on a specific server computer can be minimized. However, the prefetch access list is originally created in the order of appearance frequency, and the hit rate can be expected to be improved if the file objects are acquired in order of appearance frequency. Therefore, it is desirable to keep the order of appearance of file object names on the prefetch access list to some extent. For example, it is possible that prefetching of all file objects in the prefetch access list is not completed by the end time of prefetching. In such a case, it is better to ensure that the one with a high appearance frequency has completed the prefetch.

According to the actual measurement, the number of consecutive identical server addresses in such a prefetch access list is at most several tens. Therefore, the prefetch access list is divided into a number of rows which is somewhat larger than the number of tens, for example, a block of about 1,000 rows. Then, the order of the file object names is changed inside each of these blocks. The prefetch of the file object is executed from the head of the prefetch access list in which the rows are replaced for each block in the same manner as in the second embodiment. By doing so, the order of the rows is changed in each block, so that the same server address is less likely to appear consecutively. On the other hand, since the order of blocks is maintained, the order of frequency of appearance in the first prefetch access list is maintained at the granularity of block size. Therefore, it is possible to solve the problem that access requests concentrate on only a specific server computer at the same time while preferentially prefetching the ones that appear frequently.

The processing procedure in the third embodiment will be described below with reference to FIG.
Will be described with reference to. Set the block size to the BLOCKSIZE variable. This variable is placed in memory. In the following processing, random numbers are used when the order is changed (interleaved). Of course, this is given as an example that can be easily realized, and the replacement process may be performed using another method.

The procedure shown in FIG. 5 is replaced with step 3 in place of step 3 of the procedure of the second embodiment shown in FIG.
Use 3.1 and 3.3.3. Since other steps are the same as those in the second embodiment, detailed description thereof will not be repeated here.

In step 3.3.1, the prefetch access list is rearranged in the order of appearance frequency of the file object names, and the same file object names are combined so as not to overlap.
This process is almost the same as the process in step 3 of FIG.

Next, in step 3.3.2, the file object name is interleaved to replace the lines. More specifically, follow the procedure below.

First, a random number [i] is assigned to all the rows of the prefetch access list (i indicates the row number).
There are various possible methods for calculating the random number, but in this embodiment, the random number is calculated according to the following formula.

[0218]

[Equation 6]

However, the function rand () is assumed to be a function for generating a random number in the range of 2 32. The first term of this expression is the remainder of the BLOCKSIZE variable for the random number obtained by the function called rand ().
Therefore, the first term is a random number (integer) in the range of 0 to BLOCKSIZE -1.

The second term in the above equation has the following meaning. The prefetch access list shall be divided into blocks each containing BLOCKSIZE lines, and each block shall be given an integer block number starting from 0. The second term is
It is equal to the block number assigned in this way multiplied by BLOCKSIZE. Therefore it is an integer multiple of the BLOCKSIZE number and starts at 0 and increases. Second
If the BLOCKSIZE is 3, the term is 0, 0, 0, 3, 3, 3,
6, 6, 6, ...

Such random number [i] is added to the head field of each line of the prefetch access list. Then, each row is sorted in ascending order of the number of first fields, and the random number [i] is removed from the beginning of each row. This process creates a prefetch access list with rows randomly interleaved for each block of BLOCKSIZE rows.

As described above, at this time, the order of appearance frequency in the first prefetch access list is maintained at the granularity of the block size.

An example of interleaving of each row in the prefetch access list by random numbers will be concretely shown below.
First, assume that the original prefetch access list is as follows.

[0224]

[Table 18]

The following is obtained by assigning a random number to the first field in accordance with the above-described formula with BLOCKSIZE = 10.

[0226]

[Table 19]

Further, if this is sorted in ascending order of the numbers in the first field, the following list is obtained. In this example, since BLOCKSIZE is as small as 10, it is not sufficiently interleaved, but it can be seen that the row order is randomly changed within each 10 row unit. It should be noted that this table is only one example showing the procedure for interleaving.

[0228]

[Table 20]

After this interleaving process, prefetch child processes are executed in parallel as in the second embodiment. Since it is possible to prevent file objects including the same server computer as an address from appearing consecutively in the prefetch access list, it is possible to avoid concentration of access to a specific server computer. As a result, the required file objects can be quickly prefetched from the server computer without the load being concentrated on each server computer, and as a result, the time of the entire prefetch process can be shortened. Since the amount of cache data prefetched to the cache file can be increased, the cache hit rate can be increased.

[Fourth Embodiment] Next, a proxy server device, which is a gateway device according to the fourth embodiment, will be described. The fourth embodiment is a further improvement of the third embodiment. In the third embodiment, the activation of the number of prefetch child processes is controlled so as to maximize the MAXPROCESS number as the number of prefetch child processes. However, in such processing, the number of prefetch child processes equal to the number of MAXPROCESS described above may be activated even when there are many requests to access file objects from the client computer, and as a result, the client computer may The response to the access request to the file object may be degraded. The fourth embodiment is an improvement for solving such a problem.

In the fourth embodiment, the number of prefetch child processes to be activated is reduced during the time period when the user of the client computer uses the first proxy process to access many server computers of the external network. This prevents internal users from interfering with network usage activities.

A specific processing procedure is shown in FIG. The flowchart of FIG. 6 is replaced with steps 5.2.1 to 52.4 of the flowchart shown in FIG.
It adopts 5.4.4 and is characterized in that the number of prefetch child processes is dynamically controlled. Other steps are the same as those described in the first to third embodiments, and therefore detailed description thereof will not be repeated here.

Referring to FIG. 6, in step 5.4.1,
Extract one line from the file object name that appears in the prefetch access list. Then, one prefetch child process for accessing the file object is started as a background process by using the second proxy process.

Subsequently, in step 5.4.2, the number of prefetch child processes running in the background,
Check the number of network access relay processes started by the first proxy process and add up
Substitute in the ses variable.

Subsequently, in step 5.4.3, processes
Variable is the maximum number of prefetch child processes specified in advance
A determination is made as to whether it is less than MAXPROCESS. If it is smaller, the control returns to step 4, and the above-described new prefetch child process activation processing is performed.

The processes variable is a prespecified MAXPRO
If CESS or more, control proceeds from step 5.4.3 to step 5.4.4, pauses for a fixed time (for example, 10 seconds), and returns to step 5.4.2.

The proxy server device according to the fourth embodiment operates in accordance with the processing procedure as shown in FIG. A new prefetch child process is started so that the “total” of the number of prefetch child processes and the number of relay child processes of network access of internal users using the first proxy process is constant. Control. First
If the number of child processes of network access by the user using the proxy process is more than MAXPROCESS number,
The prefetch child process is not newly started and waits for start.

Even if the above-mentioned prefetch process is executed during a time period when many internal users are using the first proxy process, a low priority is given to the prefetch child process. You can prevent them from interfering with your activities. Starting the prefetch process during the active hours of the internal user does not prevent the internal user from using the network. In addition, even if the prefetch process breaks into the active time of the user of the client computer, it does not prevent the internal user from using the network.

As a result, prefetching can be performed for a longer time from the start to the end of the prefetch process. Therefore, it becomes possible to prefetch many file objects, and therefore the amount of cache data can be increased. As a result, the cache hit rate can be increased from the relationship between the amount of cache data and the cache hit rate.

This has the effect of increasing the average file object transfer rate because the hit rate is increased with reference to the formula shown in "Equation 1". Another way to look at this is to allow more users of the internal network to use it while maintaining the same average file object transfer rate.

[Fifth Embodiment] Next, a proxy server device as an example of the gateway device of the fifth embodiment will be described. In the first to fourth embodiments that have already been extended, during the time period when the internal user does not use the external network, such as at night,
Prefetching for n days (= n × 24 hours = N hours) was executed. For cache files, the expiration date is M
If it is time, cache files older than M hours were regularly deleted. Regarding the erasure of such old cache files, the above-mentioned Embodiment 1
Not mentioned in 4.

With the UNIX OS, a file that has not been changed for a certain period of time can be deleted by using the find command. The following is an example of a cache clear command that is activated at a fixed time every day. An old file under the directory cache that has not been changed for more than a day
It is erased by this cache erase command.

[0243]

[Table 21]

By the way, in the proxy server devices of the first to fourth embodiments, the prefetch is carried out from 9:00 pm to 8:00 am the next morning. This results in n times more traffic at night than access by internal users during the day.

In practice, when the cache has expired, the following algorithm is taken according to HTTP in a commonly used proxy process such as DeleGate.

(1) When issuing a request (GET command) for a file object existing in a server computer of an external network, the request is issued with the cache file time Tcache added to the GET command.

(2) Upon receipt of this request, the server computer compares the last modification time Torg and Tcache of its corresponding file object.

(3) When Torg> Tcache,
That is, the server computer transfers the file object to the proxy server device only when the time of the cache file is older than the time of the last modification of the file object. That is, the transfer of the file object is executed only when the original file object has been changed since the last cached time.

(4) If Torg> Tcache is not satisfied, the server computer returns a response code indicating that the cache file can be used to the proxy server device.

According to the above-mentioned algorithm, only the changed original data is actually transferred from the server computer to the proxy server device. If it has not been changed, the last modification time of the file object of the cache file is updated to the time at which such confirmation was made.

In the fifth embodiment, the data transfer amount on the network is reduced by using such an algorithm of the proxy process. Therefore, in the proxy server device according to the fifth embodiment, of the cache data of the cache file (expiration time M hours), data older than M hours is not always deleted regularly, but M <N ≦ E. The time E is set so that only file objects older than this are periodically deleted. As a result, even a file object of a cache file whose expiration date has passed may not be immediately deleted and may be effectively used within the E time. For example, when prefetching to the server computer, it is not necessary to transfer the data whose original data has been confirmed to have been changed from the server computer to the proxy server device. This is because the cached data matches the original data. At this time,
Only the time of writing to the cache file is updated. As a result, with respect to this data, the same effect as that newly transferred from the server computer at the time when the prefetch is performed can be obtained, and the data transfer amount of the network at the time of prefetch can be reduced.

Here, if N = E, the management is easy. As the time E, a value that does not overflow the cache file and is as large as possible may be adopted. N
= E, consider the case where N is taken as the subject and a large value is taken. If N is set to a large value, the number of file objects to be prefetched increases as already expanded, so the cache file storage amount should increase. However, actually, the prefetch is forcibly terminated at a certain time, so that even if the data transfer is always performed in the prefetch as in the first to fourth embodiments, the accumulated amount of the cache file does not increase with N. However, in the device of the fifth embodiment, it is possible to obtain the same effect as prefetching without performing data transfer for a file object that has not been changed. Therefore, the accumulated amount of the cache file is actually affected by the magnitude of the time E.

According to the actual measurement result, E / 24 hours =
At the time of n = 7 days, only about 6% of all the prefetch data causes reloading of the file object to the cache. The transfer amount of the last modified time information by the “conditional GET command” with the last modified time information of the cache file is small enough to be neglected compared to the actual transfer of the file data. Therefore, the traffic increased by prefetching according to the fifth embodiment is
Only for data transfers about those updated after the time stored in the cache. According to the actual measurement example, the traffic increase due to prefetching is for 7 days ×
0.06 = 0.42 days.

By improving the cache hit rate by the prefetch process performed by the proxy server device of the fifth embodiment, the file object transfer amount from the external network to the internal network due to the user's activity during the daytime is reduced by about 20%. Have been observed to exist. Therefore, even considering the increase in traffic due to the prefetch described above, the total amount of data flow from the external network due to the prefetch does not increase remarkably in a total amount per day. Therefore, there is an advantage that the contents of the cache file can be kept fresh without unnecessarily increasing the traffic of the external network.

[Embodiment 6] By the way, an example of the access log in the above embodiment is shown in Equation 4. The access log shown in Equation 4 is the De process as the first proxy process.
This is an example of an access log when using leGate software. Of these, the parts other than those already explained will be explained here. These are the status code of the server computer, the data size of the file object, the cache status, and the like.

The status code of the server computer is 3 issued by the server computer in response to a request from the client computer.
It is a response expressed as a digit. The first number in the status code indicates the category of status code, 1, 2,
3, 4, 5 are used. The 200th status code indicates that the request from the client computer is understood without any problem. For example, when the status code is 200, it indicates that the request has been normally accepted. The 300th status code indicates that the server computer needs to be retransmitted.
The status code in the generations indicates an error in the client computer, and the status code in the generations 500 indicates an error in the server computer. Also, the 100th status code is reserved for future use and is not currently used.

The data size of the file object indicates the size of the file object in bytes. The cache status indicates the relationship between server-side data and cache contents. If the cache status is "H", a cache hit occurs. If "N", the cache does not exist and a new file object is written in the cache file. If "O", the server side data is newer than the cache content. Indicates that.

The gateway device according to the sixth embodiment of the present invention uses such a file object transfer record.

The gateway device of the sixth embodiment is an improvement of the first to fifth embodiments. In the above-described gateway device of the present invention, it is assumed that the prefetch is executed during a time period when the client computer is little used. However, those hours are limited to a limited time such as night. Therefore, there is a limit in the amount of file objects that can be prefetched, and there is a problem that it may not be possible to prefetch all the file objects that are the prefetch targets. The gateway device according to the sixth embodiment aims at efficiently prefetching a file object in a short time in order to solve such a problem.

FIG. 7 shows a schematic block diagram of a proxy server device 54 which is an example of a gateway device according to the sixth embodiment of the present invention, together with a network 25 and an internal network 23 to which the proxy server device 54 is connected. 7, the proxy server device 53 shown in FIG.
The same parts as those of the above are given the same reference numerals and names. Their functions are also the same. Therefore, detailed description thereof will not be repeated here.

The proxy server device 54 shown in FIG. 7 is different from the proxy server device 53 shown in FIG.
The proxy process 34 of writes the transmission record of the prefetch to the access log 39 (40), and the second process for the file object read from the access list 30
The prefetch process 36 shown in FIG.
Same as, but this new access log 3
Second, referring to the contents of No. 9, according to the information indicating the change status of the file included in the access log, the file object whose change is detected at the previous prefetch is preferentially prefetched over the file object from the access list 30. The prefetch process 41 for controlling the proxy process 34 is included.

First proxy process 14, second proxy
The process 34 and the prefetch process 41 are both placed in the memory and processed by the CPU, as in the device of the first embodiment.

In this proxy server device 54, the second pr
The oxy process 34 records an access log 39 resulting from the prefetch of the file object. The prefetch process 41 scans the access log 39, extracts the name of the file object prefetched as a result of the previous prefetch, which has been updated after the previous prefetch, and creates the change detection list. To do. The change detection list is arranged in the order of name and frequency of appearance to remove the same name, and then merged with the front part of the prefetch list extracted from the access list 30. Then, the prefetch process of the second proxy process 34 is executed in accordance with this merged prefetch list.

It can be said that at the time of the previous prefetch, the file object changed after the previous prefetch is likely to be changed relatively frequently thereafter. Therefore, the prefetch list created as described above has the file object that is highly likely to be changed placed at the front. Therefore, these are prefetched at the beginning of the specified time. Those that are unlikely to be modified will be prefetched after these, but since the likelihood of modification is low, data that has already been fetched is likely to be sufficient. Therefore, it is possible to preferentially prefetch a file object having a high possibility of change within a limited time, and to keep the contents of the cache file 16 fresh.

The operation of the prefetch process 41 will be described in detail below. The outline of the operation is as follows.

(1) Collect logs for a certain number of days in the past from the access log 39 of the second proxy process 34. A list of file objects having information that the file object on the server computer side has been changed since the previous prefetch as a result of the previous prefetch is created as a change detection list. They are,
It is also a list of file objects written to the cache file as a result of the previous prefetch.

(2) Based on the access log file 30 of the first proxy process, the file object name of a certain past period is extracted and a prefetch access list is created. The prefetch access list itself is the same as that in the first embodiment.

(3) The change detection list created in (1) is merged with the head part of the prefetch access list created in (2).

(4) Based on the prefetch access list thus created, the second proxy process 34 is used to prefetch the file objects in the prefetch access list from the server computer of the external network.

(5) As a result, the cache file 1
In 6, the file object is written by the second proxy process 34 and kept up to date. At this time, the second proxy process 34 writes the access result in the access log 39.

The operation of the proxy server device 54 will be described in more detail with reference to FIGS. 8 and 9. Each step of the flowcharts shown in FIGS. 8 and 9 will be described as, for example, “step 1”, “step 5”, or the like.

Step 1 Collect the access log files of the second proxy process for the latest P days. The access log file shall be created as a separate file for each day. This log file is an access log by prefetch. Therefore, it has information as to whether the file object has been updated according to the result of accessing the file object of the server computer at the time of prefetching. It should be noted that, in the experience of the inventor, P of about 2 is sufficient.

The access log file name of the second proxy process can be set by the second proxy process so that the name changes every day. For example, the date can be added at the end of the log file name. In that case, change the file name to "/ usr / local / etc / delega
It can be named "ted / 10000.http.day".
As an example, the names of log files from July 1st to 10th are as follows.

[0274]

[Table 22]

Under the UNIX OS, the / bin / ls command is started with the "-t" option in order to arrange the log files in order from the latest log file. Specifically, it is as follows.

[0276]

[Table 23]

As a result, the following result can be obtained.

[0278]

[Table 24]

Since P = 2 is sufficient as already described, the log file names for the last two days are extracted from the file list rearranged in this way. To do this, use / usr / ucb / hea, which is standard on UNIX operating systems.
Use the d command. Since the head command outputs the specified number of lines from the standard input to the standard output, use "|"
Use the standard output of ls -t as the standard input of the head command. The following table 25 shows the commands for making such standard output connections.

[0280]

[Table 25]

When the above command is executed, the following result is obtained.

[0282]

[Table 26]

Step2 Here, the access log file of the second proxy process collected in step1 is a single file "obsoletelist.txt".
Put together. Note that this name is only an example, and can be appropriately determined according to the operation.

When the access log files are put together in the file "obsoletelist.txt" in this way, weighting according to the interval between the time when the file object is relayed to the user through the proxy server device 54 and the time when the prefetch is performed. Add. That is, the latest access log file is weighted more than the old access log file. For example, you could simply focus on the date in the log file and do something like this: The output obtained as a result of executing the commands shown in Table 25 is arranged in order from the log file with the newest date. Therefore UNIX
You can create the "obsoletelist.txt" file as follows using the cat command that comes standard with the OS.

[0285]

[Table 27]

By using the cat command as shown in Table 27, the access log remaining on the day of prefetching is counted as two, while the access log for the previous day is counted as one. . Therefore, the appearance frequency of recently accessed file objects is
The access log list given the higher weight is left.

Step3 The file "obsoletelist.txt" output in step 2 above
Each row of is a record of an access file object by past prefetch. Therefore, the access log including the file object name according to the following conditions is selected.

-The http command character string is a file object read request (GET), and-the protocol part of the file object name is http.-The object that matches a specific character string pattern is excluded.-The file object name is prefetched. Excludes files that are not suitable for the above-Excludes files with a large data size in the file object code-Excludes specific server computer status codes-Prefetch files whose cache is updated as a result of accessing the file object Use to extract the file object.
The list extracted in this way includes only those whose cache has been updated as a result of the previous access to the file object, and thus can be called the change detection list of the file object. For this change detection list, first sort by appearance frequency, then
Collect file objects with the same name. Also, only the field of the file object name is extracted and a prefetch access list composed of the file object names to be prefetched is created.

As a file object name that is considered unsuitable for prefetching, there is a file object name that includes "?" In the URL. The URL notation including "?" Is used when the client user manually inputs a character string and passes it to the server computer. Therefore, if you prefetch a file object with a name that includes this URL notation,
This is harmful because the machine will have entered the characters on its behalf on behalf of the user.

The extraction operation of such a file object name is performed by the following command part provided as standard in UNIX OS:

[0291]

[Table 28]

And a program written in C language or the like

[0293]

[Table 29]

It can be performed by using, for example. In Tables 28 and 29, the left side shows the commands and parameters, the command for program reading, and the right side shows the corresponding functions.

For example, by inputting the following command, the list processed in step 3 is output to the standard output.

[0296]

[Table 30]

The process will be described below. "obsol
The format of the log file in "etelist.txt" is as described above.

[0298]

[Table 31]

The cache status at the end changes depending on the access status of the second proxy process to the server computer. Specifically, it is as follows.

"O": When the file object of the server computer is newer than the file object of the cache "S": When the file object of the server computer and the file object of the cache are compared, they are the same. If it turns out that there are various other cache statuses, information about these cache statuses can be found in the second one that relayed the prefetch.
It is written in the access log 39 by the proxy process. Therefore, by outputting "obsoletelist.txt" to the standard output with the cat command and inputting it in grep 'O $',
Line endings of "O" are printed to standard output. Note that
The command called grep is also standard in the UNIX OS. In order to further process this output, "|" is used as described above, and this standard output may be used as the standard input of the next command.

By using the standard output as the standard input of "grep -v" \? ", A line not including"? "Is extracted and output to the standard output. The output of" grep "\" GET "is output. Input to standard input, so that only lines for which the command is "get" are extracted and printed to standard output.

The standard output obtained in this way is grep_
datasize DATASIZE is given to standard input. In this case, DA
In TASIZE, the upper limit value of the data size of the file object designated in advance is set. This value is set to a concrete value in an actual command, for example. As the standard output of this command, only the lines whose data size of the file object is smaller than the constant DATASIZE are extracted and the standard output is obtained.

Further, this standard output is grep_response r
Feed it to the standard input of esponse.txt. Where "response.
"txt" is a file that describes the status code of the server computer that should be extracted in advance. By executing this command, the status code of the server computer is "response.txt".
Only the lines that match the status code described in are extracted and output to standard output.

Considering a blank as a delimiter, in the access log shown in Table 31, the seventh field corresponds to the file object name. So grep_resp mentioned above
Output the onseresponse.txt to the standard input of awk '{print $ 7}' and extract only the part of the file object name from the file access log.

Further, it is necessary to extract only the line in which the protocol name of the file object name portion is http. Therefore, the output described above is further given to the standard input of "grep" ^ http: // ", where" ^ "indicates the beginning of the line.The results of the above processing are output to standard output.
The standard output thus obtained is used again as the standard input in the next step 4.

Step4 Next, from the list of file objects output in step3, those which are considered unfavorable for prefetching are removed. Specifically, the string pattern that the file object that seems to be unfavorable to prefetch has as an exclusion pattern list is "exclud
Store it in a file called "e.txt" in advance. The gateway administrator can add or change this list.

[0307] Specifically, since the file object including the contents for which it is determined that it is not necessary to update the cache by prefetching, the size is huge, so prefetching hinders prefetching of other file objects. File object, or a file object that has a problem in prefetching due to an abnormality in the server computer. This "e
Table 32 shows an example of the contents of the "xclude.txt" file.
Of course, the one shown in Table 32 is a hypothetical example. A file object having a character string that matches the character string pattern shown in Table 32 is removed from the update target list.

[0308]

[Table 32]

By the way, since the information output to the standard output in step 3 is a record of prefetch access performed a plurality of times, the same file object name may appear many times. Therefore, this list is further sorted in descending order of appearance frequency. At the same time, combine the same URLs so that they do not overlap. On the UNIX OS, this sort of work can be done using the standard commands provided below.

[0310]

[Table 33]

Therefore, the above sort operation is performed by the following command.

[0312]

[Table 34]

The processing contents of this command are as follows. First, in fgrep -v -f exclude.txt, "exclude.txt"
Only file objects that do not match the string pattern contained in the file are output to standard output. By using this as the standard input of sort, the file object names are arranged in alphabetical order. In addition, in this example, only the string pattern that the URL should match exactly is specified in the "exclude.txt" file, but as shown below, exclude the string pattern that the URL should partially match. It is also possible to prepare an "exclude.txt" file for this.

[0314]

[Table 35]

In this example, "*" indicates an arbitrary character string pattern. Therefore, the URLs shown below are also excluded.

[0316]

[Table 36]

The commands shown in Table 34 cannot be used as they are in order to exclude a character string pattern that partially matches. However, it is possible to easily realize this by combining a command group prepared as standard in the UNIX OS or a program described in C language.

In this case, the contents shown in Table 32 are "exclude.t.
The following description will be given assuming that the processing command stored in the xt "file and described in Table 34 has been used for the processing. It is assumed that the standard output of step 3 is as follows.

[0319]

[Table 37]

By the processing of fgrep -v -f exclude.txt, the 3rd and 4th lines having the character strings matching the character string patterns included in the exclusion pattern list stored in the "exclude.txt" file are obtained. Removed. As a result, the following standard output is obtained.

[0321]

[Table 38]

Furthermore, this is given to the standard input of sort, and ASCI
Sorted by I code, the result is:

[0323]

[Table 39]

Further, when this output is given to the standard input of uniq -c, the lines having the same file object name are aggregated and the appearance frequency is inserted in the first field and output. The output looks like this:

[0325]

[Table 40]

The first field of each line of the obtained list can be considered as a numerical value of the appearance frequency. Therefore, in order to sort from the one with the highest appearance frequency, Table 4
This list can be rearranged in the order of appearance frequency by giving the contents of 0 to the standard input of sort -n -r and sorting. The result is as follows.

[0327]

[Table 41]

Step5 Next, set the access log file of the first proxy process to n
Collect for one day. The specific procedure of this collection method is as follows. The access log file name of the first proxy process can be set by the first proxy so that the name changes every day. For example, if the date is added at the end of the log file name, the access log file name of the first proxy process is "/usr/local/etc/delegated/10000.ht.
You can name it "tp. day" and change it every day.

Then, for example, from July 1st to July 1st
The names of the log files up to 0th are as follows.

[0330]

[Table 42]

This file name list is arranged in order from the newest date. To do this, run the / bin / ls command with the -t option under UNIX OS.

[0332]

[Equation 7]

As a result, the following result is obtained.

[0334]

[Table 43]

Further, log file names for the last 7 days are extracted from these file lists. for that purpose,
You can extract the first 7 lines using the / usr / ucb / head command that comes standard with the UNIX OS. The head command extracts the specified number of head lines from the list given to the standard input and outputs it to the standard output.

[0336]

[Equation 8]

The results are as follows.

[0338]

[Table 44]

Step 6 Here, the access log files of the first proxy process collected in Step 5 are put together into one file "workfile.txt". At this time, each file object is weighted according to the interval between the time when each file object was last relayed by the user and the time when prefetching is performed. For example, simply focusing on the dates in the log files, it can be seen in Table 43 that the log files are ordered from the newest date. Therefore, the following processing is performed using the cat command that is standardly installed in the UNIX OS.

[0340]

[Table 45]

In this way, the access log left on the day of the prefetch is 7 times, the access log left on the previous day is 6 times, the access log left 2 days ago is 5 times, and so on. , An access log list is left that makes the frequency of recently accessed file objects appear higher.

Step 7 The "workfile.txt" file collected in Step 6 is a file in which past access file objects are recorded line by line. Therefore, from each of these lines, one including the file object name to be prefetched under the following conditions is selected.

The http command character string is a file object read request (GET), and the protocol part of the file object name is http. Excludes those that match a specific character string pattern. Prefetches the file object name. Exclude files that are not suitable for the above-Exclude file objects with a large data size-Exclude specific server computer status codes A list of file object names extracted from the "workfile.txt" file using the above conditions To create. Further, only the file object name field of each line of the created list is taken out and the list of file object names to be prefetched is output to the standard output.

By the way, a file object name that is not suitable for prefetching is, for example, one that includes "?" In the URL. This is as described above.

In the UNIX OS, the file option name extraction work described above is performed by the command group provided with the OS.

[0346]

[Table 46]

And a program written in C or the like

[0348]

[Table 47]

Can be done using. The following is an example of a command for performing the above extraction process.

[0350]

[Table 48]

The process performed by the above command will be described. The format of the log in the "workfile.txt" file is as shown in Table 31 as already described. Therefore, the "workfile.txt" obtained as standard output is "grep-
v By inputting "\?", the log is extracted so that "?" is not included and it is output to the standard output. Add it to "grep" \ "G
By giving it to the standard input of "ET", only the lines for which the command is the get command are extracted and output to the standard output.

The standard output of "grep" \ "GET" is grep_datasi
When given to the standard input of ze DATASIZE, the data size of the file object will be the constant DATASI specified in advance.
Only lines smaller than ZE are extracted and printed to standard output.
When the standard output is given to the standard input of grep_response response.txt, only the lines where the status code of the server computer matches the status code described in the file "response.txt" in advance are extracted and output to the standard output. .

Further, considering a space as a delimiter, 7 in each line is considered.
The second field corresponds to the file object name. Therefore, use awk '{print $ 7}' to extract only the file object name part. Furthermore, in order to extract only the line where the protocol part of the file object name is "http", it is filtered by "grep" ^ http: // "(" ^ "indicates the beginning of the line).

Step 8 The list output to the standard output in Step 7 is a record of access to the server computer from a plurality of client computers on the internal network. Therefore, the same file object name may appear many times in this list.

Therefore, this URL list is sorted in order of appearance frequency, and the same URLs are put together so as not to overlap. Also, as in step 4, the file "exclude.txt" should be deleted to remove undesired file object names.
The rows having the character string that matches the character string pattern of each row stored in are deleted. Under the UNIX OS, such sort work can be performed using the following command groups provided with the OS.

[0356]

[Table 49]

Using these command groups, processing is performed by the following commands, for example.

[0358]

[Equation 9]

This command is a process for arranging file object names in alphabetical order by using the standard output of the process of the previous step as the standard input of sort.

The file "exclude.txt" stores a character string pattern in which the URL partially matches the file object name, and a UR that partially matches this character string pattern.
It is also possible to exclude the file object name with L. But here the file "exclude.txt"
It is assumed that contains the character string pattern that the URL must match exactly.

It is assumed that the standard output contents of step 7 are as follows.

[0362]

[Table 50]

Then, the result of "standard output | sort" of step7 is that each row of the table 50 is sorted in alphabetical order as follows.

[0364]

[Table 51]

Further, when this output is given to the standard input of "uniq -c", the lines having the same file object name are aggregated, and the appearance frequency of the lines having the same file object name is inserted in the first field. A list like this will be output.

[0366]

[Table 52]

The first field of each line of this list can be considered as a numerical value indicating the appearance frequency of the corresponding URL.
Therefore, this first field is treated as a numerical value and given to the standard input of sort -n -r to sort in descending order. This allows each row to be arranged in order of appearance frequency. The result is as follows.

[0368]

[Table 53]

That is, the output of the command described as the equation 8 is a "workfile2.txt" file, but this file "workfile2.txt" is a prefetch in which one file object name is listed in one line in the order of appearance frequency. It becomes an access list.

Step 9 Next, the file "obsoletelist.txt" obtained as a result of step 4 and the file "workfile2.txt" obtained as a result of step 8 are concatenated into one file "pre".
fetch_url.txt ". The concatenation order is as follows.

(1) Change detection list "obsoletelist.tx
t "(2) Prefetch access list" workfile2.txt "This process can be performed under UNIX OS using the following command.

[0372]

[Equation 10]

Step10 Subsequently, interleave processing of the prefetch list is performed. This interleaving process is the same as the process already described in the third embodiment, and is a process of rearranging the prefetch access list in an order in which access is not concentrated on a specific server, instead of the original order itself. Say. By doing so, it is possible to minimize the possibility that access will be concentrated on a specific server computer. Since the details have already been described in the third embodiment,
It will not be repeated here.

Step 11 Next, the maximum constant of the number of relays in the parallel network is calculated. That is, the number MAXPROCESS, which is the maximum allowable number of child processes for parallel execution, used in the prefetch child process startup loop shown in the subsequent steps is calculated.
In the embodiments described thus far, MAXPROCESS, which is the maximum value of concurrent execution of parallel prefetch, is a predetermined fixed parameter.

That is, in the above embodiments, the number of parallel child processes of the prefetch process is regulated by the following formula.

[0376]

[Equation 11]

As described above, by regulating the number of child processes for parallel execution of the prefetch process to be activated, there is an effect that the priority of the prefetch child process with respect to other processes is lowered. This is as already described in the description of the embodiment. In addition, in all proxy network relay process number,
Contains the prefetch child process.

In this case, however, there are the following problems. For example, assume that MAXPROCESS = 20 has been set. In this case, there may be 20 or more network relay processes by the client computer user when the loop execution of the next step is started. Then, only by that, the condition shown by the above equation 11 cannot be satisfied. That is,

[0379]

[Equation 12]

Since the above condition is not established, a new prefetch child process cannot be activated. If such a situation continues, prefetching cannot be substantially performed. The present embodiment takes the following measures so as to avoid such a problem and to steadily perform prefetch regardless of the number of network relay processes. That is, when deciding whether or not to start the prefetch child process, "a certain number of MAXP
We use the one with "ROCESS" added as a new MAX PROCESS, and then make the judgment of the above equation 11. As a result, the total number of all network relay processes will be the initial constant MAXPROCE when the loop after step 11 is started.
Even if it exceeds SS, the new constant MAXPROCESS calculated as above is used in the actual comparison, so that a new parallel child process for prefetching can be newly started.

A specific implementation method will be described below.
First, let NUMPROCESS be the number of network relays by all proxies at a certain point. NUMPROCESS can be obtained from the process table of the OS. Under UNIX OS, NUMPROCESS can be calculated with the following combination of commands.

[0382]

[Table 54]

NUMPROCESS can be obtained by combining these commands as follows, for example.

[0384]

[Equation 13]

Here, the fact that the name of the network relay process is "DeleGate" as described above is used. Furthermore, MAXPROCESS is recalculated by the following formula.

[0386]

[Equation 14]

As a result, a new MAX PROCESS can be obtained. step12 Step 12.1 and step 12.2 shown in FIG. 9 are collectively referred to as step 12 here. The process after step 12.1 is st
Prefetch list "prefe" interleaved by ep10
A line buffer in memory for each line of "tch_url.txt"
It is a loop process that reads in the URLBUF and starts the prefetch child process according to the read URL. At this time, using the second proxy, one prefetch child process for prefetching the file object is activated as a background process.

To access the network, the http protocol may be generated directly from the prefetch child process. Therefore, a dedicated program should be created. The name of this program is webacces
Let s.

The program webaccess uses the socket mechanism which is the basis of the network function of UNIX OS. Make a network connection for process function communication by specifying the address of the proxy process and the port number used by TCP / IP. Further MAX PROCESS
Directly proxy GET command in http protocol
Write to the server and read the response data to memory and discard. The read response data is automatically stored in the cache file 16 (see FIG. 9) by the cache mechanism of the proxy server.

An actual description example when using webaccess is as follows.

[0391]

[Equation 15]

In this case, the address of the proxy server and the port number used are specified for webaccess. webaccess is included in the above description in response
Access the URL to read the file and discard it in the internal memory. The network address of the machine used by the proxy process is proxyserver, and the TCP / IP port number is 10001. It is assumed that the following file object names are stored in the line buffer URLBUF.

[0393]

[Equation 16]

In this case, if webaccess is started by the following command, the file object "http://www.xxx.co.jp/test/ind" of the server computer will be sent via the proxy process.
ex.html "can be accessed. At this time, a proxy child process that is a relay child process is created.

[0395]

[Equation 17]

In this embodiment, the cache file 16 managed by the first proxy process can be kept in the latest state by the cache update access for the file object (URL) described by the equation 15. In addition, "&" is added to the end in Equation 17,
This is because the child processes operate in parallel in the background.

Step 13 Subsequently, in Step 13 of FIG. 9, the number of prefetch child processes and the number of network relay child processes of the first proxy process 14 are measured, added, and added to the variable processes.
The number of these processes can be obtained from the process table as in step 11.

Step14 Subsequently, in step 14, the variable proces obtained as described above
ses is the maximum number of processes MAXPROCESS calculated in step 11
It is determined whether or not the following. Control is step if
Return to 2.1, extract the next line in the prefetch access list, and start the prefetch child process in the background.

Step15 In step14, the variable processes is the maximum number of processes MAXPROCESS
If it is determined that the value is larger than the above, the processing is paused for a certain period of time (for example, 10 seconds). When this pause for a certain period of time is controlled, the control is returned to step 13, and the parallel start of the prefetch child process is retried.

By the above loop processing, the number of prefetch child processes and the first proxy process 14
The start of the prefetch child process is controlled while limiting the total number with the number of network relay child processes of the client computer user who is using. In this case the first
When the number of child processes accessing the network by the user using the proxy process 14 is equal to or more than MAXPROCESS, the prefetch child process waits for activation.

Therefore, even if the user of the client computer executes the prefetch process in the time zone in which the first proxy process 14 is used, the prefetch child process is given only a low priority. Therefore, even if the start of the prefetch process is started during the active time period of the internal user, it does not prevent the internal user from using the network. On the contrary, even if the prefetch process slightly digs into the beginning of the activity time of the user of the client computer, it does not hinder the internal user's use of the network.

Therefore, it becomes possible to perform the prefetch process for a longer time from the start to the end of the prefetch process. Since more file objects can be prefetched, the amount of cache data can be increased. As a result, the cache hit rate can be increased. This brings about the effect of increasing the average file object transfer rate, as in the first embodiment. From a different perspective, while maintaining the same average file object transfer rate,
This has the effect of allowing more users of the internal network to use the network.

It should be noted that after step 15 in FIG. 9, O is set in order to forcibly terminate this prefetch process loop at the specified time.
Set the S timer. The OS timer can be realized by executing a specified operation at a specified time by using a command provided as standard in UNIX OS. In order to end the above process at the designated time, specifically, the following may be done. The prefetch process shown in FIGS. 8 and 9 is given a process ID number from the system at the time of execution. Therefore, using this process ID number, the UNIX OS kill command is given to the OS as follows.

[0404]

[Equation 18]

This command sends a kill command for ending the process specified by the process ID number at 08:00.
It is reserved for the OS to be issued at 0 minutes.

The above is a detailed description of the prefetch process. Then, the first proxy process 14 and the second
A cache management method in the proxy process 34 will be described. In the present invention, two proxy processes share the cache file 16 as shown in FIGS. Moreover, both cache files 16
Update in file object units. Therefore, if exclusive control cannot be performed on the cache file 16 in file object units, competition will occur between network relay processes. Therefore, the cache management method of the proxy process needs to have exclusive control.

In the embodiment of the present application, the well-known "DeleGate" is used.
It uses proxy software called. This software has an exclusive control function, and has a function sufficient to realize the embodiment of the present invention as described above.

The cache management method in DeleGate is
It is published in the form of source code. Therefore, the method will be described below along with this source code. Note that the proxy server software for implementing the present invention is not limited to this DeleGate, but prox with cache function that uses the same cache management method.
y Any server software that has an exclusive control function can be used.

According to the example above, the proxy process
It operates on a machine (system) named proxyserver, and accepts URL acquisition requests on TCP / IP port number 10001. At this time, it is assumed that the specified URL notation is stored in the buffer URLBUF. Then, for example, the prefetch process 41 shown in FIG. 9 executes the next command.

[0410]

[Formula 19]

Communication between the second proxy process 34 and the prefetch process 41 is performed using TCP / IP. As an example, "http: //www.sharp.c" in the buffer URLBUF
It is assumed that the character string "o.jp/sample/test.html" is stored.

By the prefetch process 41 executing the command shown in Expression 14, the URL acquisition request becomes the second
Submitted to the proxy process 34. In response to this, the second proxy process 34 creates a child process, leaves the actual process of acquiring the file object to the child process, and waits for the prefetch process 41 to execute the number 14 again. Second proxy process 34
The child process generated by executes the following processing. The proxy process has the cache file 16 shown in FIG. 7 as a cache file.
This cache file 16 is created as a part of a normal UNIX file system. In this case, the partition called / cache is used as the cache directory. At this time, the next character string among the character strings stored in the buffer URLBUF

[0413]

[Equation 20]

The protocol name is http, the server computer name is www.sharp.co.jp, the directory of the actual part of the file object and the file name are sample / text.
html. At this time, the cache file 16 tries to generate a cache file according to the following name rule.

[0415]

[Equation 21]

Therefore, "http://www.sharp.co.jp/samp
The directory and file name of "le / test.html" are specified as follows.

[0417]

[Equation 22]

Therefore, with reference to FIG. 10, the proxy process performs exclusive control of the cache file 16 as follows.

(1) First, when the update routine of the cache file 16 is started (A1), the specified URL is converted into the cache file name according to the above rule. In the example just described, the file name of the above formula 22 is obtained (A2). Then, it is checked whether or not the file name thus converted exists in the cache file 16. Specifically, the existence of this file is checked by checking whether this file can be opened as an existing file by the open () system call of the OS (A3).

(2) If this file does not exist (NO in A3), this cache file is opened in the file new creation mode (A4).

(3) Further, exclusive lock is applied to this cache file so that writing cannot be performed by other processes (A5). The process after this is exclusive control.

(4) Next, the file object designated by the character string in the buffer URLBUF is acquired from the designated server computer via the network (A6).

(5) The file object thus obtained is passed to the cache update process and written in the cache file 16 (A7). The cache file name at this time is as shown in Expression 22.

(6) When the writing is completed, the cache file is closed and the exclusive lock is released (A8).

If the cache file can be opened in step A3, that is, if the specified cache file already exists, the following processing is performed (A
9).

(7) Cache file / cache / http /
Obtain the last modification time Tm of www.sharp.co.jp/sample/test.html from the OS. On UNIX, the last modification time is fstat
() System call (A10). fstat
The () system call is a function that simply increases with the passage of time from the past to the future.

(8) Next, the current time is read using a system call and stored in a variable called Tonow (A11).

(9) The last modification time Tm of the cache file and the current time Tnow are compared to determine whether the difference is within the cache expiration date or not (A12).

(10) If it is within the cache expiration date (YES in A12), this cache file is shared and exclusive locked. In other words, the cache file cannot be written by other processes but can be read (A13).

(11) Next, the current cache file is read and transferred to the cache update process (A1
4).

(12) The cache file is closed and the exclusive control lock is released (A8).

On the other hand, when the cache expiration date has passed (NO in A12), the following processing is performed.

(13) Exclusively lock the cache file to prevent writing from other processes (A15).

(14) Next, the file object designated by the character string stored in the buffer URLBUF is acquired from the server computer via the network (A16).

(15) The file object thus obtained is passed to the cache update process and is written in the cache file shown in Eq. 22 (A1
7).

(16) Then, the cache file is closed and the exclusive lock is released (A18).

The above is the cache control procedure of the proxy process used in the above embodiment. In each of the embodiments of the present invention, the cache file system (for example, the cache file 16 in FIG. 7) is set to the first by using the exclusive control function by such a proxy process.
Shared by the proxy process 14 and the second proxy process 34. The second proxy process 34 has a cache expiration time = 0 as described above. The last modification time Tm of the cache file is always smaller than the current time Tnow. Therefore, the determination made in step A12 of FIG. 10 always results in NO in the server computer access via the second proxy process 34, and the cache file is always updated (A16).

As described above, also in this embodiment, the first object is obtained by prefetching the file object (URL).
Cache file 16 managed by proxy process 14
It is possible to increase the capacity of n × 24 hours / m times and keep the latest state.

In the step 5 of this embodiment, the past access log files for n days are collected. The number of n days is determined from the maximum capacity of the cache file and the file transfer record. Different for each system. 1 for a cache file device with a capacity of 1 GB
If 100 MB of file objects are stored per day, the cache file 16 will not overflow if the setting is such that n = 5, that is, the file objects are prefetched by about 500 MB.

In this way, it is easy to calculate the number n that does not cause the cache file to overflow, from the record of cache file accumulation. To do so, do the following: For example, in the case of a cache file device having a capacity of 1 GB, the minimum value of the cache storage amount MIN = 6
It is defined that 0% and maximum value MAX = 80%. If the actual amount of cache storage falls below MIN, n = n + 1 is set at the time of daily prefetch activation. Further, when the actual amount of cache storage exceeds MAX, it is reduced to n = n-2 or the like. By doing so, the cache storage capacity can be maintained between MIN and MAX.

Such a prefetch process can be activated using a time zone when the client computer user does not access the external network from the internal network very often. By doing so, the file object access by the prefetch process can be operated so as not to interfere with the external network access of the internal network user. In the above-described embodiment, the prefetch process is started at 21:00 when the usage of the client computer is low, and ended at 8:00 the next day at the latest. 8
If the prefetch process is still operating at 0:00, it is forcibly terminated as described above. As a result, even if the prefetch list takes a long time and the prefetch process does not end by 8 o'clock the next day, the prefetch process can be surely performed during the time when the use of the client computer can be started. Will be terminated. Subsequent use of the client computer by internal network users is not hindered.

By the way, in the above embodiment as well, the first
The cache expiration date of the proxy process 14 is M = 24 hours. Therefore, the content of the cache data updated at night is at least valid until 21:00 the night after the update. That is, at least a file object that has a high change frequency in the cache file is maintained as information updated within the past 24 hours. On the other hand, the capacity of the cache file is n days of the average daily access amount. The cache hit rate is 2 when the cache accumulation period is 2
It is empirically known that it becomes three times or more when n = 7 compared with the case of 4 hours. Therefore, by setting as described above, the cache hit rate can be improved while keeping the contents of the cache file fresh. Can be increased. Further, since the prefetch process is activated at a time interval equal to or shorter than the cache expiration date M hours of the first proxy process 14, the cache file is always kept within the expiration date. Therefore, the effect of prefetch is kept effective from one prefetch to the next prefetch.

[Seventh Embodiment] In the first to sixth embodiments,
It has been described that the gateway computer and the client computer are separate computers. However, FIG.
As shown in, there is no problem in activating the client process 42 in the gateway computer 55 to include the function of the client computer. On the contrary, of course, there is no problem in including the function of the gateway computer in the client computer. In FIG. 11, the same components as those shown in FIG. 7 are designated by the same reference numerals and names. Their functions are also the same. Therefore, detailed description thereof will not be repeated here.

By thus incorporating the functions of the client computer and the gateway computer in the same computer, it becomes possible to prefetch more file objects. Therefore, the amount of cache data can be increased, and as a result, the cache hit rate can be increased. The effects obtained as the gateway computer are the same as those obtained by the devices of the other embodiments already described.

Here again, the effect of each embodiment will be described. According to the first embodiment, in the proxy process used by the internal network user, the cache expiration time M
Is set to be short, for example, 24 hours.
Therefore, the content of the cached file object is guaranteed to be within the last 24 hours. That is, the stale data rate of the cache file is kept small. On the other hand, the cache file itself stores n days worth of cache files.
Since the accumulated amount of the cache is n × 24 / M times as compared with the case where the prefetch process is not performed, the cache hit rate becomes a high value corresponding to the cache expiration date of n days.

In the conventional proxy server device, it has been observed that the cache hit rate is about 12% when the cache expiration date is set to 24 hours. On the other hand, when prefetch processing is executed by the proxy server device of the first embodiment, if n = 7 days, the cache hit rate is predicted to increase as described above, and the hit rate actually becomes 35%. . This reduced the number of inquiries to the web server and improved the response speed.

In the proxy server device according to the second embodiment, the prefetch can be performed simultaneously in parallel to shorten the time from the start to the end of the prefetch. You can prefetch more file objects by the end time. Since the amount of cache data can be increased, the cache hit rate can be increased as a result.

In the proxy server device according to the third embodiment, the file object names in the prefetch access list are interleaved, and then the prefetches are simultaneously proceeded in parallel. Therefore, it is possible to prevent the access from being concentrated on a specific server computer. Since it is possible to avoid a slow response of the server computer and become a bottleneck,
The time from the start to the end of prefetch can be shortened.
More file objects can be prefetched by the designated end time. Since the amount of cache data can be increased, the cache hit rate can be increased as a result.

According to the fourth embodiment, even if the internal user executes the prefetch process during the time period in which the first proxy process is used, the prefetch child process is given a low priority. , There is an effect that it does not interfere with the network usage activities of internal users. Starting the prefetch process during the internal user's activity window does not interfere with the internal user's network usage. In addition, even if the end of the prefetch process is delayed until the internal user's activity time, it does not prevent the internal user from using the network. It is possible to use a longer prefetch time from the start to the end, and it is possible to prefetch more file objects. Since the amount of cache data can be increased accordingly, the cache hit rate can be increased as a result.

In the proxy server device of the fifth embodiment, the deadline for deleting the file object in the cache file is about n days. This deadline is selected to be longer than the expiration date of the cache file, and the expired cache file is not immediately deleted. Then, even if the file object has expired, an access request is made and the modification time of the original data of the corresponding server computer is inquired. If the original data of the server computer has not been changed, it is cached again. File object is valid. Then, by only updating the modification time of the header information of this file object, it is possible to substantially realize the prefetch of the file object without actually transferring the data of the file object. Therefore, the traffic for reloading can be reduced.

Since the cache hit rate is improved by prefetching, the file object transfer amount due to the user's activity during the daytime is sufficiently reduced, and even if the above-mentioned prefetching is performed, the traffic will not be unnecessarily increased. Further, since the contents of the cache files can all be maintained within the expiration date, the freshness of the contents can be maintained.

In the sixth embodiment, only the line indicating that there is a change in the log file of the second proxy is extracted,
It is merged at the beginning of the prefetch list extracted from the log file of the first proxy. In general, it can be said that a file object that has been changed at the time of the previous prefetch has a high change frequency. Therefore, by preferentially prefetching these, it becomes possible to effectively use the limited time and maintain the state of the cache file close to the latest state.

In any of the above-mentioned embodiments, the existing pr
It can be achieved by simply adding a prefetch process to the oxy process, and no changes are required to the existing proxy process itself. Therefore, there is an advantage that a proxy server device with an improved cache file hit rate can be easily provided. A list of references including the HTTP protocol is shown below.

[0454]

[Table 55]

[Brief description of drawings]

1 is an example of a gateway device according to the present invention, pr
It is an operation conceptual diagram of an oxy server device.

FIG. 2 is a diagram showing a display screen when an information page in a WWW server is browsed by client software.

FIG. 3 is a proxy which is a gateway device according to the first embodiment.
It is a flowchart which shows the implementation algorithm of the prefetch process performed by a server device.

FIG. 4 is a proxy which is a gateway device according to the second embodiment.
It is a flowchart which shows the implementation algorithm of the prefetch process performed by a server device.

FIG. 5 is a proxy which is a gateway device according to the third embodiment.
It is a flowchart which shows the implementation algorithm of the prefetch process performed by a server device.

FIG. 6 is a proxy which is a gateway device according to the fourth embodiment.
It is a flowchart which shows the implementation algorithm of the prefetch process performed by a server device.

FIG. 7 is a block diagram of a proxy server device according to a sixth embodiment of the present invention.

FIG. 8 is a flowchart of the first half of a prefetch process in the proxy server device according to the sixth embodiment.

FIG. 9 is a flowchart of the latter half of the prefetch process in the proxy server device according to the sixth embodiment.

FIG. 10 is a flowchart of a cache file updater process in the proxy server device according to the sixth embodiment.

FIG. 11 is a block diagram of a proxy server device according to a seventh embodiment of the present invention.

FIG. 12 is a diagram showing a configuration of a conventional gateway computer.

FIG. 13 is a conceptual diagram of a proxy server device.

FIG. 14 is a diagram showing a first example of a circuit configuration of a proxy server device.

FIG. 15 is a diagram showing a second example of the circuit configuration of the proxy server device.

FIG. 16 is a third example of the circuit configuration of the proxy server device and is a diagram showing the circuit configuration of the proxy server device which is also used as the proxy server device according to the embodiment of the present invention.

FIG. 17 is a graph showing the relationship between the cache hit rate and the average access speed.

FIG. 18 is a graph showing the relationship between cache file size and cache hit rate.

FIG. 19 is a graph showing the relationship between the cache expiration date and the cache file size.

[Explanation of symbols]

11 server computer 14 First proxy process 15 Access log 16 cache files 30 access list 34 Second proxy process 36 Prefetch Process

Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06F 12/00 G06F 13/00

Claims (16)

(57) [Claims] 1. A gateway device provided so as to intervene between a first network in which a client computer exists and a second network in which a server computer exists, wherein a gateway device from a client computer in the first network is provided. In response to an access request for the file object, the server computer designated by the file object includes a first network file relay means for making an access request for the file object, wherein the first network file relay means includes , A cache file means for temporarily storing the acquired file objects until the expiration date has passed, and for recording the last modification time in file object units, and files that have been used within a certain period in the past. of File transfer recording means for accumulating the transfer record of the object, and referring to the last modification time of the file object in response to the access request to the file object, whether the file object within the expiration date exists in the cache file means And a transfer unit for acquiring the file object from the cache file unit or the server computer designated by the file object and transferring the file object to the client computer based on the determination result. The apparatus further includes a second network file relay means for prefetching the file object in the transfer record from the server computer and storing the file object in the cache file means according to a predetermined schedule. E b devices. 2. The gateway device according to claim 1, wherein said second network file relay means simultaneously prefetches a plurality of file objects from a server computer simultaneously in parallel via said second network. 3. The gateway device according to claim 2, wherein the second network file relay means determines the prefetch order of the file objects so as to avoid simultaneous prefetch from the same server computer. 4. The second network file relay means extracts a list of file objects from the transfer record according to the frequency of appearance in the transfer record, and simultaneously prefetches file objects according to the order of the file objects in the list. The gateway device according to claim 2, which is performed in parallel. 5. The gateway device according to claim 4, wherein the second network file relay unit swaps the order of the file objects in the list, and prefetches the file objects simultaneously in parallel according to the swapped order. . 6. The second network file relay means divides the list into a plurality of blocks, exchanges the order of the file objects for each block, and stores the files according to the order on the exchanged list. The gateway device according to claim 5, wherein prefetching of objects is performed simultaneously in parallel. 7. The first network file relay means simultaneously activates a plurality of first transfer processes, each of which processes an access request for a file object from a client computer of the first network, in parallel. Each of the second network file relaying means has one
A plurality of second transfer processes for prefetching one of the file objects can be simultaneously activated in parallel, and the number of the first transfer processes is detected to detect the first transfer process. 7. The new activation of the second transfer process is controlled so that the sum of the number of processes and the number of the second transfer processes is equal to or less than a predetermined upper limit. Gateway device. 8. The gateway device according to claim 1, wherein the second network file relay means prefetches a file object only in a predetermined time zone. 9. The first network file relaying means periodically prefetches a file object at a cycle equal to or less than the expiration date of the cache file of the first network file relaying means.
9. The gateway device according to any one of 1 to 8. 10. The cache file means of the first network file relay means does not delete the file object stored in the cache file means until a predetermined deletion time longer than the expiration date has passed. The transfer means accesses the corresponding server computer in response to an access request for a file object whose expiration date has passed but has not been deleted, and the file object in the server computer is After the acquisition, it is determined whether or not it has been updated, and when it is not updated, the file object in the cache file means is transferred to the client computer as a newly acquired file object, and when it is updated, it is specified by the file object. Server computer The acquires the file object after updating issues a transfer request for file object, and transfers the acquired file object to the client computer, the gateway device according to claim 1, wherein for. 11. The first network file relay means simultaneously activates a plurality of first transfer processes for processing access requests for file objects from client computers of the first network, in parallel and simultaneously. Each of the second network file relaying means has one
A plurality of second transfer processes for prefetching one of the file objects can be simultaneously started in parallel, and the number of transfer processes already in operation at the time of starting the second transfer process can be determined. The number of transfer processes that are running and the number of second
7. The new activation of the second transfer process is controlled such that the sum of the number of transfer processes of the second transfer process is less than or equal to the sum of the number of transfer processes in operation and a predetermined constant. The gateway device according to any one of the above. 12. The second network file relay means stores a transfer record of a file object made during a certain period in the past together with information indicating whether a change of the file object is detected at the time of transfer. Further comprising a second file transfer recording means, wherein the second network file relaying means comprises:
From the transfer recording recorded by the file transfer recording means, the file objects whose changes are detected at the last fetch are extracted, and these file objects are extracted from the file objects being transferred and recorded by the file transfer recording means. 2. The gateway device according to claim 1, further preferentially prefetching from the server computer and storing in the cache file means. 13. The second network file relay means stores a predetermined character string pattern, and selects the predetermined character from the transfer records recorded by the second file transfer recording means. The gateway device according to claim 10, wherein a file object having a character string that matches a column pattern is excluded and extracted. 14. The transfer record accumulated by the second file transfer recording means further includes the file size of the acquired file object, and the second network file relay means stores the file object size of the transfer object. 11. The information for specifying the maximum value is stored in advance, and a file object having a file size larger than the maximum value is excluded from the transfer recording recorded by the second file transfer recording means and extracted. The gateway device according to 1. 15. The transfer record accumulated by the second file transfer recording means further includes a status code of the server computer relating to the file object for which acquisition is attempted, and the second network file relaying means of the server computer. A specific status code is stored in advance, and a file object having a status code that matches the specific status code is excluded and extracted from the transfer recording recorded by the second file transfer recording means. 1
0 the gateway device. 16. The gateway device according to claim 1, further comprising means for realizing a function of the first network file relay means as a client computer.