KR20010032419A - An internet caching system and a method and an arrangement in such a system - Google Patents

Abstract

The present invention relates to an internet caching system and an apparatus and method for servicing a request for an internet information file in an internet caching system. The system is formed as two tier caching systems. To reduce the load on the central file server 130, the intermediate apparatus 110 interconnects the central server 100 and the central file server 130 of the system. The device communicates with a local cache server according to the protocol used to communicate between cache servers. When requesting an Internet information file from a central file server, the device uses Structured Query Language (SQL). Thus, central file server 130 is primarily used to respond to SQL queries.

Description

An internet caching system and apparatus and method for servicing a request for an internet information file in the system {AN INTERNET CACHING SYSTEM AND A METHOD AND AN ARRANGEMENT IN SUCH A SYSTEM}

The World Wide Web (WWW), the form of the Internet and most of it, has recently been developed with numerous sources of information. Anyone can provide all the information such as text, pictures, audio and video on the World Wide Web, and the World Wide Web can be easily retrieved as long as the user has access to the Internet from anywhere.

The biggest problem facing the Internet is the increasing demand for communication capacity when users access information from around the world. World wide web traffic has already exceeded all conventional telephone and facsimile traffic over most international telecommunication lines. More transmission and switching capacity is added in succession, but this is slow, expensive, and demand outpaces supply.

The content of the World Wide Web is countless and contains millions of terabytes (as of summer 1998). However, a relatively small subset of all this information takes into account a large portion of the information that is actually accessed. Thus, in order to minimize the bandwidth and associated latency used to access information on the Internet, different caching techniques are currently used to limit the amount of information that must be transmitted over the Internet and to limit the distance over which the information is delivered.

There are basically two ways to cache WWW targets or Internet information files: client-side caching and server-side caching. The simplest form of client-side caching is currently used by virtually all WWW browsers. The browser maintains a recently accessed Internet information file in a cache on the user's computer. When a user wants to access a particular information file a second time, the browser searches in the cache rather than requesting it over the Internet.

To support neighbor users, a proxy server caching method, another form of client-side caching, may be used. In this manner, the cache is located in a WWW proxy node to which a number of neighboring users are connected, such proxy node may be, for example, a server located in a company. When a WWW client wants to access a WWW server on the Internet, the client sends an http request to a proxy node or WWW proxy server rather than directly to a server on the entire Internet. Instead, the proxy server forwards the request to the WWW server on the entire Internet, caches the response, and sends the response to the client. Thus, an information file first transmitted over the Internet is required and stored in the cache of the WWW proxy server. Next, the request for the same information file from the client connected to the WWW proxy server can be resolved locally rather than making an http request to the WWW server over the entire Internet. Proxy server caching can be used on-premises or outside of some organizations by executing the method on a local Internet cache server to which multiple clients are directly or indirectly connected.

Depending on the size and homogeneity of the user community using the cache on the server, about 20-40 gigabytes of cache storage reduces 30-50% of Internet traffic generated by the user community. As the information provided by the Internet and the WWW continues to increase, the required cache size must increase to maintain the hit rate (i.e., the percentage of information files required to be delivered from the cache server). It is also desirable for the performance of the Internet to increase the hit rate by more than 75%. According to typical end-user behavior, this not only requires a fairly large cache (currently about 200-400 gigabytes) but also requires very many members (currently hundreds of thousands) in the end-user community. This is because as the end user community grows, it is likely that non-community members have previously accessed the requested file, especially if the users share common interests.

Installing a large cache is as simple as getting the right computer and the right amount of disk space. However, the cache is required to handle all requests from participating end users. Using current technology, it is not possible for one single processor computer to handle the requests from hundreds of thousands of end users. Thus, several systems have been devised to address this issue and are described along with the name of the main advocate who created it.

The Cisco system suggested that end users connect to the backbone router, which is programmed to redirect all WWW requests to a group or to a ″ farm ″ or ″ cache engine ″ of a ″ dedicated cache device. Each cache engine processes a subset of all origin WWW servers based on the grouping of IP (Internet Protocol) addresses. This approach scales up to 32 cache engines at a time, which corresponds to handling about 500.000 subscription end users.

Inc. proposed a switch called the Layer 4 switch, which is used to redirect all requests for WWW pages to the ″ Inktomy Traffic Server ″. Clusters of powerful computers are used, all of which share the same disk storage system. This method scales up to 16 parallel workstations, which is equivalent to handling about 500.000 subscription end users. However, having several computers access the same disk storage system increases complexity and requires additional management. That is, some capacity of each computer cannot be used to handle the request.

Network Appliance has proposed two tiers of caching. The system has several local caches near the end user. These local caches use the Internet Cache Protocol (ICP) to communicate with the central cache when cache misses occur locally. If the requested file exists in the central cache, it is delivered to the local cache and sent to the end user. The requested file does not exist in the central cache, the central cache makes a request to the origin server and sends the file to the local cache, which in turn sends the file to the end user. The central cache thus handles ICP requests from the local cache and communicates with the origin server when a cache miss occurs in the central cache. For expansion, there may be several parallel central caches, each processing a subset of the origin server. This means that the local cache can handle each request for the correct central cache server. Since these protocols are not standardized, this means that all local caches must receive delivery from network appliances.

All these methods have the disadvantage that the central cache has to handle communication extended in one direction or the other. Therefore, it is difficult to obtain a high hit rate when the server utilization rate is lowered and service to hundreds of thousands of users. By adding more servers, the system becomes expensive and complex. The complexity of the system incurs additional costs and therefore does not make full use of the relatively expensive resources provided by the server.

The present invention relates to an internet caching system and an apparatus and method for servicing a request for an internet information file in the system.

1 schematically illustrates an embodiment of an internet caching system according to the present invention.

2 schematically shows another embodiment of an internet caching system according to the present invention.

FIG. 3 schematically illustrates a flow diagram of operations performed by the local cache server of FIG. 2.

4 schematically shows a flowchart of an operation performed by the feeder of FIG. 2.

FIG. 5 schematically illustrates a flowchart of an operation performed by the updater of FIG. 2.

6 schematically illustrates another embodiment of an internet caching system according to the present invention.

It is an object of the present invention to overcome the disadvantages of current known techniques for caching information files on the Internet and to provide a method for caching information files in a cost effective manner.

It is another object of the present invention to provide a method for allowing a user request for a cached information file to be served by a caching system in a fast and cost effective manner.

Another object of the present invention is to provide a cache server capable of coping with the increasing information files provided by the Internet and the World Wide Web.

Another object of the present invention is to obtain a high hit rate for an information request eight day request delivered to a caching system at minimal cost.

Another object of the present invention is to provide a scalable caching system that can be extended in a standardized manner.

These objects are achieved by an internet caching system and method of servicing a request for an internet information file in an internet caching system according to the appended claims.

According to the present invention, there is provided a method for servicing a request for an internet information file in an internet caching system, the method comprising: receiving a user request from a user for an internet information file at a local internet cache server; In response to the received request, generating a query of the information file if the information file is not cached in the local server; In response to the query, generating a file request for the information file, if the central file server storing the cached internet information file causes the information file to be cached. Is transmitted to the feeder means; And querying the central file server for the information file from the feeder means in response to the file request, thereby reducing the load on the central file server.

According to the present invention, there is provided an apparatus for reducing load on a central file server in an internet caching system comprising at least one local cache server storing at least one cached internet information file and at least one central file server. The apparatus includes a feeder in communication with the local cache server and a central file server, the feeder comprising: first means for receiving a request for an internet information file from the local cache server; Second means for deriving a query from an alphanumeric string received from the local cache server; And third means for querying the central file server for the Internet information file using a query derived by the second means.

According to the present invention, there is provided an internet caching system, comprising: a set of local internet cache servers each receiving a request from a user for an internet information file; At least one central file server included in the central cache site and storing cached Internet information files; And feeder means for interconnecting said central file server and said local cache server set, said feeder means comprising at least one feeder, said feeder being at least in accordance with a protocol used for communicating between internet cache servers. Means for communicating with one local cache server and means for retrieving an Internet information file from the central file server using a database query to reduce the load on the central file server.

The present invention is based on the concept of connecting multiple dedicated computers to a central file server or a central cache server and storing the Internet information file. In the context of a central cache server, these additional computers are lower end computers. The dedicated computer reduces the load on the central cache server by performing some of the tasks handled by the central cache server itself. In this way, the central cache server can serve a local cache server connected to the central server via a dedicated computer, more precisely in a high speed, low cost manner, connected to the central server. While making the most of the expensive hardware that forms the file store of the actual central file server and the central file server where the files are cached, the specialized machines wrapped around the file server are time consuming and perform time-critical tasks in parallel.

Thus, the feeder means or feeder of the present invention is implemented by a device separate from the device implementing the central file server. This reduces the load on the central file server, allowing more processing time to be allocated for the actual retrieval of cached information files. Thus, the central file server can efficiently serve a large community of users. As user requests through the local cache server in question are handled more efficiently, the number of user requests processed increases, allowing the central file server to obtain a high hit rate for its cache.

According to an embodiment of the invention, the feeder means communicates with the local cache server on behalf of the central file server according to the protocol used for communication between the internet cache servers. The protocol currently used is the Internet Cache Protocol (ICP) or Cache Digest but may be any future protocol used for other conventional or same purposes. Thus, by placing tasks to accept and respond to queries and / or requests for information files on devices separate from the central file server device, the load on the central file server is significantly reduced.

When the local cache server receives a request from a user for an information file, if the file is not cached at the local server, the local server begins to generate a query for the file. In one embodiment, the query is forwarded to a table or database within or directly connected to the local server. If the table indicates that the queried file is cached in the central file server, the local server requests the file from the feeder means or feeder. Such queries and requests are preferably performed in accordance with the cache digest protocol. However, depending on the request from the user for the local server, the request from the local server to the feeder may be, for example, an HTTP request according to a three-layer protocol.

In another embodiment, the query from the local server is forwarded to the feeder. Included in a query, for example an ICP inquiry, is the URL of the information file inquired. The feeder derives the inquiry number from the alphanumeric URL of the query received for the information file, which is used by the feeder to query the central file server for the information file. The feeder queries the file server for information files using standard Structured Query Language (SQL) queries. If the queried file is present in the central file server, that is, cache hit, the queried file is transferred from the central server to the local server through the feeder. The fact that the central file server initiates file delivery as a response to an SQL query rather than a response such as an ICP query from a local cache server means that the central file server can save considerable capacity.

Optionally, the query number is derived from the alphanumeric URL and header information portion included in the query. The header information portion includes the original queryer's specific user information, for example he uses and enables a central file server to respond in accordance with this specific information. The query number corresponding to the information file is derived using a hash algorithm, preferably an MD5 hash algorithm.

In an embodiment where the local server forms an internal query for an information file, the feeder derives the query number from the next request that is sent to the feeder by the local server. The alphanumeric string used to derive the query number is included in the URL of the request, for example the HTTP request. The query number is preferably used by the feeder using an SQL query when querying the central file server for the information file. It is also preferable to include at least a part of the header information file of the inquiry as a basis for deriving the inquiry number.

In order to further reduce the load on the central file server, the feeder preferably includes table storage information for each information file cached by the central file server. The table may be, for example, an MD5 index hash table residing in memory. By searching the table, the feeder can determine whether the queried information file is cached in the central file server without having to query the server, so that the feeder can quickly respond to the query to the local server.

According to another embodiment of the present invention, the Internet caching system further comprises an updater means or updater for updating the set of information files cached by the central file server. The update process consists of sending a copy of the file cached on the local server to the central server. The transferred file is a file retrieved from the origin server by the local server and cached by the origin server according to a cache miss at the central server when querying the file.

Thus, the central file server or the central cache server by itself cannot retrieve uncached files and therefore does not burden the origin server with requests for files due to cache misses when servicing the local cache server. Instead, when the feeder evaluates the inquiry from the support beam 52 for the information file and the requested file has not been cached at the central file server, the feeder passes the response to the local server requesting the response and the file is not available to the updater. Instructs you to update the central file server. Upon receiving a response indicating a cache miss, the local cache server retrieves the file from the origin server. Upon being instructed to update the central file server, the updater requests a file copy from the local server and delivers the file copy received thereby to the central cache server, where the file copy is stored. The delivery and storage process is preferably performed when the overall load on the central file server is low and when the local server is given enough time to retrieve the file from the origin server.

However, if the local server is deployed behind a firewall, the updater requests four copies from its origin server, which is stored in a central cache server. In this case, the feeder does not instruct the updater to begin the update process until a certain query number for the same specific information file is reached, where these queries originate from a local server located behind the firewall. Preferably, the updater is implemented by a device separate from the device implementing the feeder and separate from the file server device. This is desirable because file requests to the origin server, for example an HTTP request, require an unexpectedly large amount of time and thus cause an unexpected load on the device making the request. However, in a simplified system, it is possible to implement the updater on the same device as implementing the feeder separately from the central file server device. In an embodiment in which the device implementing the updater and the feeder interconnects the local cache server and the central file server, the device itself is not included in the central cache site with the central file server and is separate from these central file servers.

Some Internet information files are not suitable for caching. The file is called a dynamic information file, and the term dynamic is pasted because these files are continuously updated in the origin server. Examples of such files are files such as stock quotes and weather forecasts. One preferred way to handle dynamic files is to look at a list of non-cacheable files known at the updater or local server. In this way, communication of the system can be minimized depending on the user requesting the file.

According to another embodiment of the present invention, several central file servers are included in the central cache site, and each file server caches the information file associated with the original host name, IP-address or derived query number in a limited area. Based on the original host name, IP-address of the requested information file, or derived query number, the feeder addresses the query to a file server that caches the file to the appropriate extent. In this scalable method, each file server has its own disk system, thus minimizing overhead costs. In addition, the central cache site is scalable to third-party file servers because of the standardized protocol used at the site.

In order to speed up communication between the central file server and the low level computers (ie feeders and updaters), each low level computer is preferably connected by a dedicated wire, optionally by a dedicated network if several file servers are present. You are connected to a central file server. Such a network is a dedicated network or a public network. In the case of a public network, at least part of the capacity of the network is preferably reserved for the corresponding communication. The network used could, of course, be the non-dedicated Internet part. This type of connection between the central file server and the low level computer depends on whether the low level computer or feeder and updater are located at the same site as the central file server or at a different location than the central file server.

It is also desirable for the central cache site to serve a limited set of local cache servers, which service the linguistically and culturally alumni user community. This increases the hit rate at the central cache level, because more of the same information file is required at one time.

Using the present invention, an operator of an Internet caching system that processes information file requests in accordance with the present invention can serve a large number of subscribing customers in a fast, cheap, and efficient manner. These customers may preferably be other Internet service providers, companies or other organizations connected to the central cache site of the present invention or the feeder / updata of the present invention and having their own local cache server or formed by the central cache site. It is connected as a client to a system that includes the entire caching system of the invention and has a feeder and updater and a local cache server connected thereto. Of course, the customer may be a single user constituting a single WWW client connected directly to the system of the present invention. In addition, many companies or Internet service providers may choose to operate the system of the present invention on their own, rather than being connected to a system operated by others. In addition, since the caching system of the present invention is formed around standardized protocols (e.g., ICP and SQL), local cache servers and central file servers of a given manufacturer can be included in the system as long as these protocols are supported.

Within the scope of the present invention, a local Internet cache server is preferably interpreted as a proxy node, which is a WWW proxy node that maintains a cache for a user, or as a WWW client connected to a proxy node.

Items cached on a local Internet cache server or file server at a central cache site are non-dynamic files that can be accessed using the Internet and contain all kinds of information. Thus, many different types of files such as, for example, binary, text, picture, audio and video files, HyperText Transfer Protocol (HTTP) files, WWW files, File Transfer Protocol (FTP) files, WWW pages, WWW destinations, etc. The named file is included in the term internet information file used in the present invention. In addition to files accessed using the HTTP or FTP protocol, all files accessed over the Internet according to the three-layer protocol are also included in the term internet information file. Another example of a protocol that may be used is the Wireless Transport Protocol (WTP) protocol used within the Wireless Application Protocol (WAP) standard.

According to the present invention, the present invention includes a computer readable medium storing one or more computer instruction programs for one or more general purpose computers, and means for causing the one or more computers to perform the steps set forth in the attached claims 1-17. Include.

According to the invention, the invention comprises one or more program storage devices comprising one or more sequence of instructions for one or more general purpose computers to carry out the steps disclosed in the appended claims 1-17.

Features and the like of the present invention will be described with reference to the accompanying drawings.

An embodiment of the present invention will be described with reference to the drawings shown in FIG. In FIG. 1, a number of local cache servers 100 are shown. These local cache servers 100 are connected to feeder means 110 (e.g., feeder 110) via the Internet. The number of feeders 110 and local cache server 100 shown in FIG. 1 is merely an example, and is not limited to the number shown in the embodiments.

However, regardless of the number of feeders, each feeder is connected to one single central file server in this embodiment. In FIG. 1, feeder 110 is connected to central file server 130. This central file server includes a storage medium (not shown) in which Internet information files are stored (i.e. cached), implemented as a low level computer such as Sun Ultra Sparc or DEC Alpha Computer, and with a local cache server 100 It consists of a front end machine that handles communication between file servers 130.

The feeder 110 communicates with the local cache server 100 using the Internet cache protocol, which is a message based protocol used to communicate between cache servers over the Internet. Thus, feeder 110 responds to the ICP query for the cached Internet information file, which is received from one of the local cache servers 100 with the ICP response. This ICP response indicates a cache hit (ICP_OP_HIT) or a cache miss (ICP_OP_MISS).

According to the Internet cache protocol, the ICP query received by the feeder includes the URL of the information file queried. From this URL the feeder 110 derives the inquiry number corresponding to the information file queried using the MD5 hash algorithm. Using the query number, the memory resident MD5 index hash table 115 is searched. The feeder 110 includes a RAM 116, in which an index table is stored. Index table 115 includes an entry for each query number corresponding to an Internet information file cached at central file server 130. Searching the index table 115 includes searching for an entry for a query number that matches the derived query number. If a matching query number is found in the table, it indicates that the queried information file is cached in the central file server 130 and therefore the ICP response to the local server 100 indicates a cache hit. On the other hand, if no matching query number is found in the table 115, this indicates that the requested information file was not cached in the central file server 130, and therefore the ICP response indicates a cache miss.

Means for deriving the query number and searching the index table using the MD5 hash algorithm are the appropriate software modules and microprocessors 120 included in the feeder 110. The microprocessor executes the software module, by which the query number is derived and the index table 115 is searched. Implementation of such software modules is easy for one skilled in the art of programming.

If the response from the feeder 110 to the local cache server 100 is a cache hit, the local server requests an information file from the feeder using HTTP, which is the protocol used to access the WWW object over the Internet. That is, an HTTP request is sent to the feeder, and the request includes the URL of the requested file.

When communicating with the central file server 130, the feeder 110 uses a common SQL query. Upon receiving the HTTP request, the feeder retrieves the query number derived from the URL of the previously corresponding ICP response. Optionally, the URL of the HTTP request is used to derive the query number once again. The feeder uses the query number in a standard SQL statement sent to the central file server. In response, central file server 130 delivers the information file to the feeder, which feeds the information file to local server 100 that issues a request for the file.

If the response from the feeder 110 to the local cache server 100 indicates a cache miss, the local server makes an HTTP request to the origin server (not shown) of the requested file, caching the received file and requesting the user ( File copy).

The means for performing the internet cache protocol at the feeder 110 is the microprocessor 120 included in the feeder. The microprocessor also implements means for receiving HTTP requests from the local cache server 100 and means for querying the central file server 130 using SQL. The operations to be performed by the microprocessor are controlled by suitable software modules which are part of the aforementioned means. Implementations of these software modules are known to those skilled in the art in the programming art.

Another embodiment of an internet caching system according to the present invention is shown in FIG. The system of FIG. 2 shows that the Internet caching system includes an updater 240 (ie, update means), a feeder 210 and a local cache server 200 over the Internet connected to a central file server 130. It is different from the one shown in. Thus, FIG. 2 shows an apparatus of the present invention that includes a feeder 210 and updater 240.

In addition to being described below with respect to the elements of FIG. 2, the elements of FIG. 2 with the corresponding elements of FIG. 1 operate and interact as described previously with respect to FIG. 1. Thus, only the features of these elements related to the embodiment shown in FIG. 2 are described below.

Updater 240 updates the storage medium (not shown) associated with central file server 230 with the new cached information file. As illustrated in FIG. 1, when the local cache server 200 receives a cache miss in an ICP response from the feeder 210, as a response to a previous ICP query for it, the local cache server 200 may be configured to access the file. Form an HTTP request to its origin server (not shown). The requested file is received and cached at the local cache server 200. After a predetermined time, according to the cache miss reported in the ICP response, the feeder 210 instructs the updater 240 to update the central file server.

The updater 240 receives the URL of the file inquired from the feeder 210 and the ID of the local cache server 200 inquired about the file. HTTP requests for files are forwarded from the updater to a specific local server. Upon receiving the requested file, the updater stores, i.e., caches the file in the central file server 230. Once the file is saved, the updater instructs the feeder to add a query number corresponding to the file in the index table 215 stored in the RAM area 216.

The means for requesting the information file from the local cache server 200 and the means for caching the information file received at the central file server 230 is the microprocessor 260 along with a suitable software module included in the updater 240. Implementation of these software modules is easy for those skilled in the program arts.

An example of the operation performed by the local cache server 200 of the embodiment of FIG. 2 is described with reference to the flowchart of FIG. 3.

In step 300, the local cache server 200 receives a request for an internet information file from a client serviced by a particular local cache server. However, the file request may be received from the updater 240 operating according to the description associated with FIG. Next in step 301 the local cache server searches for a locally cached file for the requested file. If the file is found, the file is forwarded to the requesting client or updater 240, which is shown in step 302.

If the local cache server 200 does not find the requested file during the search, that is, the requested file has not been cached, it is checked in step 303 if the request was passed from the updater 240. If this condition is true, a message is sent to the updater stating that the file requested in step 304 is not available. If the condition of step 303 is false, i.e., if the request was sent from the client, then the ICP query is passed to the feeder 210 in step 305. In a next step 306, the local cache receives an ICP response from the feeder 210 indicating whether the central file server 230 cached the requested file. In step 307, the ICP response is evaluated. If the response indicates a cache miss, that is, the requested file was not cached centrally, the local cache server 200 forms an HTTP request for the file delivered to the origin server of the file. On the other hand, if the response indicates a cache hit, the local cache sends an HTTP request to the feeder 210 for the file, which is shown in step 309. In a next step 310, the local cache server receives the requested file from the feeder. Finally, at step 311, the file is delivered to the client who requested the file.

The operations performed by the feeder 210 in the embodiment of FIG. 2 are described with reference to the flowchart of FIG. 4.

In step 400, feeder 210 receives an ICP query for an Internet information file from local cache server 200 processed by the feeder. The query includes the URL of the information file inquired. From this URL, in step 401 the feeder 210 derives a query number using the MD5 hash algorithm, which is used when searching the index MD5 hash table residing in the memory 216 of the feeder 210. Used at 402.

If the number is not found in the hash table search, at step 403 the feeder sends an ICP response indicating a cache miss back to the local cache server 200 where an ICP query is received. In step 404, feeder 210 instructs updater 240 to retrieve the uncached queried file by sending the URL of the queried file to the updater. At step 405, feeder 210 adds a query number corresponding to the file queried in index hash table 215. This is done by the updater 240 notifying the feeder that the queried file has been delivered from the local cache server 200 and stored in the central file server 230. Operation of the updater 240 is described with reference to FIG. 5.

If the feeder 210 finds the query number during the search of the hash table 215 in step 402, it sends back an ICP response indicating a cache hit to the local cache server 200 where the ICP query is received in step 406. . In step 407, the feeder receives an HTTP request from the local cache server 200 that previously issued the ICP query. Similar to the ICP query, the HTTP request contains the URL of the requested information file. At step 408, feeder 210 retrieves the previously derived query number corresponding to the file. Based on this query number, in step 409 the feeder queries the central file server 230 for the requested information file using a standard SQL query. In step 410, the feeder receives the cached information file from the central file server 230 as a response and in the next step 411, the requested cached Internet information file is requested from the feeder 210. 200).

Operations performed by the updater 240 of the embodiment of FIG. 2 are described with reference to the flowchart of FIG. 5.

In step 500, updater 240 receives a command from feeder 210 indicating that a particular file should be requested. The file was already requested by the local cache server 200, but the feeder found that the central file server 230 did not cache the file. The command includes the URL of the file and the address of the local cache server 200 that requested the file from the central file server 230. In step 501 the updater checks the requested file of the command for a list of known non-cacheable files. If the list contains the requested file, the command is deleted. If the list does not contain the requested file, the command will be kept on updater 240 and will require time for local cache server 200 to retrieve the file from the origin server.

At a time appropriate for the central file server 230, i.e. associated with low load on the central server, the central server sends a message to the updater 240 indicating that an undetermined command should be performed, which is updated by the updater 240. Reception is shown in step 502. In the next step 503, execution of the command begins with requesting a copy of the file from which the updater is to be retrieved and cached locally from the local cache server 200 requesting the file. The file copy is received from the local cache server at step 504. In step 505, the received file copy is sent to the central file server 230 for caching by the central file server 230. In the last step 506, the updater 240 instructs the feeder 210 to add to the index hash table 215 the query number corresponding to the cached file to the central file server 230.

The operation of the central file server 230 is simple. Basically, this performs two tasks: responding to SQL queries from the feeder 210 by sending a cached file to the feeder 210 and storing the new information file delivered from the updater 240 in the cache.

Another embodiment of an internet caching system according to the present invention is described with reference to FIG. The system of FIG. 6 differs from the system of FIG. 2 in that it has one or more central file servers, where three central file servers 630 are shown by way of example. 6 also includes two feeders 610, each feeder connected to its own set of local cache servers 600. FIG. Feeder 610 and updater 640 are disposed with central file server 630 at central cache site 690. By the Ethernet network 680 located in the central cache site, the updater 640 and each feeder 610 are connected to all central file servers 630.

The additional number of central file servers in this embodiment allows more files to be cached and more SQL queries to be answered by the central file server as compared to the embodiment of FIG. Since the system can be fully extended, any number of feeders, updaters, or central file servers can theoretically be added to the system.

The basic difference between the operation of the system of FIG. 6 and the system of FIG. 2 is that the feeder 610 must select one server from multiple central file servers 630 to which SQL queries will be forwarded. Each central file server 630 caches the information file within the original host name within a predefined range, and the selection of one of the central file servers is included in the URL received from the local server as part of the ICP query or as part of the HTTP request. Is performed according to the specified host name. If one of the central file servers is selected by the feeder, the SQL query with the derived query number is passed to the selected file server.

The configuration and function of the elements described based on the drawings are apparent to those skilled in the art.

Although the present invention has been described with reference to specific embodiments, many variations, modifications, and the like will be apparent to those skilled in the art. Accordingly, the described embodiments are not intended to limit the scope of the invention, which is defined by the appended claims.

Claims (46)

A method of servicing a request for an Internet information file in an internet caching system, the method comprising: Receiving a user request from a user for an internet information file at a local internet cache server; In response to the received request, generating an inquiry of the information file if the information file is not cached at the local server; In response to the query, generating a file request for the information file, if the central file server storing the cached internet information file causes the information file to be cached. Delivered to the feeder means; And Querying the central file server for the information file from the feeder means in response to the file request, Reducing the load on the central file server. 2. The method of claim 1, wherein the query is performed by the local cache server according to a protocol used for communication between internet cache servers. 3. The method of claim 2, wherein said protocol is an Internet Cache Protocol (ICP). 3. The method of claim 2 wherein the protocol is a cache digest. The method of any of claims 1-3, wherein the query is forwarded by the local cache server to the feeder means, wherein the feeder means sends back the response. 6. The method according to claim 5, further comprising the step of deriving in said feeder means an inquiry number corresponding to said information file relating to said inquiry. 7. The method of claim 6, wherein said querying means comprises using said derived query number when querying said central file server for said information file. 7. The method of claim 6, wherein the query provides an alphanumeric string associated with the information file, the string being used to derive the query number. 9. The method of claim 8, wherein the alphanumeric string is a Uniform Resource Locator (URL) and the query number is derived from at least a portion of the URL and the query header information file. 5. The method of claim 1, 2 or 4, wherein the file request provides an alphanumeric string associated with an information file, wherein the string is used by the feeder means to derive a query number corresponding to the information file. Way. 11. The method of claim 10, wherein the alphanumeric string is a Uniform Resource Locator (URL) and the query number is derived from at least a portion of the URL and the query header information file. The method of any one of the preceding claims, further comprising generating an index table having an entry for each Internet information file cached at the central file server. The method of claim 12, Performing a search in the index table for the information file; And And indicating in response to the query whether the information file was found during the search. The method of any one of the preceding claims, wherein the querying step comprises using a Structured Query Language (SQL) when querying the central file server for the information file. The method of any one of the preceding claims, wherein the querying step is: Selecting one central file server from a set of central file servers for caching Internet information files with the original host name or IP-address within a preset range, based on the original host name or the IP-address of the information file; And Querying the selected central file server for an information file. 15. The method of any of claims 6-14, wherein the querying step is: Selecting one central file server from a set of central file servers each of which caches an Internet information file having a corresponding inquiry number within a preset range based on the inquiry number derived for the information file; And Querying the selected central file server for an information file. The method of claim 1, wherein At the local cache server, retrieving an information file from an origin server if the response to the query indicates that the information file is not cached at a central file server; Caching the information file at the local cache server; And Updating the central file server by requesting a copy of the information file from a private local cache server and caching the copy to the central file server. 10. An apparatus for reducing load on a central file server in an internet caching system that includes at least one local cache server and at least one central file server that stores cached internet information files. The apparatus includes a feeder in communication with the local cache server and a central file server, the feeder comprising: First means for receiving a request for an Internet information file from the local cache server; Second means for deriving a query from an alphanumeric string received from the local cache server; And And third means for querying said central file server for said internet information file using a query directed by said second means. 19. The apparatus of claim 18, wherein said first means operates in accordance with a three-layer Internet protocol. 20. The apparatus of claim 18 or 19, wherein the third means uses SQL to query the Internet information file. 21. The apparatus of any one of claims 18-20, wherein the alphanumeric string is included in the request received from the local cache server. 22. The apparatus of claim 21, wherein the query is derived from the alphanumeric string and at least a portion of a header information file of a request received from the local cache server. 23. The apparatus of claim 22, wherein the query includes a query number, the query number being derived by applying a hash algorithm to the string and a portion of the header information file. The method of claim 18, wherein the feeder is: Fourth means for receiving a query for an Internet information file from the local cache server; And And fifth means for providing a response to the received query to the local cache server. 25. The apparatus of claim 24, wherein the fourth and fifth means operate according to a protocol used for communicating between internet cache servers. 27. The apparatus of claim 25, wherein the protocol is an Internet Cache Protocol (ICP). 27. The apparatus of any of claims 24-26, wherein the alphanumeric string is included in the query received from the local cache server. 28. The apparatus of claim 27, wherein the query derived by the second means is derived from at least a portion of the alphanumeric string and a header information file of the query received from the local cache server. 29. The apparatus of claim 28, wherein the query includes a query number, the query number being derived by applying a hash algorithm to the string and a portion of the header information file. 30. The apparatus of any one of claims 24-29, wherein the feeder comprises a table having a copy of the entire index of all internet information files cached in the central file server. 31. The apparatus of claim 30, wherein the response to the received query by the fifth means is based on the contents of the table. 32. The system of any of claims 18-31, further comprising an updater in communication with the local cache server and the central file server, wherein the updater comprises: Requesting means for requesting a copy of an internet information file stored in a local cache server; And Storage means for storing the received copy on a central file server. 33. The apparatus of claim 32, wherein the requesting means requests a copy of the information file from the origin server if the local cache server storing the information file is located behind a firewall. 34. The apparatus of claim 32 or 33, wherein the updater communicates with the feeder to receive a command requesting a copy of the information file. 35. The apparatus of any one of claims 32-34, wherein the updater comprises a list of known non-cacheable information files for which no copy can be requested. 36. The apparatus of any of claims 16-35, wherein the feeder is implemented by a low level computer and the central file server is implemented by a high level computer. 36. The apparatus of any of claims 32-35, wherein the updater is implemented by a low level computer and the central file server is implemented by a high level computer. 38. The apparatus of claim 37, wherein the updater and at least one feeder are implemented with one low level computer. In the internet caching system, A set of local Internet cache servers each receiving a request from a user for an Internet information file; At least one central file server included in the central cache site and storing cached Internet information files; And Feeder means for interconnecting said central file server and said local cache server set, The feeder means comprises at least one feeder, the feeder being connected to the internet from the central file server using a database query and means for communicating with at least one local cache server according to a protocol used for communicating between internet cache servers. Means for retrieving an information file to reduce the load on said central file server. 40. The system of claim 39, wherein said feeder means is included in said central cache site. 41. The Internet caching system of claim 39 or 40, wherein each feeder means comprises a plurality of feeders, each feeder interconnecting the subset of the local cache server set and the central file server. . 42. The Internet caching system of any one of claims 39-41, wherein the central cache site services a limited set of local cache servers that serve a linguistic and cultural homogeneous user community. 43. The Internet caching system of any one of claims 39-42, wherein the protocol used is an internet cache protocol or cache digest. 44. The Internet caching system of any one of claims 39-43, wherein each feeder includes a table having a full index copy of all information files cached at the central cache site. 45. The Internet caching system of any one of claims 39-44, wherein the central file server includes a cached internet information file with an original host name within a preset range. 46. The system of any one of claims 39-45, wherein the at least one cache of the set is interconnected with the central file server to derive a copy of an internet information file from an origin server or the at least one local cache server. Further comprising updater means for storing said copy on a server.