GB2295035A - Computer network distributed data storage. - Google Patents

Abstract

A computer network having a root server 8 connected via telecommunications links 14, 16 to a plurality of intermediate servers 10, 12. Each intermediate server 10, 12 is connected to a client terminal 20, 24 via further telecommunications links 18, 22. The root server 8 stores a database composed of a plurality of data blocks. The intermediate servers store a subset of the data blocks held within the root server 8. When a client terminal 20, 22 makes a request for a particular data block, the intermediate server 10, 12 attempts to satisfy this request and only if it does not have a copy of the data block concerned will it request it from the root server 8. When the data block is returned from the root server 8, it is stored within the intermediate server 10, 12 overwriting the least recently used data block held within the intermediate server 10, 12. Subsequent requests for that data block routed via the intermediate server 10, 12 concerned are able to be satisfied by the intermediate server without recourse to the root server 8. <IMAGE>

Description

COMPUTER NETWORK DISTRIBUTED DATA STORAGE This invention relates to the field of data processing. More particularly, this invention relates to the field of computer networks in which a root server stores a database composed of a plurality of data blocks that are accessed over telecommunications links by one or more client terminals.

It is known to provide computer networks having a root server storing a database that may be accessed via telecommunication links from remote client terminals. Figure 1 of the accompanying drawings schematically illustrates such an arrangement. In this computer network a number of client terminals 2 are linked via local telecommunication lines to a network node 4 that is typically situated in the same town or city. These network nodes 4 are schematically illustrated as modems, although in practice they are more sophisticated devices. The various nodes 4 of the network are linked via long distance telecommunication lines (such as optical fibre links) to a distant root server 6 that may typically be situated in a distant town/city or even a different country.In practice there may often be other layers between the nodes 4 and the root server 6 comprising further nodes or intermediate servers.

One problem with computer networks such as illustrated in Figure 1 is that a concentration of traffic tends to occur at the root server since all data access requests and the responses to these requests for all of the client terminals must pass through these lines. Another problem with such systems is the cost of transmitting data over long distances via the telecommunication links. These links are expensive to install and maintain and capacity upon them is expensive to purchase.

Viewed from one aspect this invention provides a computer network comprising: a root server storing a plurality of data blocks forming a database; a client terminal for generating a data block request for one or more data blocks of said database, said client terminal being remote from said root server; an intermediate server coupled with telecommunications links to said root server and to said client terminal, communication between said client terminal and said root server being routed via said intermediate server, said intermediate server comprising: (i) data storage means for storing one or more previously requested data blocks of said database routed via said intermediate server; and (ii) request intercepting means responsive to said data block request for transmitting to said client terminal from said data storage means rather than from said root server those of said previously requested data blocks stored within said data storage means that correspond to said data blocks of said data block request.

The invention recognises that the probability of an access request being made to a particular data block within a database is not uniform between data blocks. More particularly, a relatively small number of the data blocks within a data base will be many times more popular than some other data blocks, e.g. in a news database, the data blocks relating to today's news will be very many times more likely to be accessed than those relating to the news in the distant past. The invention exploits this realisation by providing intermediate servers between the client terminals and the root server which are able to store a subset of the main database.The intermediate servers need not have a storage capacity sufficient to store the entire database (such a storage capacity may well be prohibitively expensive and/or impractical to provide) since a significant fraction of data block requests will relate to only a relatively small subset of the data blocks of the entire database. Accordingly, the intermediate servers positioned between the client terminals and the root server serve to intercept a portion of data block requests made to the root server and instead satisfy these locally from the data that they themselves hold.

This reduces the data transfer loads near to the root server so alleviating a potential bottleneck in system performance and reducing the amount of long distance data transmission that need take place and consequently the costs of operating the system.

It will be appreciated that a problem may arise with data consistency and integrity when a data block may be stored at more than one location within a computer network. In order to reduce the potential for such problems, preferred embodiments of the data are such that said data blocks comprise read only data that may not be modified other than at said root server.

Recognising that the capacity of the data storage means within the intermediate server is finite and unlikely to be able to hold the complete database, it is important that some mechanism is provided to manage which data blocks are to be held within this data storage means.

Accordingly, in preferred embodiments of the invention said intermediate server includes means for overwriting in said data storage means the least recently accessed data blocks with any data blocks received from said root server.

This relatively simple technique of overwriting the least recently accessed data blocks with the latest data blocks being routed via the intermediate server provides an effective mechanism for ensuring that the most popular data blocks are the ones that are stored by the intermediate server. It is possible that more sophisticated algorithms and other considerations, such as speed of response for critical data blocks, might determine that a parameter other than popularity would be better used to determine which data blocks should be held by the intermediate server.

It will be appreciated that the invention is particularly suited to networks having a hierarchical structure in which a plurality of client terminals are connected to each intermediate server and a plurality of intermediate servers are connected to a root server. It is also possible that more than one root server may be provided holding the full database with intermediate servers deciding upon which root server to access for a data block depending upon the prevailing conditions within the computer network as a whole.

In a computer network in which a given data block may be stored in more than one location, it is important to be certain of the identity of each data block. Accordingly, in preferred embodiments of the invention each data block includes an identifier unique within said computer network to a particular version of a particular data block.

The storing of data blocks within the intermediate servers may be advantageously simplified in preferred embodiments of the invention in which said data blocks have a uniform length.

As previously mentioned, the computer network may be a hierarchical system having more than one layer of intermediate servers, i.e. one or more further intermediate servers are disposed via telecommunication links between said client terminal and said root server, said data block request being routed by said one or more further intermediate servers, each further intermediate server having a data storage means and a request intercepting means.

A problem within such a computer network is how to ensure that the intermediate servers and client terminals do not intercept data block requests and service them with out of date data blocks. In order to deal with this, preferred embodiments of the invention are such that said root server, said intermediate server and any further intermediate servers include means for transmitting a deletion code indicative of which data blocks are no longer valid and should be deleted from said computer network.

The transmitting of a deletion code from the root server downwards through the network enables the flushing of out of date data blocks to be achieved. The deletion codes may be broadcast on a general basis, but preferably are requested from the root server or one of the intermediate servers by the client terminal or intermediate server immediately underlying it in the hierarchy whenever that client terminal or underlying intermediate server is logged into the system when switched on or makes a request from the system. If broadcasting of deletion codes is not undertaken, then it is preferred that intermediated servers should poll for deletion notices at a time determined by the nature of the application.

In order to reduce the amount of network traffic such that data unnecessary to the management of the data blocks is not transmitted, preferred embodiments of the invention are such that said client terminal, said intermediate server and any further intermediate servers include means for comparing a received deletion code with a stored deletion code and. if said received deletion code and said stored deletion code do not match, then generating a request for a deletion list of data blocks that are no longer valid, means for receiving said deletion list and treating any stored data blocks specified in said deletion list as invalid, and means for storing said deletion list for retransmission in response to a request for a deletion list from elsewhere.

It will be appreciated that whilst the management of the data blocks throughout the computer network using the deletion codes is efficient and fast, it presents a potentially unbounded requirement to store the deletion data for the entire history of the database since one could not guarantee a maximum period of time for which a client terminal or intermediate server is switched off or disconnected from the network and then reconnected and attempts to use an out of date data block.

Accordingly, in preferred embodiments of the invention each data block includes a persistence parameter indicative of a period of time for which any deletion list relating to said data block will be maintained within said computer network.

The inclusion of a persistence parameter acts as a fall back to the deletion code technique whereby once a data block has exceeded its period of validity it will be discarded from the network element storing it even without a deletion instruction such that deletion instructions need not be stored permanently.

It will be appreciated that the data blocks may store many different types of data, but these will typically include text data, audio data, video data and program instructions for controlling execution of the client terminals or intermediate servers.

It will be appreciated that providing the intermediate servers with a data storage capacity enables them to intercept a significant fraction of data block requests and so reduce the load upon the root server. The same technique may be additionally applied to the client terminals whereby data storage within the client terminals themselves is able to satisfy some of the data block requests and so avoid a data transmission even to the relatively nearby intermediate server.

The computer terminals may be relatively low function terminals, but will preferably be of a type having a central processing unit, a memory and a non-volatile data storage device. Having a client terminal capable of a significant degree of data processing for itself reduces the amount of network traffic.

Viewed from another aspect the invention provides a method of distributing data via a computer network, said method comprising the steps of: storing in a root server a plurality of data blocks forming a database; generating within a client terminal a data block request for one or more data blocks of said database, said client terminal being remote from said root server; routing communication between said client terminal and said root server through telecommunications links via an intermediate server; storing within a data storage means of said intermediate server one or more previously requested data blocks of said database routed via said intermediate server; and in response to said data block request, transmitting to said client terminal from said data storage means rather than from said root server those of said previously requested data blocks stored within said data storage means that correspond to said data blocks of said data block request.

A complementary aspect of the invention provides an intermediate server connected via telecommunications links to a root server storing a plurality of data blocks forming a database and a client terminal, said client terminal generating a data block request for one or more data blocks of said database and being remote from said root server, said intermediate server comprising: data storage means for storing one or more previously requested data blocks of said database routed via said intermediate server; and request intercepting means responsive to said data block request for transmitting to said client terminal from said data storage means rather than from said root server those of said previously requested data blocks stored within said data storage means that correspond to said data blocks of said data block request.

A further complementary aspect of the invention provides a client terminal for receiving data blocks of a database stored in a root server via a computer network, said client terminal comprising: means for comparing a received deletion code with a stored deletion code and, if said received deletion code and said stored deletion code do not match, then generating a request for a deletion list of data blocks that are no longer valid; and means for receiving said deletion list and treating any data blocks stored within said client terminal and specified in said deletion list as invalid.

A still further complementary aspect of the invention provides a root server for storing data blocks forming a database as part of a computer network, said root server comprising: means for transmitting a deletion code indicative of which data blocks are no longer valid and should be deleted from said computer network.

An embodiment of the invention will now be described, by way of example only, with reference to the accompanying drawing, in which: Figure 1 schematically illustrates a computer network; Figure 2 schematically illustrates a computer network incorporating intermediate servers; Figure 3 illustrates the variation in popularity of requests to different data blocks; and Figures 4 to 7 illustrate the operation of the computer network illustrating Figure 2.

Figure 2 illustrates a computer network incorporating a root server 8 storing a database comprising a plurality of data blocks of uniform length. Two intermediate servers 10, 12 are coupled to the root server 8 via telecommunications links 14, 16. Further telecommunications links 18 couple a plurality of client terminals 20 to the root server 10. Similar telecommunications links 22 couple a plurality of client terminals 24 to the intermediate server 12.

Typically, the root server 8 is a mainframe computer having a very large data storage capacity capable of holding a complete database. In contrast intermediate servers 10, 12 are smaller computers having insufficient storage capacity, either in total or when their other duties are taken into account, to hold the entire database stored within the root server. The client terminals 20, 24 are typically personal computers having their own central processing units, memories and non volatile storage in the form of computer hard discs.

In use, a user sitting in front of a clients terminal 20, 24 enters a command to access a given block of data within the database held upon the root server 8. This data block request passes via the telecommunications links 18, 22 to the associated one of the intermediate servers 10, 12. The intermediate server 10, 12 then checks to see if the data block requested is in fact already held by the intermediate server 10, 12. If the data block is so held, then it is returned to the clients terminal via the telecommunications links 18, 22 without recourse to the root server 8. If the data block is not held, then the intermediate server 10 requests the data block from the root server 8.The root server 8 holds the complete database and so is able to service this request and return the data block to the intermediate server 10, 12 from where it is passed down to the requesting client terminal 20, 24.

When the intermediate server 10, 12 requests the data block concerned from the root server 8, it does so in its own name. The root server 8 need have no knowledge of which particular client terminal is requesting that data block. The intermediate server 10, 12 itself is responsible for passing the data block received from the root server 8 down to the appropriate one of the client terminals 20, 24. This avoidance of the need for the root server 8 to individually address client terminals 20, 22 is a significant advantage since it avoids problems whereby a single user identifier is used by more than one user having access to the root server. With this technique, the requirement is merely that user identifiers should not be duplicated between client terminals sharing a common intermediate server at the layer above them.

Providing this pathway is uniquely identified, then the computer network may support multiple users having the same user identifier.

The computer network illustrated in Figure 2 has only one layer of intermediate servers 10, 12. In practice, multiple layers of intermediate servers 10, 12 may be provided between the client terminals 20, 24 and the root server 8. In addition, more than one root server 8 may be provided holding the complete database. In this case, the intermediate servers 10, 12 can make a decision as to which root server 8 to access for a given block of data depending upon the prevailing data traffic conditions.

Figure 3 illustrates a typical variation in the popularity of different data blocks within a database. Certain data blocks relate to information that is frequency accessed, whilst other data blocks are of less common interest. As an example, the data relating to a video clip of a latest news story might be frequently accessed, whereas the data relating to the text of an obscure committee report issued several years ago would be only infrequently accessed. This variation is exploited by the invention. Whilst the root server must store the entire database so as to be able to ultimately service any request for a data block issued by a user, the intermediate servers can employ a smaller data storage capacity to hold a sub-set of data blocks that are the most popular of the data blocks.Accordingly, as illustrated in Figure 3, the most popular data blocks are stored in both the intermediate servers and the root servers, whilst the less popular data blocks are stored only in the root server.

Figure 4 schematically illustrates the operation of the invention. A root server stores a plurality of data blocks having unique identifiers within the system numbered 01 to 48. The root server has sufficient data storage capacity to hold all of these data blocks. In contrast, the intermediate server has sufficient storage capacity to hold twelve data blocks. These are a subset of the data blocks held by the root server and represents those data blocks that were most recently requested.

In operation, a client terminal issues a data block request for data block 08. This data block request is passed to the intermediate server. In this case, the intermediate server has not got a copy of the data block 08 within its data storage means. Accordingly, the intermediate server itself requests this data block 08 from the root server. The root server returns the data block 08 to the intermediate server where it is stored. The intermediate server adopts a storage strategy where each data block received from the root server is stored by overwriting the least recently requested data block held within the intermediate server. The intermediate server then passes the data block 08 down to the client terminal that originally requested it.

It will be seen that of the two intermediate servers illustrated in Figure 4, certain of the data blocks stored by the intermediate servers are common. Thus, data blocks 35, 36, 37 and 38 might represent a commonly requested video clip. Data block 01 might represent a block of computer instructions that are basis to the operation of the client terminals and so are frequently requested by the client terminals. Conversely, neither of the intermediate servers initially stores the data block 08. This may mean that the data block 08 is only rarely requested or it may be that it is a new data block.

Figure 5 illustrates a data access request subsequent to that illustrated in Figure 4. In this case, a client terminal different to that in Figure 4 again requests data block 08. Thus, a data block request is passed to the intermediate server. However, since the data block 08 was recently requested by another client terminal, it is stored within the intermediate server and the intermediate server is able to immediately return the data block 08 without recourse to the root server. This saves network traffic and the expense of telecommunications.

Figure 6 illustrates a data block request made for a plurality of sequentially numbered data blocks. In this case, the data blocks are not present within the intermediate server and so the request is passed on to the root server. The data blocks 49, 50, 51 and 52 are then returned from the root server and stored within the intermediate server overwriting the least recently used data blocks within the intermediate server.

As illustrated in Figure 7, when a different client terminal (or the same client terminal), subsequently makes a request for those same data blocks, then this can be met from the intermediate server without recourse to the root server.

Claims (24)

1. A computer network comprising: a root server storing a plurality of data blocks forming a database; a client terminal for generating a data block request for one or more data blocks of said database, said client terminal being remote from said root server; an intermediate server coupled with telecommunications links to said root server and to said client terminal, communication between said client terminal and said root server being routed via said intermediate server, said intermediate server comprising: (i) data storage means for storing one or more previously requested data blocks of said database routed via said intermediate server; and (ii) request intercepting means responsive to said data block request for transmitting to said client terminal from said data storage means rather than from said root server those of said previously requested data blocks stored within said data storage means that correspond to said data blocks of said data block request.

2. A computer network as claimed in claim 1, wherein said data blocks comprise read only data that may not be modified other than at said root server.

3. A computer network as claimed in any one of claims 1 and 2, wherein said intermediate server includes means for overwriting in said data storage means the least recently accessed data blocks with any data blocks received from said root server.

4. A computer network as claimed in any one of claims 1, 2 and 3, wherein a plurality of client terminals are coupled to said intermediate server.

5. A computer network as claimed in any one of the preceding claims, wherein a plurality of intermediate servers are coupled to said root server.

6. A computer network as claimed in any one of the preceding claims, wherein each data block includes an identifier unique within said computer network to a particular version of a particular data block.

7. A computer network as claimed in any one of the preceding claims, wherein said data block have a uniform length.

8. A computer network as claimed in any one of the preceding claims, wherein one or more further intermediate servers are disposed via telecommunication links between said client terminal and said root server, said data block request being routed by said one or more further intermediate servers, each further intermediate server having a data storage means and a request intercepting means.

9. A computer network as claimed in any one of the preceding claims, wherein said root server, said intermediate server and any further intermediate servers include means for transmitting a deletion code indicative of which data blocks are no longer valid and should be deleted from said computer network.

10. A computer network as claimed in claim 9, wherein said client terminal, said intermediate server and any further intermediate servers include means for comparing a received deletion code with a stored deletion code and, if said received deletion code and said stored deletion code do not match, then generating a request for a deletion list of data blocks that are no longer valid, means for receiving said deletion list and treating any stored data blocks specified in said deletion list as invalid, and means for storing said deletion list for retransmission in response to a request for a deletion list from elsewhere.

11. A computer network as claimed in any one of the preceding claims, wherein each data block includes a persistence parameter indicative of a period of time for which any deletion list relating to said data block will be maintained within said computer network.

12. A computer network as claimed in claim 11, wherein said intermediate server and any further intermediate servers include means responsive to said persistence parameters of data blocks within their data storage means for deleting any data blocks indicated to be invalid.

13. A computer network as claimed in any one of the preceding claims, wherein said data blocks include one or more of: (1) text data; (2) audio data; (3) video data; and (4) program instructions for execution upon said client terminal to control access to said computer network.

14. A computer network as claimed in any one of the preceding claims, wherein said client terminals include client data storage means for storing previously received data blocks, least recently used data blocks within said client data storage being overwritten by received data block when said client data storage means is full.

15. A computer network as claimed in any one of the preceding claims, wherein said client terminal comprises a computer having a central processing unit, a memory and a non-volatile data storage device.

16. A method of distributing data via a computer network, said method comprising the steps of: storing in a root server a plurality of data blocks forming a database; generating within a client terminal a data block request for one or more data blocks of said database, said client terminal being remote from said root server; routing communication between said client terminal and said root server through telecommunications links via an intermediate server; storing within a data storage means of said intermediate server one or more previously requested data blocks of said database routed via said intermediate server; and in response to said data block request, transmitting to said client terminal from said data storage means rather than from said root server those of said previously requested data blocks stored within said data storage means that correspond to said data blocks of said data block request.

17. An intermediate server connected via telecommunications links to a root server storing a plurality of data blocks forming a database and a client terminal, said client terminal generating a data block request for one or more data blocks of said database and being remote from said root server, said intermediate server comprising: data storage means for storing one or more previously requested data blocks of said database routed via said intermediate server; and request intercepting means responsive to said data block request for transmitting to said client terminal from said data storage means rather than from said root server those of said previously requested data blocks stored within said data storage means that correspond to said data blocks of said data block request.

18. A client terminal for receiving data blocks of a database stored in a root server via a computer network, said client terminal comprising: means for comparing a received deletion code with a stored deletion code and, if said received deletion code and said stored deletion code do not match, then generating a request for a deletion list of data blocks that are no longer valid; and means for receiving said deletion list and treating any data blocks stored within said client terminal and specified in said deletion list as invalid.

19. A root server for storing data blocks forming a database as part of a computer network, said root server comprising: means for transmitting a deletion code indicative of which data blocks are no longer valid and should be deleted from said computer network.

20. A computer network substantially as hereinbefore described with reference to Figures 2 to 7 of the accompanying drawings.

21. A method of distributing data via a computer network substantially as hereinbefore described with reference to Figures 2 to 7 of the accompanying drawings.

22. An intermediate server substantially as hereinbefore described with reference to Figures 2 to 7 of the accompanying drawings.

23. A client terminal substantially as hereinbefore described with reference to Figures 2 to 7 of the accompanying drawings.

24. A root server substantially as hereinbefore described with reference to Figures 2 to 7 of the accompanying drawings.