GB2411018A - System for accessing data held in a plurality of storage locations - Google Patents

Abstract

A system that enables uses to access diverse data resources that are stored on a plurality of servers and data sources that are of different types and may be geographically dispersed, for example across countries or even continents. The system presents data resources to a user such that documents of the same data type are grouped together, regardless of their physical or geographical location or other factors.

Description

- 1 - 2411 18

SYSTEM

The present invention relates to systems for use in enabling users to access a multiplicity of documents and databases that may be stored on a number of servers or data stores, the data stores being of different types and geographically dispersed.

The management and transfer of data between systems is the cause of many problems within complex communications networks that are found within large organizations. It is particularly relevant to companies that are growing, either through merger or acquisition, and organizations, such as government, that are now having to make existing information available to a wider audience through new media channels such as the Internet.

The problem occurs because there is often no standard mechanism or framework to reliably transfer data from one system to another, for example transferring data from production systems that use one set of formats to management and commercial systems that use different formats. The deficiency is frequently addressed by creating an in-house solution to connect such systems together. These systems may evolve over time, and, as new interfaces are required, additional programs are added to, but not necessarily integrated with, earlier solutions. This piecemeal approach results in the inefficient duplication of equipment and makes it vary difficult to trace the data flow between different systems. A result of this is that maintenance and upgrade activities become unnecessarily complex. - 2 -

Scheduling can also become a major problem, especially where application programs are responsible for their own task management. With multiple applications, possibly requiring manual intervention, scheduling becomes, at best, crude, inefficient and difficult to manage. Simple timetabled scheduling becomes highly inefficient if the volume of data to be processed is unpredictable, leading to the time required to perform the task becoming highly variable.

According to a first aspect of the present invention there is provided a system for accessing one or more of a plurality of data resources stored on a plurality of data stores, the data resources being one of a plurality of types, the system comprising: a user interface for enabling a user to view and manipulate one or more data resources; a plurality of domains, each domain comprising a representation of one or more data resources, each domain being associated with a single type of data resource; and a system management entity for holding a record of each data resource stored on the plurality of data stores and providing a record to each domain of each data resource of the data resource type associated with that domain.

This enables a user to see all documents of the same type being brought together in a single viewing location regardless of the location of the server storing the file.

Such a system is significantly less complex than known solutions and provides more effective network utilisation and enables scheduling of network operations.

Preferably, the system comprises a plurality of system management entities, with each domain having a unique - 3 associated system management entity. It is further preferred that the or each system management entity comprises a resource area and one or more resource location adaptors.

One or more of the plurality of domains may be connected to one further domain by a conduit, the conduit comprising configuration data that comprises the data resource type associated with the two domains connected to the conduit and furthermore one or more of the plurality of domains may be connected to a plurality of further domains by a plurality of conduits, with each further domain being associated with a unique conduit. Preferably a conduit controls the transmission of data resources between the domains connected by the conduit, such that the data resource is converted from the data resource type associated with the transmitting domain to the data resource type associated with the receiving domain. The conduit may send the transmitted data resource to a conversion application process to convert the data resource to the data resource type associated with the receiving domain and to forward the converted data resource to the receiving domain.

The system preferably further comprises a system database comprising data records regarding each of the plurality of domains, the or each system management entity and the one or more conduits.

The invention and its method of operation will be described by way of illustration only and with respect to the accompanying drawings, in which - 4 - Figure 1 shows a schematic depiction of a system according to the present invention.

Figure 2 is a flowchart illustrating the operations comprising a user operation; and Figure 3 is a schematic depiction of a further system according to the present invention.

Figure 1 shows a schematic depiction of a system according to the present invention. The system 10 comprises a user interface 100 that interfaces with a plurality of domains 200, 210, 220, .... For ease of comprehension only three domains are shown in Figure 1 but it will be understood that the system 10 may comprise a lesser or greater number of domains. Domain 200 is connected to domain 210 by conduit 215 and domain 210 is connected to domain 220 by conduit 225.

Each domain may be connected to one, some or all of the other domains in the system by a conduit, depending on the requirements of the system. Each domain comprises a respective database 202, 302, 402,..., referred to as a document log, that comprises data concerning the documents associated with that domain.

Domain 200 is connected to resource area 300 by interface 305 and in turn resource area 300 is connected to resource location adaptor 400 by interface 405. The interface 405 operates in conjunction with data file 402. Domains 210 and 220 are connected to respective resource areas 310, 320 and resource location adaptors 410, 420 in an similar manner.

Each of the to resource location adaptor 400, 410, 420 are connected to one or more of data stores 500, 510, . - 5 The user interface 100 enables a user to view documents that are stored across the data stores. In conventional computer operating systems, a folder or directory stores one or more files that the user stores together. In the present invention, the domains present to the user, via the user interface, documents or data resources of the same type (such as, for example, HTML documents or SQL databases) that are stored across the different data stores. This enables a user to access any HTML document, for example, from HTML document domain 200 without needing to consider which of the data stores the document is stored upon or having to login to the data store or any intermediate network element.

A request for a document from the user interface will be passed to the domain associated with the document type; this request will be forwarded to the associated resource area and then to the associated resource location adaptor which retrieves the document from the data store on which it is located. The document is then returned to the domain, via the resource location adaptor and the resource area, so that it can be presented to the user via the user interface. If it is desired to convert a document from one format to another, for example from HTML to Adobe Acrobat PDF, then the conduit that connects the two domains associated with the source and target formats can be used to effect the conversion. These processes will be explained in greater detail below.

Scripts can be defined so that, for example, changes made to a database in domain 210 are captured and converted to different formats, for example HTML documents in domain 200 and JPG documents in domain 220. The conduits connecting the - 6 - domain process the change in formatting, either by performing the conversion by altering the content of the document or by sending the document to an OS utility that performs the conversion and returns the document in the format of the target domain.

The user interface 100 provides a user with a range of commands that are similar to the conventional commands found within computer operating systems. Examples of such commands are given below in Table 1 (for the sake of clarity and to reduce complexity, the command names and parameters shown in Table 1 and elsewhere are simplified versions of the actual ones used in the software): - 7 -

Command Description

copy( source, target) Copy the source document to the target move (source, target) Move the source document to the target.

delete (name) Remove the document identified by name.

list() Return a list all the resources in this location.

openDocument (Document, Open the Document using one of the mode) modes read, write or append. If the Document does not already exist and the mode is either write or append, the Document is created. The command returns a handle to the open Document.

closeDocument (Document) The Document is closed.

Table 1: User Interface Commands The user interface is preferably a graphical user interface (GUI) as is in conventional in modern computer OSs, but it may in some circumstances be a command line interface (CLI).

These commands may be provided as an Application Programming Interface (API) and comprise the interface 205 between a domain 200 and its resource area 300. The API may use a program to act as a proxy for the user, for example it may comprise a modified CLI that exposes its commands as functions. - 8

The resource areas link the virtual view presented by the domains to physical data sources. The interface 305 between the resource area 300 and the resource location adaptor 400 is implemented as a set of functions (preferably these functions are not exposed as an API), which are shown below

in Table 2: _.

Function Description

open(name, location, mode) Open the document identified by name and location in mode (read, wri te or append) and create a Document object. If the Document does not already exist and the mode is either wri te or append, the Document is created. The function returns the Document object.

close(name, location) Close the document identified by name and location.

read(name, location, data) Read data from name.

write(name, location, Write data to name. data)

delete(name, location) Delete the document identified by _ name and location.

Table 2: Resource Area Functions Any document can be identified using two parameters: its name and its Resource Location. The location is provided by the domain using the document log. When a document is created the resource area allocates one of the resource locations that it manages to that document. This information is passed - 9 - back to the domain for storage in the document log. Although the resource area interface is generic, some data resources will not support all the operations; for instance, an interface to an Internet server might be read-only, in which case it will not be possible to create, modify or remove documents on the server. For this reason each resource area has a capability list that determines what it can and can't do i.e. can read/can write/can list/can delete. This capability list is configured when the resource area is created and may be re-configured if the resource area requirements change.

The Resource Location Adapter interface comprises an API and a physical data-file. The content and format of the data file is determined by the RLA and is not restricted by the interface. The API exposes the following functions: - 10

Function Description

get(name, path) Retrieve the resource identified. by name to the data file identified by path.

put(path, name) Store the data file identified by path to the resource identified by name.

remove(name) Remove the resource identified by name.

list() Return a list all the resources in this location.

copy(source, Copy the resource identified by source to target) the resource identified by target.

rename(source, Move the resource identified by source to target) the resource identified by target.

_ _

exists(name) Check if the resource identified by name exists.

capabilities() Tell the client what this RLA can and can't do.

connect() Connect to location.

disconnect() Disconnect from location.

Table 3: Resource Location Adapter Functions A Resource Location Adapter can be configured to provide the required capabilities, which may be selected from: Read, Write, List, Delete, Connect & Disconnect. The Resource Location Adapter capabilities should coincide with the capabilities of the Resource Area, and this may be enforced by ANDing the capabilities of a Resource Location Adapter and a Resource Area.

An example illustrating how a Resource Location Adapter may provide an interface to a relational database will now be given. When RLA 400 is initialized it loads a configuration data file 402 that contains the skeleton SQL (Structured Query Language) statements that enable the RLA to communicate with the database. The configuration file need not have a standard format, as it is only read by the RLA it is intended for use with. The file may contain a number of statements,

for example,

create a new record in the database.

select the content of an existing record.

modify the content of an existing record.

delete a record.

check that a record exists.

list all records.

These statements contain parameters that are replaced with information from the database. A simple name and address database may have the following record structure:

ADDRESS

customer_id first_name last_name address postcode where customer_id is a number that uniquely identifies the record. A select statement for this record might look like: select customer_id, first_name, last_name, address, postcode from ADDRESS where customer_id='$customer_id'; When the RLA receives a get (name,path) command, it parses the document name, which might have the format: <customer_id>-clast_name>-cfirst_name>.csv to extract the customer_id, which is then used to replace the $customer_id parameter in the select statement. The result of the select statement is then used to create a text file in the location identified by the path variable. The RLA might use the document extension to determine the format of the text file, in this case comma-separated value.

Similarly, when a put (path,name) command is received the RLA parses the document name for the customer_id and extracts information from the file located by path. The RLA will expect the information in the file to conform to a pre- determined format (or one of a number of determined formats with a specific format being determined by information contained in the document name, such as the extension).

The process of creating records in the database is slightly more complex than querying them. If a record is being created it might not be possible to supply a document name. This might be because the database is responsible for allocation of new customer ids. In this case the document name is created by the RLA and passed back to the parent domain via the resource area. In other cases the document name will be supplied by the domain. In this instance, the RLA must first check to see if a record for that customer already exists. If it doesn't it will create one, if it does it will update the existing record. - 13

The skeleton SQL statement for the insert command is: insert into ADDRESS ( customer_id, first_name, last_name, address, postcode) values ( '$customer_id', $first_name', $1ast_name', $address', $postcode '); The skeleton SQL statement for the update command is: update ADDRESS set first_name = '$first_name', last_name = '$1ast_name', address = '$address', postcode = '$postcode' where customer_id = '$customer_id'; In this instance the values of address and postcode are extracted from the data-file located by path. Similar techniques can be used to implement the delete() and exists() commands. The list() command is slightly different as it returns an array of document names generated from the result

of an SQL statement of the form: - 14

select customer_id, first_name, last_name, address, postcode from ADDRESS; It is preferred that the specific details of the mapping between the document name and the query is entirely under the control of the RLA designer. As long as the information can be encoded in the document the designer is free to create any scheme that is appropriate for the resource that is being accessed. This approach may appear to be too open or even simplistic, but this provides one of the major benefits of the present invention, because it is extremely easy to understand and very flexible.

This flexibility does place restraints on the overall system designer, as document transfers must result in a document with name and content that is compatible with the Domain that it is being transferred to. Documents transferred in the wrong format will generate errors from the RLA. RLAs that use different document name and data-file formats cannot be mixed in a Resource Area, because the RA assumes that all RLAs are equal and can potentially choose any of them when creating a new document.

An example of the execution of a system command will now be described. To use the system, the operator logs on from a suitable terminal. Once the user has been authenticated by the system, the operator is presented with all of the active domains and the conduits that connect them. When a domain is selected, a list of documents in that domain is displayed.

This list of documents is obtained from the selected domain's document log. - 15

The domain and resource areas load their configurations from a system database. A resource area is associated with a domain by means of a tag value; preferably a tag is a alphanumeric string that does not contain spaces and has a maximum length of 64 characters. Similarly, resource location adapters are associated with a resource area by means of a class name. Preferably a class name is an alphanumeric string, containing no spaces, having a maximum length of 32 characters. When a resource area is initialized it loads all its RLAs from dynamic link libraries, creating an object for each RLA that it is associated with. When the RLA is loaded it is initialized and a connection is opened to any required data stores: if this requires any authentication credentials, these may either be obtained from the system database or from the RLA initialization file. Any required conduit definitions are also loaded from the system database. Each conduit has one input and one output domain and is associated with each domain by a tag value.

To perform an operation on one of the documents the operator performs the required operation in the GUI (or enters an appropriate CLI command), for example copying a document from a source domain 200 to a target domain 210. The process performed by the system is now explained with reference to Figure 2.

Step Action S10 The user interface 100 generates the command Copy(document, source domain, target domain) and sends it to source domain 200; the user interface displays a visual confirmation that the Copy operation is being - 16 executed.

S20 The conduit definition for the conduit 215 between the source domain 200 and the target domain 210 is loaded.

S30 The source domain 200 queries the document log 202 to obtain the loca tion of the source document.

S40 The source domain 200 issues an open() command to resource area 300 for the source document S50 The resource area 300 uses the get() command to obtain temporary copy of the source document from the resource location adaptor 400 identified by location.

S60 The resource area 300 creates a document object to manage the temporary file and adds this object to a processing list, enabling the resource area to manage multiple requests for the document.

S70 The resource area 300 passes a reference to the document object created in S60 to the source domain 200.

S80 The source domain 200 issues an OpenDocument() command to the target domain 210 to create a new Document.

S90 The target domain 210 issues an Open() command to resource area 310.

S100 Resource area 310 allocates a resource location adaptor 410 for the new document.

S110 The target resource area 310 creates a temporary file and an associated object to manage the temporary file and adds the new document to a processing list.

S120 The new location id and a reference to the document object are sent to the target domain 210: the location id is added to the document log 202.

S130 The target domain 210 passes the document object reference created in S120 to the source domain 200.

S140 The source domain 200prompts a conduit object to be 17 created for the link between the source domain 200 and the target domain 210 and a reference to the source document object is passed to the conduit object for processing.

S150 The processed source document is written to the temporary file created by the target resource area 310 in step 110.

S160 The source domain 200 issues a CloseDocument() command to the target domain 210.

S170 The target domain 210 issues a Close() command to resource area 310.

S180 Resource area 310 issues a put() command to resource location adaptor 410 to create the new document.

S190 Resource area 310 removes the document object from its processing list, destroys the object and deletes the associated temporary file.

S200 Target domain 210 creates a document log entry fro the new document.

S210 Source domain 200 issues a closet) command to resource area 300.

S220 Resource area 300 removes the document object from its processing list, destroys the object and deletes the associated temporary file.

S230 The user interface 100 confirms that the action has been completed.

For the sake of clarity the above does not include notifications that are sent to prompt the next entity to take an action or error states that may occur.

It will be understood that the entities contained within dotted line 600 of Figure 1 form the core of the system. - 18

Figure 3 shows a schematic depiction of a further system according to the present invention which shows that these entities may be deployed in a platform 700 that interfaces a plurality of user terminals 800 with the data stores and servers 500, 510, . ,shown in Figure 1. The terminals, platform and data stores may be connected by a combination of network transports and protocols. Given the nature of the present invention it is likely that the terminals and data stores may be located across a country or even a country.

The platform also comprises a system database 710 that is used to manage use identities, user authentication and the resource areas, resource location adaptors, conduits, etc. required for the operation of the system. The platform may comprise a single computer or a plurality of distributed computers. The platform may be implemented using any conventional server OS and this is not essential to the operation of the invention.

The user terminals may be a conventional personal computer that can run a client application for connecting to the platform 700 and providing the user interface 100.

Alternatively a conventional web browser may be used to connect to a suitable interface to the platform (it may be necessary to use one or more plug-ins to provide the required functionality for the browser) or dedicated terminals may be provided.

It is preferred that the functionality of the RA and RLA are provided by separate entities as this allows RAs to load RLAs as required and therefore it is possible to add new RLAs by changing a configuration file rather than having to alter the code defining the RA. It is also possible for an RA to use - 19 more than one RLA to access data and for end-users to create and configure RLAs as required, either through the user interface or via a further control/management utility. It will be understood that whilst it is possible to integrate the RAs and the RLAs to provide a single entity to interface between the domains and the data servers, this approach is not preferred as any changes to the interface entity become more complicated, hindering system maintenance and upgrades. - 20

Claims (9)

1. A system for accessing one or more of a plurality of data resources stored on a plurality of data stores, the data resources being one of a plurality of types, the system comprising: a user interface for enabling a user to view and manipulate one or more data resources; a plurality of domains, each domain comprising a representation of one or more data resources, each domain being associated with a single type of data resource; and a system management entity for holding a record of each data resource stored on the plurality of data stores and providing a record to each domain of each data resource of the data resource type associated with that domain.

2. A system according to claim 1, wherein the system comprises a plurality of system management entities, with each domain having a unique associated system management entity.

3. A system according to any preceding claim, wherein the or each system management entity comprises a resource area and one or more resource location adaptors.

4. A system according to any preceding claim, wherein one or more of the plurality of domains is connected to one further domain by a conduit, the conduit comprising configuration data that comprises the data resource type associated with the two domains connected to the conduit. - 21

5. A system according to claim 4, wherein one or more of the plurality of domains is connected to a plurality of further domains by a plurality of conduits, with each further domain being associated with a unique conduit.

6. A system according to claim 4 or claim 5 wherein a conduit controls the transmission of data resources between the domains connected by the conduit, such that the data resource is converted from the data resource type associated with the transmitting domain to the data resource type associated with the receiving domain.

7. A system according to claim 6 wherein the conduit sends the transmitted data resource to a conversion application process to convert the data resource to the data resource type associated with the receiving domain and to forward the converted data resource to the receiving domain.

8. A system according to any preceding claim system further comprising a system database comprising data records regarding each of the plurality of domains, the or each system management entity and the one or more conduits.

9. A system substantially as described hereinbefore and with reference to the Figures.