WO2003060724A2 - Method and system for storage and retrieval of arbitrary content and application data - Google Patents

Abstract

Systems and methods for improving the performance of a data storage and retrieval system by enabling dynamic switching from one internal data structure to another in response to detecting conditions indicating that a switch would improve performance. In one embodiment, a network proxy is implements a cache using a first internal data structure. The cache objects comprise Web pages, and the cache keys comprise URLs corresponding to the Web pages. The proxy monitors cache usage and periodically determines costs associated with usage of the first data structure and an alternative data structure. If the costs associated with the alternative data structure are less than the costs associated with the first data structure, the proxy creates the alternative data structure, migrates data from the first data structure to the alternative data structure, begins using the alternative data structure for the cache, and deletes the first data structure.

Description

DESCRIPTION A DESIGN FOR STORAGE AND RETRIEVAL OF ARBITRARY CONTENT AND

APPLICATION DATA

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent Application No. 60/349,378, entitled "OneCache: An Abstract Design for Storage and Retrieval of Arbitrary Content and Application Data," by Jeremy S. de Bonet, Todd A. Stiers, Jeffrey R. Annison, Phillip Alvelda VII,^' and Paul M. Scanlan, filed January 18, 2002, U.S. Provisional Patent Application No. 60/349,344, entitled "Modular Plug-In Transaction Processing Architecture" by de Bonet et al., filed January 18, 2002 and U.S. Provisional Patent Application No. 60/349,424, entitled "Network Proxy Platform that Simultaneously Supports Data Transformation, Storage, and Manipulation for Multiple Protocols" by de Bonet et al., filed on January 18, 2002, which are hereby fully incorporated by reference herein. Additionally, U.S. Patent Application Serial No. , entitled "Method and System of Performing Transactions Using Shared Resources and Different Applications," by de Bonet et al., filed January 14, 2003 is incorporated by reference herein.

REFERENCE TO APPENDICES

Appendices are included in this application by way of attachment, the totality of which is hereby incorporated by reference for all purposes as an integral part of this application. Appendix 1 is entitled "Network Proxy Platform and Its Use" and includes 17 pages of text and 8 sheets of drawings.

BACKGROUND OF THE INVENTION Technical Field

This invention generally relates to storage and retrieval of electronic entities. More particularly, this invention relates to automatically changing the data structure of a storage and retrieval system based on the detection of conditions indicating that a different structure would provide improved performance.

Related Art

Some data storage and retrieval mechanisms use lookup keys to store and identify data. Such mechanisms include caches, associative arrays, and databases. The keys are associated with the corresponding data according to a specific internal data structure. These internal data structures may, for example, comprise trees, hashes, heaps, and lists. Each of these data structures enables the storage of data in a different manner and therefore provides different performance characteristics. Some of these data structures are described briefly below. A list is simply an unordered set (a list) that enumerates all of the keys and corresponding data. A hash is an associative array in which a key is converted to a hash table entry. The hash table entry defines the position in a hash table in which the corresponding data is stored. The hash table is static and may be only partially filled. A tree is a hierarchical data structure in which keys and their associated data are stored in a sorted manner. A heap is a tree which is only partially sorted. Hybrid structures may combine, for instance, a first layer of trees or heaps with a second layer of hashes or lists.

Different data structures are optimal for different uses. Consequently, the selection of a data structure for use in a particular application typically depends upon the manner in which the data is expected to be used, as well as the amount of the data to be stored, and the type of access to the data that will be needed. The greater particularity with which these factors can be specified, the more accurately a developer can select an "optimal" data structure for the application.

It is therefore apparent that one of the problems with selecting a data structure that will provide the best performance in an application is identifying the conditions under which the data structure will be used. While it may be relatively easy to identify factors such as the type of data that will be stored and the types of access that will be needed, it is typically much more difficult to identify things like the frequency of accesses, or any patterns with which the accesses are made.

As a result of the difficulty in predicting some of the factors which form the basis for determining which data structure is "optimal," a software developer may simply have to make an educated guess as to which type of data structure will ultimately provide the best performance. This guess may turn out to be accurate, or it may not. If the developer has selected a data structure that is not actually optimal, the performance of the application may be substantially degraded by the less- than-optimal performance of the selected data structure.

SUMMARY OF THE INVENTION

One or more of the problems outlined above may be solved by the various embodiments of the invention. Broadly speaking, the invention comprises systems and methods for improving the performance of a data storage and retrieval system by enabling dynamic switching from one internal data structure to another in response to detecting conditions indicating that a switch would improve performance. These systems and methods provide a mechanism for gathering statistics on the system as it is being operated and modifying the internal data structure of the system as necessary to provide optimal performance for the current usage. These systems and methods are applicable to any storage and retrieval system that uses keys to store and identify data and are particularly applicable to Web caching. One embodiment of the invention comprises a method in which a first internal data structure is provided for storing a plurality of objects. The first internal data structure is used during operation of a system in which it is implemented, and the usage of the first internal data structure is monitored. Periodically, a cost associated with usage of the first internal data structure is determined. A cost associated with usage of an alternative internal data structure is also determined, based upon either empirical usage data or statistically estimated usage data. The cost associated with the alternative internal data structure may also include the cost of transitioning from the first internal data structure to the alternative internal data structure. The costs of the first and alternative internal data structures are then compared to determine whether or not the system should switch to use of the alternative internal data structure. If not, the first internal data structure remains in use. If so, the alternative internal data structure is created, data is migrated from the first internal data structure to the alternative internal data structure, operations using the alternative internal data structure are begun, and the first internal data structure is deleted.

Another embodiment of the invention comprises a system in which a method similar to the foregoing method is implemented. In one embodiment, the system comprises a network proxy having a data processor and memory, wherein the data processor is configured to implement a cache using a first internal data structure in the memory. Each entry in the cache comprises a Web page as the data object and a corresponding URL as the key. The data processor is configured to monitor usage of the cache and to periodically determine costs associated with usage of the first internal data structure and an alternative internal data structure. The cost associated with usage of an alternative internal data structure is determined based upon either empirical usage data or statistically estimated usage data and includes the cost of switching from the first internal data structure to the alternative internal data structure. The data processor compares the costs of the first and alternative internal data structures to determine whether or not the system should switch to use of the alternative internal data structure. If indicated by the comparison, the data processor creates the alternative internal data structure, migrates data from the first internal data structure to the alternative internal data structure, begins using the alternative internal data structure for the cache, and deletes the first internal data structure.

Another embodiment of the invention comprises a software application. The software application is embodied in a computer-readable medium such as a floppy disk, CD-ROM, DVD- ROM, RAM, ROM, database schemas and the like. The computer readable medium contains instructions which are configured to cause a computer to execute a method which is generally as described above. It should be noted that the computer readable medium may comprise a RAM or other memory which forms part of a computer system. The computer system would thereby be enabled to perform a method in accordance with the present disclosure and is believed to be within the scope of the appended claims.

Numerous additional embodiments are also possible. BRIEF DESCRIPTION OF THE DRAWINGS Other objects and advantages of the invention may become apparent upon reading the following detailed description and upon reference to the accompanying drawings.

FIGURE 1 is a diagram illustrating an exemplary architecture for a network system employing a network proxy.

FIGURE 2 is a diagram illustrating the basic configuration of a computer suitable for use as a network proxy in accordance with one embodiment of the invention.

FIGURE 3 is a flow diagram illustrating a simple method in accordance with one embodiment of the invention. FIGURE 4 illustrates the detailed steps that present cache system takes in one embodiment when changing the internal structure of a network proxy cache from a list to a heap.

While the invention is subject to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and the accompanying detailed description. It should be understood, however, that the drawings and detailed description are not intended to limit the invention to the particular embodiment which is described. This disclosure is instead intended to cover all modifications, equivalents and alternatives falling within the scope of the present invention as defined by the appended claims.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT A preferred embodiment of the invention is described below. It should be noted that this and any other embodiments described below are exemplary and are intended to be illustrative of the invention rather than limiting.

Broadly speaking, the invention comprises systems and methods for improving the performance of a data storage and retrieval system by enabling dynamic switching from one internal data structure to another in response to detecting conditions indicating that a switch would improve performance. These systems and methods provide a mechanism for gathering statistics on the system as it is being operated and modifying the internal data structure of the system as necessary to provide optimal performance for the current usage. These systems and methods are applicable to any storage and retrieval system that uses keys to store and identify data and are particularly applicable to Web caching.

One embodiment of the invention comprises a dynamically self-modifying Web cache system implemented in a network proxy. The Web cache is configured to store Web pages using URLs as keys. The Web cache initially uses a list data structure to store the Web pages. As the network proxy operates, Web pages are stored in the Web cache and retrieved from the Web cache. Web pages in the cache may also be updated or removed. Operations on the Web cache are monitored to determine the cost of operation of the cache using the current (list) data structure. Periodically, the current cost of operation is compared to a cost of operation that is computed for one or more altemative data structures. The cost associated with the alternative data structure may be computed on the basis of estimated usage, or empirically determined usage. The Web cache system also computes a cost associated with a switch from the currently-used data structure to the alternative data structure. If the cost of the currently-used data structure is greater than the cost of the alternative data structure, plus the cost of switching to the altemative data structure, the Web cache will initiate a switch to the lat data structure. This essentially comprises the creation of the alternative data structure, the migration of data from the current data structure to the alternative data structure, and the transfer of operations from the formerly current data structure (the list) to the now-current alternative data structure. The formerly current data structure is then deleted. When the Web cache begins operation with the alternative data structure, the cost of operation of the cache using this data structure is less than the cost of operation using the previous data structure under the current usage conditions. The Web cache continues to monitor its operation, however, and may switch back to use of the previous data structure or another data structure if the usage conditions change. Thus, the Web cache optimizes its performance by switching from a less optimal data structure for the conditions to a more optimal data structure.

It should be noted that, although the present disclosure focuses on embodiments of the invention that are implemented in a Web cache in a network proxy, the invention is more broadly applicable to any storage and retrieval system that uses keys to store and identify data. For example, another embodiment may comprise a cache configured to store parameter sets. The parameter sets may contain such information as configuration data (e.g., parameter values and corresponding names), or network connection data (e.g., protocols that are used for a connection and the system and port to which the connection is made). Such an implementation may provide greater performance improvements than a Web cache implementation because, while the storage patterns involved in the caching of Web pages are relatively well understood, the usage of parameter sets can vary widely from application to application, and even from user to user. The parameter set cache may therefore be able to take greater advantage of the present systems' and methods' adaptability to changing usage patterns. An embodiment implemented in a file system cache is another example of an implementation that may take greater advantage of the adaptability of the present systems and methods. Nevertheless, the present disclosure will focus on network proxy implementations, as they may present more easily understandable embodiments of the invention.

As noted above, a preferred embodiment of the invention is implemented in a network proxy. Referring to FIGURE 1 , a diagram illustrating an exemplary architecture for a network system employing a network proxy is shown. In this figure, the architecture comprises a client 12 which is coupled to a network proxy 14, which is in turn coupled to a Web server 16. Network proxy 14 includes a cache subsystem 18. Client 12 is coupled to proxy 14 via a first network 13. Proxy 14 is coupled to Web server 16 by a second network 15. It is contemplated that at least one of networks 13 and 15 comprises the Internet. The other of these networks may comprise a network which is either internal or external to a particular enterprise. It should be noted, however, that the coupling of client 12, proxy 14 and Web server 16 need not be configured in any particular manner for the purposes of the invention.

A proxy handles communication between a client device or program, such as a Web browser, and a server device or program, such as a Web server. In a Web-based system, the proxy handles the clients' requests for Web content, as well as the Web content provided by the Web server in response to these requests. In handling these communications, the proxy is responsible for emulating the Web server and thereby reducing the loading on the system (both on the Web server and on the network itself). The proxy does this by storing some of the content provided by the Web server and, when possible, providing this stored content to clients in response to requests for the content. In this manner, the proxy relieves the Web server of the burden of serving a portion of the clients' requests. Referring to FIGURE 2, a diagram illustrating the basic configuration of a computer suitable for use as a network proxy in accordance with one embodiment of the invention is shown. Server 14 is implemented in a computer system 100. Computer system 100 includes a central processing unit (CPU) 112, read-only memory (ROM) 114, random access memory (RAM) 116, hard disk drive (HD) 118 and input output device (I O) 120. Computer system 100 may have more than one CPU, ROM, random, hard disk drive, input-output device or other hardware components. Computer system 100 is nevertheless depicted as having only one of each type of component. It should be noted that the system illustrated in FIGURE 2 is a simplification of an exemplary hardware configuration, and many other alternative configurations are possible. A more detailed description of an exemplary architecture is described in U.S. Patent Application No. (Attorney Docket No. IDETl 130- 1), by inventors Phillip Alvelda VU, Todd A. Stiers and Jeremy S. de Bonet, filed on January 14,

2003 and entitled entitled "Method And System Of Performing Transactions Using Shared Resources And Different Applications," which is which is incorporated by reference as if set forth herein in its entirety.

Portions of the methods described herein may be implemented in suitable software applications that may reside within memories such as ROM 114, RAM 116 or hard disk drive 118. The software applications may comprise program instructions that are configured to cause the data processor in which they execute to perform the methods described herein. These instructions may be embodied in (stored on) internal storage devices such as ROM 114, RAM 116 or hard disk drive 118, other, and external storage devices, or storage media readable by a data processor such as computer system 100, or even CPU 112. Such media may include, for example, floppy disks, CD-ROMs, DVD ROMs, magnetic tape, optical storage media, and the like. In an illustrative embodiment of the invention, the computer-executable instructions maybe lines of compiled C^1"1", Java, or other language code. Other architectures may be used. For example, the functions of any one of the computers may be performed by a different computer shown in FIG. 2. Additionally, a computer program or its software components with such code may be embodied in more than one data processing system readable medium in more than one computer.

In the hardware configuration above, the various software components may reside on a single computer or on any combination of separate computers. Li alternative embodiments, some or all of the software components may reside on the same computer. For example, one or more the software component(s) of the proxy computer 100 could reside on a client computer or server computer, or both. In still another embodiment, the proxy computer itself may not be required if the functions performed by the proxy computer are merged into a client computer or server computer. In such an embodiment, the client computer and server computer may be directionally coupled to the same network.

Communications between any of the client, server and proxy computers can be accomplished using electronic, optical, radio-frequency, or other signals. For example, when a user is at a client computer, the client computer may convert the signals to a human understandable form when sending a communication to the user and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by, the proxy or server computers. Similarly, when an operator is at the server computer, the server computer may convert the signals to a human understandable form when sending a communication to the operator and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by the computers.

As explained above, the proxy is responsible for storing information previously provided by the Web server so that this information can be provided to clients in response to their requests. This information is stored in the Web cache of the proxy. The network proxy provides a mechanism for gathering statistics on the operation of the Web cache using a current type of data structure and determining a cost associated with usage of this data structure. A cost associated with the usage of an alternative type of data structure is also determined for the same usage conditions. If it is determined that the alternative type of data structure would operate more efficiently than the type currently in use, the internal data structure of the Web cache is modified to the alternative type of data structure.

Referring to FIGURE 3, a flow diagram illustrating a simple method in accordance with one embodiment of the invention is shown. This figure depicts a series of steps that are taken periodically during operation of the Web cache. This may occur more or less frequently, depending upon the needs of the system. For example, if the usage patterns are very irregular, it may be desirable to repeat the steps of the method frequently. If the usage patterns change more slowly, it may be better to repeat the steps less frequently. The method depicted in the figure assumes that the usage of the Web cache is continually monitored so that, at any given time, usage information is available for use in determining the costs associated with the different data structures. The method then comprises the computation of costs, comparison of the costs and switching to an alternative data structure if necessary. The first step in this method is determining the "cost" of operating the current data structure.

"Cost," as used here, refers to resources that are used in the operation of the Web cache, including processing time, memory and possibly other types of resources. The determination of the cost of operation is accomplished by associating costs with each of the operations on the Web cache and computing the total cost based on the operations that have been performed on the Web cache. The operations on the Web cache may be incorporated into the computation through the use of empirical data or statistical data on the operations. The total cost may also incorporate other factors, such as the size of the stored data set, the amount of resources available, and so on.

The next step is to determine the cost of operation of the alternative data structure, plus the cost of a potential transition from the current data structure to the alternative data structure. The cost of operation of the alternative data structure is determined is much the same way as for the current data structure, using either empirical or statistical data on the Web cache operations.

Li addition to determining the cost of operation of the alternative data structure, it is also necessary to determine the cost of transitioning to the alternative data structure from the current data structure. This is true because, in order to start using the alternative data structure, it will be necessary to first create the structure and then populate it with the data from the current data structure. A small improvement in operational cost may therefore be outweighed by the transition costs, making it impractical to switch to the alternative data structure.

After the costs associated with the current and alternative data structures (including transition costs) have been determined, these costs are compared. If the costs associated with the alternative data structure are lower than the costs associated with the current data structure, then a switch from the current data structure to the alternative data structure will be initiated. It should be noted that, although the comparison is depicted in the figure as a simple "less than" comparison between the costs, more complex functions may be used in other embodiments to determine whether the switch to the alternative data structure is initiated. Such functions may incorporate the costs described above and/or may take into account various other factors to determine whether it is desirable to switch to the alternative data structure.

If the cost of operation of the current data structure is less than the expected cost of switching to and operating the alternative data structure, no action is taken, except to repeat the process at a later time. If the expected cost of switching to and operating the altemative data structure is less than the cost of operation of the current data structure, then the switch to the alternative data structure is initiated. The switch entails creating the alternative data structure, migrating the data from the current data structure to the alternative data structure, and beginning operation using the alternative data structure. The formerly-current data structure is typically deleted after the switch to the alternative data structure.

When deciding whether to change the internal data structure of a particular cache instance, algorithms in the preferred embodiment consider five factors, though others could also be considered: How many objects are currently stored in the cache? How many lookups (or reads) does the system perform? How many stores (or writes) does the system perform?

How much memory is available on the current system? How much of the available memory should the storage and retrieval system use?

What would be the cost of reconfiguring the current data structure? To consider the relative costs (i.e., CPU time and RAM used) of alternate data structures, algorithms in the current cache system consider the costs of performing the following actions, though other costs could be easily incorporated into the cache as well: Looking up (or reading) a piece of data from the cache using the current data structure.

(Lookup time is a function of the number of objects stored in the cache. It also depends on the internal data structure of the cache; for example, a hash table can look up and store objects in constant time.)

Storing (or writing) a piece of data to the cache using the current data structure Restructuring the internal data

The current cache system can estimate these costs for two or more data structures based on the current usage. It can perform calculations on costs by using the theoretical computational complexity of the storage algorithms, by using empirical measurements, or by using some combination thereof. If the cache makes the decision to change the internal data structure of its storage and retrieval system, it internally allocates a new object, then copies and stores data from the old structure to the new one. After migrating all data to the new structure, the cache system deletes the old structure.

The present cache system's monitoring of usage statistics and making decisions based on the monitored usage incurs a minor cost (in CPU time and RAM) to the system. This cost depends, for example, on how often the cache system checks the usage statistics and how many different internal data structures it considers. For cases in which a data structure does not need to change at all, a developer can eliminate the cost entirely by configuring the program not to perform any checks, in which case the cache system^is exactly equivalent to the internal data structure that it is currently using.

In a preferred embodiment of the invention, a cache system collects usage statistics and applies algorithms to select a structure which is optimal for the actual usage of a storage and retrieval system, then modifies the internal data structure of the system to adopt the optimal structure. The cache system can thereby dynamically shift the internal data structure for the storage and retrieval system among the data structures that are defined for the system. These data structures may include trees, hashes, heaps, lists, and hybrid structures, such as heaps of lists.

In addition to internal usage statistics, the present cache system can consider the type of data being stored and the type of key used to access the data. Because types of data and keys can affect usage, and because the cache system can alter its internal data structure based on internal usage statistics, the cache system allows a single programming construct to handle multiple types of data and keys. The cache system is novel in its ability to create a cache for any type of key and data. This is in distinct contrast to caching or other storage systems in the prior art, which must be built for a specific type of data that uses a specific type of key.

The dynamically self-modifying cache described herein may be considered a superset of a number of internal data structures, any one of which can be used by the cache object to store and retrieve data. In one embodiment,' the cache is implemented using C++ templates to create the different internal data structures. Although, in the preferred embodiment, C++ is used to represent the programmatic structures of the cache system, most Turing complete programming languages with macro support could be used. A C++ template makes it unnecessary to write separate bodies of code to accomplish similar tasks on differing data types. It makes the tasks abstract, allowing one set of C++ code to be applied to different types of data. To accomplish a specific task, the template can be instantiated using the particular types that the task calls for. In the case of the template for the present cache system, cache keys and values of any type can be used to instantiate the template.

This cache system makes it unnecessary for programmers to understand before building a storage and retrieval system how the system will be used. Once a storage and retrieval system in accordance with this disclosure is created, it is not necessary to review statistics manually, create new data structures, or require developers or system administrators to migrate data. In particular, this means that if programs using the present cache system are designed to be used under one set of circumstances, and those circumstances change, the entire program continues to run optimally, and it does not need to be rewritten as a prior art program would.

An additional benefit of the present cache system is its ability to handle any type of data or key. Within a computer, many types of information need to be stored and retrieved. A cache has many advantages over more general storage and retrieval methods when retrieval time is more important than storage time. Before the creation of the present cache system, however, development of caches was specialized to the type of content being stored and the type of key used to reference that data. By encapsulating the underlying methods needed by a cache and abstracting the functionality particular to the type of key and type of content, the present methodologies allow for the creation of caches of arbitrary key and data type with a single programming construct. In the preferred embodiment, the definition of the cache system is completely recursive. That is, one instance of a cache in the system can refer to another instance of a cache in the system. This allows for the easy creation of multi-tiered cache systems (caches-of-caches) as described in commonly owned U.S. Patent Application No. , (attorney docket IDET1220-1, entitled "A Multi-Tiered Caching Mechanism for the Storage and Retrieval of Content Multiple Versions," by inventor Jeremy S. de Bonet, filed on January 16, 2003, which is incorporated by reference as if set forth herein in its entirety.

In another embodiment, a cache object in accordance with this disclosure can be used to create a method for protecting shared resources across multiple threads. In this case, the key used is the name of the resource (or variable), and the value is the shared resource itself. This embodiment is described in detail in commonly owned U.S. Patent Application No. , (attorney docket

IDET1190-1), entitled "A Method for Protecting Shared Resources Across Multiple Threads," by inventor Jeremy S. de Bonet, filed on January 15, 2003, which is incorporated by reference as if set forth herein in its entirety. In the preferred embodiment, the present cache system is used as a Web cache on a network proxy, storing Web pages and using URLs as keys. FIGURE 4 illustrates the detailed steps that present cache system takes in one embodiment when changing the internal structure of a network proxy cache from a list to a heap. This embodiment is exemplary, and other internal data structures could be used in alternative embodiments. For instance, in another embodiment, cache system may store connection and DNS information. In that case, the key used is the name of a server, and the value is the IP address of the server.

The cache system of the preferred embodiment runs as a network proxy cache with a particular internal data structure which is, in this case, a list (1010). Each time a get, set, or remove operation is performed on the cache (1020), the cache system collects usage statistics (1030). If the operation is a get (1040), the cache system performs a check to determine whether to change its internal data structure from a list to a heap (1050). If the operation is not a get (1040), the system continues to run with a list as its internal data structure (1010). The selection of checking on a get operation here is purely exemplary, and the check could be triggered by any other action taken on the structure or by some other external trigger (e.g. a timer). If the cache system determines that it should not change its internal data structure (1060), the system continues to run with a list as its internal data structure (1010). If the cache system determines that it should change its internal data structure (1060), the get function triggers the change. The cache system creates a new object (in this case a heap) for the new data structure (1070), then migrates data from the old data structure (the list) to the new data structure (the heap) (1080). The cache system then deletes the old data stmcture of the list (1090). While these changes are taking place, users can perform get and set functions on the cache. The operations are performed on both the old data structure (the list) and the new data structure (the heap). After the changes have taken place, the cache system runs as a network proxy cache with a heap as its internal data structure (1100).

The preferred embodiment implements the following algorithm to determine whether to switch from the current internal data structure to an alternative data structure. In this embodiment, the computational cost of performing the indicated operation is defined as follows:

G(x,n) is the cost of a get operation using data structure x currently containing n elements.

S(x,n) is the cost of a set operation using data structure x currently containing n elements if the element already exists and a developer is simply changing its value. A(x,n) is the cost of a set operation using data structure x currently containing n elements (and going to n+1) if the element does not exist and a developer must add it. R(x,n) is the cost of a remove operation using data structure x currently containing n elements (and going to n-1). F(x,n) is the cost of freeing a data structure. W(xι, ₂,n) is the cost of switching from one data structure to another.

The cost of switching from one data structure to another is given by: m=l W(xι, x₂, n) = Σ ( G(_Xl,n) + A(x,m) ) + F(_Xl,n) n To determine whether it is worthwhile to switch from one data structure to another, the cache system looks at the following usage statistics: g = number of get operations performed s = number of set operations performed (not requiring new elements to be added) a = number of set operations performed (which add new elements) r = number of remove operations performed

A developer could design the program to check the statistics at a configurable time interval or every time a get or set function is performed. In the preferred embodiment, these options are fully configurable by the system administrator

In one embodiment, these statistics are adjusted to compensate for how long ago each operation was performed. For example, one way to adjust the statistics is to update each by performing an operation similar to the following examples every time a get is performed: g' = g * .9 + l s' = s * .9 a* = a * .9 r' = r * .9 ϋi this example, ALL of the values are modified when ANY operation is performed. The 0.9 multiplier represents the depreciation due to time.

When determining whether to switch from one data stmcture to another, the cache system uses the statistics it collects to predict how the system will behave in the future. A switch is worthwhile if the total future cost of access using the new data structure (get, set, add, free) plus the cost of switching is less than the cost of access using the old structure.

In the preferred embodiment, the cache system uses the following approximation or prediction to determine whether to switch from one structure to another. It determines whether the cost of handling current usage patterns plus the cost of switching: g*G(x₂,n) + s*S(x₂,n) + a*A(x₂,n) + r*R(x₂,n) + W(x_h x₂, n)

is greater than, less than, or equal to the cost of handling current usage patterns using the current structure, without switching: g*G(x,,n) + s*S(x,,n) + a*A(x,,n) + r*R(x,,n)

Other ways to do this might involve keeping more elaborate statistics, and then, for example, trying to predict when gets or adds will occur.

In deciding whether to switch data structures, the cache system also may need to take other factors into account, such as memory and CPU availability.

When one embodiment of the cache system determines that it should change its internal data structure, the process may begin in one of two ways. If the cache system is configured to review its statistics every time a specific function (e.g., get) is called, then that function may be able to execute the change. If the cache system is configured to check its statistics in a separate thread, then that thread call a function that can execute the change.

When executing a change of its internal data structure, the cache system first generates the new data structure, then migrates the data from the old stmcture to the new stmcture. After migrating all data, the cache system deletes the old stmcture. While the program is executing these changes, the data can be both read- and write-available by other threads which need to access the data, though some embodiments may force the the changes to be made synchronously. When a get operation is performed concurrently with a restructuring, data may be read from either the old or the new structure. When, a set is performed during a restructuring, it will typically need to be written to both structures.

The benefits and advantages which may be provided by the present invention have been described above with regard to specific embodiments. These benefits and advantages, and any elements or limitations that may cause them to occur or to become more pronounced are not to be construed as critical, required, or essential features of any or all of the claims. As used herein, the terms 'comprises,' 'comprising,' or any other variations thereof, are intended to be interpreted as non- exclusively including the elements or limitations which follow those terms. Accordingly, a system, method, or other embodiment that comprises a set of elements is not limited to only those elements, and may include other elements not expressly listed or inherent to the claimed embodiment.

While the present invention has been described with reference to particular embodiments, it should be understood that the embodiments are illustrative and that the scope of the invention is not limited to these embodiments. Many variations, modifications, additions and improvements to the embodiments described above are possible. It is contemplated that these variations, modifications, additions and improvements fall within the scope of the invention as detailed within the following claims.

APPENDD NETWORK PROXY PLATFORM AND ITS USE Reference is made in detail to exemplary embodiments of the network proxy platform and its use, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts (elements). A method and system can comprise a software architecture that allows different applications in the same or different communications protocols to interact with shared resources within a computer. More specifically, code for a computer program may be written to increase the amount of code that is generic to (i.e., shared by) more than one application or communications protocol and reduce the amount of code that handle application-specific or protocol-specific actions. In one embodiment, a transaction may be broken down into a set of discrete actions. The discrete actions may include functions that are common to more than one network application. These functions may be performed by the shared resources.

For each action, code that is specific to a particular protocol or application may be written as part of a software plug-in module with function calls to functions of the shared resources. Each software plug-in module may substantially act similar to a manager for the action, where common tasks are delegated to the shared resources and the module performs specialized functions. Each protocol may have its own set of software plug-in modules for the discrete actions. New applications and support for new protocols can be added by developing a new set of plug-in modules instead of writing an entirely new program. New applications for the same protocol may be developed by replacing or editing as little as one plug-in module from a different application in the same protocol. The software architecture can reduce development time, increase the likelihood that new applications may be developed quickly with fewer changes from an existing application, more protocols will be properly supported, and reduce the burden on hardware and software resources. A few terms are defined or clarified to aid in understanding the descriptions that follow. A network includes an interconnected set of server and client computers over a publicly available medium (e.g., the Internet) or over an internal (company-owned) system. A user at a client computer may gain access to the network using a network access provider. An Internet Service Provider ("ISP") is a common type of network access provider. As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having" or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a method, process, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such method, process, article, or apparatus. Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is tine (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

The term "software component" is intended to mean at least a portion of a computer program (i.e., a software application). An example includes a software plug-in module or the like. Different software components may reside in the same computer program or in different computer programs on the same computer or different computers.

Before discussing embodiments of the network proxy platform, an exemplary hardware architecture for using network proxy platform is described. FIG. 1 illustrates such an exemplary hardware architecture and includes client computer 120, proxy computer 140, and server computer 160. Client computer 120 and proxy computer 140 are bi-directionally coupled to network 11, and proxy computer 140 and server computer 160 are bi-directionally coupled to network 13. Each of networks 11 and 13 may be an internal network or an external network (e.g., the Internet). In one embodiment, networks 11 and 13 may be the same network, such as the Internet. Computers 140 and 160 may be bi-directionally coupled to databases 14 and 16, respectively. Client computer 120 can include a desktop computer, a laptop computer, a personal digital assistant, a cellular phone, or nearly other device capable of communicating over network 11. Other client computers (not shown) may also be bi-directionally coupled to network 11. The proxy computer 140 can be a server computer, but in another embodiment may be a client computer. Other server computers (not shown) similar to server computer 160 may be bi-directionally coupled to network 13.

In an alternative embodiment, each of proxy computer 140 and server computer 160 may be replaced by a plurality of computers (not shown) that may be interconnected to each other over a network or a combination of networks. For simplicity, a single system is shown for each of proxy computer 140 and server computer 160. The client computer 120 can include central processing unit ("CPU") 122, read-only memory

("ROM") 124, random access memory ("RAM") 126, hard drive ("HD") or storage memory 128, and input output device(s) ("I/O") 129. I/O 129 can include a keyboard, monitor, printer, electronic pointing device (e.g., mouse, trackball, stylus, etc.), or the like. Proxy computer 140 can include CPU 142, ROM 144, RAM 146, HD 148, and I O 149, and server computer 160 can include CPU 162, ROM 164, RAM 166, HD 168, and I/O 169.

Each of the computers in FIG. 1 may have more than one CPU, ROM, RAM, HD, I/O, or other hardware components. For simplicity, each computer is illustrated as having one of each of the hardware components, even if more than one is used. Note that FIG. 1 is a simplification of an exemplary hardware configuration. Many other alternative hardware configurations are possible and known to skilled artisans. Each of computers 120, 140, and 160 is an example of a data processing system. ROM 124, 144, and 164; RAM 126, 146, and 166; HD 128, 148, and 168; and databases 14 and 16 can include media that can be read by CPU 122, 142, or 162. Therefore, each of these types of memories includes a data processing system readable medium. These memories may be internal or external to computers 120, 140, or 160.

Portions of the methods described herein may be implemented in suitable software code that may reside within ROM 124, 144, or 164, RAM 126, 146, or 166, or HD 128, 148, or 168. The instructions in an embodiment may be contained on a data storage device, such as HD 148. FIG. 2 illustrates a combination of software code elements 204, 206, and 208 that are embodied within a data processing system readable medium 202, on HD 148. Alternatively, the instructions may be stored as software code elements on a DASD array, magnetic tape, floppy diskette, optical storage device, or other appropriate data processing system readable medium or storage device.

In an illustrative embodiment, the computer-executable instructions may be lines of compiled assembly, C, C**, Java, or other language code. Other architectures maybe used. For example, the functions of any one of the computers may be performed by a different computer shown in FIG. 1. Additionally, a computer program or its software components with such code may be embodied in more than one data processing system readable medium in more than one computer.

In the hardware configuration above, the various software components may reside on a single computer or on any combination of separate computers. In alternative embodiments, some or all of the software components may reside on the same computer. For example, one or more the software components) of the proxy computer 140 could reside on the client computer 120, the server computer 160, or both. In still another embodiment, the proxy computer 140 and database 14 may not be required if the functions performed by the proxy computer 140 are merged into client computer 120 or server computer 160. m such an embodiment, the client computer 120 and server computer 160 may be bi-directionally coupled to the same network (not shown in FIG. 1).

Communications between any of the computers in FIG. 1 can be accomplished using electronic, optical, radio-frequency, or other signals. For example, when a user is at client computer 120, client computer 120 may convert the signals to a human understandable form when sending a communication to the user and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by, computers 140 or 160. Similarly, when an operator is at server computer 160, server computer 160 may convert the signals to a human understandable form when sending a communication to the operator and may convert input from a human to appropriate electronic, optical, radio-frequency, or other signals to be used by computers 120, 140, or 160.

Attention is now directed to the methodology of developing a software architecture for the software in accordance with one embodiment. The method can comprise breaking down a transaction into a set of discrete actions. The actual definitions used for separating the transaction into the discrete actions is variable and may be selected by skilled artisans in manners that best suit their particular transactions, hardware requirements, and software requirements. The method can also include determining which functions within the set of discrete actions are common to more than one application. As more are identified, the number of shared resources can increase and the amount of application-specific code can be decreased. Therefore, skilled artisans are encouraged to examine the software from many different levels of abstraction to discover potential shared resources that may otherwise be missed.

The method can further comprise generating software components for the discrete actions. A set of software plug-in modules can correspond to the different discrete actions for the transaction. Each application may have its own set of software plug-in modules. The amount of code within each software plug-in module should be kept relatively low if the identification of shared resources was performed properly. To the extent code for any shared resources does not currently exist, code for the shared resources should be generated to maximize its ability to be used by as many different plug-in modules as possible. At least two of the software plug-in modules for different applications, whether they use the same or different protocols, can make function calls to any one or more of the shared resources. For different applications using the same protocol, only a request manipulation plug-in module, a content manipulation plug-in module, or both may be the only modules changed. Therefore, creating new application for the same protocol may be simplified because other plug-in modules used for the application may be copied from another application using the same protocol. These other plug-in modules may be substantially the same between the applications. By replacing or editing the request manipulation plug-in module, content manipulation plug-in module, or both, new applications may be developed very quickly.

Regarding applications in different protocols, each protocol may have a module that performs substantially the same action as any or all of the similar module(s) for the other protocol(s) though reducing this duplicative code by combining the common functionality is preferable.

Attention is now directed to the software architecture of the software in accordance with one embodiment. The software architecture is illustrated in FIGs. 3 and 4 and is directed towards an electronic transaction that can be performed over a network. A basic idea behind the architecture is to allow programming code for shared resources to be commonly used by as many different network applications as possible. Note that all of the resources may or may not be shared by all the applications. The programming code for each application-specific plug-in module may include code to connect the incoming communication in any supported application to the shared resources. By limiting the code within the plug-in modules, a user of the software architecture can reduce development time, increase the likelihood that more applications in the same or different protocols will be properly supported (especially proprietary protocols that may be used by only a limited number of computers or users), and reduce the burden on hardware and software resources for different applications because only relatively small plug-in modules may be used.

In FIG. 3, each row of boxes 3200, 3400, and 3600 represents different applications in the same or different protocols. For example, row 3200 may represent a first application using HTTP, row 3400 may represent a different application using HTTP, and row 3600 may represent yet another application in a different protocol, such as POP, SNMP, WAP, and the like. Note that the series of dots between rows 3400 and 3600 indicate that many other applications in the same or different protocols may be present. Additionally, the architecture may be configured to allow the addition of future applications. The software architecture easily supports at least three different and potentially many more protocols.

Referring to row 3200, each of the boxes 3202 through 3214 represents different stages (actions) that may occur during an electronic transaction. For example, box 3202 may represent a request reception plug-in module, box 3204 may represent an authorization plug-in module, box 3206 may represent a request manipulation plug-in module, box 3208 may represent a content retrieval plug-in module, box 3210 may represents a content manipulation plug-in module, box 3212 may represent a content delivery plug-in module, and box 3214 may represent a post-response communication plug-in module (e.g., acknowledgement, billing, etc.). Each module may correspond to one or more of the discrete actions. Details about the individual plug-in modules are described later in this specification. Note that the other rows 3400 and 3600 include corresponding boxes for substantially the same types of actions except that they are designed for different applications. More specifically, box 3402 represents an incoming message reception plug-in module for a different application using the same protocol as box 3202, and box 3602 represents an incoming message reception plug-in module for yet another application using a different protocol compared to box 3202. New applications that make use of already-supported protocols can be developed with a minimum of effort. This is achieved by creating a new row, which makes use of protocol specific plug-ins used in another row and combines them with other plug-ins developed for the specific application at hand. Some plug-in modules may be substantially the same for many different applications in the same protocol. In different protocols, the plug-in modules for at least some of the different applications may provide substantially the same functionality, although the code within those plug-in modules may be different compared to similar modules for the other protocols.

Within the software architecture, shared resources are illustrated as planes 3102, 3104, and 3106 that lie beneath each of the rows 3200, 3400, and 3600. Referring to FIG. 4, interfaces may be made to each of the shared resources for each plug-in module. Specifically referring to box 3214, functional connectivity 4102 links module 3214 and shared resource 3102. Likewise, functional connectivity 4104 links module 3214 and shared resource 3104, and functional connectivity 4106 links module 3214 shared resource 3106. Links 4102, 4104, and 4106 can be achieved by function calls to the shared resources. Examples of the shared resources may include a content cache, a parameter cache, a connection pool, a domain name server cache, a clock, a counter, a database, a global variables space (e.g., a logging database), or the like. A list of potential shared resources is nearly limitless. Note that not all shared resources may be connected to all modules along a row. For example, modules 3202 and 3204 may not need access to the content cache because they may not receive or process content returned for a request. Each connection from a client may be handled independently on its own thread. However in other embodiments, fewer threads or a single thread can be used to operate all connections to a specific row that supports a particular application or protocol. Unless stated to the contrary, the method below is described from the perspective of proxy computer 140.

FIG. 5 includes a flow diagram of a method of performing an electronic transaction that corresponds to the software plug-in modules that lie along any of rows 3200, 3400, and 3600. Note that all modules are not required and that functions of some modules may be combined with others (e.g., authorization may be part of processing an initial request). The process flow diagram will be briefly covered followed by a more detailed description of each module.

The method can comprise receiving a request from a client computer using a request reception plug-in module (block 502) and performing authorization using an authorization plug-in module (block 504). The method can also comprise manipulating a request using a request manipulation plug-in module (block 512). The method can further comprise retrieving content using a content retrieval plug-in module (block 522). The method can yet further comprise manipulating returned content using a content manipulation plug-in module (block 532) and sending the modified content to the client computer using a content delivery plug-in module (block 534). The method can still further comprise processing post-response communications using a post-response plug-in module (block 542). Note that not all of the activities described in the process flow diagram are required, that a limitation within a specific activity may not be required, and that further activities may be performed in addition to those illustrated. Also, some of the activities may be performed substantially simultaneously during with other activities. After reading this specification, skilled artisans will be capable of determining what activities can be used for their specific needs. Attention is now directed to the protocol-specific plug-in modules along the rows 3200, 3400, and 3600 and how they are related to the activities illustrated in FIG. 5. Although the discussion is directed to row 3200, the corresponding modules along other rows can provide similar functionality. Also, in the example below, client computer 120 is sending a request for content to proxy computer 140, and server computer 160 is providing content in response to the request. The flow of information could be in the opposite direction (server computer 160 seeking information from client computer 120). The method can comprise receiving a request from client computer 120 using request reception plug-in module 3202 (block 502 in FIG. 5). Request reception plug-in module 3202 can be used when a request from client computer 120 is received or accessed by proxy computer 140. Module 3202 can initially generate an associative array from portions of the header of the request. Part or all of the associative array may be used by the other modules along the same row. The associative array may provide information that can be part of function calls to the shared resources. Any or all the data (including the associative array) may be passed from any prior plug-in module (e.g., module 3202) to any or all the subsequent plug-in modules along the same row (e.g., 3204, 3206, 3208, 3210, 3212, or 3214). The method can also comprise performing authorization using authorization plug-in module

3204 (block 504). The authorization plug-in module 3204 is optional and can be used for determining whether a user at client computer 120 has proper authorization. The authorization modules may be based on an Internet Protocol ("IP") address or a name and a password. Module 3204 may send the IP address or name and password to a shared resource to determine if the user is allowed access. The method can further comprise manipulating the request using request manipulation plug-in module 3206 (block 512). Request manipulation plug-in module 3206 may be used to modify, replace, or otherwise manipulate the request. For example, proxy computer 140 may have code to redirect a URL within a request to a different URL. More specifically, proxy computer 140 may make a function call to that shared resource using the requested URL. The shared resource may pass the different URL back to module 3206. Module 3206 may have the logic to put the different URL in the correct protocol, so that it will be understood by a computer that may receive the redirected request.

The method can yet further comprise retrieving content using content retrieval plug-in module 3208 (block 522). Content retrieval plug-in module 3208 may be used to send the request and receive or access content in response to the original request or manipulated request. More specifically, a request originating from client computer 120 may have been processed by proxy computer 140 before being received by server computer 160. Content from server computer 160, in response to the processed request from proxy computer 140, would be processed using module 3208. Similar to module 3202, the code may parse the content from server computer 160 into a header portion and a content portion and append that information onto a previously generated associative array. The method can still further comprise manipulating returned content from the server computer using content manipulation plug-in module 3210 (block 532). Content manipulation plug- in module 3210 may be used to add or modify content before sending it to client computer 120. More specifically, proxy computer 140 may add advertisements or supplementary information from third parties to the content provided by server computer 160. In an alternative embodiment, part or all of the content originating from server computer 160 may be deleted or replaced with other content. The method can comprise sending the modified content to the client computer using content delivery plug-in module 3212 (block 534). Content delivery plug-in module 3212 may be used to route the content, after manipulation, if any, to client computer 120. Some of the information in the associative array generated when the original request from client computer 120 was processed may be used by module 3212 when sending the outgoing content to client computer 120.

The method can also comprise processing post-response communications using post-response plug-in module 3214 (block 542). Post-response communication plug-in module 3214 may be used for acknowledgement, billing, or other purposes. For example, after content is successfully sent to client computer 120 from module 3212, module 3124 could then charge the user's credit card for that transaction. Alternatively, module 3214 may look for a signal that service to or from client computer 120 or server computer 160 is being terminated for the current transaction. Such post-response processing may be helpful in avoiding invoices or other bills sent to a user at client computer 120 if a product or service was either incomplete or defective or to properly reflect the connect time for a transaction. Along similar lines, one of the planes as illustrated in FIG. 3 may include global space variables that may need to be used by other shared resources, proxy computer 140, or the plug-in modules. System statistics are examples of information that may be within a global variable space. This information may be useful to proxy computer 140 or another computer, such as client computer 120 or server computer 160, in monitoring activity. The statistics may include how many computers are connected to proxy computer 140, the amount of time each of those computers are connected to proxy computer 140, the amount of or time lapsed during transactions being processed through proxy computer 140, or the like.

These global variables may be used in conjunction with a module, such as authorization module 3204. If too many users are currently logged into proxy computer 140, authorization may be denied even if the computer attempting a connection to proxy computer 140 has proper security clearance. After another transaction by another client computer is terminated, a signal from module 3214 can be sent to the logging system within the shared resources. A new client computer may now gain access to the services provided by proxy computer 140 after the connection from the other transaction is terminated. Attention is now directed to more specific activities that may be performed by a specific module, and how that specific module may interact with other modules for the same transaction using a specific application. The process flow diagram illustrated in FIGs. 6-8 is used to describe some of the specific activities. Again, unless stated to the contrary, the method is primarily described from the perspective of proxy computer 140. To aid in understanding the method in FIGs.6-8, a specific example is used and occasionally referenced. In the example, an incoming communication may be a request from client computer 120 sent to proxy computer 140 for www.yahoo.com. The client computer 120 is communicating using HTTP, using a Netscape™ browser (of AOL Time Warner, Inc. of New York, New York), and has a MacOS X™ operating system (of Apple Computer, Inc. of Cupertino, California).

Referring to FIG. 6, the method can comprise receiving an incoming communication for a specific application (block 602). The communication can comprise a request, a message, or other form of communication. The communication can be sent by client computer 120 and received or accessed by proxy computer 140 via network 11. Proxy computer 140 can access or read at least a portion of the incoming communication and determine the specific application for the communication. In the example, the incoming communication is a request from client computer 120 sent to proxy computer 140 for www.yahoo.com. The incoming communication will also contain other information within the header of the request. In the example, the other information can include the browser and operating system of client computer 120.

After determining the application for the communication, proxy computer 140 can determine which row 3200, 3400, 3600, or other or row of plug-in modules will be used for the transaction. At this point in the method, proxy computer 140 may activate any or all of the plug-in modules for the row corresponding to the specific application. In one embodiment, plug-in modules within each row may be activated only as they are first used. Referring to the example, the request is for an application corresponding to row 3200. Therefore, plug-in module 3202 may be activated. If the communication is for another application, plug-in module 3402 or 3602 may be activated for the particular application.

The method can further comprise routing the incoming communication to a first software plug-in module for the specific application (block 604). Proxy computer 140 can route the request to request reception software plug-in module 3202 because the incoming request uses the application corresponding to row 3200. The method can comprise parsing the incoming communication into a header portion and a content portion (block 622). The parsing can be performed by module 3202 to obtain information from the request.

The method can also comprise generating an associative array using information contained within the header portion (block 624). The associative array can include nearly any finite number of rows. Each row can comprise a key and a value. The key can comprise a parameter within the header portion, and the value can comprise a value for that parameter. In general, the header portion may include one or more lines of a command followed by a command argument. The command may be a key, and the command argument may be the corresponding value for the key. The associative array may be searched by the key or the value. By knowing conventions used by each of the protocols for incoming communications and the characteristics of headers used for those protocols, formation of the associative array can be performed without complicated coding requirements. The associative array is flexible regarding the number of rows and allows different sizes of associative arrays to be used for different protocols.

For HTTP, one of the lines within the header may include a line with "User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv:l.l) Gecko." The key will be "User-Agent," and the value will be " Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US; rv: 1.1) Gecko." For POP, a line may include "RETR 27," where 27 is an object identifier for is a particular item to be retrieved. The key will be "COMMAND," and the value will be "RETR." A second entry will be made with a key of "ARGUMENT" and a value of "27." For SNMP, a line may include "get 47.12.112.38," where 47.12.112.38 corresponds to an object identifier. The key will be "COMMAND", and the value will be "GET," and a second entry will have the key "ARGUMENT'' and the value "47.12.112.38."

The content may or may not be part of the associative array. If it is, the associative array can include a key of "CONTENT" and the entire content data block as the value. For an image, the content may be a very large amount of data. Alternatively, the associative array may be paired with a data pointer that points to the data block, rather than incorporating it directly into the associative array.

Turning to the example, the associative array may include information as shown in Table 1 below. Descriptive names are used instead of actual names to aid in understanding the associative array. Also, the associative array may include many more rows. Because the associative array may be searched by key or value, the order of the rows is unimportant.

Table 1. Exemplary associative ar ay

The method can also comprise generating a function call to at least one of the shared resources using data within the associative array (block 702 in FIG. 7). In the example, proxy computer 140 can make a function call to a shared resource, more specifically to a clock (shared resource) and a logging system (another shared resource) to get the time and log the beginning of the transaction. The logging information may include the time and a transaction identifier. Note that some of the information within the associative array could be sent with the function call to the shared resource. The method can further comprise receiving data from the function call (block 704). In the example, the transaction identifier may be passed back to module 3202. The method can still further comprise processing data from the function call with other code within the first software module (block 706). Module 3202 may be more focused on processing the incoming message rather than processing data coming back from the function call. Other modules, such as the content deliver plug- in module 3212, may perform such data processing. Note that the application-specific processing may occur before, during, or after function call(s), if any, are made to the shared resource(s).

A determination may be made whether the first software plug-in module is the last software plug-in module (diamond 722). If so, the method may end. Otherwise, the method may continue with passing any or all of the data (including the associative array) from a prior software plug-in module to the next software plug-in module (block 802 in FIG. 8). Li the example, the next software plug-in module is authorization module 3204. Authorization module 3204 may use some of the information that was collected or generated by module 3202. Passing the information reduces the load on hardware by not sending a communication from proxy computer 140 to another computer (e.g., client computer 120) or making the same or similar function call to a shared resource for the same information.

The method can also comprise generating a function call to at least one of the shared resources using data within the associative array (block 822). Authorization module 3204 may make a function call to the parameter system to determine if the user has proper authorization, whether access can be granted (whether number of users currently connected to proxy computer has exceeded its limits), priority of connection (level or speed of service to be provided), etc. Module 3204 may pass user name and password when making the function call to the logging system. Module 3204 may also make a function call to the shared clock to obtain a time for the action.

The method can also comprise receiving data from the function call (block 824). The data may include information regarding whether user at client computer 120 has proper security clearance, whether the connection could be made, priority of the connection, and the like. The method can further comprise processing data from the function call with other code within the current software plug-in module (block 826). An example may include sending a communication from proxy computer 140 to client computer 120 informing the user whether the connection was made. Alternatively, no further processing may occur with module 3204.

A determination may be made whether the current software plug-in module is the last software plug-in module (diamond 842). If so, the method may end. Otherwise, the method may continue with block 802 in FIG. 8 and proceed in an iterative manner until the last software plug-in module is reached. The remaining modules along row 3200 will be addressed to complete the example transaction to give a better understanding of actions within the modules and some function calls that those modules may make. More or fewer modules may be used. Also, more, fewer, or different function calls may be made by the modules.

Data can be passed to request manipulation software plug-in module 3206. A function call can be made to a shared resource to determine if the request should be changed. The function call may pass information that a request for www.yahoo.com has been received or accessed. The shared resource may include logic to replace the original client request with www.google.com. The associative array may be changed to replace www.yahoo.com with www.google.com or be appended to note that the manipulated request is www.google.com.

Module 3208 may perform the content retrieval. A function call can be made to a content cache (shared resource) at proxy computer 140 to determine if the content cache includes a network page for www.google.com specifically formatted for a computer having a Netscape™ browser and a MacOS X™ operating system. Note that the browser and operating system information can be obtained from the associative array. If the content cache has the network page, it can be passed to module 3208. Otherwise, module 3208 may formulate an HTTP request to server computer 160 requesting the network page for the specific browser and operating system of client computer 120. After proxy computer 140 obtains the proper network page from server computer 160, module 3208 may send a function call to the content cache at proxy computer 140 to cache the network page. The proper network page and other information previously collected may be sent to module 3210.

Content manipulation module 3210 may delete, add, or replace some or all of the content within the proper network page returned. For example, when the proper Google network page is received or accessed, module 3210 may add advertisements) around the border(s) of the page. A function call can be made to a shared resource to determine which advertisements) should be added. The logging system may keep track of which advertisement is being added, whose advertisement it is, and how many times the advertisement has been added during the current billing cycle. The logging system, which is a shared resource, may access the counter (another shared resource) by itself. In other works, some or all of the shared resources may interact with each other without requiring an application-specific software plug-in module to intervene. The manipulated content and other information may be passed to module 3212.

Content delivery software plug-in module 3212 may take the Google network page formatted for a Netscape™ browser and MacOS X™ operating system and the advertisement(s) from module 3210 and prepare a communication using HTTP. The communication can be sent from proxy computer 140 to client computer 120. Function calls can be made to the logging system to note the actual content sent to client computer 120 and time sent. Any or all information collected or generated by modules 3202-3212 may be passed to module 3214. Post-response communications module 3214 may be used to track usage or billing information. At the end of a transaction, module 3214 may make a function call to the clock to determine the current time, and make another function call to the logging system to determine how much time lapsed during the transaction and record any billing information. The billing information may be within a shared resource managed by an accounting department. Billing information for the user at client computer 120 may be passed from one of the shared resources to module 3214, which may return some of the information for the user at client computer 120. Proxy computer 140 may send a message to client computer 120 similar to "You were connected for 2.1 minutes and were charged $1.27. Thank you for using our service." Alternatively, no message may be sent and the method may end.

Note that not all of the activities described in the process flow diagram in FIGs. 6-8 are required, that a limitation within a specific activity may not be required, and that further activities may be performed in addition to those illustrated. Also, some of the activities may be performed substantially simultaneously during with other activities. After reading this specification, skilled artisans will be capable of determining what activities can be used for their specific needs.

The power of creating new applications for the same protocol may be better understood with the flow diagram in FIG. 9 and an example. In one embodiment, different applications may be generated for different priorities of users for a network site. The communication protocol may use HTTP. The method can comprise developing a first set of plug-in modules for a first application (block 902). The set may correspond to row 3200 and be directed to premium users of a network site. A new application may need to be developed for regular users of the network site. The communication protocol may also use HTTP. The method can comprise copying the first set of plug- in modules to form a second set of plug-in modules (block 922).

For the new application, only the request manipulation plug-in module, the content manipulation plug-in module, or both may be replaced. The remainder of the plug-in modules may be unchanged and be substantially the same as the remainder of the plug-in modules for the first application.

The method may comprise replacing a first request manipulation plug-in module with a second request manipulation plug-in module for a second application (block 924). For example, the premium user may have access to some network pages that the regular user may not. If the regular user requests a premium page, the second request manipulation module may direct the regular user to another network page for which the regular user has proper access.

The method may also comprise replacing a first content manipulation plug-in module with a second content manipulation plug-in module for the second application (block 926). The premium user may have only 10 percent of his or her window occupied by advertisements, whereas the regular user may have 50 percent of his or her window occupied by advertisements. The second content manipulation module may reformat the retrieved content to allow for more advertising space. The second content manipulation module may also access the shared resources to obtain the advertisements and keep track of which advertisements were used. Device dependent optimization of network pages (desktop computer vs. cellular phone, etc.) can be achieved by plugging in a module which transcodes content using settings developed for the particular device that made the initial request. After one or both of the request manipulation and content manipulation modules are replaced, the method can still further comprise executing the second application using the second set of plug-in modules (block 942).

Note that while the example focused more on replacing specific modules, in other embodiments, those modules may be generated by editing code within the corresponding modules within the first set for the first application.

After reading this specification, skilled artisans will appreciate that entirely different applications, using the same network protocol, can be developed by simply inserting new plug-in module(s) at the request manipulation location, the content request location, or both locations.

In other embodiments, the method and system may be used for nearly any other network communications. As an example, client computer 120 may make a request for information within a database located at server computer 160. The request may be handled in a manner similar to a request for a network page. If the user does not have proper authorization to all information within a request, the request manipulation module may request only that information for which the user has property access or the content manipulation module may add information stating that the user does not have proper access to some or all the information.

In another embodiment, the multiple-protocol software architecture and plug-in modules may be installed in client computer 120 or server computer 160. Not all modules in proxy computer 140 may be needed by client computer 120 or server computer 160. Authorization modules 3204, 3404, and 3604 may not be used or can be coded to allow authorization (always authorized) at client computer 120. The content manipulation modules 3210, 3410, and 3610 may not be used by the server computer 160. After reading this specification, skilled artisans are capable of determine which modules are needed and which ones can be eliminated or bypassed (module exists but passes information through without performing any other significant activity).

The software components can be designed to maximize their ability use shared resources while minimizing the amount of code used for application-specific operations. Therefore, relatively smaller plug-in modules (compared to the shared resources) may be used to access the shared resources illustrated in the planes below the modules. L this manner, less code needs to be written for a new protocol compared to the prior-art method of writing or copying and modifying an entire program for a specific protocol. For applications in the same protocol, the specific coding requirements may be much less. Furthermore, protocols are more likely to be supported because the coding requirements are less, and therefore, may be generated for protocols that have relatively fewer users compared to other protocols. The method and system are significantly more efficient in both time and cost compared to existing prior-art methods dealing with the problem of many different applications in the same or different protocols.

Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any elements) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be constmed as a critical, required, or essential feature or element of any or all the claims.

Claims

1. A system comprising: a data processor; and a memory coupled to the data processor; 5 wherein the data processor is configured to implement a cache system that stores a plurality of data objects and corresponding keys within an internal data stmcture in the memory, monitor usage of the cache system, determine whether usage of the cache system indicates that an alternative internal 10 data structure should be used in the cache system, and if usage of the cache system indicates that alternative internal data stmcture should be used in the cache system, create the alternative internal data stmcture, migrate data from a current internal data stmcture to the alternative internal 15 data structure, and begin operation using the alternative internal data stmcture.

2. The system of claim 1 , wherein the data processor and memory reside within a network proxy.

20.

3. The system of claim 2, wherein the data objects comprise Web pages and the corresponding keys comprise uniform resource locators (URLs).

4. The system of claim 1 , wherein the system is configured to store network connection and 25 DNS information by storing data objects comprising IP addresses and keys comprising server names corresponding to the IP addresses.

5 The system of claim 1, wherein the system is configured to store configuration parameters by storing selected configuration data as objects and parameter names as keys corresponding to the

30 objects.

6 The system of claim 1, wherein the system is configured to store shared resources as objects and resource identifiers as keys corresponding to the objects.

7 The system of claim 1, wherein the system is configured to store open network connections as objects and systems, ports and communication protocols corresponding to the open network connections as keys corresponding to the objects.

8. The system of claim 1 , wherein the internal data structures in the cache system comprise at least two types selected from the group consisting of: lists; hashes, trees, heaps and hybrid data structures.

9. The system of claim 1, wherein the cache system is multi-tiered, and at least one tier of caches is configured to store references to caches as data objects.

10. The system of claim 1 , wherein the data processor is configured to create one or more caches in the cache system using a C++ template.

11. The system of claim 1 , wherein the data processor is configured to determine whether usage of the cache system indicates that the alternative internal data stmcture should be used by determining a cost associated with a current internal data stmcture and a cost associated with the alternative internal data structure and comparing the costs.

12. The system of claim 11, wherein determining the cost associated with the alternative internal data structure includes determining a cost for transitioning to the alternative internal data structure.

13. The system of claim 12, wherein transitioning to the alternative internal data structure comprises creating the alternative internal data structure, migrating data from the current internal data structure to the alternative internal data stmcture, and beginning operation using the alternative internal data structure.

14. A method: comprising providing a first internal data structure for storing a plurality of objects; using the first internal data structure; monitoring usage of the first internal data stmcture; determining a cost associated with usage of the first internal data structure; determining a cost associated with usage of an alternative internal data stmcture; comparing the cost associated with usage of the first internal data stmcture to the cost associated with usage of the alternative internal data stmcture; and if the cost associated with usage of the first internal data stmcture is greater than the cost associated with usage of the alternative internal data structure, creating the alternative internal data structure and migrating the plurality of objects to the alternative internal data structure.

15. The method of claim 14, wherein determining the cost associated with usage of the alternative internal data structure comprises determining a cost associated with creating the alternative internal data structure and migrating the plurality of objects to the alternative internal data stmcture.

16. The method of claim 12, wherein determining the cost associated with usage of the alternative internal data structure further comprises determining a cost associated with storing objects in the alternative internal data stmcture and retrieving objects from the alternative internal data structure.

17. The method of claim 14, further comprising periodically repeating the steps of determining the cost associated with usage of the first internal data stmcture, determining the cost associated with usage of the alternative internal data structure and comparing the cost associated with usage of the first internal data stmcture to the cost associated with usage of the alternative internal data stmcture.

18. The method of claim 14, wherein determining the cost associated with usage of at least one of the first internal data structure and the alternative internal data stmcture comprises computing a cost based on estimated usage.

19. The method of claim 14, wherein determining the cost associated with usage of at least one of the first internal data structure and the alternative internal data stmcture comprises computing a cost based on empirical usage data.

20. The method of claim 14, further comprising, if the cost associated with usage of the first internal data structure is greater than the cost associated with usage of the alternative internal data structure, deleting the first internal data structure after migrating the plurality of objects to the alternative internal data stmcture.

21. The method of claim 14, wherein providing the first internal data structure for storing a plurality of objects comprises providing a cache system having the first internal data stmcture in a network proxy.

22. The method of claim 21 , wherein using the first internal data structure comprises storing and retrieving entries in the first internal data stmcture, wherein each entry includes an object comprising a Web page and a corresponding key comprising a uniform resource locators (URL) corresponding to the Web page.

23. The method of claim 21, wherein using the first intemal data stmcture comprises storing and retrieving entries in the first internal data structure, wherein each entry includes an object comprising an IP addresses and a corresponding key comprising a server name corresponding to the IP address.

24 . The method of claim 21 , wherein using the first internal data stmcture comprises storing and retrieving entries in the first internal data stmcture, wherein each entry includes an object comprising selected configuration data and a key comprising a parameter names.

25 The method of claim 21, wherein using the first internal data structure comprises storing and retrieving entries in the first internal data structure, wherein each entry includes an object comprising a shared resource and a key comprising a resource identifier.

26 The method of claim 21, wherein using the first internal data structure comprises storing and retrieving entries in the first internal data stmcture, wherein each entry includes an object comprising an open network connection and at least one key comprising one or more of: a system and a port to which the connection has been made and a communication protocol used for the connection.

27. The method of claim 14, wherein the first and alternative internal data structures comprise at least two types selected from the group consisting of: lists; hashes, trees, heaps and hybrid data structures.

28. The method of claim 14, wherein the first internal data structure comprises a cache within a multi-tiered cache system, and wherein at least one tier of caches in the multi-tiered cache system is configured to store references to caches as data objects.

29. The method of claim 14, wherein providing each internal data structure comprises creating a corresponding cache using a C++ template.

30. A software product comprising a plurality of instructions embodied in a medium readable by a data processor, wherein the instructions are configured to cause the data processor to perform the method comprising: providing a first internal data stmcture for storing a plurality of objects; using the first internal data stmcture; monitoring usage of the first internal data stmcture; determining a cost associated with usage of the first internal data stmcture; determining a cost associated with usage of an alternative internal data structure; comparing the cost associated with usage of the first internal data stmcture to the cost associated with usage of the alternative internal data structure; and if the cost associated with usage of the first internal data stmcture is greater than the cost associated with usage of the alternative internal data stmcture, creating the alternative internal data structure and migrating the plurality of objects to the alternative internal data stmcture.

31. The software product of claim 30, wherein determining the cost associated with usage of the alternative intemal data stmcture comprises determining a cost associated with creating the alternative internal data stmcture and migrating the plurality of objects to the alternative internal data stmcture.

32. The software product of claim 31 , wherein determining the cost associated with usage of the alternative internal data stmcture further comprises determining a cost associated with storing objects in the alternative internal data structure and retrieving objects from the alternative internal data structure.

33. The software product of claim 30, wherein the method further comprises periodically repeating the steps of determining the cost associated with usage of the first internal data stmcture, determining the cost associated with usage of the alternative internal data stmcture and comparing the cost associated with usage of the first internal data stmcture to the cost associated with usage of the alternative internal data stmcture.

34. The software product of claim 30, wherein determining the cost associated with usage of at least one of the first internal data structure and the alternative internal data stmcture comprises computing a cost based on estimated usage.

35. The software product of claim 30, wherein determining the cost associated with usage of at least one of the first internal data stmcture and the alternative internal data stmcture comprises computing a cost based on empirical usage data.

36. The software product of claim 30, wherein the method further comprises, if the cost associated with usage of the first internal data stmcture is greater than the cost associated with usage of the alternative internal data stmcture, deleting the first internal data structure after migrating the plurality of objects to the alternative internal data stmcture.

37. The software product of claim 30, wherein providing the first internal data stmcture for storing a plurality of objects comprises providing a cache system having the first internal data stmcture in a network proxy.

38. The software product of claim 37, wherein using the first intemal data structure comprises storing and retrieving entries in the first internal data stmcture, wherein each entry includes an object comprising a Web page and a corresponding key comprising a uniform resource locators (URL) corresponding to the Web page.

39. The software product of claim 37, wherein using the first internal data stmcture comprises storing and retrieving entries in the first internal data stmcture, wherein each entry includes an object comprising an IP addresses and a corresponding key comprising a server name corresponding to the IP address.

40 The software product of claim 30, wherein the internal data structures are configured to store configuration parameters by storing selected configuration data as objects and parameter names as keys corresponding to the objects.

41 The software product of claim 30, wherein the internal data structures are configured to store shared resources as objects and resource identifiers as keys corresponding to the objects.

42 The software product of claim 30, wherein the internal data structures are configured to store open network connections as objects and systems, ports and communication protocols corresponding to the open network connections as keys corresponding to the objects.

43. The software product of claim 30, wherein the first and alternative internal data structures comprise at least two types selected from the group consisting of: lists; hashes, trees, heaps and hybrid data structures.

44. The software product of claim 30, wherein the first internal data stmcture comprises a cache within a multi-tiered cache system, and wherein at least one tier of caches in the multi-tiered cache system is configured to store references to caches as data objects.

45. The software product of claim 30, wherein providing each internal data stmcture comprises creating a corresponding cache using a C++ template.