KR20020003859A - Web server content replication - Google Patents

Abstract

The present invention relates to managing a plurality of web servers, and more particularly, to a web service system and method by which a system operator can distribute content to each web server in a web service system. In one embodiment, a method of replicating changes in a source file set of a destination file system identifies the changes in the source file set, stores the identified changes in a modification list, and stores the modification list in the destination file system. And sending to the agent accessing the agent. In another embodiment, a method of replicating changes in a raw file set of a destination file system identifies changes in the raw file set, stores the identified changes in a modification list, and stores the modification list in a plurality of webs. This includes sending to the server. In another embodiment, a web service system includes a host including an administrator managing the web service system, a web server communicating with the administrator to receive web page requests and agents, and providing the content change to the host. It includes a content distributor. In another embodiment, the content distributor includes an identification module that identifies changes in the raw file set, a modification list that stores the identified changes, and a transmitter that sends the modification list to an agent accessing the destination file system. Include.

Description

Web server content replication {WEB SERVER CONTENT REPLICATION}

In a computer communication network environment, a web server is used to respond to a web page request of a user transmitted through the computer communication network. Web page requests, also called content requests, are typically made by a browser running on a user's computer. The web server monitors one or more computer network address / port endpoints to request a web page, and sends the web page to the requestor in response to the web page request. Web servers can be special purpose devices and can also be run as software programs running on general purpose computers. The service capacity of the web server limits the number of web page requests that can be received and responded for a predetermined time.

There is more than one web server in a web service system. In general, when there is more than one web server in a web service system, the web service system is designed such that the plurality of web servers each responds to a web page request. Typically, a user's web page request is sent to one of the web servers, and the web server responds to the web page request. Also, there are many web servers in a web service system that are typically designed to receive many web page requests.

In general, in a system with multiple web servers, system operators or operators manage content provided by various web servers. The system operator may wish to adjust the content on the system so that, for example, the content of various web servers can be consistently matched, or some content is available from one web server, and other content is available from another web server. May be. This is difficult to implement. It is difficult to implement, especially when content updates are made apparent to the potential user while talking to a system having a series of related web pages.

Content management also has a problem with caching servers. The caching server later caches or temporarily stores the result of the delayed request from the browser to the web server to satisfy the same request. The problem with cache server design and operation is to determine when the stored (cached) content does not match the content of the original server at all, ie when the cached content is invalid. The hypertext transfer protocol (" http ") includes a mechanism for the origin server of some content to specify how long the cache server should retain a copy of that content. However, for some content the origin server cannot accurately determine in advance how long the content will remain valid, and may unexpectedly update the content earlier than the expiration time specified by the original server.

This application is incorporated by reference in US Patent Application No. 09 / 086,821, US Patent Application No. 09 / 086,836, US Patent Application No. 09 / 086,874, and US Patent Application No. 09 / 874,263, filed May 29, 1998. US Patent Application No. 60 / 117,674, filed Jan. 28, 1999, US Patent Application No. 09 / 377,611, filed Aug. 19, 1999, and US Patent Application No. 09 / 376,017 was claimed as priority.

The present invention relates to managing a plurality of web servers, and more particularly to a web service system that allows a system operator to distribute content to each web server of the web server system.

1 is a block diagram of one embodiment of a web service system in accordance with the present invention;

2 is a detailed block diagram of one embodiment of a web service system.

3 is an operational flow diagram of an embodiment of the content distributor of FIG.

4 is a comparison flow diagram of a current source file set and a previous file set.

5 illustrates one embodiment of a manifest entry.

6 is an exemplary view of a list of changed files.

FIG. 7 is an exemplary diagram of a manifest showing the modified file of FIG. 6. FIG.

8 is an embodiment diagram of old version maintenance.

9 is a block diagram of one embodiment of a web service system in communication with a caching server.

10 is a flow diagram of one embodiment of a method of distributing content.

11 is a flowchart of operation of another embodiment of a content distributor.

12 is an operational flow diagram for another embodiment of the content distributor.

FIG. 13 illustrates an example of information provided to a script in the embodiment of FIG. 12. FIG.

In a web service system having one or more web servers, systems and methods for managing and distributing content in the one or more web servers are useful for system operators. For example, it is desirable that the content update be executed quickly from time to time. Use scheduling and automation to update content on the server in an efficient and consistent manner. It also identifies content update failures, removes the rogue server from service, repairs it, and returns it to the service of the web service system as soon as possible. Even when an update occurs, it is desirable to make full use of the Web site while reducing transaction corruption. Changes may require a server restart, for example, if the content is provided using a shared library that will not be unloaded (and / or updated) until the web server process is finished.

The web service system according to the present invention accurately and efficiently updates the changed content in one or more web servers of the system so that the changes are constant between the web servers and do not require excessive network bandwidth.

This ensures that the content changes are not prominent in the browser interworking with the web server, and that the content versions are preserved to match the transaction initiated using the content, and the web server can return to the old content if there is a problem with the update. do. The web service system of the present invention may track content changes and may notify the appropriate caching server that the caching content becomes invalid.

In general, in one aspect, the present invention is directed to a system and method for replicating changes in a source file set on a destination file system. Changes to the source file set are identified. The change is stored in the revision list. The modification list is sent to the agent accessing the destination file system. In one embodiment, the modified files are sent to the agent. In another embodiment, the modified files are installed on the destination file system. In another embodiment, the changes are identified by examining the file set and comparing the file set with the initial record set. In other embodiments, the changes are identified by installing a device driver to perform file operations and writing the changes to the source file set by the device driver. In other embodiments, the changes are identified by receiving, in the source file set, a manifest listing the changes, and the files are compared by comparing file attributes with the file attributes of the initially recorded set. do. In one embodiment, the file attribute comprises at least one attribute selected from the file size, file authorization, file ownership, modification time and the set of file hashes. In other embodiments, the method includes invoking a script prior to identifying the change. In other embodiments, the method includes calling a script after sending the change. In other embodiments, the method includes invoking a script after determining whether the transfer has been successfully terminated.

In general, in other aspects, the present invention is directed to a system and method for replicating changes in a source file set on a destination file system. The identified changes are stored in a modification list that includes uniform resource locators (URLs) that specify the changed files. The modification list is sent to at least two web servers. In one embodiment, the transmission is performed by multicasting. In one embodiment, the changes are identified by examining the file set while simultaneously comparing the file set with the initially recorded set. In other embodiments, the file set is compared with the initially recorded set by comparing the file attributes with the file attributes of the initially recorded set. In other embodiments, the file attribute includes at least one attribute selected from a set of file size, file authorization, file ownership, modification time, and file hash.

In general, in other aspects, the invention relates to a web service system. The system has an administrator for managing the web service system. The system also has a web server that communicates with the administrator to receive web page requests and agents. The system has a content distributor for providing content changes to the host. In one embodiment, the content distributor includes an identification module for identifying changes in the source file set, a modification list for storing the identified changes, and a transmitter for transmitting the modification list to an agent accessing the destination file system. There is. In another embodiment, the system has a transmitter for transmitting the modified file to the agent. In other embodiments, the agent has an installer that installs the modified file on the destination file system.

In general, in other aspects, the invention relates to a content distributor. The content distributor has an identification module that identifies changes in the source file set. The content distributor also has a transmitter for transmitting the modification list to an agent accessing the destination file system. In one embodiment, the content distributor has a transmitter that sends the modified files to the agent.

Other objects, features and advantages of the present invention will become more apparent from the following description.

A system serving web pages has a plurality of web servers and provides a system operator with features and tools for coordinating the operations of the plurality of web servers. The system has only one web server, but typically there is more than one web server. Since the system sends a web page request, typically initiated from a web browser, on a client computer to an available web server, traffic can be managed by balancing the web page request service load among the plurality of servers. The system collects data upon a web page request, the web server can respond to the web page request, and provides automatic and manual analysis tools as well as reporting of the data. The system can monitor certain events and operate automatically upon the appearance of such events. The events include an estimate or threshold that indicates an impending system problem. The system has a crisis management capability that automatically recovers from errors and guides the system operator through possible actions that can be recovered from events such as component failures or network environmental problems. The system may present current information about the system operation to a system operator. The system can manage content duplication with version control and data updates. Some or all of these functions are provided in certain embodiments.

Referring to FIG. 1, the entity of the web service system 90 receives a web page request from the browser 1. In this content, a web page is electronic content that can be used in the computer communication network 2 in response to a web page request. Requests are typically initiated from the web browser 1. Examples of web pages are executable or interpretable information, graphics, voice, text and / or which may be displayed, executed, played, processed, streamed, stored, and may include links or pointers in other web pages. Or video. The request is communicated via the communication network 2. In one embodiment, the communication network 2 is the Internet. In other embodiments, the communication network 2 is a global communication network known as the Internet. The browser 1 is operated by a user for a web page request, can be operated by a computer or a computer program, and automatically makes a request based on the programming of the computer. The web page request is made using HTTP format and may be made using other protocols with request capability.

Referring to FIG. 2, the entity of the web service system 90 includes various components 100-126. The components communicate over one or more networks. The physical location of the components does not compromise the capability or performance of the system as long as the communication link between the various components has sufficient data communication power. The web service system 90 may operate through variously designed firewalls and may be remotely deployed and managed.

The web service system 90 manages one or more hosts 100. One host 100 is shown as an example. The entity of the web service system 90 may have any number of hosts 100. Each host 100 may use a multi-threaded operating system such as UNIX or Windows NT, and may be a commercially available computer system. Each host 100 may have at least one network connection to a computer communication network, such as the Internet or an intranet, or any other communication network to which the host 100 may provide web page data in response to a web page data request.

The web server 102 may be a web server that provides a web page in response to a web page request received through a computer communication network. Two examples of such web servers are the NETSCAPE ENTERPRISE SERVER from Mountain View Netscape Communications Corporation and the MICROSOFT INTERNET INFORMATION SERVICES SERVER from Redmond Microsoft. The web server 102 may receive a web page request from a web client, referred to as a browser and / or web page request. Web page requests from the browser are referred to as content requests, and web page requests from the view point of the web server are referred to as "hits." Often a web page request is part of communicating with web server 102 which includes some requests and responses. Such a series of sessions, referred to as sessions, extends the conversation with the web server. Short conversations, such as purchasing an item, are referred to as transactions. The session may include some transactions. The user communicates with the web server 102 at the initial request of the web server 102, thereby sending the web page in response. The web page contains information and also points to other requests for the user to make a web server 102 or other web server. Sometimes the requests are for information that must be retrieved from the database, and sometimes the request includes information stored in the database. Occasionally, the request requires processing by the web server 102 or a conversation with another computer system. Complex web servers and browsers can communicate in a variety of ways.

The set of related web pages is presented to the user as a set of web pages on related topics or from a particular source, but is not generally represented from the same web server 102 and is called an application. One example of an application is a set of pages that provide information about recall. Another example of an application is a series of pages that a user can transact with his savings bank. Two sets of web pages can be considered as one application or as two separate applications. For example, a web page set provides information about a bank, and a customer service set of web pages allows business transactions with the bank. Whether a web page set is one or several applications is determined by the application designer. The web service system 90 may deliver one or more applications to a user. The web service system 90 may be arranged such that some auxiliary set of the web server 102 may exclusively provide one application. In one embodiment, some web servers 102 service an auxiliary set of available applications, and other web servers 102 service other applications. In other embodiments, all web servers 102 service all available applications.

The web page that appears to the user in response to a web page request from the user's web browser may be stored in a host 100 or file system that may access the web server. Some or all web page content may be generated by the web server 102 by processing data available to the web server 102. For example, for a web page with a document about a topic, the web page may be generated and stored in the web server 102 file system. In response to a web page request, such web page may be sent to the user as stored in the file system. However, in a bank transaction system, information about a user's bank account may be stored in a database. The web server 102 may generate a web page including account information of the user every time a user requests a page. Occasionally, web pages are partially stored in the file system and, when requested, are partially generated by the web server 102.

Various techniques are used to store state information, which is called the state of the web server 102 and the user session. The user may develop a state during the conversation with the web server 102 via a request made to the web server 102 and a web page received in response to the request. As one example, the user's status includes information identifying the user. As another example, the status may include information specifying a web page that the user has already requested, or an option selected by the user in conversation with the system. As another example, the status may include an item that the user selects to purchase from the sales application. In general, some information identifying the state of the session or some information about the state may be stored in a client web browser, such as a cookie to be described later, and some information may be stored in the web server 102.

The host 100 may determine the number of executions of the web server 102 in consideration of capacity, performance, and cost. In one embodiment, the host 100 includes one web server 102. In other embodiments, the host includes one or more web servers 102. One web server 102 on host 100 is a simplified example and does not limit the number of possible web servers 102. Each web server 102 monitors at least one network address and port, also referred to as an endpoint. Certain addresses and ports are called endpoints because they are the actual endpoints for communication, and network connections are made between one address / port endpoint and another. The web server 102 receives a request forwarded to one of its endpoints and responds to the request with the data in the form of a web page.

A web server 102 that accepts requests at multiple network address / port endpoints may run as if there are multiple web servers 102 even though one web server is actually running. Such a web server is referred to as a plurality of endpoint web servers. For this discussion, a plurality of endpoint web servers can be described as if there are actually a plurality of web servers 102 with each web server 102 receiving a request on a network address / port endpoint. In one embodiment, such a plurality of endpoint web servers has one web server interface 104 with all the plurality of endpoint interfaces.

Each web server 102 is coupled with a web server interface 104. The web server interface may be a plug-in, filter or other software associated with the web server 102 serving as an interface between the web server 102 and other components of the web service system 90. In this content, the term web server interface is distinguished from a network interface that may appear on the host 100. For example, the web server 102 has a web server interface 104. Each web server interface 104 may communicate with an agent 106 on each callout 100.

Host 100 includes agent 106. The agent 106 provides a web service system 90 interface to the host 100. The agent 106 couples the web server interface 104 with the web service system 90. The agent also couples the host 100 with the web service system 90. Even on a host with multiple web servers, there is only one agent 106 running on the host 100, but may be more than one. Each agent 106 accesses a database 108 that contains information about the system components.

Agent 106 on host 100 communicates with web services system manager 110. The manager 110 receives information from the agent 106 regarding the status of the host 100 and the web server 102. The manager 110 sends commands to the agent 106 to place the host 100, to start, stop or stop the web server 102, and to manage load on the web server 102. Can be. The manager 110 accesses a logging database 114 that is used to log system activities and events. The manager 110 also accesses a management object database 112 that stores information about various components of the system. The manager 110 also communicates with one or more consoles 116A- 116X, and is generally referred to as 116. The console 116 provides a user interface to the system operator. The system operator can monitor the status of the system and deploy the system via a console. The manager 110 may run on the same host 100 as one of the web servers 102 or other web service system 90 components such as traffic manager 120.

The manager 110 communicates with the traffic manager 120 and is referred to as an interceptor. The traffic manager 120 instructs the web server to request a web page. The invention is not limited to a particular type of traffic manager 120, but rather works with any kind of traffic manager 120 that directs a web page request to the web server 102.

In one embodiment, the traffic manager 120 receives information and commands from the manager 110. In addition, the traffic manager 120 receives information and commands from the control program 122. The traffic management control program can be the same computer system as traffic manager 120, or can run on other systems. The traffic manager 120 receives the web page request and references the request to one of the web servers. Part of the management capability of the web service system 90 is performed by monitoring web page requests made on the web server 102 and the results on the web server 102 and the host 100. The web page request is to balance the load between the web servers 102. In one embodiment, the traffic manager 120 is a first point of contact for a user. The traffic manager 120 receives a web page request from a user and sends the user's web browser to a web server 102 suitable for the request. The user's web browser is sent in response to a web page request sent as a web page to a suitable web server 102. This transmission capacity is performed with the capacity included in http, but cannot be performed in other ways. The user may or may not know that the web browser is sent to the appropriate web server 102. A user accesses an application on the web server 102 and simultaneously receives the web page request response from the web server 102. In one embodiment, when the web server 102 is overloaded, the web server 102 in the direction of the manager 110 is inverse to the traffic manager 120 or other web server 102 capable of delivering the application. To the user.

The traffic manager 120 receives the requests from the user and sends the user's requests back to the web server 102. In one embodiment, the traffic manager 120 is used to direct all users to one web server 102, such as other traffic managers 120 or one endpoint. In this way, the traffic manager 120 acts as a shunt, which means sending all requests sent to one or more web servers on a host to another web server 102. In other embodiments, the traffic manager 120 receives status information from the manager 110 and directs the user back to the user using the information. The state information includes server availability and load, administrator changes, and application or web server 120 startup and shutdown operations. The traffic manager 120 is designed for speed and security. The traffic manager 120 is the front door of the system and, as a result, its performance affects the perceived performance of the overall web service system 90. It may be useful to locate the traffic manager 120 that is closed upon detecting the network topology as possible in the backbone. Inevitably it is the most exposed part of the web service system 90.

In one embodiment, the traffic manager 120 runs in hardware. In another embodiment, the traffic manager 120 is a software program running on a host computer. In one software entity, the traffic manager 120 is an independently operable program that runs on a server class computer capable of running a multithreaded operating system. Under UNIX, for example, the traffic manager 120 can run as a daemon. Under Windows NT ^™ , the traffic manager 120 can run as a service.

In another embodiment, the traffic manager 120 is an Internet protocol bridge or router that directs requests originating from one endpoint to an endpoint belonging to the web server 102. In this way, the traffic manager 120 sends a web page request to one or more web servers 102. One example of such a traffic manager is LOCALDIRECTOR, available from Cisco Systems, Inc. of San Jose, California. In other embodiments, the traffic manager 120 is a web switch, such as CONTECT SMART WEB SWITCH, available from Aeropoint Communications, Westford, Massachusetts. The traffic manager 120 receives each web page request and sends the request to the web server based on the request.

The web service system 90 includes a version controller referred to as content distributor 125. The content distributor 125 manages versions and content replication, including content updates of various web servers 102 in the web service system 90. A system operator interface versus the content distributor 125 is provided to content control 126. In one embodiment, the content distributor 125 and content control 126 are independently operable processes operating on the host 100. In other embodiments, the content distributor and content control 126 operate on the same host as the manager 110. In still other embodiments, content distributor 125 and content control 126 can operate on the same host or on other hosts. In other embodiments, the content distributor 125 is included in the functionality of the manager 110 or other components of the system 90.

The content distributor 125 transmits information to the agent 106 and to the web server interface 104 via the agent 106. The transmitted information describes changes to the file and content directories referred to as the raw file and is generally organized in a hierarchical directory structure. This source file is a "master" copy of the content for the web server. Together, the directories and files are referred to as raw files or set of raw files. The set of raw files is stored in a raw host, referred to as a stacking server, which is a common but unnecessary host that the content distributor 125 executes. To simplify the description, in the following discussion, the content directory and files in one of the source files or host 100 are generally referred to as files. The source file set includes a directory and a file, and changes to the file may include adding, deleting, and modifying a directory.

Referring to Figure 3, changes occur over some time period in the set of source files (step 180). Such changes include changes such as creating new files, deleting old files, replacing existing files with new content (i.e. modifying file contents) and modifying file attributes such as authorization restrictions and ownership. Again, even if changes to the file are described, the changes are included in the directory. Changes are made by editing the file using conventional editing techniques and tools, including application programs and application system utilities. Changes can be tested and certified in the development and approval process. In fact, there are no file changes at any given time, and the system can recognize this. In such a case, after identifying the absence of changes, no further processing can be done until the next time that can be identified.

The content distributor identifies the change as the file (step 181). Changes to the file can be identified in many ways. In one embodiment, the content designer provides a manifest, which is a list of changes. The designer keeps track of the changes and either manually identifies the changes to the content distributor using the content control user interface 126, for example, or manually delivers the manifest to the list distributor 125. To identify. In other embodiments, software and systems used by content designers to design the content track track changes and provide a manifest of changes delivered to the content distributor 125.

In other embodiments, the content distributor 125 gathers in an operating system file server to monitor changes to the set of source files. In such an embodiment, a software device driver " monitor " is installed that is called by the operating system to perform file operations. The device monitor acts as a device driver pass path that passes data between the operating system and the actual device driver (ie, hard disk driver software). In addition to the disk I / O commands passing through the disk driver, the device monitor also observes the I / O commands and records the changes in the set of source files. In one embodiment, the device monitor is integrated into the content distributor 125. In other embodiments, the device monitor generates a manifest, which is delivered to the content distributor 125.

In other embodiments, the content distributor 125 compares the current source file set with the previous source file set to determine changes to the source file set. The content distributor maintains a list of recently displayed files that are updated for comparison purposes. This list includes file names and other file characteristics such as file size, date / time the file was last modified, hash code authorization and / or access control restrictions on the content, and user / group ownership. In one embodiment, the list of files includes a complete copy of all files in the previous set of source files, but in this embodiment is very inefficient for large file systems. In a more efficient embodiment, only a list of previous files and specific file attributes is needed. As described later, the step of identifying the change (step 181), sending the change (step 182), and installing the change (step 183) is sometimes a reference to the update.

Referring to FIG. 4, in one embodiment, the comparison between the source file set and the previous file set occurs by file analysis of the source file set. For each file in the source file set (step 190), the content distributor compares the file in the source file set in the form of the aforementioned list with information describing the previous source file set in one embodiment. For each file, the content distributor determines if the file is in the list of previous source files (step 191). Otherwise, the file is listed as a new file indicating that the file is to be added to the set of source files (step 192), and a hash is calculated on the file (step 193). If it is a list of files, the file list is displayed to indicate that the file is still included in the file set (step 194). This is later used to determine whether any file in the previous list has been lost or deleted.

The size of the file is compared with the information in the list (step 195). If the file size is different, the file is considered modified and listed as such (step 196). The new hash of the file is calculated from the list and stored (step 193). If the magnitudes are the same, the date / times are compared (step 197). If the date / time is different, the hash of the file is calculated and compared to the hashes included in the previous list. In this list, the hash is calculated on the file content due to one number associated with the file content. Such a hash is referred to as a message integration code. Examples of hash codes are checksums and circular redundancy codes. If the hash of the file is different, the file is listed as modified (step 196) and the new hash is stored in the list (step 193). Alternatively, if the file attribute (owner, group, etc.) is changed (step 199), the file is listed as the attribute changed (step 200).

In one embodiment, if the file content does not change, the system determines whether file attributes are to be changed (step 199). In other embodiments, the attribute information is sent as part of the content update. Once these steps are completed for all files in the source file set, the notation from step 194 checks whether any files previously listed are now missing, meaning that they have been deleted from the file set (step 200 ). Files not marked are listed as deleted (step 202).

The change to the source file is the same as the content distributor 125 of the change list referred to as the manifest. In one embodiment, the manifest contains entries indicating changes to the file, such as additions, deletions, content modifications and / or attribute modifications. For example, the additional statement "ADD source / dir / dir / file1" indicates that the file: file1 "is added, and the additional statement" CHMOD source / file2 "indicates that the authorization associated with the file" file2 "has been changed. Other statements include "MODIFY" to indicate the change of content and "DELETE" to indicate the deletion of the file.Additional attribute differences can also be made using CHMOD statements, CHOWN for ownership change or CHGRP for group association change, etc. It can have its own specific statements.

Referring to FIG. 5, in one embodiment, the manifest entry 300 includes at least eight information elements 301-308. In one embodiment, each entry is stored in text format such that the information can be read by a system operator with a simple text editor or interpreted by other software. In other embodiments, each entry is stored in a binary format containing the same information as the text format, but much more compact. The binary format can be converted to the text format by a conversion tool. The text format can also be converted to binary format.

In one embodiment, the manifest list entry 300 includes a file type field 301. Possible file types are the end of a directory, file, sign link, pod link, or directory marker, followed by the last file in a directory. In a text entity, each file is described by three character codes such as "DIR" for a directory, "END" at the end of a directory marker, "NRM" for normal files, "LNK" for signed links, and "HLK" for hard links. do. The unaltered file will be described using the three character codes "NOC". For binary entities, each choice is represented by an integer code. For example, in one embodiment, the added code is represented by the number 1, the changed content is represented by 2, and so on.

In addition, the manifest list entry 300 includes an authorization field 303 to describe file access authorization and other information about the file. In a text entity, the authorization is stored in UNIX style, such as three 3-item binary entries (i.e., rwxrwxrwx), where the first three-entry entry describes whether the executable program will be executed by the user, and then 3 Entries appear red and white, and perform authorization to their owner, group and general, respectively. In the binary entity, the authorization information is stored as a binary integer which is a 4 digit octal number indicating the authorization.

The manifest entry 300 also includes a file size field 304 to describe the size of the file. In text entities, the file size is written in ASCII characters. In binary entities, the file size is stored as an integer.

In addition, the manifest entry 300 includes a date / time stamp 305. In text entities, the date and time are written in ASCII characters. In binary entities, the date / time is stored as an integer representing the flow of seconds during the night of January 1970.

The manifest entry 300 includes ownership information 306. This information includes the user identifier of the file owner and the group identifier of the group owner. In a text entity, the owner and group identifier are stored as ASCII strings. In binary entities, the owner identifier and the group identifier are each stored as an integer.

The manifest list entry 3000 includes a checksum 307 or a hash result. In text instances, this is stored as an ASCII string. In a binary entity, the check sum is stored as an integer. The manifest also contains the relevant pathname 308 of the file. In two text and binary entities, the associated pathname 308 is stored as an ASCII string.

Referring to FIG. 6, an example of a change list includes an additional file (aaa) (directory), file agents (reg, questid.reg and sqlserver.reg) of the directory (aaa), and files (bbb and ccc) (directory). have. The directories ddd and files abc and efg of the directory ddd are not changed. The directory efg is also unchanged. Directories (mmm, xxx and yyy) are deleted, and file agents (reg, questd, reg and sqlserver.reg) are deleted from directory (xxx).

Referring to FIG. 7, an example text instance of the drop list of the format of FIG. 5 reflects the changes listed in FIG. 6. In the first entry (ENTRY 1), the directory (aaa) is listed in the file name (308). The file aaaa is listed as type 301 "DIR". Operation 302 for this file is "ADD" to indicate that the file is to be added. The authorization 303 is listed in UNIX style, indicating that the file is a directory and that the owner reads, records, and executes the authorization. The file size 304 is listed zero. The date / time stamp 305 of the file is listed at 17:12 May 27. The user and group identifier 306 is listed as identifier 0. The check sum of the file 307 is zero.

In the second entry (ENTRY 2), the file of the file name 308 is agent.reg. Since ENTRY 2 is between the aaa directory entry (ENTRY 1) and the END statement (ENTRY 5), this file is in the directory aaaa. The file (agent.reg) is of normal type 301 and is added as a new file as shown by operation element 302. The authorization 303 indicates whether the owner reads and writes the authorization. The file size 304 is 569 bytes. The date 305 of the file is May 11, 1998. The user and group identifier 306 are both zero. The check total 307 is 3564886220.

The fifth entry ENTRY 5 is a type 301 END marker and indicates the end of the listing for the directory aaaa. The sixth entry ENTRY 6 and the directory bbb are not auxiliary directories of aaa because they are a list after the aaa END marker.

Referring again to FIG. 3, in one embodiment, the content distributor generates a manifest by determining changes using the method of FIG. 4. In other instances, the manifest is sent to a content distributor. In one embodiment, the manifest is delivered by storing the manifest at a specific location (ie, directory and file name), the content distributor being distributed to find it. In other embodiments, the manifest is delivered by transmitting to content distributor 125 via a communication network.

After identifying the change (step 181), for example, using the entity of FIG. 3, the changes are communicated to the agent (step 182). Sometimes, these changes are mapped to specific directories on the host file system. In one embodiment, the source file is mapped to the file systems of the various hosts shown in the example of Table 1.

Example of Raw Directory to Destination Host Directory Mapping Raw host Raw directory Destination host Destination directory stagig.streve.com / usr / netscape / app1 www1.atreve.com / usr / netscape / app1 www2.atreve.com / dos / current / app1 www3.atreve.com / usr / netscape / docs / a1

As can be seen in Table 1, the system administrator can map files of the raw file set from the stacking server file system to the file system on the various web servers. The use of actual file system identifiers other than URLs allows mapping of shared libraries and other files that are not equal to 'URL'. For example, a CGI script cannot be mapped when using a URL, because the files This is because it is not normally available from the web server via a web page request and does not have a URL In the case of using URLs, only files accessible from the web server through the web page request can be accessed.

In one embodiment, the files of the raw file set directory hierarchy are combined into one storage file, and the storage file is compressed. Any file larger than the deployable file is divided into small blocks for more efficient communication across the network to the host system, while reducing impact on the network. Dividing the files into small blocks reduces the amount of retransmissions in case of timeout. An example of such a dispositionable value is 1 megabite.

A change list (load list) containing the file size of the change file is sent to each agent receiving the content. The agent determines whether there is sufficient disk space for the update based on the file size of the change list and the calculations that the agent needs to execute. For example, the agent may determine whether there is enough space in the file system to store two copies of the compressed and decompressed data of the content update, namely, a temporary copy and a stored copy. In this way, the agent may stop transmitting before the host is overloaded with the new content.

When a predetermined number of servers receive the updated content, the data is sent from the content distributor to the agent. If too many agents are unable to receive the content, the content distributor may abort the task. The proportion of agents requiring delivery is deployable. For example, the system operator can determine that 50% of the agents are not sent if they cannot accept the change. In this way, it is possible to prevent having too few servers with updated content.

In one embodiment, the delivery occurs using standard TCP / IP delivery, such as a file delivery protocol ('FTP'), etc. In other embodiments, delivery is done using a reliable multicast protocol. The block is resent If the delivery fails repeatedly, the delivery is stopped and various actions are taken to change the system operator Again, the content distributor has fewer agents than expected because of the unpredictable circumstances. If delivered, the change installation can be aborted, for example, if many agents lose their network connection, the number of agents that can provide new content may be insufficient.

Referring to FIG. 1, if an update for a particular agent 106 fails, the web service system 90 will attempt to "peripheral path" to the affected portion of the system. In an embodiment where the content distributor is separated from the manager, the content distributor 125 notifies the manager 110 that the agent 106 is bad. The manager 110 communicates with the traffic manager 120 such that the traffic manager does not forward the web page request to the web server 102 in communication with the agent 106. The manager 110 will point traffic away from the web server 102 on another server.

In one embodiment, the content distributor 125 will generate a "catch up" update package that is the target of the rogue destination agent. The catch up update package contains changes that are reached but not reached by the agent, and initially, this is a list of changes that were first updated due to a transmission failure. This catch up package can increase if subsequent updates reach the same agent. When the update package is bad, the agent will not receive another update until the "catch up" update ends. This is because propagation initially depends on new updates of the failed changes.

Referring to FIG. 3, when the changes are sent to the agent 182, the changes are installed (step 183). The agent 106 and web server interface 104 cooperate with the content installation. When the new content is received, the agent reassembles the update package and then decompresses it to place the data in a temporary data store. The agent waits for a signal provided from the content distributor to begin copying files to the server content directory. When the signals are received, a change is made to the server content directory. Also, the list files in the directory are removed at this time. If the update is successful, the agent informs the content distributor that continuously updates the file. In one embodiment, the old files are first renamed and stored in an alternate directory. This quickly reverses the change, but results in a transaction that requires a lot of data storage. If for some reason the replication fails, the agent will change the content distributor in question. In addition, either the agent or the content distributor will change the manager instructing the traffic manager to avoid the failed agent. In one embodiment, the agent maintains a version identifier that indicates a constant state of the web server. In one embodiment, the version identifier is an integer value.

In one embodiment, the web server is suspended or stopped during the update process. When the files are copied to the web server content directory, the web server prevents the web page request from being processed so that the request does not receive certain content. This is performed by the administrator instructing the traffic manager that the web server has not requested. At the same time, requests arriving at the web server 102 may be re-issued by the web service interface from the web server 102 to the traffic manager 120 or to the other web server 102. In the middle of a transaction the user completes the transaction before being instructed again for a given amount of time. When the request is flushed to the web server, the update will occur. If the application binary file does not change, the web server then flushes the requester and then the requesters instruct the server when content delivery ends. However, if the application binary files are modified, the web server can be restarted.

In some cases, there is no need to disable the web server to update the content. The process of temporarily instructing the browser again is time consuming and the time combined with the time required for file delivery and web server update is unacceptably large. In order to maximize server availability, the content may be updated while the web server is consistently responding to the request. This can be done by duplicating the file to be modified and duplicating the file in a location accessible to the web server. The agent instructs the web server interface to intercept all requests for files to change. Requests for files that do not change may be directed to the general content area. Requests for change files are directed to a stable copy, while the new files are installed in the general area. After the agent finishes modifying the file, the web server interface signals to use the normal area again. Users involved in a transaction that includes the associated web page at the time of conversion from the old content to the new content need to access the old content continuously for several hours after the conversion until the transaction ends. For example, a user purchasing at an initial price should not end a transaction with a web page that uses a different updated price.

In one embodiment, each transition is assigned to a version identifier in order to confirm the flawless status of the transaction. In one embodiment, the version identifier, called the checkpoint identifier, is an integer that changes by incrementing with each content update. The web server interface intercepts a web page request based on the version expected by the browser and provides the version to the content. To do this, the web server needs to know which version the browser wants. In one embodiment, this information is included in the URL. However, this has the disadvantage of making the URLs and more complex structures. Also, the version change is less obvious to the user.

In one embodiment, the web server has a cookie in the browser that includes a version identifier that specifies in the browser the current version of the web page provided by the server. A cookie is a specific text file that a web server sends to a browser that allows the web server to remember something about the browser. Using HTTP on the web, each request for a web page is independent of all other requests. For this reason, without a mechanism such as a cookie, the web server does not know about the page previously sent to the web browser regarding the browser's previous visit. Cookies are a mechanism by which the server stores its own information about the browser on the browser's own computer and accesses that information when the user makes a web page request. Cookies may be provided during the extinction period, meaning that they are discarded by the browser after time.

In one embodiment, one example of the cookie is executed with the code "Set Cookie; atreveBCD = 37; expries = Thursday, 20-January-1999 12; 32: 34 GMT; path = /". This code is sent from the web server to the browser. The browser includes a line "Cookie: AtreveBCD = 37" that all requests route through 12:32:34 GMT on January 20, 1999 when communicating with a web server in the same area. The "Set-Cookie" and "Cookie" parts of the protocol are headers according to the HTTP specification. "AtreveBCD" is the name of the cookie. Another name is used, and AtreveBCD is bad. "37" is the value of the cookie that the server currently has a checkpoint, and will continue to provide until its expiration time (due to the web server interface). The code "Expires" sets the expiration time. As an example, the extinction time is January 20, 1999. The "Path" code is used to limit the page providing the cookie, but the "/" indicates to cover all files.

The cookie that the web server generates in the browser is valid for the browser to request the content from the web server efficiently and indicate a version identifier for the content. Thus, if the browser makes six requests with a specific cookie specifying a particular URL, even if the agent delivers a new version of the content to the Ube server 102 for a while, the six requests will be constant on their own. .

In one embodiment, the initial content is not discarded until all cookies that specify the content are discarded. The content is stored in a "backup area" containing a change from one version to the next. For example, files that change in a transaction from version 2 to version 3 will be stored in the version 2-3 backup area. The web server interface will direct the web page request for version 2 to be changed in a transaction from version 2 to version 3 to the desired content in the appropriate backup area.

Referring to FIG. 8, as a simple example, a web server has content with a current version identifier 34. There are still versions (33 and 34) of previous versions of cookies that are still valid. The current version, i.e., the version 34, includes the files A and B modified in the version 34, the file C modified in the version 28, and the file D modified in the version 25. Include. File A is modified with all versions shown in the figure, including version 32, version 33 and version 34. File B is modified in version 30, version 32, and version 34 described above. Changes from version 31 to version 32 include changes to files A and B and deletion of file E created in version 1. Therefore, in the 31-32 backup area including the file changed in the transition from version 31 to version 32, the version 31 of the file A, the version 30 of the file (B), and the file Version (1) of (E) is stored. The change from the version 32 to the version 33 includes the modification of the file A, and in the 32-33 backup area, the version 32 of the file A is stored. Changes from version 33 to version 34 include modification of file A and deletion of file F (added to version 10), as a result of which files ( The version 33 of A), the version 32 of the file B and the version 10 of the file F are stored.

When the web server requests the content, for example, by changing a file between the old check point and the current check point by indicating a cookie indicating a content version other than the current version, the web server interface is connected to the web server. Instructed to have content from an area other than the current version area. In one embodiment, the web server interface reviews the change list to determine whether the request file has changed and the backup area in which it is located has changed. For example, referring to the example of FIG. 8 in which the current version is version 34, when the web server receives a request for file A, and when the browser instructs to find version 32, The web server interface will know that file A is modified as part of version 33. Thus, version 32 of file A is located at a 32-33 backup location. When the web server receives a file (B) request from the same browser (as the same version 32 cookie), the web server interface is said to have changed the version of file (B) and file (B) to version 34. 34 You will decide what is stored in the backup area. When the web server receives a request for file D, the web server interface will determine that the file has not been modified since version 32, and that the file can be obtained from the current version area.

In one embodiment, the content distributor provides a change update of the planned set of source files at a specific time or time interval. The content distributor may also be updated upon manual instruction by the system operator. In one embodiment, the update is a content mapping indicating where files are to be copied (i.e., a set of raw files), which should be copied (i.e., the file system of the file server), and initiating / The date time or time interval, whether only the change file should be updated (as described above) or whether all files should be copied, the failed action, whether the server should restart the content update, and the content Whether the server should be stopped or stopped during an update, whether the update should be enabled or disabled (where the system operator can disable the planned update), and which server is required to accept the update before the change command. It is specified by the ratio of.

For example, it can update every 10 minutes, update the changed file, and notify the system operator of the specific problem. You can also specify an update once a month. In one embodiment, the system turns off servers while copying the files, updates all change files, and reboots the server when its content is updated.

9, some web service systems 90 have one or more associated caching servers 200. Caching server 200 is a type of web server located between requester 201 (eg, a browser) and web service system 90. The caching server 200 initially lacks specific content. Rather, it requests a web page from the requester 201 and stores the web page in the cache. An example of a cache server purchase is NETCACHE, a product of Network Appliance of Santa Clara, California. The caching server 200 receives a web page request from a browser for content located in the web service system 90. Based on the request and its placement, the caching server 200 requests a web page from an initial web server (web service system 90) having the web page. The caching server 200 receives a web page from the original web server 90 and transmits the web page to the browser 201. The casing server 200 stores a web page for a predetermined time. The time at which the web page is stored is a time period determined by the content provider (original web service system 90) or the caching server 200 system administrator by specifying an extinction period for delivering a document according to the HTTP protocol. If a second request from the same requestor 201 or another requester arrives at the caching server for a web page within an expiration period, the caching server does not request the content from the original server and responds to the request without the web page. You can send it again. Thus, the request will be filled without making a request on the original server or network of the second request.

When files change on the web service system 90, as noted, files cached in the caching server 200 cannot be destroyed. In that case, the caching server 200 continues to provide the initial version of the web page rather than the updated version available to the web service system 90. Therefore, it is useful for notifying the caching server 200 that a file is changed by using a change list such as the above-described list.

Referring to FIG. 10, in a system where file changes are coordinated with caching server 200, files are changed (step 180), changes are identified (step 181), and changes are sent to the web server ( Step 182), the changes are installed (step 183). The caching server is further notified of the change (step 184). An additional step of notifying the caching server (step 184) allows the caching server to ignore (flush) useless web pages. There are many caching servers or many caching servers and can be notified through various communication techniques. For example, in one embodiment, a broadcast message is used. In other embodiments, a separate message is sent to each caching server. In yet another embodiment, the file is stored in a directory available to one or more caching servers.

Referring to FIG. 11, the embodiment of the method of FIG. 6 makes a change (step 180), identifies the change (step 181), and notifies the cache (step 184). In one embodiment, the notification (step 184) converts the change list into a URL to identify content (step 186). The browser uses the URLs to identify the content. As mentioned above, in one embodiment, the change list is described as a directory and file in the host file system, because it is more convenient for the web server and can point to all files in the best file system. . When providing changes to the caching server, it is useful for converting a list of files, a change list (and a directory), into a format that contains a URL. The source directory may correspond to a specific prefix of possible URLs. For example, the content directory and file "C: \ WEBSPECTIVE_HOME_ABOUT.HTML" correspond to the URL "http://web.webspective.com/home/about.html". This implies that the prefix must be modified (in this case, C: \ WEBSPECTIVE is replaced with http: //www.webspe ctive.com), and the directory splitter, in this case, the backslash "\" The URL standard forward slash is replaced with "/". In addition, characters not allowed in URLs such as spaces are encoded according to specified rules. Examples of such character substitutions are described in the document IETF Network Working Group RFC-1783 document entitled "Uniform Resorce Locators" vy T. Berners-Lee et al. For example, the space can be encoded as "% 20", and this translation is linked to the URL prefix.

When converting the list item to URL format, the list of modified URLs is sent to the caching server (step 187). The caching server uses this list to discard old content. As shown in Fig. 7, there is no need to propagate the content for the notification. For example, even with a web server and many caches, the system of FIG. 7 will inform the one or more caching servers of the change. As shown in FIG. 7, the change is made (step 180), the change is identified (step 181), and the change list is converted to URLs (step 186). Such changes are communicated to the caching server (step 187). In other embodiments, the translation to URLs (step 186) occurs between the time when the change occurs (step 180) and the change identification step 181. After the change (step 181), the deployable script of the user is executed (step 280). After identifying the changes (step 181). A user deployable script is executed (step 281). The script is executed concurrently with the transmission of the changes (step 182). After sending the changes (step 182), the user deployable script is executed again (step 282). User deployable scripts are used to perform content distribution tasks that specify the customer's performance. The user deployable script is executed via a call appropriate to the operating system function.

Referring to FIG. 13, in one embodiment, information is provided to the call script. In one embodiment, the information is provided to the script on standard input. In other embodiments, the information is provided in a file. The script returns to either a success (zero) status code or a nonzero error code, and in one embodiment will discard the operation.

In one embodiment, the information 400 is provided in the next format that initiates the BEGINPARAMS statement 401 and ends the ENDPARAMS statement 420. The information 400 includes a version of the information block verno 402 which is used interchangeably. The information 400 includes the name of the job 403, which is, for example, EXAMPLEJOB.

The information 400 includes the status of the job 404, which can be one of BEGIN, IN PROGRESS and END. The BEGIN value indicates that it is called a script before identifying the change (step 280 of FIG. 11). The IN PROGESS value indicates that after identifying the change, it is called into the script before sending the change (step 281 in FIG. 11). The END value indicates the end of the transmission (step 282 of FIG. 11). The information 400 includes the name of the mapping 405, which in this case is ALLIMAGES. The information 400 includes the path name of the manifest file 406, such as /TMP/STAGING/23423.TXT. As an example, the manifest is a text file. This implies that it is the text entity described above. In one embodiment, the manifest is stored as a binary entity that is converted to a text file before calling the script.

The information 400 can include a force file set directory 406 and a return directory 408 to maintain the file version of the renunciation (before the change) or include content that should be automatically returned. The destination 409 array is indicated. Each array element contains information about the file designation. In this example, there are two destination hosts, OOLONG and DARJEELING. The first element of the array begins with a DEGINELEMENT statement 410 and ends with an ENDELEMENT statement 414. The array element contains the name 411 of the destination host, in this case OOLONG. Status The array element contains a destination directory 412 on the destination host, such as / HOME / HTTPD-OOLONG / DOC / IMAGES. The array element includes a delivery state 413, an example of which is MORMAL. The second element of the array begins with a BEGINELEMENT statement 415 and ends with an ENDELEMENT statement 419. The second array element contains the name 416 of the destination host, in this case DARJEELING. The array element includes a destination directory 417 on the destination host, such as / HOME / HTTPD-DARJEELING / DOC / IMAGES. The second array element includes the transfer 418 state, an example of which is NORMAL. Thus, with the information provided to the user script, the same script can be called at different times during task execution, and the script can take different actions depending on the provided variables.

Various modifications, changes and implementations may be made by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, the invention is not to be limited by the spirit and scope of the foregoing description and the appended claims.

Claims (25)

In the method of replicating changes in the source file set of the destination file system, (a) identifying changes in the source file set, (b) storing the identified changes in a modification list; and (c) transferring the modification list to an agent accessing the target file system. 2. The method of claim 1, further comprising: (d) transferring the modified file to an agent accessing the target file system. 3. The method of claim 2, further comprising: (e) installing the changed files in the destination file system. The method of claim 1, wherein the identifying step comprises: examining a set of files; Comparing the file set with an initially recorded set. The method of claim 1, wherein the identifying comprises: installing a device driver to execute a file operation; And recording, by the device driver, the changes to the source file set. 2. The method of claim 1, wherein the step of identifying comprises receiving a manifest listing describing changes in the source file set. 5. The method of claim 4, wherein the comparing step comprises comparing a file attribute with a file attribute of the initial record set. 8. The system of claim 7, wherein the file attributes comprise at least one attribute selected from a file size, file authorization, file ownership, modification time, and the set of file hashes. How to. 2. The method of claim 1, further comprising invoking a script prior to said identifying step. 2. The method of claim 1, further comprising invoking a script prior to said transferring step. 2. The method of claim 1, further comprising invoking a script after the transferring step. 12. The method of claim 11, further comprising determining whether the transfer step has been successfully completed, wherein the invoking step occurs after the determining step. . 12. The method of claim 11, wherein invoking the script occurs after knowing that the transfer is successful. In the method of replicating changes in the source file set of the destination file system, (a) identifying changes in the source file set, (b) storing the identified changes in a modification list; (c) transferring the modification list to a plurality of web servers. 15. The method of claim 14, wherein the transferring step comprises a multicasting step. 15. The method of claim 14, wherein the step of identifying comprises: examining the file set; And comparing the file set with the initial record set. 17. The method of claim 16 wherein the comparing step includes comparing file attributes and file attributes of the initial record set. 18. The method of claim 17, wherein the file attribute is a file size, file. And at least one attribute selected from the file ownership, modification time, and the set of file hashes. The administrator who manages the web service system, A host including a web server in communication with the manager to receive web page requests and agents; A content distributor for providing content changes to the host. 20. The web service system of claim 19, further comprising a traffic manager for directing a web page request to the web server. The method of claim 19, wherein the content distributor, (a) an identification module that identifies changes in the source file set, (b) a modification list that stores the identified changes; and (c) a transmitter for transmitting the modification list to an agent accessing the destination file system. 22. The web service system of claim 21, further comprising a transmitter for transmitting the modified file to the agent. 22. The web service system of claim 21, wherein the agent includes an installer that installs the modified files on the destination file system. (a) an identification module that identifies changes in the source file set, (b) a modification list that stores the identified changes; and (c) a transmitter for transmitting the modification list to an agent accessing the destination file system. 25. The content distributor of claim 24 including a transmitter for transmitting the modified file to the agent.