WO2005120102A1

WO2005120102A1 - Dynamic connection structure topologies and methods for facilitating the peer-to-peer transfer of digital files

Info

Publication number: WO2005120102A1
Application number: PCT/US2004/034101
Authority: WO
Inventors: Vincent Addessi; Jamie Addessi; Chris Bick; Ken Hausam; Kirk Feathers; Gregory Kerber
Original assignee: Wurld Media, Inc.
Priority date: 2004-05-19
Filing date: 2004-10-18
Publication date: 2005-12-15
Also published as: WO2005119477A3; WO2005119477A2; EP1752002A1; WO2005119476A3; WO2005119476A2; EP1776643A2

Abstract

Dynamic Connection Structures (DCS) utilizing many topologies and routing means and methods for transferring data files to pluralities of computers or nodes on a network are provided. Many types of meshes, graphs, rings, and ordered binary tree algorithms are provided which are configured to adapt and change their topologies and routing schemes in peer-to-peer architectures, depending upon a number of significant attributes of the nodes, including their relative positions, capacities, content status, and their respective relationships to other nodes in the DCS. At least one traffic cop node is arranged and enabled for overseeing, adapting and maintaining the topology of the DCS. Advantageously, nodes of the invention are adapted for simultaneously sending and receiving chunks of a file set transmitted over the DCS. Additions and removals of nodes can be achieved at any time by contacting affected nodes, and modifying their connections to reflect the changes determined by the algorithm. Thus, nodes of the invention are adapted for efficiently changing their topologies and for routing content between and among the nodes of the DCS.

Description

DYNAMIC CONNECTION STRUCTURE TOPOLOGIES AND METHODS FOR FACILITATING THE PEER-TO-PEER TRANSFER OF DIGITAL FILES

RELATED APPLICATIONS

[0001] The present application is a Continuation-In-Part of U.S. Appl. Serial No.

09,742,661, filed December 21, 2000. The present application also claims the benefit of

U.S. Provisional Patent Application No. 60/511,665, filed October 17, 2003, and of U.S.

Provisional Patent Application No. 60/572,229, filed May 19, 2004.

FIELD OF THE INVENTION

[0002] The present invention relates to the creation and maintenance of a dynamic centrally organized network topology and the use of that topology in conjunction with a routing scheme for multi-media file distribution via a multi-media protocol for transferring files, or Multi-Media Transfer Protocol (MMTP). The invention organizes nodes into one or more types of Dynamic Connection Structure (DCS) such that the available bandwidth for each node, both upstream as well as downstream, is utilized to the fullest extent necessary in order to maximize the efficiency of file transfers. Moreover, enabled nodes of the invention are adapted and configured so that they can both receive and send portions, or chunks, of a file virtually simultaneously.

BACKGROUND OF THE INVENTION [0003] It is common for many computers distributed across a network to each require a copy of the same file. For example, many people connected to the internet may all need a copy of an available program, document, or music file. A file server can only send a file to a limited number of clients simultaneously, and the speed of the file transfer decreases as new clients are added. Because of these limitations, multiple servers are frequently necessary in order to efficiently send a file to a large number of clients. [0004] The most widely used method of transmitting files across a network is File

Transfer Protocol (FTP). FTP allows a server to transmit a file to a client by breaking the file into packets and sending each packet independently. For an FTP server to send a file to multiple clients, the server is required to separately send the same packet to each client. This is disadvantageous because it requires work repetitions and therefore causes time delays.

[0005] Recent art makes use of peer-to-peer technology, in which a client receives a file and then sends it on to other clients. Use of this technology decreases the load, on the original file server by utilizing the processing and transmitting capabilities of the clients. Unfortunately, conventional embodiments of such conventional technology require the entire file to be transmitted to a client before that file can be retransmitted to other clients, thereby producing undesirable delays. [0006] File sharing programs such as Napster and Gnutella use peer-to-peer technology to transmit files between clients. In these systems, clients search for available files located on other client computers. A client is allowed to transfer a desired file via FTP from another client. These systems do not make provisions to transfer one file efficiently to a large number of clients. Instead, they merely give a client a wide selection of other clients from which a file may be received, to thereby increase the probability that an available source is found.

[0007] Several recent attempts to increase the efficiency of file transfer over networks have concentrated on determining one or more efficient routes between clients. Some such attempts have used algorithms including spanning tree algorithms to determine the optimal route over a set of potentially active connections. See U.S. Patent No. 4,905,233, U.S. Patent No. 6,098,107, and U.S. Patent No. 6,044,075. However, such art has not provided a means of creating and removing connections between clients as data packet transmissions take place over said connections. Nor has such art yielded a method by which clients in a peer-to-peer communication network are able to retransmit a file before that file is first received in full.

[0008] One method of transferring files by cascaded release describes an application of peer-to-peer technology that allows a file to be transmitted from a server to a number of clients. See U.S. Patent No. 5,995,982. This method allows a group of clients to receive a file from the server and then send the file on to other clients. This method arranges a list of clients into groups, and does not allow a client of the group to begin transmission until a client in the previous group has fully received the file. The organization of connections between clients in the list remains static once the list is created. Thus, this method does not make provisions for dynamically redefining the distribution list by adding, removing, and repositioning clients while files are being transmitted between clients in the list. In addition, for a client to retransmit a file after receiving it, the client computer is required to remain connected to the network after it has received the file, a condition which is often not desirable. No existing technology allows clients to retransmit packets of a file to other clients contemporaneously as packets of the file are being received. [0009] U.S . Patent Publication 2003/0115823 "Content Request Routing Method" by Barbir et al. describes a method of redirecting content requests among content distribution network peers. However, with Barbir et al., the entire request must be redirected and the entire file must be transferred. [0010] U.S. Patent Publication Number 2002/0147815 "Distributed Network Data Storage System and Method" to Tormasov et al. describes a highly scalable wide area peer-to-peer network data storage system and method. However, the server connected to a client collects/ requests file pieces so that it appears to the client that the file came from a single source. No topology or routing is described. Files, whose content can be modified, are distributed as opposed to digital content making this system a data storage and retrieval system.

[0011] U.S. Patent Publication Number 2003/0139934 "Requesting and Providing Services via a Registry" to Mandera describes a method for transferring files in a peer-to- peer network wherein requests are split up into sub-requests and a registry server is used to find peers that can fulfill the sub-requests. However, no firewall provisions are described and the responding peer is responsible for further forwarding further sub- requests.

[0012] U.S . Patent Publication Number 2002/0007350 "System and Method for On- Demand Data Distribution in a P2P system" by Yen describes an encrypted peer-to-peer on-demand file distribution and payment system wherein an index of distributable data files stored at peers is used to inform other peers of the location of requested files. However, the responding peer must have the entire file.

[0013] U.S. Patent Publication Number 2003/0125964 "System and Method for Controlling Distribution of Digital Copyrighted Material using a Multi-Level Marketing Model" by Chang et al. describes a method for distributing files in a peer-to peer network wherein a copyright owner distributes to level 1 consumers and level 1 consumers are permitted to distribute the files further to level 2 consumers for a reward. Level 2 consumers are further permitted to distribute the file to level 3 consumers for a reward etc. However, the upstream bandwidth is unused. [0014] U.S. Patent Publication Number 2002/178255 "Incentive Program for a Peer-to- Peer network" by Hobart describes an incentive program for a peer-to-peer network wherein incentives are offered to peers for actively or passively contributing to network resources. However, no routing for file distribution is described nor is there any mechanism to prevent the pirating of intellectual property. No distribution of digital content is described and multiple connections with multiple peers are required to stay connected. '

[0015] U.S. Patent Publication Number 2003/0149620 "System for Offering Services Using Network of Unowned Computers" by Gaither describes the use of computers by vendors of computers after they have been sold in order to advantageously use their processing power. However, in Gaither, unlike the present invention, an entire file must be distributed from one computer. Moreover, no firewall protection is provided by Gaither et al., and there is no attempt to prevent pirating of intellectual property. [0016] U.S. Patent Publication Number 2002/0169700 "Digital Content Subscription Conditioning System" by Huffman et al. describes distribution services in a peer-to-peer system in which individual users store content on their local computers making the content available to other users for free or for a fee. However, a Digital Service Provider (DSP), which is a central authority, is required to host the digital content. [0017] U.S. Patent Publication Number 2002/0138362 "Digital File Marketplace" by Kitze et al. describes a financial model for the digital file marketplace which includes generating revenue from the delivery of files and paying royalties to content owners from the revenue collected. However, no topology is described and no routing is described. While there is a decrease in unauthorized copies of the digital files in Kitze, there is no guarantee or attempt to guarantee that transmitted content constitutes an authorized copy. Thus, the copyright owner must be proactive. [0018] U.S. Patent 5,884,031 "Method for Connecting Client Systems into a Broadcast network" by Ice describes a private network built by allowing a predetermined number of clients to connect directly to a server. After this occurs, additional client systems requesting connection are furnished with the addresses of client systems already connected within the private network. Each of the additional client systems then makes connections with a multiple number of client systems to receive information from the server system. Each of these client systems subsequently accepts connections from up to a second pre-determined number of client systems to which it transmits information received from the server system. [0019] While the private network described by Ice does connect nodes in an organized structure that attempts to minimize resource requirements on the server, there are many differences in Ice when contrasted to the present invention. For example, the present invention, describes a dynamic connection structure (DCS) in which nodes connect and disconnect dynamically. The DCS of the present invention is highly intelligent and considers geography, firewalls, subnets, speed (upstream and downstream), bandwidth etc. in situating a node connecting to the network as well as in DCS (topology) reorganization/maintenance. The present invention also dynamically re-organizes the topology of the DCS. The present invention packetizes files and transfers those files as packets, and retransmits those packets virtually simultaneously, and as necessary, among and between the nodes of the DCS. As an additional advantage, systems of the present invention perform data routing and message routing between nodes. The present invention simultaneously routes files, data and messages between nodes of the DCS. In organizing and re-organizing (maintaining) the DCS, a special node communicates node instructions regarding connections to affected nodes. [0020] Additionally, Ice describes a redundant tree or pyramid architecture in contrast to the binary heaps and meshes of the present invention. In the present invention, any node can be a server/seed node/root server, that is, any node of the present invention can distribute content. This is in stark contrast to the single server described by Ice. Moreover, the server described by Ice performs both network organization and routing functions with routing of data accomplished via broadcast in contrast to the routing of packetized files between nodes of the present invention.

[0021] U.S. Patent 6,584,073 "Network Topologies" by Steele et al. describes network topologies selected to improve network performance including adding nodes and removing nodes from the network. In contrast, the mesh of the present invention is not organized (or re-organized) in the same manner as the network topologies described by Steele. The meshes of the present invention are organized using criteria analysis algorithms. When determining the topology of node connections in a DCS, the traffic cop node of the present invention looks at a large number of nodes and selects those that fit the criteria best. [0022] U.S. Patent Application Publication 2003/0051051 entitled "System for Distributing Content over a Computer Network and Method of Arranging Nodes for

Distribution of Data Over a Computer Network" by O'Neal et al., describes a tree or heap type topology through which files are propagated. The present invention propagates packets or chunks of files. In contrast, the nodes of O'Neal propagate data/files linearly from start to finish. The server described by O'Neal does not dictate the structure of the network but rather passes addresses of nodes within the network. Then, the nodes described by O'Neal manage the structure themselves (decentralized) by sending messages around. For example, if a node determined that it was not connected in an optimal way, it would send messages to its neighbors and facilitate connection changes itself to try to improve its situation in contrast to the centralized organization and re- organization of the topology of the present invention.

[0023] BitTorrent is a system in which the publisher of a file may effect peer-to-peer transfer. In the BitTorrent system, a tracker helps peers find each other in order to obtain fixed size pieces of an available file by requesting the same pieces of files from multiple peers. BitTorrent supports no central resource allocation. [0024] There is thus a need in the field to mitigate the time bottleneck associated with the transfer of a file to a number of other computers (or nodes) and the limitations in the number of simultaneous file transfers possible with conventional systems. There is also a need to provide clients in a network with the means to dynamically change their connections with other clients at any time. There is a similar need to provide a client in a network with the ability to retransmit packets to other clients virtually simultaneously as those packets are being received. Ideally, such an invention should ensure that no client receives packets from another client with a slower network connection speed. [0025] Until the present invention, no system or methods available address the use of a dynamic topology or the use of a routing scheme as provided herein. Similarly, none of the currently available resources sufficiently address the efficient use of bandwidth and other system software and hardware components.

SUMMARY OF THE INVENTION [0026] The invention provides a method for transferring a file from one computer to many computers distributed across a network. A program called a server is executed on a computer connected to the network. A program called the root client is executed on a computer connected to the network which has direct access to the file which is to be transferred. A program called the client is executed on each computer which requires receipt of the specified file.

[0027] The MMTP is an application layer protocol as is file transfer protocol (FTP). MMTP is a combination of the DCS (topology) and routing scheme described herein. The combination of topology and routing address the problem of copyright infringement as well as better utilizing both unutilized and under-utilized upstream and downstream bandwidth of nodes connected, for example, to the Internet or cable. The present invention is adaptable for use in related applications such as the re-engineering of business processes related to the distribution of audio, video, streaming media, interactive chat, interactive games, etc. Moreover, the present invention provides the enabling technology, which concurrently addresses the issues of efficient use of bandwidth and system resources and further provides for delivery of multiple files over the same DCS. [0028] Topologies of the present invention will be illustrated by describing them herein in terms of two exemplary embodiments of a DCS, in particular, a binary heap and a mesh. A DCS is a peer-to-peer network of nodes in communication with each other and with a traffic cop node, which organizes and maintains (re-organizes) the topology of the network. One important aspect of the present topology systems is a data structure containing connection change information, which is described in greater detail hereinbelow with respect to the packet chain portion of the multimedia transport protocol (MMTP). [0029] In the routing portion of the MMTP, multi-media files are broken into chunks. There are several alternative schemes by which chunks are requested from peers. For example, rarest first, randomly, and most needed first. The present invention will be described in terms of using the most needed first approach. The use of a hashing algorithm insures the integrity of the data. Routing employs Access Maps, which create an intelligent awareness among peer nodes of the proximity of each chunk of the requested multi media file, and request maps, which guide the data flow to maximize the efficiency of each chunk transmission.

[0030] There are measures and guarantees built into the present invention regarding quality of service (QoS). Levels of service are determined, and the minimum amount of data is requested from the root server or seed node with the bulk of the data (chunks) requested from peer nodes participating in the DCS. Accordingly, it is an object of the invention to provide means and methods for efficiently transferring digital files in peer- to-peer networks, such as the Internet. It is a further object of the invention to provide dynamic connection structure topologies, routing means and methods, and nodes enabled for simultaneously receiving and sending portions of one or more digital files. It is a further object of the invention to provide managerial "traffic cop" nodes that are enabled to manage a DCS of the invention by being connected to that DCS only on an as needed bases. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0031] In reply to: reply to the following detailed description, specific embodiments of the invention are shown by way of illustration. These embodiments described in sufficient detail to enable those skilled in the art to make and to use the invention, and it is to be understood that many other embodiments may be specified that fall within the scope of the present invention. The following detailed description should not be construed to limit the scope of the invention as defined by the appended claims. [0032] In accordance with the above-stated and other objects, the present invention provides a series of dynamic connection structures which are configured to adapt and change their topologies depending upon a number of significant attributes of the relative positions, capacities and content statuses of individual nodes, and their respective relationships to other nodes in the DCS. In one preferred embodiment, a dynamic connection structure (DCS) of the invention comprises a plurality of nodes, wherein the plurality comprises at least one first node, the first node being connected to at least one second node; and a traffic cop node for overseeing, adapting and maintaining the topology of the DCS, wherein the DCS forms a peer-to-peer architecture. The topology of a DCS of the invention refers to the nodes and the connections and interconnections between or among any two or more nodes of a DCS, or between any node and the nodes to which it is connected.

[0033] In one aspect, each of the nodes is enabled to perform the functions of self- insertion, that is, inserting itself into an existing DCS in accordance with instructions received from the traffic cop node; and of reorganizing itself positionally in the DCS with respect to other nodes of the plurality, wherein the reorganization is in accordance with instructions received by that node from the traffic cop node. In another aspect, each of the nodes is enabled to perform the self-insertion and the reorganization of the DCS simultaneously. Advantageous, each of the nodes of a DCS of the invention is enabled to function as both a server and a servant, and in certain respects can further perform these functions simultaneously, and can function to provide content suitable for distribution to one or a plurality of the nodes in the DCS, and which content is distributed by way of the nodes in, over, by way of, or through the DCS. Thus, with respect to any file or piece of content, or any group of files or group of content, for example an audio file or a video file, any of the plurality of nodes, such as at least one first node and at least one second node, are enabled to function both as a server to provide that content suitable for distribution to one or a plurality of the other nodes in the DCS, and as a servant to receive that content, or other content, which is distributed by way of the nodes in the DCS.

Moreover, each or any of the plurality of nodes, such as, for example a first node, and at least one second node, are enabled to function both as a server and as a servant with respect to first content and with respect to additional content and both the server functions and the servant functions can be performed simultaneously with respect to the first content and with respect to the additional content.

[0034] In another key aspect of the invention, any of the nodes of the DCS, or any group of nodes of the DCS, are continually organized and reorganized in accordance with instructions conveyed from the traffic cop node. Advantageously, the organization and re-organization is centrally controlled and is a function of the software and information possessed or accessed by the traffic cop node. Information necessary for the organization and the re-organization of the DCS can be provided to the traffic cop node in a number of ways, such as from a centrally maintained database in the traffic cop node itself, or which is stored in one or more of the traffic cop node, and one or more digital media accessible to the traffic cop node such as a traffic cop server or a central server functionally connected to the traffic cop. Thus, information and communications necessary to organize and re-organize the DCS is decentralized and is among the plurality of nodes. Among other advantages of the invention, when the communication with one or more nodes is protected by a firewall, a buddy connection is effected. [0035] Preferably, the traffic cop node processes all connection change information. In order to do so, the present invention is configured and arranged such that the' traffic cop node maintains a connection list of all active transmission connections and, preferably, the connection list is grouped in accordance with one or more universal resource nodes (URN's). Moreover, the traffic cop node maintains a connection list of all active transmission connections, and wherein the connection list is grouped in accordance with one or more universal resource nodes (URN's). In another aspect, connections between a first node and a second node of the DCS can be established simultaneously while connections between the first node and a second node of the DCS are broken. In addition, the DCS can be organized and re-organized simultaneously with the establishment and breaking of the connections. In this way, peer level nodes of the invention are not necessarily always connected to the traffic cop, but can connect, disconnect and reconnect to the traffic cop intermittently as needed to send and receive information regarding a requested or actual change in the status or position of one or more nodes. [0036] Another advantage pertains the aspect where each connection that is established and each connection that is broken uses separate threads in order to effect simultaneity. A dynamic connection structure (DCS) of the invention can be of any topology, for example a binary heap, an incomplete binary heap, a binary tree, an m-ary tree, a graph, a chain, a mesh, a ring or a bus. In a similar aspect, a DCS of the invention can be two or more interconnected systems selected from the above-mentioned list of topologies. DCS systems of the invention are organized and re-organized according to a number of attributes to maximize efficiency by maximizing usage of upstream and downstream bandwidth capacity, for example, network connection speed, firewall status, geographic location, IP sub-net address, location of chunks of content by sub-net address and user account information. As yet another significant aspect, a DCS of the invention may comprise almost any number of nodes, for example at least a third, fourth and fifth node, or at least dozens or hundreds of nodes, for example least three hundred nodes, or at least six hundred nodes or at least one thousand nodes, or more.

[0037] Methods of the invention include those for changing or adapting the topology or topologies to add or delete nodes, to connect nodes to additional nodes, and to disconnect a node from one node and connect that node to another node, and to disconnect one node from one of a group of connected nodes while maintaining its connections to the other nodes of the group. In one aspect, the invention includes a method for processing a connection change in a dynamic connection structure (DCS), the method comprising the steps or actions of adding a new connection to a connection list for a specified universal resource node (URN); validating a timestamp to ensure the connection change is more recent than a last processed connection change; validating a security signature; locating a data object corresponding to the connection change, the data object having a matching local session ID for the specified URN and storing connection change information in the data object; and propagating a connection change message to a connection identified in the data object. The method may further comprise the act or step of storing a null value in the data object if no valid entry is located, and the step or act of initiating a connection to a specified node if the data object is not null, and the local session ID does not match an existing connection. [0038] As another advantage, the act or step of initiating may further include the initiation of an outgoing connection, and the addition of the outgoing connection to the connection list. Also, the step or act of initiating may further comprise initiating a buddy connection if the node is firewall-protected, that is, if the node is behind a software firewall. Further acts or steps of the invention may comprise the acts or steps of removing the connection from the connection list when, for any existing connection for the specified URN, there is no data object, and transmitting a message to the connection indicating connection deletion.

[0039] The present invention also includes methods for creating a connection between a first node and a second node of a dynamic connection structure (DCS). In one preferred embodiment, the method comprises the steps or acts of: sending, by the first node, a connection request to the second node; acknowledging, by the second node, receipt of the connection request from the first node; establishing the connection between the first and the second node; requesting, by the first node, access to a file set and forwarding, by the first node, access information to the second node; validating, by the second node, the access information forwarded by the first node; granting, by the second node to the first node, access to the file set; and forwarding, by the first node, the access information of the first node to all other nodes in the dynamic connection structure that are affected by the creation of the connection between the first and second nodes. As another aspect, the method may further comprise the additional steps or acts of: denying access to the file set if the access information is not validated, and deleting the established connection. [0040] Advantageously, the first node and the second node can each be one of a server (or "seed node") and a servant, and each of the first and the second nodes may simultaneously be a server (or seed node) and a servant with respect to the same or different portions of one or more file sets. Thus, a node of the invention can receive a first chunk, or portion, of a first file set from a neighboring or distant node in the DCS, and simultaneously send the same first chunk, or portion, of that file set to yet another node with which it is directly or indirectly connected in the DCS. Similarly, a node of the invention is enabled to send a portion, or chunk, of a first file set while simultaneously receiving a portion of a second, that is, different, file set. In a similar manner, the functions of receiving and sending can occur simultaneously in a particular node with respect to various related or unrelated file sets traveling across a DCS of the invention. Thus, this simultaneity increases substantially the speed of file transfer between and among nodes of the DCS. [0041] The present methods include also those for deleting one or more nodes from a dynamic connection structure. In one preferred embodiment, the method comprises the acts or steps of receiving, by a traffic cop node, a request from the node to leave the DCS; and forwarding, by the traffic cop node, a connection list to nodes affected by disconnection of the node from the DCS. Thus, disconnection of a particular node in a DCS of the invention includes communications conducted by and through the traffic cop node such that nodes to which the disconnecting node is connected are informed of the disconnection and reorganization.

[0042] Numerous nodes can be enabled in accordance with the present invention to participate in one or more dynamic connection structures. In another preferred embodiment, a DCS of the invention comprises; at least one first node connected to a plurality of second nodes; and a traffic cop node configured and arranged for overseeing the topology of the DCS, wherein the DCS forms a peer-to-peer architecture, and wherein each of the at least one first node and each of the plurality of second nodes are enabled to perform the functions of i) inserting itself into the DCS in accordance with instructions received from the traffic cop node; and ii) reorganizing itself positionally in the DCS in accordance with instructions received from the traffic cop node. Advantageously, each of the nodes of a DCS of the invention is enabled to perform the self-inserting and the reorganizing simultaneously. Thus, a node can be inserted simultaneously as the DCS is being reorganized.

[0043] As in other preferred embodiments, each of the nodes is enabled to perform the functions of self-insertion, that is, the insertion of itself into an existing DCS in accordance with instructions received from the traffic cop node; and of reorganizing itself positionally in the DCS with respect to other nodes of the plurality, wherein the reorganization is in accordance with instructions received by that node from the traffic cop node. In another aspect, each of the nodes is enabled to perform the self-insertion and the reorganization of the DCS simultaneously. Advantageous, each of the nodes of a DCS of the invention is enabled to function as both a server and a servant, and in certain respects can further perform these functions simultaneously, and can function to provide content suitable for distribution to one or a plurality of the nodes in the DCS, and which content is distributed by way of the nodes in, over, by way of, or through the DCS. Thus, with respect to any file or piece of content, or any group of files or group of content, for example an audio file or a video file, any of the plurality of nodes, such as at least one first node and at least one second node, are enabled to function bottυas a server to provide that content suitable for distribution to one or a plurality of the other nodes in the DCS, and as a servant to receive that content, or other content, which is distributed by way of the nodes in the DCS. Moreover, each or any of the plurality of nodes, such as, for example a first node, and at least one second node, are enabled to function both as a server and as a servant with respect to first content and with respect to additional content and both the server functions and the servant functions can be performed simultaneously with respect to the first content and with respect to the additional content. [0044] In accordance with still other objects of the invention, a dynamic connection structure (DCS) of the invention comprises a plurality of nodes, all of the nodes being adapted for efficiently routing content between and among the nodes of the DCS. For example, one embodiment of a DCS of the invention is where at least one first node is connected to at least one second node, and wherein the at least one first node and the at least one second node are adapted for efficiently routing content between and among the at least one first and the at least one second node of the DCS, and wherein the DCS forms a peer-to-peer architecture. In one aspect of the invention, each of the at least one first node and the at least one second node is capable of simultaneously sending and receiving chunks of a file set over the DCS. [0045] In another aspect, each of the at least one first node and the at least one second node creates and maintains an Access Map, the Access Map being useful for, and being used for locating chunks of the file set within the DCS. In another aspect, each of the at least one first node and the at least one second node creates and maintains Neighbor Access Maps, the Neighbor Access Maps being useful for, and being used, for locating chunks of the file set within the DCS. In a similar fashion with respect to locating chunks of the file set, each of the at least one first node and the at least one second node creates and maintains a Request Map, the Request Map being useful for, and being used, to determine which chunks of the file set are needed by each the creating and maintaining node. Preferably, the at least one first node and the at least one second node create and maintain Neighbor Request Maps, the Neighbor Request Maps being useful for, and being used, to determine which chunks of the file set are needed by each of the creating and maintaining nodes, that is, the nodes which create and maintain the Request Maps. In addition to generating its own Access Map and Request Map, a node generates an Access Map and a Request Map for each node (neighbor/subnode/peer) to which it is directly connected. These neighbor/subnode/peer Access Maps and Request Maps are communicated as appropriate to other nodes (neighbors/subnodes/peers) to which the node is connected. This is done in order to propagate access information and requests for chunks of a file farther than just the directly connected nodes. These Access Maps and Request Maps for neighbors/subnode/peers are called Neighbor Access Maps and Neighbor Request Maps. As another advantage with respect to maximizing the efficiencies of transfer of the file set, each of the at least one first node and the at least one second node propagates its Access Map to nodes to which it is directly connected, that is, to its Neighbor Nodes, and each of the at least one first node and the at least one second node propagates its Request Map to nodes to which it is directly connected, that is, to its Neighbor Nodes. [0046] In accordance with other advantageous aspects of the invention, each node which is enabled to function within a DCS of the invention is provided with a plurality of sockets over which chunks of a file set are communicated. Thus, each of the at least one first node and the at least second node has a plurality of sockets over which chunks of a file set are communicated. Even more advantageously, each of the sockets is enabled to operate simultaneously with each other socket, that is, each socket of each of the at least one first node and the at least one second node operates simultaneously with each other socket of each of the at least one first node and the at least one second node. Thus, a node in a DCS of the invention is enabled such that, as messages arrive on one or more sockets of the respective nodes, the messages are added to a message queue, and the messages can include one or a plurality of Access Maps, Request Maps and chunks of a file set forwarded from other nodes. Thus, for example, when messages arrive on one or more sockets of each of the at least one first node and the at least one second node, the messages are added to a message queue, each node having its own message queue, and the messages typically include Access Maps, Request Maps and chunks of a file set forwarded from other nodes.

[0047] In accordance with other aspects of the invention, each node of a DCS is provided with at least one message handling thread which repeatedly removes messages from the message queue and processes those messages. Thus, each node of the at least one first node and the at least one second node has a message handling thread which repeatedly removes messages from the message queue and processes the messages. [0048] Preferably, each Access Map is transmitted as a compressed table of chunk ranges and each Access Map is transmitted as changes to a last Access Map. In a similar fashion, each Request Map preferably is transmitted as a compressed table of chunk ranges and is transmitted as changes to a last Request Map. Furthermore, each node preferably regenerates and transmits its Access Map at a predetermined interval or whenever a change occurs, updated Access Maps are transmitted. To further increase the efficiency of a DCS of the invention, each node regenerates and transmits its Request Map at a predetermined interval, or whenever a change occurs to the Request Map, updated Request Maps are transmitted. [0049] Preferably, an Access Map is maintained in a storage system of each of the at least one first node and of the at least one second node as a table, and a size of the table is equal to a number of chunks in the file set. Preferably, the size of the table is equal to, or directly related to, the number of chunks in the file set. Preferably, a Request Map is maintained in a storage system of each of the at least one first node and of the at least one second node as a table, and a size of the table is equal to a number of chunks in the file set. Preferably, the size of the table is equal to, or directly related to, the number of chunks in the file set. In computing, a storage system is generally defined as any device in or on which information can be kept, where the term kept includes placing the information in or on the device and updating or maintaining such information as required. For purposes of the present invention, a storage system includes anything on or in which data/information/content can be stored. This would include memory and storage means such as hard drives, any of the forms of RAM (SRAM, DRAM, PCRAM, CAM, etc.) as well as any form of removable media such as floppies, magnetic tapes, CDs, flash memory, sticks, etc.

[0050] In a DCS according to the invention, each Access Map is uncompressed by a receiving node upon receipt by that node, wherein the receiving node is one of the at least one first node and the at least one second node to which a transmitting node is directly connected, and wherein the transmitting node is one of the at least one first node and the plurality of second nodes. In a similar fashion, each Request Map is uncompressed by a receiving node upon receipt, wherein the receiving node is one of the at least one first node and the at least one second node to which a transmitting node is directly connected. [0051] In accordance with other aspects, each entry of the Access Map table indicates whether there is access to a particular chunk of the file set and, if there is access to a particular chunk of the file set, the distance in hops to that particular chunk of the file set. In accordance with other aspect, each entry of the Request Map table indicates whether there is a need for a particular chunk of the file set.

[0052] Preferably, updated Access Maps and updated Request Maps are each generated by each node of the at least one first node and the at least one second node at periodic intervals, where the periodic intervals are predetermined intervals. In accordance with other advantages of the invention, methods are provided for increasing and maximizing the efficiency of the transfer of files between and among nodes in a Dynamic Connection Structure. In one embodiment, the invention provides a method for enabling nodes to generate one or more Access Maps. Thus an Access Map generating node can generate an Access Map to be transmitted from that node to a node directly connected to the generating node.

[0053] The method comprises the steps of 1) initializing all entries in the Access Map table, where the entries each represent a chunk of a file set, 2) determining which, if any, chunks of the file set are available on the generating node; 3) re-setting any entries in the Access Map table corresponding to chunks of the file set maintained on the generating node; 4) determining, using Access Maps of other nodes maintained on the generating node, which, if any, chunks of the file set are available on those others nodes, and also the identity of nodes on which the chunks of the file set are available, and a distance from the generating node, and 5) re-setting any entries in the Access Map table to a number corresponding to the distance from the generating node. Preferably, the distance is a number of hops away from the generating node.

[0054] The present methods include also a method for generating a Request Map table to be transmitted from a node generating the Request Map table to a node directly connected to the generating node. Thus a Request Map generating node can generate a Request Map to be transmitted from that node to a node directly connected to the generating node. The method comprises the steps of a) validating all incoming Request Map tables; b) using an Access Map table for the generating node, setting each Request Map table entry to a first predetermined value for each chunk of a file set that is maintained on the generating node; and c) using an Access Map table for the generating node, setting each Request Map table to a second predetermined value for each chunk of a file set that is not maintained on the generating node.

[0055] A method for requesting a chunk of a file set is also provided. One embodiment of the method for requesting a file chunk comprises the steps of i) verifying pending requests for chunks of the file set; ii) determining if any new requests for chunks of the file set overlap pending requests for chunks of the file set; and iii) forwarding an intersection of the pending requests and the new requests if there is an overlap. This method for requesting a chunk of a file set may also include the step of iv) one of forwarding new requests for chunks of the file set to a first node that can be reached in a minimum number of hops and forwarding new requests for chunks of the file set to a second node that has a most needed chunk of the file set. This embodiment includes also the optional step of v) removing requests when one of all chunks of the file set in the request have been received and all nodes contributing to the request have been disconnected. [0056] The present invention includes also methods for transmitting a file set over, across or through a Dynamic Connection Structure (DCS) from one or more nodes to one or more other nodes. This method comprises the steps of A) determining a size and a number of chunks of the file set based on a size of the file set; B) generating an Access Map for a first node of the DCS based on an Access Map forwarded to the first node by at least one second node of the DCS upon establishment of a connection between the first node of the DCS and the at least one second node of the DCS; C) generating a Request Map for the first node of the DCS based on the first node's Access Map; and D) requesting at least one chunk of the file set from the at least one second node of the DCS based on one of requesting a most needed chunk of the file set first and requesting a closest chunk of the file set first. This embodiment may include also the further steps of E) updating the first node's Access Map; F) forwarding the first node's updated Access Map to the at least one second node; G) updating the first node's Request Map; and H) forwarding requests for additional chunks of the file set based on the first node's updated Request Map. [0057] Updates to the various Maps of the invention can be effected in any manner which contributes to the efficiency of the performance of the DCS. For example, updates can be provided to the first node's Access Map and the first nbde's Request Map upon the occurrence of one or more events. Such events include the establishment of a new connection between the first node of the DCS and another node of the DCS such as a connection to an additional second node, the disconnection of one of the second nodes from the DCS, the reception of the requested chunk by the first node of the DCS, the response by the first node to a successful receipt of the chunk of the file set, reception of a requested chunk of the file set, the reception by the first node of the DCS of the updated Request Map from the second node of the DCS, the reception by the first node of the DCS of the updated Access Map from the second node of the DCS. Moreover, updates are made to the first node's Access Map and the first node's Request Map at one or more predetermined intervals.

[0058] The present invention provides also for the validation of Request Maps. Validation can be provided by any means or method which contributes to the operational efficiency of the DCS. For example, one method to validate a Request Map forwarded to a first node of a Dynamic Connection Structure (DCS) by at least one second node of the DCS includes the steps of: setting a request to null and setting a forwarding target to null if the request is empty; determining if a chunk of a file set which fulfills the request is held locally; determining if forwarding target is not null; determining if an overlapping request exists and forwarding the request to the same node for which there is an overlapping request; and forwarding a non-overlapping request to a second node of the DCS based on one of requesting a most needed chunk of a file set first and requesting a closest chunk of the file set first.

[0059] The present invention provides also for the generation of Access Maps which show the inventory of chunks of the file set in neighboring nodes. Generation of such maps can be provided by any means or method which contributes to the operational efficiency of the DCS. For example, one method for generating an Access Map by a first node of a Dynamic Connection Structure (DCS) for a second node of the DCS comprises the steps of: initializing each entry of the Access Map for the second node of the DCS; resetting each entry of the Access Map that corresponds to a chunk of a file set that is maintained locally; and setting each entry of the Access Map that corresponds to a distance based on a number of hops from the first node.

[0060] In a further aspect, the present invention provides also for the generation of Request Maps which show the chunks of the file set needed by neighboring nodes. Thus, the methods of the present invention include a method for generating a Request Map by a first node of a Dynamic Connection Structure (DCS) for at least one second node of the DCS, the method comprising the steps of 1) initializing each entry of the Request Map for the at least one second node of the DCS; 2) resetting each entry of the Request Map to indicate whether a chunk of a file set is needed by the at least one second node; 3) determining if the Request Map for the at least one second node of the DCS has overlapping requests with the Request Map of the first node of the DCS; 4) updating the Request Map accordingly to produce a Resulting Request Map; and 5) forwarding the resulting Request Map to a target node.

[0061] In one preferred embodiment, the invention uses two Java programs, a j-server and a j-client program. Any reference to a j-server implies an executing instance of the j- server program. Any reference to a j-client implies an executing instance of the j-client program. Any reference to a root j-client implies an executing instance of the j-client program configured to be a root. Any reference to a general j-client implies an executing instance of the j-client program not configured to be a root. [0062] A user of a computer system makes a decision to use the DCS of the present invention and the corresponding routing scheme of the present invention. The user needs to have special software to use the present invention. This special software may be downloaded from a website, such as a multi-media vendor, or may be distributed free on a CD when a video, e.g., DVD, or music CD is purchased, or by any other convenient means. Such means include a display in a multi-media vendor where the software may have a nominal cost attached thereto, which upon installation and registration, is rebated or credited back towards purchases. The node software to operate the present invention will be hereinafter referred to as Network Interface Software (NIS). [0063] The present invention comprises two major components - the DCS topology and the routing scheme. While these two components may each be able to be used with other corresponding components, the present invention works best when they are used together. Use of the DCS topology and the routing scheme of the present invention ensures maximum effective use of a node's bandwidth - both upstream and downstream and system resources, in general. [0064] The DCS can be a mesh or graph, a binary heap, a ring, a chain, a bus or of any desired topology. Salient features include wherein the structure is dynamic, and thus changes when a node disconnects/connects from the topology for any reason, for example, when a download is complete, failure of the node's link to the DCS, or when requesting a service or product. The DCS is centrally organized and centrally reorganized whenever an event occurs such as a node connecting/disconnecting. A DCS of the invention is a peer-to-peer network of nodes in communication with each other and with a traffic cop node, which organizes and maintains (re-organizes) the topology of the network as needed or as required.

[0065] The underlying network may be the Internet, a cable network, fiber, hybrid- fiber-coax, wireless satellite communications or any other technology capable of supporting topologies of the DCS, and the many possible routing schemes described herein.

[0066] In addition to the dynamic nature of the connection structure, and the ability of nodes to connect and disconnect from it easily, nodes may be mobile. That is, a node may be in a moving vehicle, such as a car, bicycle, RV, ship, boat, laptop computer, train, airplane, or handheld device.

[0067] Significant to an understanding of the invention, and particularly of the DCS topologies, it would be helpful to describe the functions that a node in the topology can assume. The topology of the present invention uses three different and distinct views or perspectives of node functionality. A node can be a servant in which case it requests content from the network via the routing scheme and the associated Request Maps and Access Maps. A servant is a client and server which perform tasks normally associated with both clients and servers. A node can also be a subnode (neighbor or peer) to another node. A node can be a root server (server/seed node or seed server). In this case the node is the provider of content to the network. A seed server is a central place for getting specific content. Its only job is to deliver the data that is requested by the servants. A client is, a node that requests services or data of another node. That is, a client is a node that receives shared services and data from a server.

[0068] A peer, or peer node, is any of the devices in a layered communications network that operate on the same protocol level. In network architecture, a peer is any functional unit that is in the same layer as another entity. A peer is a corresponding node or entity. A peer may manifest in the form of a processor, a process or a device. A peer may be anything with a digital heartbeat, including sensors, servers, PC's, manufacturing and medical equipment, phones and cellular phones. In order to interact with other peers (e.g. to form or join peer groups), the peer needs to be connected to some kind of network (wired or wireless), such as IP, Bluetooth cable, or Havi, among others.

[0069] In Open Systems Interconnections architecture, peer entities are entities in the same or different open systems that are in the same layer, where a layer is a network protocol layer or level. A peer node is a node that is a member of the same peer group as a given node. A peer group is a set of logical nodes, which are grouped for purposes of creating a routing hierarchy.

[0070] Peer protocol is the set of rules defining the procedures for communication between like entities. The identical entities may be devices or software modules at specific layers in an architecture implementation. Peer-to-peer communications pertains to data communication between two nodes that have equal status in the interchange, where either node can begin the exchange and where equal status means nodes that are in the same protocol layer or level.

[0071] A traffic cop is a software component that manages the topology of the DCS. The primary functions of a traffic cop are to provide information to servants who wish to join the DCS, and to adjust the topology of the DCS when servants leave. The traffic cop also helps a servant to determine if that servant is behind a firewall or not. The traffic cop (or j-server) is a software program used to: 1. Organize dynamic connection structures ("DCS's"); 2. Reorganize dynamic connection structures; and 3. Communicate organizational changes to affected nodes in the DCS.

[0072] To organize a Dynamic Connection Structure, the traffic cop utilizes algorithms that allow it to intelligently organize a DCS. Some examples of organizational determinants include: 1. Is the client high-speed? If so, give it a higher number of connections. 2. Is the client low-speed? If so, give it a lower number of connections. 3. Is the client behind a firewall? If so, don't connect it to other clients that are behind firewalls. 4. Through what ISP is the client connected to the Internet? ISP connection/identity information can be obtained, for example, via lookup in an ISP database by Internet Protocol address. Preferably, clients having the same ISP are connected to one another. 5. What is the client's geographic location? This information can be obtained, for example, via lookup in a geographic database by Internet Protocol address. Connect the client to other clients that are in nearby geographic locations.

[0073] The traffic cop is adapted and configured to communicate organizational changes to affected nodes. Once the traffic cop algorithms have determined the desired topology of the DCS, one or more messages are constructed which contain topology instructions. These instructions can be interpreted by the affected nodes (by special client software installed on the clients' computers), and these nodes will carry out the instructions dictated by the traffic cop. Generally, organizational changes are made in response to an action/event that is transmitted to the traffic cop. The client notifying the traffic cop of the action/event will receive the instruction message(s) from the traffic cop, make connections that apply to itself, and propagate the message to other nodes that are affected by the instructions. The other nodes will similarly make connections that apply to them, and propagate the message farther if necessary.

Examples of actions/events to which the Traffic Cop responds include: 1. Client wishes to be added to a specific DCS . 2. Client wishes to be removed from a specific DCS. 3. Client in a DCS disappeared. 4. Client attributes changed. For example, a client may notify the traffic cop that the client has received the entire file, and the traffic cop may decide to give the client additional connections. [0074] Preferably, a client inteφrets a Traffic Cop instruction message in accordance with specific criteria, including: 1. Does this message have a valid security signature? If yes, proceed. 2. Does this message contain directions for any of the peers that I am currently connected to? If yes, forward the message to those peers. 3. Does this message dictate connection changes for me? If yes, make new connections as specified and sever existing connections as specified. 4. Does this message contain directions for any of the new peers that I have just connected to? If yes, forward the message to those peers. 5. Does this message contain any additional instructions for me (called non-topology messages)? If so, handle them appropriately.

[0075] In another aspect, the invention provides means and methods for assembling a DCS comprising a plurality of nodes which can perform the desired file transfer and file re-transfer functions even when not connected to a server or seed node. This partial autonomy is accomplished, in pertinent part, by means of software (NIS) which are provided to each node prior to or as that node is connected to the DCS. Typically, the following acts or functions are enabled in a node possessing the NIS: 1) Receiving at least one portion of a first file (a chunk) from another node in the DCS or from a seed node/server in the DCS enabled to contribute content to one or more nodes of the DCS; 2) Receiving the remaining portions of the first file from one or more other nodes in the DCS or from a seed node/server in the DCS, enabled to contribute content to one or more nodes of the DCS; 3) Transferring the at least one portion of the first file to one or a plurality of other nodes in the DCS; 4) Transferring the remaining portions of the first file to one or a plurality of other nodes in the DCS; 5) Assembling, at each node desiring/requesting a complete first file, the portions of that first file into a complete first file;

6) Receiving requests for one or more portions of a first file from one or a plurality of other nodes in the DCS; 7) Assessing the node's own inventory with respect to all portions of the first file; also assessing its neighbors'/peers' inventories of all portions of the first file;

8) Transmitting requests to other nodes, seed nodes/servers for portions of the first file needed by the node;

9) Responding to requests from other nodes in the DCS by a node having any portion of the first file by one or more of a) satisfying the received requests by transmitting the requested first file portions from the node's own inventory, and/or b) forwarding the received request to other nodes in the DCS;

10) Communicating the status of its own first file portions inventory to one or more of a) other nodes, b) seed nodes, and servers enabled to contribute content to one or more nodes of the DCS ;

11) Integrate itself into the existing DCS topology in accordance with instructions contained in the NIS and via the "traffic-cop" node;

12) Re-organize itself and the DCS topology in accordance with the NIS, and via the traffic cop node including a) removal of self from DCS, b) removal of seed node/server from DCS, c) removal of another (neighbor/peer/sub-) node from DCS;

13) Simultaneously i) receiving a portion of the first file from a node in the DCS and ii) transmitting that portion to one or more nodes in the DCS;

14) Communicating the status of its own ongoing transmission and reception of specific first file portions to one or more of a) other nodes, b) seed nodes/server enabled to contribute content to one or more nodes of the DCS;

15) Receiving file portion location information from other nodes, seed node/server;

16) Identifying and sorting separate portions of the first file;

17) Assessing the topology of the DCS with respect to the respective inventories of portions of the first file and providing preferred paths for transferring portions of the file from the other nodes to requestors, including an estimate of the number of hops to the requested file portion. 18) Maintaining associations between respective portions of the first file and Digital Rights Management data and code accompanying those portions; and; 19) Forwarding those respective portions with the corresponding DRM data and code.

[0076] A dynamic connection structure of clients is established comprising all clients involved in the file transfer and their connections to each other. This dynamic connection structure is generated using an incomplete binary heap algorithm (defined below) as follows. For each client added to the dynamic connection structure, a node is added to the heap. For each client removed from the dynamic connection structure, the node corresponding to that client is removed from the heap. There is always a direct mapping between nodes in the heap and clients in the dynamic connection structure. [0077] Nodes in a binary heap can have a parent and up to two children. Henceforth, any reference to a given client's parent implies the client which corresponds to the parent node of the node which corresponds to the given client. In other words, if Node A in the heap corresponds to Client A, and Node B corresponds to Client B, and Node A is the parent of Node B, then Client A may be referred to as the parent of Client B. The same is true of references to a given client's children. [0078] The server program is responsible for creating and maintaining the heap. When a new client requires addition to the dynamic connection structure, the client contacts the server and requests addition. The server then adds a node representing the client to the heap using the incomplete binary heap addition algorithm, and sends the client an encoded representation of its corresponding resultant subheap. The client then inserts itself into the dynamic connection structure by contacting necessary clients, providing them with connection information derived from the server's message, and cooperatively establishing the specified connections. When a client has received the entire file or when a client crashes (hangs), a removal from the dynamic connection structure is accomplished similarly. The parent of the client contacts the server, the server removes the corresponding node from the heap, and sends information which includes a description of the new subheap to the parent. The parent then makes the necessary communications to enable the dynamic connection structure to reorganize itself in a way which contains no connections to the removed node. Thus, the server program facilitates the file transfer by organizing and specifying appropriate alterations to the connections over which the packets are transferred.

[0079] The root client is responsible for dividing the file into packets, and these packets are passed sequentially over the open connections in the dynamic connection structure. Packets are always passed from a parent client to a child client. The root client passes a packet to its child clients. Each of these clients passes the packet to their child clients, and the process continues until every client has received the packet. Packets are transferred between two connected clients without intervention. In other words, if two clients have an open connection, they will always repeatedly transfer packets until disconnected. The present invention represents a dramatic improvement on current file transfer technologies because it allows client programs to send packets to their children while receiving packets from their parents. As soon as a client has received a packet, it is able to receive the next packet. Essentially, every client in the dynamic connection structure can be receiving the file simultaneously.

[0080] To model the dynamic connection structure, this embodiment employs a modified binary heap data structure as the most appropriate type of binary tree. The dynamic connection structure is considered dynamic because additions and deletions are permitted any time after the heap is initialized and before the heap is destroyed. The embodied heap sorting requirement is that any j-client in the dynamic connection structure must always have a slower network connection speed than its parent. Each j- client's network connection speed is determined prior to the insertion of a node corresponding to that client in the heap, and is used as the sort key for the heap. [0081] In this embodiment, the heap properties are modified as follows: 1. the heap is no longer required to satisfy the completeness property; 2. the repositioning of nodes in the heap following an addition or deletion of a node, should never cause a node corresponding to a j-client to be repositioned more than one step away from its previous position. [0082] The completeness property of a binary tree requires that every level except the last is full, and that on the last level, all internal nodes are to the left of external nodes. In a standard binary heap, the completeness property must always be satisfied. Following the addition or removal of nodes, reorganization of the heap is often required to maintain the completeness property.

[0083] The removal of the completeness requirement is desired in this implementation, because we prefer reorganization of the heap to be kept to a minimum. In the dynamic connection structure, the work done processing modified connections and contacting other j -clients can cause time delays, and these delays should be minimized. By eliminating this requirement, the heap removal algorithm is simplified to require only a down-heap following a delete. It is no longer necessary to swap the positions of two nodes prior to a delete. This is desirable, because swapping nodes on different levels of the heap may result in situations where a j-client has a set of packets that the j-client' s parent does not have yet. For example, consider a j-client at level 2 of the heap which has received packets 0 through 10, and a j-client at level 6 of the heap that has received packets 0 through 5. Swapping the positions of these j -clients in the heap could require the j-client now at level 2 to request packets 6 through 10. This would obviously create unnecessary packet transfers. [0084] By introducing these modifications to the heap requirements, we are able to simplify the heap addition and removal algorithms to the following: heap addition (^'insertion') Algorithm insert(w) { p = the first open position in the heap heap(p) = w call upHeap (p)

} heap removal (deletion Algorithm delete(w) { p = the position in the heap of node w ball downHeap (p) }

[0085] The first open position in the heap is specified in the insert algorithm as the leftmost position not occupied by a node in the highest level in the tree which is not full. The upHeap and downHeap algorithms remain the same as in a standard binary heap algorithm.

[0086] These simplifications ensure that the addition or removal from the heap of a node corresponding to a single j-client will never cause a node corresponding to another j-client to be moved further than one step along a path in the tree. Thus, these simplified algorithms satisfy the second requirement that a node corresponding to aj-client is never repositioned more than one step away from its previous position in the heap.

[0087] The modified binary heap is used to model the dynamic connection structure of j -clients by using a node in the heap to represent a j-client in the dynamic connection structure. Each node in the heap can have a parent and up to two children. In the dynamic connection structure, each j-client can have a parent and up to two children. The parent of aj-client is aj-client which is responsible for sending data packets to that j- client. The children of a j-client are j-clients to which that j-client will send data packets. Thus, the representation of the heap at a given time can be used to determine which network connections for a given j-client should be actively sending and receiving data packets at that time. [0088] For the puφoses of this specification, the term incomplete binary heap or incomplete binary heap structure or incomplete binary heap data structure will be defined as a data structure possessing the characteristics of a binary heap modified in the manner described above. The term incomplete binary heap algorithm will be defined as an algorithm possessing the characteristics of a binary heap algorithm modified in the manner described above. The terms incomplete binary heap addition algorithm and incomplete binary heap deletion algorithm will be defined as the addition (insertion) algorithm and the removal (deletion) algorithms described above. [0089] While it may be possible to further refine these algorithms or to use or modify other tree organization algorithms to enhance the performance of the invention, the use of any binary tree or m-ary tree structure to model the dynamic connection structure described herein would not fall outside of the scope and spirit of the present invention. [0090] For the puφoses of this specification, a data packet or packet shall be defined as an individual segment of information resulting from the division of a larger set of information into one or more parts. [0091] For the puφoses of this embodiment, a data packet ox packet shall be defined as an individual segment of data bytes resulting from the division of a file. [0092] For the puφoses of this specification, any references to transferring a file at the packet level shall imply dividing the file into packets, and transferring the packets individually. [0093] Socket implementations in a programming language are common in the art. A socket is used in a computer program to connect to, receive connections from, send information to, and receive information from other computers. All communications between computers on a network in this embodiment are accomplished using sockets. Other utilities used to transmit data packets from one computer to another, would not fall outside of the scope and spirit of the present invention.

[0094] In this specification, any reference to connections or active connections indicates the set of open network connections between aj-client or j-server and another j- client or j-server. Any connection described in this embodiment indicates open TCP socket connection between two programs, which implies open TCP connection between the computers on which the programs are running. If the two programs are running on the same computer, this still holds true.

[0095] In this specification, any reference to a program opening a connection, opening a network connection, or connecting to another program implies the following steps: 1. The computer on which the program is running uses a socket to contact a computer on a specified port, and requests that a connection be established. 2. The contacted computer accepts, rejects, or does not respond to the request. 3. If the contacted computer accepts the connection within a certain amount of time, the connection is considered open and the sockets are considered connected. 4. If the computer to which contact is attempted rejects or does not respond to the request within a given amount of time, the connection is considered closed, and the sockets remain unconnected. [0096] Any references to sending or transmitting data, a message, or a packet from a source program to a destination program implies the following: 1. A network connection is open between the sender and receiver. 2. The source program transforms the message into data bytes as specified by a protocol. 3. The source program transmits the message over the open connection. [0097] A dynamic connection structure of clients or a dynamic connection structure is defined for the purposes of the specification of the present invention as a set of clients and a set of open network connections between these clients, of which the following are true: • information is capable of being transmitted across included network connections at any time; • the set of network connections may be modified at any time.

[0098] All programs described herein imply computer programs or processes executing on a computer connected to a network. The specific computers may vary in any way, but they must all be connected to the same network. This network will henceforth be referred to as Network A.

[0099] A data file may be transmitted from a source computer to a number of other computers over a network using dynamic connection structure means. The methods described pertain to transfer of a specific file, which may or may not be identified prior to initiation of these mechanisms. In this embodiment, the file to be transferred will be predetermined, and this file will henceforth be referred to as File A.

[00100] For the purposes of this specification, the definition of binary heap is consistent with the commonly understood definition in the art. [00101] For the puφoses of this specific embodiment of the invention, the term binary heap or heap designates an incomplete binary heap structure containing one node for each j-client in the dynamic connection structure. The heap provides a model of the dynamic connection structure and is used to determine connection information.. [00102] If Node A in the heap, which has children Node B and Node C, corresponds' to j-client A in the dynamic connection structure, and Node B and Node C in the heap correspond to j-client B and j-client C in the dynamic connection structure, respectively, it is implied that j-client A is responsible for sending data packets to j-client B and j- client C. Information stored in the heap for each j-client is: • Unique j-client ID used to identify each j-client in the dynamic connection structure. • j-client IP address. • j-client network port base (defined below). • j-client connection speed to Network A. [00103] For the purposes of this specification, a client is defined as a computer program capable of connecting to, receiving connections from, sending information, and receiving information from other computer programs. An external client is a client that is not included in a specified dynamic connection structure. [00104] For the purposes of this specific embodiment of the invention, the definition of a j-client shall be customized to refer to a program executing on a computer connected to Network A that is responsible for receiving packets from a source and sending those packets on to its children, and is also responsible for listening¹ for connection change information, and making the specified connection changes when they are received. [00105] The root j-client is a specific type of j-client. This program must be executing on a computer that has direct access to File A. It is preferred that the root j-client is executed on a computer that holds File A on its local disk. Because it is the reference source of File A, the node corresponding to the root j-client is always positioned at the top of the tree that models the dynamic connection structure. The root j-client is responsible for extracting packets from the local copy of File A, and sending them to its children. In this embodiment, only one root j-client will be used. However, additional root j-clients could be added; for instance, to provide failover protection. A root j-client is always assigned a connection speed of 0, which the binary heap sort key recognizes as the fastest defined network connection speed. This ensures that the no general j-client will ever be repositioned such that the root j-client is its child. [00106] A general j-client refers to any j-client that is not a root j-client.

[00107] For the puφoses of this specification, a server is defined as a computer program which is capable of connecting to, receiving connections from, sending information to, and receiving information from other computer programs, and that performs administrative tasks necessary to the overall implementation. The distinction between a client and a server should not be construed to mean that clients are incapable of performing administrative tasks, the distinction is drawn simply for the purpose of clarifying the specification. An external server is a server that is not included in a specified dynamic connection structure. [00108] For the puφoses of this specific embodiment of the invention, the definition of a j-server ox file j-server shall be customized to refer to a program executing on a computer connected to Network A, which may or may not be executing on the same computer as the root j-client, and which is responsible for the following: • Maintaining the dynamic connection structure, using incomplete binary heap addition and removal algorithms. • Servicing requests from j-clients to be inserted into the dynamic connection structure, by adding a node corresponding to the j-client to the heap, and responding to the j-client with new connection information as determined by the resultant heap structure. • Servicing requests from j-clients to remove j-clients from the dynamic connection structure, by removing a node corresponding to the specified j-client from the heap, and responding to the j-client with new connection information as determined by the resultant heap structure. [00109] A j-server program maintains one socket and one network port. A j-server must listen for connections from j-clients, and process the messages received. The j-server listens for these connections on a network port on the computer it runs on. This port is referred to as the j-server port. The socket which facilitates communication on the j- server port is referred to as the -.server socket.

[00110] A general j-client maintains four sockets and four network ports. The first port is used to listen for connections, and to receive messages regarding connection changes. This port is referred to as the info port. The second is used to connect to a parent j-client, and to receive packets from the parent j-client. This port is referred to as the parent port. The next two are used to receive connections from up to two child j-clients, and to send packets to these child j-clients. These ports are referred to as child ports. The sockets that facilitate communication on the info, parent and child ports are referred to as the info socket, the parent socket, and the child sockets respectively.

[00111] By convention, the four ports used by aj-client will be sequentially assigned starting with a base port number. The info port is assigned the base port, the parent port is assigned the base port + 1, the child ports are assigned the base port +2 and base port + 3. Aj-client program is responsible for finding 4 adjacent open ports prior to its initial contact to the server.

[00112] For the puφoses of this specification, connection information or connection information for a client is defined as information which indicates to a client the appropriate connections which the client should establish with other clients and connection information for other clients. [00113] The root j-client, which acts as the source of File A, is responsible for partitioning the file into one or more manageable chunks, referred to as data packets. The packet size is arbitrary, but should be chosen to be relatively small, so that no message takes excessively long to be decoded by j-clients. Each packet is assigned a packet number (packet num). [00114] When a new j-client is inserted in the dynamic connection structure, it does not request any packets from its parent until it receives packet requests from all of its children. It then requests the earliest packet required to be transferred to its children. If a j-client is inserted to a position where it has no children, it requests ANY JPACKET fxo its parent. If the parent has no other child, it will begin transmitting with the earliest packet it possesses. If the parent has another child, it will begin by transmitting the packet that it is currently transmitting to the other child. This technique insures that child j-clients do not have to wait for an inserted parent to receive packets that they already have. When aj-client is inserted, its children will have to wait a minimal amount of time before they can continue receiving their packets. [00115] The root j-client will begin packet transmissions to its first child with packet number 0. If File A contains n packets, the packets are numbered 0 to n-l. The root j- client will send each packet sequentially as they are requested until packet n-l is sent. The root j-client will then return to packet 0 and repeat the whole process. [00116] Aj-client in the dynamic connection structure always receives packets in sequential order, but not necessarily starting with packet 0. After packet n-l is received by aj-client, the j-client will wrap back around to packet 0, and begin receiving packets. If a j-client starts receiving packets with packet m, it will receive sequentially packets m through n-l, then 0 through m-l. The j-client will then transmit a final packet to its children, close connections with its children, and be removed from the dynamic connection structure.

[00117] This technique insures that aj-client in the dynamic connection structure will never need to request a packet that each of its children already has. In addition, aj-client is able to leave the dynamic connection structure (and disconnect from Network A) almost immediately following receipt of the last packet. [00118] The protocol implemented in this embodiment provides a means of communication between j-client and j-server programs. A message transmitted from one program to another in this system is structured as follows: • Message ID • Message type • Message parameter list • Included data packet size (if necessary) • Included data packet as an array of bytes (if necessary)

[00119] Each of the above message sections are separated by an end-of-line character. The last two are only necessary if a data packet is contained in the message. In the embodied protocol, there are only two message types that require a data packet be included. The message ID is a numeric id that is incremented each time a message is sent on a specific connection. This ID is used to insure that messages are not processed more than once, and to ensure that messages do not arrive out of sequence. [00120] In certain situations, a message is used to inform aj-client that its active connections should be changed. Because insertions and deletions may require multiple j- clients to change their connections, and these affected j-clients always end up in a chain, we choose, in this implementation, to notify only a single j-client of the changes, and pass these changes down through the dynamic connection structure. This takes the load off of the j-server, by making it unnecessary for the j-server to contact many j-clients to handle a single insert or delete. To generate connection information relating to aj-client, an inorder traversal of the subheap is performed, starting at the node representing the specified j-client. The j-client ID of each j-client in the subheap is appended to a delimited string. This string, henceforth called an encoded tree, can be used by any included j-client to determine the manner in which their connections should be changed. [00121] Given two j-clients j-client A and j-client B, the following convention is used. When j-client A sends an encoded tree to j-client B, the encoded tree is the encoded version of the subheap starting at j-client B. When aj-client needs to send new connection information to a new child, that j-client will generate an encoded tree for its child, which is the encoded version of the subheap starting at that child.

[00122] The following is a list of the valid message types, and their parameters: REQUEST NSERT j-clientld ' REQUEST_LNSERT_ROOT j-clientld, filename, fileLength INSERT NFO encoded tree, filename, fileLength NEW_PARENT_TREE encoded tree, parentPort SET_CHILDREN encodedTree REQUEST_PACKET packetNum CHILD_INFO_RECIEVED packetNum, packets ize, byte Array YOU_ARE_THE_ROOT no params ALREADY HAVE ROOT no params WAITING_FOR_ROOT no params PACKETJHERE packetNum, packetSize, byteArray PACKET_ HERE_BYE packetNum, packetSize, byteArray INVALID_PACKET_REQUEST packetNum ιNVALID_PACKET_REQUEST B YE no params DELETE_CHILD , clientld DELETE_INFO encoded tree OK_BYE no params

[00123] The REQUEST NSERT and REQUEST_INSERT_ROOT message types allow a general j-client or a root j-client to request insertion into the dynamic connection structure. The unique j-client ID is included in each, and it includes the j-client's connection speed and network port base.

[00124] The messages YOU_ARE_THE_ROOT allows a j-server to notify a root j- client that it has been added to the root of the heap. [00125] The message ALREADY_HAVE_ROOT allows a j-server to notify a root j- client that it was not added to the tree, because a root j-client already exists.

[00126] The INSERTJNFO message is sent from a j-server to a newly added j-client.

This message includes the encoded tree of the j-client's new parent, and information about the file being served. [00127] The NEW_PARENT_TREE message is used to inform a j-client that it must change its child and parent connections. The encoded tree contains the j-client's new connection information, and new connection information for all j-clients included in the given j-client's representative subheap. The parent port specifies the network port on which to contact the new parent. [00128] The message SET_CHILDREN is identical to the NEW_PARENT_TREE message, except it implies that only the j-client's children are changing, and does not need to send a parent port.

[00129] The message REQUESTJPACKET allows a child to request a specific packet from its parent. PacketNum is the unique identifier of the packet needed. [00130] The message CHILD_INFO_RECEIVED is used identically to the message

REQUEST_PACKET, except it implies that newly sent connection information was received.

[00131] The message PACKET_HERE is used to send a packet from a parent j-client to a child j-client. It includes the packet number, the packet size, and the data packet as an array of bytes.

[00132] The message PACKET_HERE_BYE is identical to the message

PACKET_HERE, except it implies that the connection is being closed after acknowledgement of its receipt. [00133] The message INVALID_PACKET_REQUEST allows a parent j-client to notify a child j-client that it does not have the requested packet.

[00134] The message INVALID_PACKET_REQUEST BYE is identical to the message

LNVALID_PACKET_REQUEST, except it implies that the connection is being closed after acknowledgement of its receipt. [00135] The message DELETE_CHILD allows aj-client to notify a j-server that its child j-client requires deletion from the connection structure.

[00136] The message DELETE_INFO is sent from a j-server to aj-client which requested deletion of a j-client. This message includes a new encoded tree for the j- client. [00137] The message OK_BYE is used to acknowledge receipt of a message, and to notify the recipient that the connection is being closed.

[00138] The following 4 sequences of communication are available: 1. A j-client contacts a j-server to request addition to the dynamic connection structure • J-client uses info port, j-server uses j-server port • J-client sends REQUESTJNSERT or REQUESTJNSERT_ROOT. • J-server responds YOU_ARE_THE_ROOT, ALREADY JLAVEJROOT, or INSERTJNFO. 2. A j-client contacts a j-server to request that one of its child j-clients be removed from the dynamic connection structure J-client uses info port, j-server uses j-server port J-client sends DELETE_CHILD. J-server responds DELETEJNFO. -client A transmits information about changing connections to J-client B J-client A uses info port or parent port, J-client B uses info port J-client A sends SET_CHILDREN, or NEW_PARENT_TREE. J-client B responds OK_BYE or CHILD JNFOJIECIEVED. client A attempts data packets transmission to J-client B J-client A uses a child port, J-client B uses parent port J-client B sends REQUEST J^»ACKET. J-server responds PACKET JIERE, PACKET J EREJB YE, INVALID JACKET_REQUEST, or INVALID_PACKET_REQUEST BYE.

[00139] This protocol is obviously just one example of a protocol that could be used to transmit data packets and to facilitate changes to the dynamic connection structure described herein. Any other protocol used to facilitate similar changes to a dynamic connection structure would not fall outside the scope of the present invention. [00140] The j-server program does the following when executed: 1. Initialize the binary heap to empty. 2. Initialize the j-server socket and begin listening for connections on the j-server port. 3. Wait for a connection on the j-server port. 4. When a connection is received, wait for a message and process it. 5. Send an appropriate reply. 6. Close the connection and continue listening on the j-server port. 7. Go back to step 3 and repeat.

[00141] When any of the above steps refers to processing a message, it implies the following: • If the message is a request to be added to the heap, the j-server program adds a node corresponding to the j-client to the heap in its memory, generates an encoded tree for the j-client's highest level ancestor for which connections change, and sends an appropriate reply. • If the message is a request to remove a specified node from the heap, the j-server program removes the nod corresponding to the specified j-client from the heap in its memory, generates an encoded tree for the j-client, and sends an appropriate reply.

[00142] The j-client program does the following when executed: 1. Assess the network connection speed of the computer on which it is executing. 2. Generate a unique j-client ID, which includes IP address, connection speed, base port, and a timestamp. 3. Use the local info port to connect to the remote j-server port and request addition to the tree. 4. Receive connection information from the j-server. 5. Close the connection on the local info port. 6. Use the local parent port to connect the info port of the heap ancestor specified by , the j-server. 7. Send the new connection information over this connection. 8. Close the connection. 9. Spawn threads. 10. Begin listening on the local info port, and child ports. 11. Check the info port for a connection, and if a connection has been requested, do the following: • Accept the connection • Receive new connection information over the info port. • Close the connection on the info port and continue listening. 12. If new parent information has been received, do the following: • Disconnect the parent port from the current parent. • Connect the parent port to the new parent on the remote port specified. 13. If the parent port is connected, send the parent a request for the next needed packet. 14. Check the parent port for a message from the parent. 15. If a message has been received from the parent, process it. 16. If any children have changed, do the following for each changing child: • Disconnect the child port from the current child. • Connect the new child on their info port. • Send the new child their new connection information, which can be derived from the last received encoded tree. • Close the connection on the child port, and continue listening. 17. Check each child port that is not connected for a connection, and if a connection has been requested, accept the connection. 18. For each child port that has an open connection, check for a message. 19. Process each message that has been received from children. 20. Go back to step 10 and repeat until all packets in File A have been received. 21. Assemble all packets into a complete copy of File A.

[00143] When any of the above steps refers to processing a message, it implies the following: • If the message contains a data packet, the data packet is saved to disk. • If the message contains connection information, this connection information is saved to memory. • If the message contains a BYE, the connection is immediately closed.

[00144] Each of the steps from 10 to 18 are sequence independent. In other works, any step can be run at any time, and it will not disturb the integrity of the process. To make the info, parent, and child processes less dependent on each other, Java threads are used. Java threads allow a program to execute multiple processes simultaneously, so that delays in one process do not delay other processes. In this implementation, aj-client spawns 3 threads at step 9. The first thread handles all parent port communications, the second and third handle all communications over the first and second child ports, respectively. The original process continues to handle all communications, over the info port. This is desired, because sending and receiving different messages over different ports will not interfere with each other.

[00145] This programming solution is obviously just one example of a solution that could be used to implement the file transfer method described herein. Any other programming solution used to facilitate similar changes to the described dynamic connection structure would not fall outside of the scope and spirit of the present invention.

[00146] The j-server program is executed on a computer connected to Network A. A j- client program is executed on a computer connected to Network A and with local access to File A, and it is configured to run as the root j-client. This program contacts the j- server, and requests addition to the dynamic connection structure, as the root j-client. The j-server adds the root j-client to the heap, and replies, signifying that the addition was successful, and closes the connection.

[00147] Any number of j-client programs are then executed any number of times on computers connected to Network A. Each time aj-client program is executed, it contacts the j-server and requests addition to the dynamic connection structure. The j-server adds a node corresponding to the j-client to the heap, which results in the repositioning of a number of nodes along a path in the heap. The j-server replies, signifying that the addition was successful, and sends to the added j-client an encoded tree relating to the j- client at the highest level in the tree whose connections are affected by the addition (the ancestor). The j-client contacts this ancestor, sends the new connection information generated by the j-server, and closes the connection. The ancestor then modifies its own connection information, by closing appropriate connections and contacting new children. It sends its new children their new connection information, which is extracted from the encoded tree, and then closes the appropriate connections. This process repeats; each j- client whose connections are affected closes the appropriate connections, contacts its new parents if necessary, and sends new connection information to its children. [00148] If aj-client receives a message from a child signifying that the child has received all packets in File A, or if a child or parent does not communicate for a certain amount of time (crashed or hung), the hung j-client is removed from the tree. To accomplish this, the j-client formerly connected to the hung j-client contacts the j-server and indicates to the j-server which j-client must be deleted. The j-server deletes the node corresponding to the j-client requiring deletion from the heap, which results in repositioning of a number of nodes along a path in the heap. The j-server replies, signifying that the deletion was successful, and sends to the j-client a new encoded tree. Similarly to j-client addition, new connections are established as the information is passed down the heap.

[00149] While all of this is happening, all j-clients that have open parent and child connections are sending and receiving data packets of File A.

[00150] Because changes to the heap in the j-server' s memory do not immediately take effect (they are passed down the heap over time), the actual dynamic connection structure does not always exactly match the heap representation in the j-server' s memory. In addition, because many additions and deletions may be performed by the j-server sequentially, and these changes are constantly filtering down the heap, the actual dynamic connection structure may never exactly match that of the heap. The heap is thus considered to always represent an ever-changing goal state that the dynamic connection structure is working towards. To ensure that additions and deletions of j-clients never conflict with each other, the j-server affixes a timestamp to every encoded tree it generates. When aj-client generates an encoded tree to be sent to a child j-client, the newly generated encoded tree is assigned the same timestamp as the encoded tree which was used to generate it. If aj-client ever receives connection information that includes a timestamp which is earlier than the timestamp on the last applied connection changes, the information is ignored.

[00151] While topology and routing can be used independent of each other or more correctly with other routing and topology schemes, as indicated earlier they are most effective when used together as will be illustrated below. Aspects of the invention can also be understood with reference to Figs. 11(a) through 13(c) Referring first to Figs. 11(a) through 11(d), Node N requests authorization to enter/connect to the DCS for puφoses of file transfer. In this case asiwill be described below the DCS is a mesh. In Fig. 11(a), node/peer/client N connects to traffic cop S. In Fig. 11(b), peer N requests access to a file. Traffic cop S then grants access and forwards access information, file information, a signature and a connection list to peer N as shown in Fig. 11(c). Peer N then disconnects from traffic cop S as shown in Fig. 11(d).

[00152] Once it has been established that peer N has been granted access to connect into the DCS, peer N inserts itself into the DCS by connecting to one or more peers that are already in the DCS. Peer N forwards access information and connection list information that was received from traffic cop S. As shown in Fig. 12(a), peer N connects to peers A and B. Connection list information is then sent to all affected peers. Affected peers are peers that must change their connections as a result of the newly established connection with peer N. Fig. 12(b) show peer B forwarding connection list information to peer G because that connection will be broken. Also shown is peer A forwarding connection list information to peer R because that connection will also be broken as a result of the entry of peer N into the DCS. Fig. 12(c) shows the breaking or disconnection of peer R from peer A and peer G from peer B. Fig. 12(d) shows the formation of a new connection between peer R and peer G as a result of peer N entering the DCS. The information for the new connection was sent to peers R and G along with the disconnection information. All of the above access information and connection lists are further described below in the description of the Packet Chain Protocol and the object schemas. [00153] With respect to a smaller DCS, Fig. 13(a) illustrates peer A notifying each neighbor by sending Access Maps of all chunks of the file that peer A has and what chunks of the file that it and its neighbors need. Referring to Fig, 13(b), peers B, C, D and E request chunks of the file that they and their neighbors need from peer A via Request Maps sent to peer A. Peer A then requests chunks from its neighbors (not shown) and forward chunks of the file that it has that are needed by peers B, C, D and E to them. For example, peer C might have neighbors CI and C2 (not shown) and peer C forwards a Request Map to peer A that includes the chunks that peers CI and C2 need as well. Referring now to Fig. 13(c), if peer A does not have any chunk of the file that peer B needs then peer A notifies peer C, for example, of chunks of the file that both peer A and peer B need via updated Request Maps.

[00154] Upon the completion of a file transfer, peer N connects to traffic cop S and notifies traffic cop S that the file transfer is complete and that peer N wishes to disconnect from the DCS. Traffic cop S then sends connection list information to peer N. Peer N then disconnects from traffic cop S. Connection list information is sent to all affected peers/nodes that must change their connection because peer N wishes to disconnect from the DCS. Connections that are not needed are broken including all of peer N's connections. Peer N exits the DCS and any new connections required as a result of peer N leaving the DCS are formed.

[00155] Additional aspects of the invention can also be understood with reference to Figs. 1-26.

Fig. 1 illustrates an exemplary DCS having a mesh topology, which is an extension of a binary heap. That is, in a mesh topology, a node may have any number of children (subnodes, called "children" or "child nodes" in a binary heap) to which it is connected, and is not limited to two children nor is there any limit on completeness. In fact, the number of subnodes and connections may vary from node to node. This parent-→child relationship as described above also includes parent -→-child, making the relationship more equal. A mesh or binary tree DCS may be organized and re-organized by the traffic cop node according to node attributes such as network connection speed, firewall status, geographic location, the IP address in the sub-network, where the parts (chunks) of the files are by sub-network class, user account information (e.g. most money in account, greatest number of shared files, spending habits, always online etc.). That is, the nodes of the DCS are organized to maximize efficiency using each node's available upstream as well as downstream capacity. The above attributes may also be useful to the traffic cop node in the creation of a connection list, which will be described in greater detail hereinbelow. [00156] While organization and re-organization of the DCS is centrally controlled and the database memorializing this structure is centrally maintained, decentralized communication is used to form and maintain (organize and re-organize) the DCS. If any of the nodes are protected by a firewall, then communications with such nodes is via a buddy connection. A buddy connection is a connection to a firewall protected node via a "safe/buddy" node. This feature increases involvement of firewall protected nodes in the DCS.

[00157] The traffic cop node (also called j-server) is a computer executing specialized software, which is designed, used and formulated to fulfill three primary functions: 1) organize dynamic connection structures; 2) reorganize dynamic connection structures, and 3) communicate organizational changes to affected nodes in a DCS. [00158] In accordance with the present invention, a traffic cop node utilizes software algorithms that allow it to intelligently organize dynamic connection structures in accordance with certain criteria, including the speed of the node, the node's firewall status, and the node's geographic location. If the node is high-speed, then the node is given a higher number of connections. If the node is low-speed, then the node is given a lower number of connections. For example, if the node is behind a firewall, then it is not connected to other nodes that are behind firewalls. The traffic cop will use Internet Service Provider (ISP) and geographic information to optimally manage meshes. The traffic cop node will maintain three attributes for each node: ISP, state, and city. The order of priority will be ISP, state, city. The ISP for a node is determined by looking in a database of ISPs by Internet Protocol Address and nodes are connected to each other, if possible and to the extent possible, through the same ISP. The geographic location of a node is determined by looking in a database of ISPs by Internet Protocol Address and nodes are connected to each other, if possible and to the extent possible, in nearby geographic areas. For example, two nodes using the same ISP in different cities will be connected before two nodes using different ISPs in the same city. [00159] Once the traffic cop software algorithms have determined the desired topology of a DCS, one or more messages are constructed which contain topology instructions. These instructions can be inteφreted by the affected nodes (by special node instruction software (NIS) installed on the nodes' computers), and these nodes will carry out the instructions communicated by the traffic cop node. Generally, organizational changes are made in response to an action/event that is transmitted to the traffic cop node. The node, notifying the traffic cop node of the action/event, will receive the instruction message(s) from the traffic cop node, make connections that apply to itself, propagate the message to other nodes that are affected by the instructions. The other nodes will similarly make connections that apply to them, and propagate the message farther, if necessary. Once again, it is apparent that the topology is centrally maintained and communication of the topology is decentralized. [00160] Actions/events communicated to the traffic cop node by a node of the DCS include a node that wants to be added to a DCS, a node that wants to be removed (deleted) from a DCS, a node that is reported by another node to be missing (disappeared) or non-responsive (lost a connection) from a DCS and attributes of a node that have changed. The node has received the entire file so the traffic cop node may give that node additional connections so that the file may be disseminated more quickly and efficiently. [00161] Upon receiving a topology instruction message from the traffic cop node, a node will verify that the message has a valid security signature. If the security signature is invalid the message will be ignored. If the security signature is valid then the node will establish new connections and break/sever existing connections as specified in the topology instruction message. If any of the directions in the message pertain to any peers to which the node receiving the topology instruction message is connected then the topology instruction message is forwarded/communicated to the affected peers. If the topology instruction message contains any direction for new peers with which the node receiving the topology instruction message was instructed to establish a connection, then the topology instruction message is forwarded/communicated to the new peers. If the message contains any non-topology instructions then the non-topology instructions are handled appropriately. Non-topology messages include an invitation/directive to join another DCS, a chat message, a configuration message or a promotional message/advertisement. A node might be invited to join another DCS because the node is needed as a file upload source in the other DCS or the other DCS is transferring content in which the node may be interested or the other DCS may be a chat group in which the node may be interested.

[00162] As is readily apparent, a DCS can be used for more than file transfers. A DCS can be used in any scenario that resources of nodes are needed. The configuration/topology of the DCS is organized specially depending on the types of resources needed. For example, a DCS can be used to facilitate message delivery into computers/nodes that are behind a firewall. Many firewall type security provisions currently are used which block connections into computers. A DCS would be structured as illustrated in Fig. 2. A DCS may be created for a group of nodes that wish to chat together, and chat messages can be routed between these nodes. A DCS may be created for a group of nodes and information passed among the nodes in order to utilize the CPU processing power of the nodes. This is called grid/utility computing. A DCS may be created for a group of nodes and information passed among the nodes in order to utilize the aggregate storage (e.g., hard-drive) power of the nodes. This is called distributed storage.

[00163] In order to manage meshes/DCSs a balance ratio is defined as follows:

AvgUpHighSpeed* NffighSpeed + AvgUp_LθwSpeed* N_LowSpeed

QUoHighSpeed J^DownloadingHighSpeed ^"■" ^^LowSpeed -^DownloadingLo Speed

In distributed storage, one goal is to keep the balance ratio of each mesh slightly greater than 1. A ratio of 1 indicates sufficient upstream bandwidth within the mesh to meet the desired quality of service for all the downloaders. Nonetheless, it makes sense to provide higher capacity, however, so that if any of the nodes fail, or if any of the nodes need to be pulled into another mesh, the current mesh/DCS will not suffer. [00164] Thus, the traffic cop node calculates the balance ratio of each mesh periodically. Periodically may be as frequently as each time an addition or deletion to the DCS is made, and each time a node reports itself completed. When the balance ratio of the DCS increases above a certain threshold, completed nodes are removed. When the balance ratio of the DCS decreases below a certain threshold, new upload sources are added to the mesh. Balance ratio thresholds are set using an algorithm in the traffic cop node. A range for the balance ration might be a minimum of 1.20 and a maximum might be 1.40. More accurate balance ratio calculations can be made by summing the average bandwidths reported by each node rather than using generic averages for each speed group.

[00165] When the mesh balance ratio exceeds the upper threshold, nodes will be removed. Nodes with the greatest number of shared files will be removed first. This helps to keep the greatest variety of content available at all times. A popularity index for each node could also be used. The popularity index is calculated as the sum of the popularity indices of each piece of content that node had shared. A popularity index for a piece of content would be proportional to the number of nodes sharing it. [00166] The traffic cop communicates with nodes via mesh specific CCIL messages (described in greater detail below), XML documents specifying lists of connections that need to be established/broken within that mesh. CCIL messages allow the traffic cop to relay instructions to any number of nodes within a mesh regarding how their connections should change. On-demand meshes are supported by performing the following steps: 1. Every node, when logging on to the system, will automatically insert itself into a holding mesh (see Fig. 3). The holding mesh is a non-transfer mesh (no content is being transferred within it), and it exists only to provide message propagation routes to nodes. In this way, messages can be communicated to a node when it is time to transfer a node into a mesh/DC S. a. All firewalls will be connected to two non-firewalls (buddies), and these buddies will serve as relay points for future messages directed to the firewalls. b. Non-firewalls will be given connections to firewalls as buddies are needed, but many non-firewalls will not need to be given connections in the holding mesh, (since the non-firewall nodes can be reached directly with future messages) 2. To notify a target node that the target node should join a mesh, an insert message is embedded in a holding mesh CCIL. The traffic cop node can build this CCIL message just like any existing topology message in the holding mesh, but the insert message is embedded in the XML section intended for the target node. [00167] A DCS is initiated when a node requests access to a resource, e.g. file, stream, data etc., for which no DCS currently exists. A node using NIS and content provider administration tools is directed to an appropriate traffic cop node of the plurality of traffic cop nodes. The content provider administration tool uses and Content Management Database (CMDB) to determine which traffic cop node is appropriate. If no traffic cop node exists then a new traffic cop node is cloned to handle the topology of the new DCS. The traffic cop node makes a determination that it needs to start a new DCS and insert the requesting node into the new DCS. In the case of a file transfer DCS, the traffic cop node also determines that one or more servers/root nodes/seed nodes are also needed in the new DCS. The traffic cop node has access to the CMDB which tracks servers and their locations for all known/available content. The traffic cop node communicates with the CMDB and requests new seed nodes/servers whenever necessary. Seeds (content) can be a web/http/ftp server which will be used to seed data content and provide data to the mesh. In this case, seed nodes are optional. Seed nodes/servers can be other nodes. In this case, the nodes need to be invited (directed to join) the new DCS (mesh) in order to upload the requested file. DCSs frequently overlap. If a node is downloading two or more different files then the node may be in two or more meshes simultaneously. Two DCSs could also be combined if they were separately distributing the same content.

[00168] A list of sources is obtained via communication with the Content Management Database (CMDB) by specifying the content desired by ID and the number of sources (nodes) having that content. It is desirable to obtain many more potential sources than are actually needed, so that an optimal set can be chosen from those returned. For example, if three new sources are needed in a mesh, a query to the CMDB might request thirty candidates having the specified content be returned, so that the three best candidates/nodes can be added to the mesh. A request to the CMDB for even more candidates could be made so that a list is available in the traffic cop node and a subsequent query to the CMDB need not be made.

[00169] To choose the optimal nodes/sources from a list, the following algorithm is used:

1. Sort the candidate sources/nodes by a. primary: ascending[«wrøber of meshes a node is in] b. secondary: ascending[popularity index] 2. Iterate the possible sources in order and examine each as follows: a. If the chosen source is in less than the maximum number of meshes (e.g., 5) then add the source/node to the current mesh, otherwise: i. Determine which of the node's current upload-only meshes has the highest balance ratio, and determine what that mesh's balance ratio would be if the node were removed, ii. If the balance ratio of the mesh would still be above the minimum balance, remove the node from that mesh and add it to the current mesh. b. If the desired number of sources has been obtained, then exit the loop, If the desired number of sources/nodes has not been obtained then continue looking for more nodes/sources. 3. If the desired number of sources has not been able to be located, then obtain another (non-overlapping) set of sources from the CMDB and go back to step 1. 4. If no more sources exist then a limited number of sources of this content are in high- demand across the meshes. It is best to spread the nodes/sources thinly (as evenly as possible) until their content propagates. To accomplish this: a. Repeat the entire above process, but ignoring the maximum mesh requirement of step 2a. b. Extend step 2a to balance out nodes that are in high demand by adding part iii as follows: iii. Repeat (go back to i) until this source is back down to a maximum number of meshes or until all of its meshes have been checked. [00170] The Topology Service (resident in the traffic cop node) is responsible for processing all connection change Information (CCl) Lists (CCILists) as they arrive. It will operate in its own thread, and it will run a repeating loop, which waits for a

CCIMessage to arrive, processes it once it arrives, checks firewall timeouts, and repeats itself.

[00171] The Topology Service must maintain the following values in memory: Last Processed CCIList - The last valid CCIList message processed. Connection List - A list of all active transmission connections. Because more than one content transmission can be active at a time, the list is grouped by Universal Resource Node (URN). For example: • ConnectionList o URNConnectionList 0 ^■ URN 0 - identifier of the download 0 ^■ Connection 0 ^■ Connection 1 ■ ^■ Connection n o URNConnectionList 1 ^■ URN 1 - identifier of the download 1 ■ Connection 0 ■ Connection 1 ■ ■ Connection n o URNConnectionList m ■ URN m — identifier of download m ■ Connection 0 ■ Connection 1

■ Connection n

[00172] When a new CCIList is received, the following CCl processing algorithm is used:

1. If this is a new incoming connection, add the connection to the connection list for the URN specified in the connection handshake. 2. Validate the timestamp by checking if it is greater than lastProcessedTimestamp. If it is not, perform the Connection Cleanup Algorithm and dispose of the message. 3. Validate the signature using the RSN public/private key. If it fails, dispose of the message. 4. Find the ConnectionChangelnfo object in the CCIList which matches the local Session ID for this URN. Store it in localChanges, and if no valid entry is found, store null. Remove this CCl object from the CCl List. 5. If localChanges is not null, for each connection listed in localChanges: If the session ID/URN does not match that of an existing connection, initiate a connection to the specified node as follows: o If the node is specified as outgoing, initiate a direct connection, and add it to the connection list. o If the node is firewalled, initiate a buddy connection. 6. For each existing connectiqn that has a corresponding CCl entry in the CCl List, propagate the connection change message to that connection. 7. Perform the Connection Cleanup Algorithm as follows: For any existing connection for this URN that is not listed in localChanges, remove the connection from the connection list, set the connection's isClosingGracefully flag to true, and transmit a BYE message to the connection.

[00173] To initiate a connection, a separate thread must be used. The process above does not block or wait for the results of the connection. Connection errors will be handled by the connection code itself. The topology manager assumes that all connections are successful, because unsuccessful connections will trigger other events (deletions) which will lead to a new CCl object being received.

[00174] When a connection is no longer necessary, the topology manager initiates transmission of a BYE message. All nodes must obey the following with regards to this message: 1. When a BYE message is received, a boolean flag isClosingGracefully is set to true, and a BYE R.ESPONSE message is transmitted back. 2. When a BYE_RESPONSE message is received, the connection is closed. [00175] The topology service can be illustrated using flowcharts. Fig. 4 is a topology service flowchart depicting the creation of a connection between a traffic cop node and a node specifically highlighting the communications between the traffic cop node and the node from the perspective of the traffic cop node. At step 205 the traffic cop node receives a connection request from a node. The traffic cop node responds to the node at step 210 by sending the node an acknowledgment of the connection request. A connection is established at step 215. The traffic cop node then receives an access request from the node as well as the node's access information, such as speed, upstream and downstream bandwidth, id, speed, subnet classification, firewall status etc. at step < 220. The traffic cop node then validates the received access information at step 225 and grants or denies access to the node at step 230. Based on a grant of access, the traffic cop node sends the node access information for the node's connection list including when the node becomes a peer or neighbor node. The above flowchart assumes a grant of access. If access is denied then the connection will be deleted. [00176] Fig. 5 is a topology service flowchart depicting the creation of a connection between a traffic cop node and a node specifically highlighting the communications between the traffic cop node and the node from the perspective of the node. At step 305 the node sends a connection request to the traffic cop node and then receives an acknowledgment of the connection request from the traffic cop node at step 310. A connection is established at step 315. The node then requests access to a file set and provides its access information, such as id, speed, bandwidth, firewall status, subnet classification etc. to the traffic cop node at step 320. The traffic cop node then grants or denies access upon validation of the node's access information. Based on a grant of access, the node receives access at step 325. The above flowchart assumes a grant of access. If access is denied then the connection will be deleted. [00177] Fig. 6 is a topology service flowchart of the creation of a connection between a node and a peer node from the perspective of the peer node. A node and a peer node are both nodes. The terminology is used merely to distinguish between the two nodes. A "first node" and a "second node" could just as easily serve to distinguish between the two nodes. At step 405 the peer node receives a connection request from a node and responds at step 410 with an acknowledgment of the connection request. A connection is established at step 415. The peer node then receives the node's access information, such as id, speed, bandwidth, firewall status, subnet classification etc. at 420. The peer node validates the node's access information at step 425 and based on validation and a grant of access, propagates the access information and connection list to its peer nodes at step 430. Connections to the peer node's peer nodes are established or broken as needed based on business rules and the establishment of a connection between a node and this peer node. The above flowchart assumes a grant of access. If access is denied then the connection will be deleted. [00178] Fig. 7 is a topology service flowchart of the creation of a connection between a node and a peer node from the perspective of the node. At step 505 a request is made to establish a connection and a response is then received acknowledging receipt of the connection request at step 510. A connection is established at step 515. A determination of the chunk size of the file set is determined at step 520 and an Access Map is generated at step 525. A Request Map is generated at step 530. The Access Map, access information and connection list including file set information is sent to affected peer nodes at step 535.

[00179] Fig. 8 is a topology service flowchart of the deletion of a node from the DCS from the perspective of the traffic cop node. At step 605 the traffic cop node receives a request from a node for that node to leave/disconnect from the DCS. The traffic cop node then sends the connection list for the affected peer nodes to the disconnecting node at step 610.

[00180] Fig. 9 is a topology service flowchart of the deletion of a node from the DCS from the perspective of the node. At step 705 the node sends a request for permission to leave/disconnect from the DCS to the traffic cop node. The traffic cop node sends the node the connection list for the peer nodes that will be affected by the nodes disconnection at step 710. The node thereafter propagates the connection list for the affected peer nodes at step 715 and disconnects from the DCS at step 720. [00181] The routing service is responsible for initiating the generation and transmission of Request Maps and Access Maps to neighboring peer nodes whenever necessary. Access Maps are propagated between nodes to map out the locations of different data chunks in the DCS. Request Maps are propagated to maximize the data flow throughout the DCS. [00182] The routing service will run in its own thread, and it is capable of handling routing for multiple transmissions simultaneously. Thus, every node in the DCS can receive the file or portions of the file simultaneously. The software implementing the present invention is structured so as to run or process multiple threads simultaneously. This allows communication over the ports to occur simultaneously so that different messages over different ports occur simultaneously and on a non-interfering basis. When an applicable message arrives (Access Map, Request Map, data chunk), it will be added to a message queue. The routing service thread will repeatedly pulls messages from the queue and process messages from the queue.

[00183] In all known peer-to-peer (P2P) systems conceived prior to the present invention, a file (piece of content) could be transmitted from one client to another client ("peer-to-peer" or "P2P") in its entirety. The present invention adapts and configures peer-to-peer file transfer to the packet, or "chunk" level. The packet level is a logical breakdown of a file into many packets/chunks so that each chunk may be retransmitted by a client as soon as it is received. In previous P2P systems such as Napster, for example, a client would have to receive an entire file before it could retransmit the file to another node. Because the efficiency of a P2P system is often limited by the amount of client upstream bandwidth that is utilized in the transfer, and the amount of time necessary to disseminate each file, these earlier systems are considerably less efficient than those employing the present invention. [00184] This can clearly be seen by the following example regarding the speed of transfer and re-transfer of a file. The speed of simultaneity of the many embodiments of the present invention depend upon on a number of factors. These factors include, for example, how much data a client node waits for, or receives, before deciding to retransmit, and the chunk size (packet size) of the portions of the file being received and re-transmitted to another node. In the following example, server S will transmit a file to client A and client A will transmit the file to client B.

S -> A -> B

For example, assuming the file is 10 MB in size, and transmissions between all nodes occur at a rate of 100 KB/sec (EXAMPLE 1):

EXAMPLE 1(a) - The old way: S transmits the entire file to A, seconds 0 through 100; A transmits the entire file to B, seconds 100 through 200; Total time: 200 seconds.

EXAMPLE 1(b) - The new way: Choose a logical chunk size of 100 KB;

S transmits 100 KB of the file ("chunk 1") to A ), during second 1;

S transmits a different 100 KB ("chunk 2") to A, while simultaneously A transmits chunk

1 to B, during second 2;

S transmits a different 100 KB ("chunk 3") to A, while A transmits chunk 2 to B, during second 3; ... and so on;

S transmits the final 100 KB ("chunk 100") to A, while A transmits chunk 99 to B, during second 100;

A transmits chunk 100 to B, during second 101. [00185] Thus, in Example 1, the entire file propagates to both nodes A and B in 101 seconds, saving 59 seconds of transmission time when compared with conventional systems.

[00186] Fig. 10 is a state transition diagram of the routing process. Commencing at state SO four events can occur. If a new node connects to the DCS then the response is to send the new peer/node an Access Map and proceed to state S 1. From state S 1 a server can receive an Access Map from the new node. The response to this event is to update the

Access Map and the Request Map tables (up to 7 hops) and return to state S 1 and determine which nodes/clients get updated Access Maps and Request Maps, send the updated Access Maps and Request Maps to all appropriate peers/nodes and request a chunk from a node if available. From state SO a chunk can be received. In this case the response is to request another chunk from the same node and update the Access Map and Request Map tables and proceed to state S2 from state S3. From state SO an existing per/node may be disconnected. In this case the response is to update the Access Map and Request Map tables and proceed to state S2. From state SO, a watchdog timer may trigger an update/regeneration of the Access Map and Request Map tables. [00187] Access Map Information is transmitted as a compressed table of chunk ranges in one exemplary embodiment, Access Map Information may be implemented as follows:

• Chunk sizes are determined using a common algorithm based on file set size. The larger the file set, the larger the chunks are. • An Access Map table is maintained in memory (includes disk, hard drive etc.) such that the size of the Access Map table is equal to the number of chunks in the file set. • Each entry in the table is specified as follows: -l=this node has no known accessible route to this chunk 0= this node has this chunk locally 1= this node has access to this chunk 1 hop away • 2= this node has access to this chunk 2 hops away n= this node has access to this chunk n hops away • To transmit Access Map Information, the Access Map table is compressed prior to transmission and uncompressed upon receipt. • All Access Map updates are' transmitted as differences from the previous transmitted Access Map state to conserve bandwidth. Nodes, therefore, effectively transmit only changes to the Access Map.

[00188] Access Map Information is transmitted among nodes to convey to which chunks (portions) of data in the file set that node has known access. Each node will keep an Access Map representation in memory which will reflect the last map sent to that node. When two nodes connect, each should transmit its Access Maps to the other node as soon as possible. Updates (patches) should be sent whenever changes occur. In addition to generating its own Access Map and Request Map, a node generates an Access Map and a Request Map for each node (neighbor/subnode/peer) to which it is directly connected. These neighbor/subnode/peer Access Maps and Request Maps are communicated as appropriate to other nodes (neighbors/subnodes/peers) to which the node is connected. This is done in order to propagate access information and requests for chunks of a file farther than just the directly connected nodes. These Access Maps and Request Maps for neighbors/subnode/peers are called Neighbor Access Maps and Neighbor Request Maps. [00189] For node C to generate an Access Map to be transmitted to another node N, Map_c→N: • Initialize all indices/entries to — 1, where an index represents a chunk. • Set any indices, which correspond to C's local chunks, to 0. • For each connection M other than the one to which this map is directed: o For each chunk i in M's Access Map MapM[i] where MapM[i] > 0: ^■ If Mapc→N [i] < 0, set Mapc→N [i] = Map_M[i] + 1 ^■ Else if Mapc→N [i] > 0 and Map_M[i] + 1 < Mape→N [i], set Map_c→N [i] = Map_M[i] + l

[00190] The goal of a Request Map is to facilitate the transmission of chunks of a file set between nodes via connections so that each node receives every chunk in minimal time. Chunk transmission is driven by the Request Map messages, which can be interpreted as "flow requests". By attempting to download chunks that satisfy neighbor "flow requests", data will flow across the DCS in the direction of the nodes that need it.

[00191] Request Map Information is transmitted as a compressed table of chunk ranges, and may be implemented as follows: • Chunk sizes are determined using a common algorithm based on file set size. The larger the file set, the larger the chunks are. • A Request Map table is maintained in memory such that the size of the Request Map table is equal to the number of chunks in the file set. • Each entry in the Request Map table is specified as • 0=this node does not need this chunk • l=this node needs this chunk • To transmit the Request Map Information, the Request Map table is compressed prior to transmission and uncompressed upon receipt. • All Request Map updates are transmitted as differences from the previous transmitted state to conserve bandwidth. Nodes therefore effectively transmit only changes to the map.

[00192] Request Map Information is transmitted among nodes to convey which chunks of data in the file set the node needs. Each node will keep a Request Map representation in memory which will reflect the last map sent to that node. Once an initial handshake protocol is complete, any node may send Request Maps to another node. Updates (patches) should be sent whenever changes occur.

[00193] The default Request Map is generated using the local access map. For each chunk that is held locally, the Request Map entry will have a value of 0, and for each chunk that is not held locally, the Request Map entry will have a value of 1. [00194] A node must store the most recent Request Map received from each of its neighbor nodes. These Request Maps should be frequently validated against the local Access Map to determine if some needed chunks are available locally. If a request cannot be fulfilled locally, then it must be forwarded to another node. [00195] To determine which node to which to forward a request, a check is first made for any pending forwarded requests. If any current forwarded requests intersect the new one (have any common needed chunks), then forward the intersection of the requests instead. If there are no current forwarded requests that overlap, choose the connection that can fulfill the request in the minimum number of hops (determined from its Access Map) or the most needed chunk. There may be additional schemes used to select the next requested file chunk. [00196] When storing forwarded maps, each node also tracks which connection or connections contributed to the forwarded request. A forwarded request is removed when all chunks in the request have been received or all nodes contributing to the request have disconnected. [00197] The routing service must maintain the following values in memory: Local

Access Map List (including a list of local Access Map/Universal Resource Name (URN) combinations) and a Peer Map List (a list of each known node, and its information, for example, Session ID, URN, Last sent Access Map, Last sent Request Map, Last received Access Map, Last received Request Map, Forwarding target, and Last chunk requested) [00198] The following event notifications are relevant to the routing service: 1. New peer connection has been established - create a new record in the peer map list, and initialize: a. Session ID = the peer's session ID b. URN = the peer's URN c. Last sent Access Map = null d. Last sent Request Map = null e. Last received Access Map = null f. Last received Request Map = null g. Forwarding target = null h. Last chunk requested = -1 2. A peer is to be disconnected via disconnectingPeer by removing the disconnecting! *eer 's record from the peer map list and checking all remaining entries in the peer map list, and if any forwarding target equals disconnectingPeer, set the forwarding target to null. 3. If a chunk (chunkNum) is received then set local AccessMap[chunkNum] = 0, set last chunk requested = -1 for the peer, initiate a new chunk request for the peer, and send Access Map updates to all peers except for the peer from which the chunk was downloaded. 4. If a chunk (chunkNum) has been successfully sent to a neighbor node then update the peer's last received Request Map to reflect the change (lastReceivedRequestMap[chunkNum] ~ 0). 5. If a Request Map (newRequestMap) is received from sourcePeer then update the last received Request Map for sourcePeer to newRequestMap. 6. If an Access Map (newAccessMap) is received from sourcePeer then update the last received Access Map for sourcePeer to newAccessMap. [00199] At periodic intervals (e.g., every two seconds), new Access Maps and Request Maps are generated for each node/peer. The update is only transmitted if it differs from the last map sent to the peer. Also, new chunk requests may be made at this time, since Access Map updates may allow chunks to be requested that could not previously be requested. Thus every two seconds incoming requests are validated, Request Maps are regenerated for every peer and transmitted if the regenerated Request Map differs from the last Request Map sent, Access Maps are regenerated for every peer and transmitted if the regenerated Access Map differs from the last Access Map sent and new chunk requests may be made. [00200] Before outgoing Request Maps can be generated, all incoming Request Maps must be validated. This is necessary since unserviceable incoming requests can be forwarded, which will change the outgoing requests. To validate an incoming Request Map:

1. If the request is empty (all bytes are 0), set the request to null, and set the forwarding target to null. The validation passes. 2. Else if any chunk is held locally which fulfills the request (localAccessMap[i] is 0 and request[i] is -1), then the validation passes 3. Else if the forwarding target is not null, then the validation passes (already forwarded). 4. Otherwise the validation fails. If the validation fails, then the request must be forwarded: a. Look for an overlapping forwarded request: i. Loop through all entries in the Peer Map List for this URN in which the forwarding target is not null. It is unnecessary to check the peer from which this request originated. ii. For each non-null URN found, check to see if its incoming Request Map overlaps the Request Map that is to be forwarded. (It overlaps if any common index exists such that both Request Maps are -1 at that index.) iii. If a non-null URN is found, then forward the current Request Map to the same peer. b. If an overlapping forwarded request is not found then find the best peer to forward this request to: i. Loop through all entries in the Peer Map List for this URN that are not forwarding targets. (Note: this is easier to do if a list of forwarding targets is compiled while looking for overlaps in step a above. That way, it is only necessary to check against that list.) Also, it is unnecessary to check the peer from which the present request originated, ii. For each non-null URN found, compute its ability to fulfill the request one of the following ways: a) Find the minimum number of hops to any chunk needed in the request. a) Compute the hop sum of all chunks in the request, adding the maximum hop count for any chunk that is inaccessible through the peer. c. Forward the request to the node that qualifies best to service the request based on the results of step b. [00201] When a chunk download completes, Access Map updates should be sent immediately to all peers except the peer from which that chunk was downloaded (rather than waiting until the next periodic, e.g. two second, interval). This method is preferred because it allows peers to become aware of chunks that can be downloaded as soon as chunks become available. [00202] The following algorithm is used to generate a peer Access Map: 1. If the last Access Map received from the peer shows that it already has every chunk, it is unnecessary to send a new Access Map. 2. Create an Access Map newAccessMap, and initialize it to all -Is. 3. For every chunk index chunkNum that is already held locally (localAccessMap[c zw«ANw»7] is 0), set newAccessMap[chunkNum] = 0. 4. For every element of the peer Access Map in which the URN matches the target peer URN and the session ID does not match the target peer session ID: a. Let peer Map be the last Access Map received from the peer. b. For every index for which newAccessMap[index] is -1, set newAccessMap[index] = peerMap[index] + 1. c. For every index for which newAccessMap[index] > peerMap[index] + 1, set newAccessMap[index] = peerMap[index] + 1. The following algorithm is used to generate a peer Request Map: 1. Create a Request Map newRequestMap, and initialize it to be identical to localRequestMap. 2. For every element of the peer Request Map list in which the URN matches the target peer URN and the forwarding target is the target peer: a. Let peerMap be the last Request Map received from the peer. b. For every indexfor which newRequestMap[index] is -1 and peerMap[index] is 0, set newRequestMap[index] — 0 (note that this is the Request Map intersection routine) [00203] Chunk requests can be made in two ways: 1. When a chunk is successfully received from a peer, attempt to send a new chunk request to that peer. 2. After new Request Maps and Access Maps are generated, check all peers to see if a new chunk request can be made. This is necessary since a chunk download may complete and the peer may not be aware of additional chunks to request at the time, but become aware of valid chunks to request later when an Access Map is received.

[00204] The following algorithm is used to make chunk requests: 1. Insure that the last requested chunk for the peer is -1. If it is not, then a chunk request is already pending. 2. Create an empty list possibleChunks. 3. For every chunk index, add the index to possibleChunks if all of the following are true: a. The chunk is not held locally. (localAccessMap[index] is -1) b. A peer has the chunk locally. (lastReceivedAccessMap[index] is 0). c. The chunk is not pending from any other node. (lastRequestedChunk is not index for any peer with the current URN) d. The chunk was in the Request Map last sent to the peer. (lastSentRequestMap[index] is -1) 4. If no valid chunks are found (possibleChunks.sizeQ is 0), repeat step 3, but exclude requirement d above. This means that the peer is not yet able to service its forwarded request, but it does have chunks that are needed locally. 5. For a static transmission, choose a chunk index at random from possibleChunks, and request it. 6. For a streaming transmission, choose a chunk at random from the subset of possibleChunks that includes only chunks that are within streamingBuffer a predetermined number of chunks of the first element of possibleChunks, for example within ten chunks .

[00205] While topology and routing can be used independently of one another or more correctly with other routing and topology schemes, as indicated earlier they are most effective when used together as will be illustrated below. Referring first to Figs. 11(a) through 11(d), Node N requests authorization to enter/connect to the DCS for purposes of file transfer. In this case as will be described below the DCS is a mesh. In Fig. 11(a), node N connects to traffic cop S. In Fig. 11(b), peer N requests access to a file. Traffic cop S then grants access and forwards access information, file information, a signature and a connection list to peer N as shown in Fig. 11(c). Peer N then disconnects from traffic cop S as shown in Fig. 11(d). [00206] Once it has been established that peer N has been granted access to connect into the DCS, peer N inserts itself into the DCS' by connecting to one or more peers that are already in the DCS. Peer N forwards access information and connection list information that was received from traffic cop S. As shown in Fig. 12(a) peer N connects to peers A and B. Connection list information is then sent to all affected peers. Affected peers are peers that must change their connections as a result of the newly established connection with peer N. Fig. 12(b) show peer B forwarding connection list information to peer G because that connection will be broken. Also shown is peer A forwarding connection list information to peer R because that connection will also be broken as a result of the entry of peer N into the DCS. Fig. 12(c) shows the breaking or disconnection of peer R from peer A and peer G from peer B. Fig. 12(d) shows the formation of a new connection between peer R and peer G as a result of peer N entering the DCS. The information for the new connection was sent to peers R and G along with the disconnection information. All of the above access information and connection lists are further described below in the description of the Packet Chain Protocol and the object schemas. [00207] Using a smaller DCS, Fig. 13(a) illustrates peer A notifying each neighbor by sending Access Maps of all chunks of the file that peer A has and what chunks of the file that it and its neighbors need. Referring to Fig, 13(b), peers B, C, D and E request chunks of the file that they and their neighbors need from peer A via Request Maps sent to peer A. Peer A then requests chunks from its neighbors (not shown) and forward chunks of the file that it has that are needed by peers B, C, D and E to them. For example, peer C might have neighbors CI and C2 (not shown) and peer C forwards a Request Map to peer A that includes the chunks that peers CI and C2 need as well. Referring now to Fig. 13(c) , if peer A does not have any chunk of the file that peer B needs then peer A notifies peer C, for example, of chunks of the file that both peer A and peer B need via updated Request Maps.

[00208] Upon the completion of a file transfer, peer N connects to traffic cop S and notifies traffic cop S that the file transfer is complete and that peer N wishes to disconnect from the DCS. Traffic cop S then sends connection list information to peer N. Peer N then disconnects from traffic cop S. Connection list information is sent to all affected peers/nodes that must change their connection because peer N wishes to disconnect from the DCS. Connections that are not needed are broken including all of peer N's connections. PeerN exits the DCS and any new connections required as a result of peer N leaving the DCS are formed. [00209] A node is capable of efficiently downloading a file set from one or more sources. The node will facilitate the download by connecting into a network DCS (dynamic connection structure), in which it can receive non-overlapping chunks of the file set from its neighbors/peers and send chunks to its peers/neighbors. The protocol of the present invention to facilitate this process is denominated Packet Chain Protocol

(PCP) and is part of the MMTP. [00210] A file set refers to one or more files which should be downloaded. It is assumed that the files have a predetermined sequential order (used for indexing data chunks), but they do not necessarily need to arrive in that order. A file set is data/content which includes content that is digital and/or multi-media.

[00211] A specification of the file set should include the number of files in the file set, their order, their names, their hash values, their relative paths, their sizes and a flag indicating whether the file set should be delivered sequentially (streaming) or segmented

(downloads).

[00212] Chunks of data will be transmitted in chunks of a predetermined size.

Receiving nodes will always tell transmitting clients which chunk should be sent next. To determine which chunk to request next: • If the last Request Map transmitted to the node was a forwarded request, find all chunks that the peer has locally that are in the last request, choose one at random. • If the last Request Map was not a forwarded request, find all chunks that the peer has locally that the node does not have locally, choose one at random. • If streaming, choose either of the above at random from only the lowest n qualifying chunks, where n is the streaming buffer size. • Never request a chunk that is pending from another peer.

[00213] Whenever an event occurs which requires some reorganization of node connections in the DCS, connection change messages are used to communicate the changes to the affected nodes.

[00214] It is assumed that a non-firewall node's IP address and listening port can be extracted from the node's ID to conserve bandwidth. Because firewalled nodes cannot accept incoming connections, their node IDs are arbitrary and are used for identification purposes only. [00215] The connection change message has the following structure: ConnectionChangelnformationList includes a timestamp, a signature and a connection list which contains connection change information (ConnectionChangelnfo) for each node that is affected by the current changes. ConnectionChangelnfo includes a session ID, Client Connectionlnfo includes, for example, for each of the node's connections, a session ID which is the unique identifier of the corresponding node's download session, a source URL which is the URL location of the node (not specified for firewall nodes) and buddy information which is the session ID and URL of the node's buddies if the node is firewalled. [00216] This object is always generated by the traffic cop node in response to an event notification, and it is passed to the node notifying the traffic cop node of the event. Because a connection change message contains CCl objects for each affected node, only one message is sent by the traffic cop node. The message is then propagated through the affected nodes. When forwarding a connection change message, each node saves bandwidth by retransmitting only the necessary parts of the message.

[00217] An exemplary Connection Change Algorithm/process is invoked when a node receives a CCl message, as follows: 1. Validate the timestamp and signature, dispose of the message if it fails either validation. 2. Find the ConnectionChangelnfo object in the structure which matches the local Client ID (local CCl) and remove the CCl from the object. (This object represents changes to be applied locally, and needs not be propagated any further.) 3. For each CCl in the message if the session ID matches that of an existing connection, propagate the connection change message to that connection. 4. For each connection in the local CCl if the session ID does not match that of an existing connection, initiate a connection to the specified node and propagate the connection change message to the node. 5. For each existing connection not in the local CCl, disconnect. [00218] Forming connections includes both initiating a connection and accepting a connection initiated by another node. In initiating a connection, if the connection is already pending as an incoming request, it can be taken directly from the incoming connection manager. When a new connection is required to a non-firewalled node, the node's IP and port are extracted from its ID, and the connection is formed directly. For a firewalled node, the buddy is contacted, the connection change message is transmitted to the buddy, and the buddy is disconnected.

[00219] In accepting a request for a connection, when a node connects, a standard handshake is used to validate that the client is OK. A new incoming connection will always transmit a connection change message initially. The connection should be held temporarily as a pending incoming, so that it may be used for transmission once it is validated.

[00220] Traffic cop node protocol messages are messages that are exchanged between the traffic cop node and a node. A Request Add Message is sent to the traffic cop node by a node desiring to connect to the network. A Request Add Message includes, for example, the node ID which is the unique identifier of the node, the URN which is the unique identifier of the file set being requested, firewall status (firewalled or non- firewalled), the node URL which is the node's URL location and is not necessary for firewalled nodes, the average upstream bandwidth which is the node's average upstream bandwidth and the logon which is the ID/Password combination.

[00221] The Successful Add Information Message is sent from the traffic cop node to any node after a successful Request Add. The Successful Add Information Message includes, for example, the session ID which is a unique identifier generated by the traffic cop node and which will identify the node during the transmission, the signature which is an authorization code for the node to enter the dynamic connection structure (DCS), file set information and connection change information.

[00222] The Can Not Add Message is sent from the traffic cop node to a node when an add request fails. The Can Not Add Message includes, for example, an error code which is a number representing the type of error which prevented insertion of the node into the DCS and an error description which is a description of the error. [00223] The Request Connection Update Message is sent to the traffic cop node by a node, whose connections are outdated or have been lost. This is a request for a current ConnectionChangelnfo object and includes, for example, the session ID and the URN. [00224] The Connection Update Message is sent from the traffic cop node to any node after a connection update request and includes, for example, connection change information.

[00225] The Delete Node Message is sent to notify the traffic cop node that a node should be deleted from the DCS. The Delete Node Message includes, for example, the delete ID which is the ID of the node being deleted from the DCS, the node ID which is the ID of the node requesting the deletion from the DCS, the delete code which is the number representing the type of situation which caused the deletion and the delete description which is a description of the conditions which caused the deletion from the DCS. [00226] The Delete OK Message is sent from the traffic cop node to a node after a delete request has been completed and includes Connection Change Information.

[00227] Every node-to-node connection should start with a handshake in which each node transmits their Node/Client ID and signature. Each node has the ability to validate that the other node's signature was originally generated by the traffic cop node. [00228] Encryption mechanisms are optionally available, which guarantee that transmitted data could not be inteφreted by an outside party.

[00229] The dynamic connection structure is always maintained by only the authorized traffic cop node, and nodes are never able to cause unauthorized connection changes. Every ConnectionChangelnfo object is signed by the traffic cop node, and nodes will ignore messages without valid signatures. [00230] On an initial connect, the ConnectionChangelnfo object is always be transmitted from the connecting node to the node accepting the connection. The accepting node will then respond with its Access Map. Updates (patches) are sent periodically whenever changes occur, with a minimum frequency (e.g. every ten seconds). [00231] Currently the most effective content distribution systems are positioned at the edges of the Internet and consist of many high powered computers that have access to an almost limitless bandwidth. This works well is most cases, but is very costly to the content provider and the Quality of Service (QoS) to users may be degraded during peak use. The Packet Chain Protocol (PCP) of the present invention can reduce the amount of hardware and bandwidth needed to distribute content to millions of people on the Internet without reducing/diminishing losing any QoS. This is accomplished by forming a dynamic content structure (DCS) network of computers and utilizing their unused bandwidth (upstream as well as downstream) and resources. By linking computers together maximum efficiency for content distribution over the Internet can be achieved. Use of the PCP of the present invention, thus, allows users to receive content faster and allows content providers to use fewer resources.

[00232] Below are common object schemas that will be used in messages between network elements. File Set Info (FSI) and File Info (FI) object schemas are used for routing. The balance of the common object schemas are used for topology. [00233] The File Set Info (FSI) contains information about the specified content. The content can contain one file or a list files. If the FSI has more than one file it is considered a bundle and the node will receive all files that are contained in the bundle. It is assumed that the files have a predetermined sequential order (used for indexing data chunks), but the files do not necessarily need to arrive in that order. An exemplary FSI object schema is illustrated below. [00234] Fig. 27 shows a system of ports and sockets adapted for the simultaneous/ contemporaneous reception and transmission of files according to the invention [00235] Fig. 28 shows the relative positions and transport layer relationships between and among the Multimedia Transport Protocol and applications of the present invention.

<delivery> is the type of transfer to use when delivering a FSI, for example, download and streaming. <hash> is the digest of the entire FSI. If there is only one file in the FSI it is equal the file's hash. <file> is the list of files in the file set.

[00234] The File Info (FI) contains information about a specific file. If the <φath> is present it contains what relative directory to save the file in. (For example, rpathX downloads/ docs, root=C:I client means that the file would be saved/stored in C:/client/downloads/docs). An exemplary FI object schema is illustrated below.

<index> index of this file in the FSI <name> is the name of the file. <hash> is the digest of the file. <size> is the size of the file. <φath> is the relative location to save the file. <type> is the MIME type of the file. <chunkHashes> the file is broken up into small pieces and each piece is hashed . This field is the concatenation of those hashes. <metadata> is any additional information about the file.(For example, if the MIME type is mp3 this field might contain ID3 tag information.) Connection Change Info was described above.

<sessionid> is a unique id that identifies a new session with the traffic cop network element. <src> is the source URL For example, pcp://123.456.44.44:5457 <connection> is a list of connections that <src> is connected too. <sessionid> is a unique id that identifies a new session with the traffic cop for the specified connection. <src> is the source URI. For example, pcp://123.456.44.44:5457 <root> identifies that this connection is to a root server node. <buddy> identifies that this connection uses a buddy connection, not a direct connection.

[00235] A Connection Change Info List (CCIL) has a list of Connection Change Infos that a servant uses to connect into the DCS. The CCIL also contains other CCIs that are passed to the connected servants so they can be properly connected into the DCS. He CCl and CCIL object schemas pertain to the topology of the DCS.

[00236] The traffic cop network element accepts requests via HTTP(S). There are four types of requests (topological) that the traffic cop accepts: insert into the DCS, delete from the DCS, reposition elsewhere in the DCS and new connection info in the DCS. The response to one of the above requests contains an XML message with the appropriate information. Both the request and response will contain standard HTTP headers along with custom headers. The value for the content-type response header will always be of MIME type xml/text. [00237] In the "insert into DCS" request, a servant makes an insert request to the traffic cop node in order to obtain information on how to be added into the DCS, so that the servant can start receiving the specified content. If the traffic cop node determines that a requesting node's firewall status has not been established, it will make a request back to the node, so that the node can determine its firewall status. [00238] The firewall test is the connection back that is described above. When the traffic cop node attempts to connect back to the node, if it is successful, it assumes that the node is not behind a firewall. If it is a unsuccessful, it is assumed that the node is behind a firewall. [00239] Below are exemplary URL, request headers and the response body schema that are used for an "insert into DCS" request. URL http://<host address>:<listen port>/trafficcop/insert?<urn of content>

Body Data The body of the response will contain an XML message that will have information about how the requesting node can join the DCS and detailed information about the specified content. Once the servant receives this message it can then make the necessary TCP/IP connections to other servants that can deliver the specified content. Insert into DCS Response Schema

Example: Insert into DCS- http://lxsystems.com:2222/trafficcop/insert?urn:shal:EDEDEDEDEDEDEDEDEDEDEDEDED EDEDED GET /trafficcop/insert?urn:shal :EDEDEDEDEDEDEDEDEDEDEDEDEDEDEDED HTTP/1. Kcr><lf>

PCP-Client- Version: 0.3 Need-Firewall-Test: True<cr><lf> Firewall: Unknown<cr><lf>

Listen-IP: 127.0.0. l:3434<cr><lf>

Host: 127.0.0.1

<cr><lf HTTP/1.1 200 OK<cr><lf Remote-IP:23.23.24.24<cr><lf> Content-Type :text/xml<cr><lf> Content-length=1234<cr><lf> <cr><lf> <?xml version=" 1.0">

<!DOCTYPE Message>

<pcp: IDCSR> <FSI><delivery>download</ delivery> <hash> urn:shal:EDEDEDEDEDEDEDEDEDEDEDEDEDEDEDED </hash> <φath>/mydocs/books/</φath> <name>My Book</name> <file> <name>chapter l</name> <hash>urn:shal:VFRVFRVFRVFRVFRVFRVFRVFREDEDEDE </hash> <size>123567</size> <rpath></φath> <type>application/pdf</type> <metadata></metadata> </file> <file> <name>chapter 2</name> <hash>urn:shal :ERTERTERTRTTRTRTRTRTTVFREDEDEDE </hash> <size>456783</size> <rpath></rpath> <type>application/pdf</type> <metadata></metadata> </file> </FSI> </pcp:IDCSR> <pcp:CCI> <clientid>4567</clientid> <src>pcρ ://23.23.24.24:3434</src> <timestamp>123456789</ timestamp> <sig>34jfh5611gllflsllslslgh</sig> <noConnectionChange>Y</noConnectionChange> <maintainConnection>N</maintainConnection> <cci> <clientid>4556</clientid> <src>pcρ://123.45.67.77:5678</src> <timestamp>343455455</ timestamp> <sig>34jfh5611gllflsllslslgh</sig> <noConnectionChange>Y</noConnectionChange> <maintainConnection>N</maintainConnection> cci> <cci> <clientid>5678</clientid> <src>http://12.56.78.55:3452</src> <timestamp>453543555</ timestamp> <sig>34jfh5611gllflsllslslgh</sig> <noConnectionChange>Y</noConnectionChange> <maintainConnection>N</maintainConnection> </cci> </pcp:CCI> </pcp: IDCSR> [00240] In a "delete from DCS" request, a servant makes a delete request to the traffic cop network element so that it (the servant) can be removed from the DCS. A delete request is called when either a servant has received all the specified content or a servant has detected that one of its neighbors has been disconnected. Below are exemplary URL, request headers and response body schema that are used for a "delete from DCS" request.

URL pcp://<host address>:<listen port>/trafficcop/delete?<urn of content>

Request Data - Headers

Body Data [00241] The body of the response will contain an XML message that will have information about how the other peers/nodes that the node being deleted is connected to should form other connections in the DCS. [00242] In the "Reposition elsewhere in the DCS" request, a servant makes a reposition request to the traffic cop network element when it finds itself in a non-optimal position in the DCS. The response information contains information on where a servant can connect to be in a more optimal position for receiving the specified content. Below are exemplary URL, request data header and response body schema that are used for a "reposition elsewhere in the DCS" request. URL pcp://<host address>:<listen port>/trafficcop/reposition?<urn of content>

Re uest Data - Headers

Body Data [00243] The body of the response will contain an XML message that will have information about how the requesting node can reposition in the DCS.

[00244] In the "New Connection Info in the DCS" request, a servant makes a new connection info request to the traffic cop node when it gets into a state where its connection information is outdated and it (the servant) cannot connect into the DCS. Below are exemplary URL, request headers and response body schema that are used for a "new connection info in the DCS" request.

URL pcp://<host address>:<listen port>/trafficcop/newConnecionInfo?<urn of content>

Request Data - Headers Name Value Description PCP-Client-Id 32 byte hex string Unique identifier for a node Body Data [00245] The body of the response will contain an XML message that will have information about new connection information in the DCS.

New Connection Change Information in DCS Response Schema

[00246] The PCP Client is capable of efficiently downloading a file set from one or more sources. The node will facilitate the download by connecting into a network DCS, in which the node can receive non-overlapping "chunks" of the file set from its neighbors and send "chunks" to its neighbors. Below are the messages (PCP object schemas) used to communicate between nodes:

[00247] A PCP client connects itself to the DCS by making an "insert into the DCS" request to traffic cop node for information on what node to connect to and detailed information about the desired content. The traffic cop node responds with an IDCSR message (response body schema) containing this information. The node then extracts the information from the message and goes through the follow steps: 1. Validate the timestamp and signature and dispose of the message if it fails either validation. 2. Find the ConnectionChangelnfo object in the structure which matches the local node ID. Remove the Connection List from the object. (These lists represent changes to be applied locally, and need not be propagated any further.) 3. For each node ID in the Connection List: o If the node ID does not match an existing connected node, initiate a connection. o Extract that node's connection change sub-tree/information from the message and prppagate it to the connected node. 4. If the NoConnectionChange flag is specified as true, verify that the timestamp of the message is greater than the last applied child connection change message timestamp and forwarding the traffic cop node a connection update message.

[00248] If the appropriate data items are validated/verified then the servant makes a TCP/IP request to the non-firewalled servants listed in the response message. If there are any servants behind a firewall a TCP/IP connection is made to buddy servants of the firewalled (target) servants. A message is sent over the buddy connection with information on which neighbor should receive this message. When the target servant finally receives the message it extracts the information needed to make a TCP/IP connection back to the requesting servant. [00249] Fig. 14 is a schematic diagram illustrating connecting to a firewall servant. Servant E needs to connect to Servant A, but a direct connection cannot be established because Servant A is behind a firewall. Servant E then connects to Servant A's buddy Servant B sending a message which should be routed to Servant A. When Servant A receives the message it makes a direct connection to Servant E. [00250] Once a TCP/IP connection has been established between two nodes a PCP handshake occurs to validate the two nodes. The handshake in structured much like a HTTP request:

URL pcp://<host address>:<port>/contentmgr/get?<urn of content> Request Data - Headers

[00251] Once a node has successfully connected to another node, it communicates by sending and receiving PCP object schemas/messages.

Messa e Header

Type: 0x1 Data 0x2 Data Range Change 0x3 Connection Change 0x4 Bye 0x5 Access Map 0x6 Request Map Payload length is the length of the payload following this message header. Data Payload Binary data

[00252] Data messages contain the "chunks" of the desired digital content. The header Payload length defines how much raw data is in a message. Nodes use their neighbor's Access Map and Request Map to decide what "chunks" of data to send to each other. These messages are sent with the payload containing the sequential data. Access Map and Request Map updates are also sent if there are any changes. If the data range changes, a Data Range Change message is sent telling the receiving node that a new range of data is coming.

Data Range Change New Range 4 bytes

[00253] New Range is the new starting offset into the data stream for the next data message. [00254] The Data Range Change message tells the receiving node what the next starting offset into the data stream it is going to send will be. The offset in stored in the New Range section of the message. After the Data Range Change message, data messages can be sent with the appropriate range of data.

New Connection Information Payload Null term [00255] Payload is a null terminated XML string that represents the new connection information. New Connection Info is sent to a node when its connections need to be changed.

[00256] The Bye message is sent to a node's neighbors tell them why it is about to disconnect from them. [00257] Components of the PCP Client solution will be used to track content distribution, and to manage content provider information. Fig. 15 illustrates an exemplary architecture. The content provider administration 9005 interfaces with the database 9010 (described below) as do the plurality of traffic cops 9015a - 9015c ... A node using NIS and content provider administration tools is directed to an appropriate traffic cop node of the plurality of traffic cop nodes. The content provider administration tool uses a Content Management Database (CMDB) to determine which traffic cop node is appropriate. If no traffic cop node exists then a new traffic cop node is cloned to handle the topology of the new DCS. [00258] The content provider administration tools allow providers to manage/monitor their content distribution via a web interface. Content provider administration tools use JSP pages which run on an Apache/Tomcat system hosted by the present invention. A MySQL database can be used to store all provider/content related information. Many traffic cop components will run using JSP pages, but these components will run on processors separate from the content provider administration tools, and they may or may not be hosted as part of the present invention. [00259] An exemplary database is modeled as depicted in Fig. 16 and includes the following exemplary tables: • The Provider Table (10005) lists information about each provider. This table includes provider names, billing information, contact information, etc. • The User Table (10010) lists information about each user. This table is necessary for user security. Each user is associated with a provider. • The Role Table ( 10015) lists the available roles that the Content Management system uses for security. These include: o Provider - The provider role only provides access to add, edit and remove content for the users own provider. Providers also have access to edit the details of their own user account. o Provider-admin - The provider-admin role provides the same functionality as the provider role plus the ability to add, remove and edit user information under their own provider. o System-admin - The system-admin role provides the same functionality as a provider-admin but with the ability to work under any provider in the system. • The User Role Table (10020) associates a user with a particular role. • The Content Table (10025) lists information about a specific piece of content on the network. o URN - for static content, this is the SHA hash of the content, for streaming content, this is a unique identifier automatically assigned to the stream. o Access ID - links to a record in the ACCESS table, which determines the rules for accessing the content. Most content will link to access type = 0, which represents free content that anyone can download. Access types also represent subscription- based content, price-based content, etc. o Provider ID - the provider responsible for the content. o User ID - the user responsible for the record. o Default Delivery Type - specifies whether the content should be delivered static or streaming. A negative one (-1) implies static. A positive number implies streaming, where the number is the streaming buffer size. o Default HS QOS - the default quality of service for high speed users. o Default LS QOS - the default quality of service for low speed users. o Promo Link - this field is used to specify promotional links which display to users while the download is in progress. o Metadata - metadata associated with the content. o Name - a name given to the piece of content. For content with only one file, it defaults to the filename. o Description - a text description of the content. o Status - active, inactive or pending. Active content is available to users. Inactive content may not be downloaded. Pending content is unavailable to the user because it is being processed by the system (SHA generation, for example). o Creation Time - the time the record was inserted.

• The Root Server Table (10030) lists information about each known root server/seed node on the network. A physical server may be referenced by more than one record if the server hosts more than one content object. o Content JD - a link to the content that the root server hosts. o URL - the URL location at which the content can be downloaded. o Type - the type of server. This is usually zero for all servers, implying HTTP reliable server. This field may, however, represent failover servers, or even track user machines that allow uploads. o Description - a text description of the root server. o File ID - a link to the associated file data.

• The Access Table (10035) references the types of download access supported on the network. Most access will be free access (= 0). Other access types include subscription- based content, price-based content, etc.

• The File Table (10040) contains information about each file that is accessible on the network. Content ID is specified as the specific piece of content to which this file belongs. Most often there will be a one-to-one mapping of files to content. (Each content object contains exactly one file). However, a content object may contain multiple files (bundles) which allows multiple file records to exist for the same content ID. o File Index - the index that this file holds in a piece of content. For pieces of content that hold only one file, this will be 0. o File Name - the name of the file. o Size - the size of the file in bytes (initialized to 0 until the file is processed by the system). o URN - the SHA hash of the entire file. o Metadata - metadata associated with the file. o Content ED - the ID of the associated content.

• The Chunk Table (10045) holds information about the individual chunks into which the file is broken. o Chunk Index - the index that this chunk holds in a file. o URN - the SHA hash of the chunk. o File ID - the ID of the associated file.

• The Mesh Table (10050) tracks all meshes that are created. When a traffic cop starts a mesh, a row will be inserted in this table recording the following: o Start Time -the time the mesh was created. o Finish Time - the time the mesh finished. o Status - either active(O), finishing(l) or closed(2). Meshes will always start as active, and move to closed when they finish. If a piece of content is edited, the status for all active meshes will be changed to finishing. Additional active statuses specify that a mesh is full (cannot accept any more nodes.) o QOS - the quality of service ensured within the mesh. For high-speed meshes, this will generally equal the high speed QOS specified in the content table for the serving content. For low-speed meshes, this will generally equal the low speed QOS specified in the content table for the serving content. o Traffic Cop ID -the ID of the traffic cop controlling the mesh. o File ID - the serving file. o Delivery Type - specifies whether the content should is delivered static or streaming. A negative one (-1) implies static. A positive number implies streaming, where the number is the streaming buffer size. • The Traffic Cop Table (10055) tracks all known traffic cops on the network. When a traffic cop starts, it adds a record for itself to this table (if not already present), and set the status to active. When a traffic cop finishes, it updates its record to reflect that it is inactive. o URL - the URL location of the traffic cop. o Status - active or inactive. • The Download Stats Table (10060) records statistics of PCP transmissions. Most provider reports will be run against data in this table. Data is stored with respect to set time slices. One hour time slices are the most commonly used. Records are added to the table for each active mesh at intervals of one time slice. Each record will reflect all statistics gathered by the mesh's traffic cop during that time slice. o Mesh ID - the ID of the mesh to which the stats refer. o Time Collected - the timestamp of the last logged entry for the mesh. o Root Gigs Served - total gigabytes transmitted from root servers to peers that completed the transmission during the time slice. o Peer Gigs Served - total gigabytes transmitted between peers that completed the transmission during the time slice (file data only, not protocol messages). o Avg Transmission Time - average number of seconds nodes/clients spent downloading (aggregate for all clients/nodes completing the transmission during the time slice). o Total Clients Completed - total number of clients completing the transmission. o Total Clients Cancelled - total number of clients cancelled during transmission. o Total Clients Started - total number of clients starting transmission. o Total Clients Deleted - total number of nodes deleted during transmission.

[00260] The content provider administration tool allows providers to manage/monitor their content distribution via a web interface. Content provider administration tools use JSP pages which run on an Apache/Tomcat system hosted by the present invention. This interface is secure (SSL). Its components may be categorized as follows: 1. Login 2. Content Management 3. Account Maintenance 4. Statistics

[00261] The Login screen serves as the entry point to the web system, allowing providers to log in to the Provider Administration System. Unsuccessful logins display an error message to the user and allow them to try again. Successful logins initiate a session for the user, and take the user to the Content Management screen. Multiple independent sessions may exist for the same Provider ID simultaneously. An exemplary Login screen is illustrated in Fig. 17. It should be noted that the User ID for the Login screen does not correspond to User ID in the database, which is a unique numeric key. The User ID for the Login screen corresponds to Login ID in the database, which is a text login assigned manually to new users. [00262] The content management main screen lists all content for the provider in table format. Links to the other areas of the system are provided. For each content object, the name, size, delivery method, current node count, total node count, and creation date are displayed in table format. By default, the table is ordered by name. For each object, buttons allow the user to edit, activate/deactivate, or view statistics for the content. The edit button takes the user to the Content Modification screen. Active content has a deactivate button. Inactive content has an activate button. Pressing the activate button takes the user to the Activate Content Verification screen. Pressing the deactivate button takes the user to the Deactivate Content Verification screen. Pressing the stats button takes the user to the Content Distribution Statistics screen. Also, an Add New Content button takes the user to the New Content Creation screen. Fig. 18 is an exemplary content management main screen.

[00263] The activate content verification screen verifies that a user wants to activate a piece of inactive content. Both OK and Cancel take the user to the Content Management Main screen, but OK activates the content first. Fig. 19 is an exemplary activate content verification screen. [00264] The deactivate content verification screen verifies that a user wants to deactivate a piece of active content. Both OK and Cancel take the user to the Content Management Main screen, but OK deactivates the content first. Fig. 20 is an exemplary deactivate content verification screen. [00265] The new content creation screen allows a user to add new content to the system. The user will be able to enter the content name, URL, delivery type, high-speed QOS, and description. Successful entry will take the user to the Content Management Main screen. Unsuccessful entry will return the user to this screen, and display an error message. Fig. 21 is an exemplary new content creation screen. When a piece of content is added, the contents' Status is set to Pending, and an asynchronous process is started that downloads the actual file and computes the appropriate SHA Hash value(s) for the file. When this process is completed, the Status is set to Active and an email is sent to the user who created the content to alert them of its availability. [00266] The content modification screen allows a user to edit existing content on the system. The screen pre-populates with the current values of the selected content. The user is able to change any of these values except the content name. If the user clicks the apply button and the URL does not change, the changes are applied and the user returns to the content management main screen. If the URL changed, the user is taken to the URL change verification screen. Selecting the cancel button takes the user to the content management main screen. Fig. 22 is an exemplary content modification screen.

[00267] The URL Change Verification screen is necessary because changing root server URLs is potentially dangerous to the system. If the file is different then the user must let the system know. Pressing the "yes" button initiates a background hashing, applies the changes, and takes the user to the content management main screen. Pressing the "no" button just applies the changes and takes the user to the content management main screen. Pressing the "cancel" button returns the user to the content modification screen. Fig. 23 is an exemplary URL change verification screen.

[00268] The account maintenance main screen allows a user to modify user information. Links to the other areas of the system are provided. Initially, only password change functionality will be available. Once we begin tracking more specific provider information (billing, contact info, etc.), it will become editable here. Pressing apply changes saves the changes and returns the user to the content Management Main screen, unless errors occur which returns the user to this screen with an error description. Pressing cancel returns the user to the Content Management Main screen. It should be noted that the Provider ID for the account maintenance main screen does not correspond to Provider ID in the database, which is a unique numeric key. The provider ID for the account maintenance main screen corresponds to Login ID in the database, which is a text login assigned manually to new providers. Fig. 24 is an exemplary account maintenance main screen. [00269] The Provider Distribution Statistics screen allows users to view statistics for all of their content. This screen will summarize all data delivered for the provider. This screen also includes a detailed provider reporting component. Fig. 25 is an exemplary Provider Distribution Statistics screen. Additionally, these same statistics may be generated for a specified time period. [00270] The Content Distribution Statistics screen allows users to view statistics for a specified content object. This screen also includes a detailed content reporting component. Fig. 26 is an exemplary content distribution statistics screen. Additionally, these same statistics may be generated for a specified time period. [00271] Numerous other embodiments of the present invention are clearly within the scope of the present disclosure. For example, in some preferred embodiments of the invention, the tree representing the dynamic connection structure may be organized using a different algorithm. For example, an ordered binary tree may be implemented in such a way that j-clients located geographically near one another would be grouped together on branches of the tree. This may be shown to further minimize file transfer time. Another embodiment of the invention could include a checksum feature to ensure the accuracy of the file transfer, or include provisions for packet encryption or other security measures. In addition, the method could be modified to stream an audio or video file in such a way that each packet is played on the j-client computer as it is received. [00272] As an additional advantage, a DCS of the present invention can be used in any scenario or configuration where resources of clients are needed. The configuration/ topology of the present Dynamic Connection Structures can be organized specially depending on the types of resources needed. These include, for example, message delivery across firewalls, chat message routing, on-line game playing, grid computing/ utility computing and distributed storage.

105. Message Delivery EXAMPLE - Many firewall type security provisions currently are used which block connections into computers. A DCS can be used to facilitate message delivery into firewalled computers. The DCS would be structured as:

Firewall Firewall Firewall Firewall

106. Chat - A DCS is created for a group of clients that wish to chat together, and chat messages are routed between these clients. 107. Grid computing/utility computing - a group of clients are connected into the DCS and information is passed among them in order to utilize the CPU processing power of the clients. 108. Distributed storage - a group of clients are connected into the DCS and information is passed among them in order to utilize the aggregate hard- drive storage power of the clients.

[00273] Non-topology messages can also be effected through systems utilizing the present Dynamic Connection Structures. In Message Delivery EXAMPLE 105 above, a

DCS is used to facilitate message delivery to a set of clients. Many types of messages could be delivered, such as: 1. An invitation/directive to join another DCS a. Maybe the client is needed as a file upload source in the other DCS; b. Maybe the other DCS is transferring content that the client may be interested in; c. Maybe the other DCS is a chat group that the client may be interested in 2. A chat message; 3. A configuration message; or 4. A promotional message/advertisement.

Numerous modifications to the present invention fall within the scope of the specification and claims advanced herein. The present application thus encompasses any adaptations or variations of the present invention. Therefore, it is manifestly intended that the scope of this invention be limited only by the interpretation by one of skill in the relevant art of the specifica-tions and claims of the present and related applications.

Claims

"TOPOLOGY" Claims 1-56 - and ABSTRACT as filed Oct 18 04

1. A dynamic connection structure (DCS) comprising: A) a plurality of nodes, wherein said plurality comprises i) at least one first node, wherein said first node is connected to ii) at least one second node; and iii) a traffic cop node for overseeing the topology of said DCS, wherein said DCS forms a peer-to-peer architecture.

2. The DCS according to claim 1, wherein each of said nodes is enabled to perform the functions of: a) self-insertion of said node into said DCS in accordance with instructions received from said traffic cop node; and b) reorganization of said node positionally in said DCS with respect to other nodes of said plurality, wherein said reorganization is in accordance with instructions received by said node from said traffic cop node.

3. The DCS according to claim 2, wherein each of said nodes is enabled to perform said self-insertion and said reorganization simultaneously.

4. The DCS according to claim 1, wherein any of said plurality of nodes, said at least one first node and said at least one second node are enabled to function as a server to provide content suitable for distribution to one or a plurality of said nodes in said DCS.

5. The DCS according to claim 1, wherein any of said plurality of nodes, said at least one first node and said at least one second node are enabled to function as a servant to receive content which is distributed by way of said nodes in said DCS.

6. The DCS according to claim 1, wherein any of said plurality of nodes, said at least one first node and said at least one second node are enabled to function both as i) a server to provide first content suitable for distribution to one or a plurality of said nodes in said DCS, and ii) a servant to receive said first content which is distributed by way of said nodes in said DCS.

7. The DCS according to claim 6, wherein any of said plurality of nodes, said at least one first node, and said at least one second node are enabled to function both as a server and as a servant with respect to said first content and with respect to additional content and wherein said server function and said servant function can be performed simultaneously with respect to said first content and with respect to said additional content.

8. The DCS according to claim 1, wherein any of said at least one first node, said at least one second node and said plurality of nodes of said DCS are organized and re-organized in accordance with instructions of said traffic cop node.

9. The DCS according to claim 8, wherein said organization and said re-organization is centrally controlled and is a function of said traffic cop node. Information and software

10. The DCS according to claim 9, wherein information necessary for said organization and said re-organization of said DCS is provided from a centrally maintained database.

11. The DCS according to claim 9, wherein information necessary for said organization and said re-organization of said DCS is stored in one or more of said traffic cop node, and a digital medium accessible to said traffic cop node.

Such as a or disk attached to said traffic cop 12. The DCS according to claim 6, wherein communication necessary to organize and re-organize said DCS is decentralized and is among said plurality of nodes.

13. The DCS according to claim 12, wherein said communication with either of said at least one first node and said at least one second node is protected by a firewall and is via a buddy connection.

14. The DCS according to claim 11, wherein said traffic cop node processes all connection change information.

15. The DCS according to claim 11, wherein said traffic cop node maintains a connection list of all active transmission connections, and wherein said connection list is grouped in accordance with one or more universal resource nodes (URN's).

16. The DCS according to claim 11, wherein said traffic cop node maintains a connection list of all active transmission connections, and wherein said connection list is grouped in accordance with one universal resource nodes (URN).

17. The DCS according to claim 1, wherein connections between one node and another node of said DCS are established and connections between said one node and said second node of said DCS are broken simultaneously.

18. The DCS according to claim 1, wherein said DCS is organized and re-organized simultaneously.

19. The DCS according to claim 17, wherein each connection that is established and each connection that is broken uses separate threads to effect simultaneity.

20. The DCS according to claim 1, wherein said DCS is one of a binary heap, an incomplete binary heap, a binary tree, an m-ary tree, a graph, a chain, a mesh, a ring and a bus.

21. The DCS according to claim 1, wherein said DCS is two or more interconnected systems selected from the group consisting of a binary heap, an incomplete binary heap, a binary tree, an m-ary tree, a graph, a chain, a mesh, a ring and a bus.

22. The DCS according to claim 1, wherein said DCS is organized and re-organized according to attributes to maximize efficiency by maximizing usage of upstream and downstream bandwidth capacity, wherein said attributes comprise network connection speed, firewall status, geographic location, IP sub-net address, location of chunks of content by sub-net address and user account information.

23. The DCS according to claim 2, wherein said plurality of nodes comprises at least one third node, and at least one fourth node, and at least one fifth node.

24. The DCS according to claim 23, wherein the plurality of nodes comprises at least one hundred nodes, or at least two hundred nodes, or at least four hundred nodes or at least one thousand nodes.

25. A method for processing a connection change in a dynamic connection structure (DCS), said method comprising the steps or actions of: a) adding a new connection to a connection list for a specified universal resource node (URN); b) validating a timestamp to ensure said connection change is more recent than a last processed connection change; c) validating a security signature; d) locating a data object corresponding to said connection change, said data object having a matching local session ID for said specified URN and storing , connection change information in said data object; and e) propagating a connection change message to a connection identified in said data object.

26. The method according to claim 25, said act of locating further comprising storing a null value in said data object if no valid entry is located. '

27. The method according to claim 25, further comprising the step or act of f) initiating a connection to a specified node if said data object is not null and said local session ID does not match an existing connection.

28. The method according to claim 27, wherein said act or step of initiating further comprises: i) initiating an outgoing connection, and ii) adding said outgoing connection to said connection list.

29. The method according to claim 27, wherein said act of initiating further comprises iii) initiating a buddy connection if said node is firewall-protected.

30. The method according to claim 25, further comprising the acts or steps of: g) removing said connection from said connection list when, for any existing connection for the specified URN, there is no data object and h) transmitting a message to said connection indicating connection deletion.

31. A method for creating a connection between a first node and a second node of a dynamic connection structure (DCS), comprising the steps or acts of: A. sending, by said first node, a connection request to said second node; B. acknowledging, by said second node, receipt of said connection request from said first node; C. establishing said connection between said first and said second node; D. requesting, by said first node, access to a file set and forwarding, by said first node, access information to said second node; E. validating, by said second node, said access information forwarded by said first node; F. granting, by said second node to said first node, access to said file set; and G. forwarding, by said first node, said access information of said first node to all other nodes in said dynamic connection structure that are affected by said creating of said connection.

32. The method according to claim 107, further comprising the steps or acts of: H. denying access to said file set if said access information is not validated; and I. deleting said established connection.

33. The method according to claim 32, wherein said first node and said second node are one of a server/seed node and a servant.

34. The method according to claim 32, wherein each of said first and said second nodes are simultaneously a server/seed node and a servant with respect to different file sets.

35. A method of deleting a node from a dynamic connection structure (DCS), said method comprising: receiving, by a traffic cop node, a request from said node to leave said DCS; and forwarding, by said traffic cop node, a connection list to nodes affected by disconnection of said node from said DCS.

36. A dynamic connection structure (DCS) comprising: i) at least one first node connected to a plurality of second nodes; and ii) a traffic cop node configured and arranged for overseeing the topology of said DCS, wherein said DCS forms a peer-to-peer architecture, and wherein each of said at least one first node and each of said plurality of second nodes are enabled to perform the functions of: inserting itself into said DCS in accordance with instructions received from said traffic cop node; and reorganizing itself positionally in said DCS in accordance with instructions received from said traffic cop node.

37. The DCS according to claim 36, wherein each of said nodes is enabled to perform said self-inserting and said reorganizing simultaneously.

38. The DCS according to claim 36, wherein at least one of either of said at least one first node and said at least one second node is configured and arranged as a server node to provide content to be distributed over said DCS.

39. The DCS according to claim 36, wherein at least one of either of said at least one first node and said at least one second node is configured and arranged as a servant node to receive content distributed over said DCS from other nodes of said DCS.

40. The DCS according to claim 36, wherein one or more of said nodes is configured and arranged both as a server to provide a first content set to other nodes of said DCS and simultaneously configured and arranged as a servant to receive a second content set from other nodes of said DCS.

41. The DCS of claim 40, wherein said first content set and said second content set are different from one another.

42. The DCS of claim 40, wherein said first content set and said second content set are from the same content file.

43. The DCS according to claim 40, wherein substantially all of said nodes are configured and arranged both as a server to provide first content to other nodes of said DCS and simultaneously configured and arranged as a servant to receive second content from other nodes of said DCS.

44. The DCS of claim 43, wherein said first content set and said second content set are different from one another.

45. The DCS of claim 43, wherein said first content set and said second content set are from the same content file.

46. The DCS according to claim 36, wherein the relative positions of said nodes are organized and re-organized in accordance with instructions from said traffic cop node.

47. The DCS according to claim 36, wherein said organization and said reorganization is centrally controlled in accordance with instructions from said traffic cop node.

48. The DCS according to claim 36, wherein information necessary for said organization and said re-organization of said DCS is stored in one or more of said traffic cop node, and a digital medium accessible to said traffic cop node.

49. The DCS according to claim 36, wherein communication necessary to organize and re-organize said DCS is decentralized and is among said plurality of nodes.

50. The DCS according to claim 36, wherein said communication with either of said at least one first node and said at least one second node is protected by a firewall and is via a buddy connection.

51. ' The DCS according to claim 36, wherein said traffic cop node processes all connection change information.

52. The DCS according to claim 36, wherein said traffic cop node maintains a connection list of all active transmission connections, and wherein said connection list is grouped in accordance with one or more universal resource nodes (URN's).

53. The DCS according to claim 36, wherein said traffic cop node maintains a connection list of all active transmission connections, and wherein said connection list is grouped in accordance with one universal resource nodes (URN). i

54. The DCS according to claim 36, wherein connections between one node and another node of said DCS are established and connections between said one node and said second node of said DCS are broken simultaneously.

55. The DCS according to claim 36, wherein said DCS is organized and re-organized simultaneously.

56. The DCS according to claim 36, wherein each connection that is established and each connection that is broken uses separate threads to effect simultaneity.

57. The DCS according to claim 36, wherein said DCS is one of a binary heap, an incomplete binary heap, a binary tree, an m-ary tree, a graph, a chain, a mesh, a ring and a bus.

58. The DCS according to claim 36, wherein said DCS is organized and re-organized according to attributes to maximize efficiency by maximizing usage of upstream and downstream bandwidth capacity, wherein said attributes comprise network connection speed, firewall status, geographic location, IP sub-net address, location of chunks of content by sub-net address and user account information.