WO2007133791A2

WO2007133791A2 - Data partitioning and distributing system

Info

Publication number: WO2007133791A2
Application number: PCT/US2007/011679
Authority: WO
Inventors: Richard Kane
Original assignee: Richard Kane
Priority date: 2006-05-15
Filing date: 2007-05-15
Publication date: 2007-11-22
Also published as: WO2007133791A3

Abstract

A system for distributed storage on networked computers comprising: a plurality of storage devices; means for partitioning data into a plurality of fragments; means for distributing said fragments among said plurality of storage devices; means for retrieving said fragments from among said plurality of storage devices; and means for reassembling said data from said fragments.

Description

DATA PARTITIONING AND DISTRIBUTING SYSTEM

BACKGROUND

[0001] The invention relates generally to data storage, and more particularly to storage of small fragments of a file on a plurality of hard drives.

[0002] 1. Field of the Invention:

[0003] The present invention relates generally to the field of data storage and computer systems. More specifically the present invention relates to remote data backup systems in which data is partitioned and distributed in fragments to geographically scattered receiving computer servers, and then retrieved and reassembled when needed.

[0004] 2. Description of the Prior Art:

[0005] In recent years there have been backup systems for computer data to secure the data at a remote location in the event that the primary server is damaged or destroyed. A problem with this form of backup has been that the data might be accessed in the remote location by an unauthorized user, even if encrypted.

[0006] It is thus an object of the present invention to provide a remote computer backup system that partitions and. distributes fragments of data to several geographically remote computer servers so that only a small portion can be lost in a given server due to a local catastrophe. [0007] It is another object of the present invention to provide such a backup system where, also as a result of partitioning and distribution of data fragments, no individual system member other than the originator of the data has more than a small part of the data, making the data highly secure.

[0008] It is still another object of the present invention to provide such a backup system that is two-tiered, including backup on a local disk drive such as at a member home, for additional security against catastrophic loss.

[0009] It is finally an object of the present invention to provide such a backup system that utilizes minimal band-width after initial backup, and is fast as a result of retrieving data fragments from multiple servers.

[0010] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion

in this section.

BRIEF DESCRIPTION

[0011] The invention provides a remote data backup system in which data is partitioned and distributed in fragments to geographically scattered receiving computer servers, and then retrieved and reassembled when needed, so that no more than a small fragment of the data at a given location remains vulnerable to a local catastrophe such as a hurricane. In addition, fragmenting of the data both reduces the storage demand on individual receiving servers while at the same time providing highly effective security, because no single receiving server has a significant portion of the entire backup file. The

data may be confidential business or personal files, movies, digital photographs or any other data reducible to digital form. Once partitioned and distributed, the backup of revisions normally requires transmission only of update data, rather than of all data, so that minimal band width is needed from that time forward. The data preferably is encrypted and compressed prior to distributing, and preferably prior to partitioning. The receiving servers selected for inclusion in the present system should be continuously on, and preferably are connected through DSL. The receiving servers in the system are also sending servers, since the present system is primarily intended as a data storage space sharing mechanism. Each server in the system sends data fragments for storage in other servers, and also receives and stores data fragments from other servers. One system member allow other to backup his or her data, and system members rely on the given system member to backup their data. This system is a world-wide load and barter system, because every member effectively trades computer drive space. An additional advantage of the present system is that data can be retrieved faster because pieces of the data are pulled from multiple servers at once.

[0012] The system preferably provides two-tier backup, one being the above- described method of partitioning and distributing to geographically scattered receiving servers, and the other being storing on a local disk drive such as at a system member home. The system preferably is virtual. DRAWINGS

[0013] These and other features, aspects, and advantages of the present invention will

become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0014] FIGURES 1-16 illustrate the present backup system generally in accordance with an embodiment of the present invention. FIGURES 2-16 show one implementation of the end user client. FIGURE 2 shows one draft of the selection of pricing model.

DETAILED DESCRIPTION

[0015] As required, detailed embodiments of the present invention are disclosed

herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention which may be embodied in various forms. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present invention in virtually any appropriately detailed structure.

[0016] Reference is now made to the drawings, wherein like characteristics and features of the present invention shown in the various FIGURES are designated by the same reference numerals.

[0017] First Preferred Embodiment [0018] Referring to FIGURES 1-16, a remote data backup system is disclosed, in

which data is partitioned and distributed in fragments to geographically scattered receiving computer servers, and then retrieved and reassembled when needed, so that no

more than a small fragment of the data at a given location remains vulnerable to a local catastrophe such as a hurricane. In addition, fragmenting of the data both reduces the storage demand on individual receiving servers while at the same time providing highly effective security, since no single receiving server has a significant portion of the entire backup file. The data may be confidential business or personal files, movies, digital photographs or any other data reducible to digital form. Once partitioned and distributed, the backup of revisions normally requires transmission only of update data, rather than of all data, so that minimal band width is needed from that time forward. The data preferably is encrypted and compressed prior to distributing, and preferably prior to

partitioning. The receiving servers selected for inclusion in the present system should be continuously on, and preferably are connected through DSL. The receiving servers in the system are also sending servers, since the present system is primarily intended as a data storage space sharing mechanism. Each server in the system sends data fragments for storage in other servers, and also receives and stores data fragments from other servers. One system member allows others to backup his or her data, and system members rely on the given system member to backup their data. This system is a worldwide load and barter system, because every member effectively trades computer drive space. An additional advantage of the present system is that data can be retrieved faster because pieces of the data are pulled from multiple servers at once. [0019] The system preferably provides two-tier backup, one being the above- described method of partitioning and distributing to geographically scattered receiving servers, and the other being storing on a local disk drive such as at a member home. The system preferably is virtual.

[0020] Substantial band width normally is required only for initial data fragment transmission and storage. From that time forward, only update data normally need be transmitted to the scattered servers, requiring minimal band width. The initial data may be received by the initial server through telefaxing or scanning a document. The data may then be made keyword searchable. A virus scan preferably is also conducted prior to sending and receiving data.

[0021] The system members preferably have smart disk drives in their homes. The members can share data with their friends by providing them with data encryption keys. The public INTERNET preferably is not used, so that there is no need to wait for the data.

[0022] The system provides key word searches to find data and files, and is a document processing engine. The present system automatically detects degraded documents and data. The system can be faster and more efficient than INTERNET search engines because it only searches customer documents, rather than a universe of websites. A series of boxes showing document pages can be provided together on a screen for selection. This system effectively stores filing cabinets. [0023] Method

[0024] In practicing the invention, the following method may be used. PCl, a computer to be backed up or a source of shared content to the network, contacts the ESS server; the ESS server identifies target computers to distribute this data based on (a) service level, (b) geography / diverse storage to maximize survivability, (c) bandwidth, (d) reliability, (e) availability: storage and access/up time; the number or distributed target computers and the location of these computers are transmitted to PCl; PCl compresses then encrypts the target file; PCl breaks the file into components with some overlap, this overlap providing for data redundancy among the distributed targets, such that one or more distributed targets could fail without loss of data; PCl transmits the file components directly to the target machines, thereby distributing the data while minimizing network traffic at the host site; where the content is licensed to be distributed, and is requested, and is authorized via digital rights management, then all machines storing segments of the content will stream the content to a target machine for reassembly and delivery, this machine either being or not being the originator of the content to receive the content; in the context of backup and restore, the restoration of data may occur to a replacement computer in the event of a crash, theft or other data loss of the computer being backed up, and the transfer of the DRM key to the new computer; in the context of media distribution, the number of authorized recipients of the data stream may be quite large, and the delivery may involve hundreds to thousands of machines. [0025] Business Model

[0026] Computer owners may purchase a hosted storage service on a usage basis, and have the service provider (the ESS server in FIGURE 1 ) store locally managed copies of their data. In this case the data being backed up is only stored at the service provider, and is only restored from the service provider to the customer computer.

[0027] Computer owners may sell the storage and network bandwidth associated with their machines back to the service provider on a barter, points, or cash basis. This would typically be in exchange (a barter) for an inexpensive backup of their own data across the distributed network, but may also involve hosting third party data (media content, downloads, home pages) for mass distribution on a barter for services basis (points), e.g., help us distribute five movies or TV episodes this month, and view one free. Another aspect of the model may be the purchase for cash of a portion of the locally available storage.

[0028] The service itself will be marketed directly to consumers, as well as through sales channels as a private label product (e.g., the service will be targeted to ISPs). FIGURES 2-16 show one implementation of the end user client. FIGURE 2 shows one draft of the selection of pricing model.

[0029] Small fragments of a file can be stored on a plurality of hard drives. Advantages of this approach are perfect encryption and redundancy. It is desirable to compress data prior to distribution so that it can be stored non-sequentially. Distributed storage works similar to the way peer to peer file sharing works. While the invention has been described, disclosed, illustrated and shown in various terms or certain embodiments or modifications which it has assumed in practice, the scope of

the invention is not intended to be, nor should it be deemed to be, limited thereby and such other modifications or embodiments as may be suggested by the teachings herein are particularly reserved especially as they fall within the breadth and scope of the claims here appended.

Claims

CLAIMS:

1. A system for distributed storage on networked computers comprising:

a plurality of storage devices;

means for partitioning data into a plurality of fragments;

means for distributing said fragments among said plurality of storage devices; means for retrieving said fragments from among said plurality of storage devices; and means for reassembling said data from said fragments.

2. The system for distributed storage on networked computers of claim 1 wherein

said a plurality of storage devices comprise geographically remote computer servers.

3. The system for distributed storage on networked computers of claim 1 further comprising means for restricting access to said means for retrieving or reassembling said data from said fragments.

4. The system for distributed storage on networked computers of claim 3 wherein, due to said partitioning and distribution of data fragments, an individual without access to said means for retrieving or reassembling said data can access only a small part of said data.

5. The system for distributed storage on networked computers of claim 3 wherein

said data is encrypted prior to said distribution.

6. The system for distributed storage on networked computers of claim 3 wherein said data is encrypted prior to said partitioning.

7. The system for distributed storage on networked computers of claim 3 wherein said means for restricting access to said means for retrieving or reassembling said data from said fragments comprises a data encryption key.

8. The system for distributed storage on networked computers of claim 1 wherein said system is two-tiered, such that said data file is also stored on a local storage device.

9. The system for distributed storage on networked computers of claim 1 wherein said data file is partitioned such that any part of said data file exists in a plurality of said fragments and said entire data file may be reassembled even if at least one of said fragments is unavailable.

10. The system for distributed storage on networked computers of claim 1 further comprising means for updating distributed data.

11. The system for distributed storage on networked computers of claim 1 wherein said file is compressed prior to said distribution.

12. The system for distributed storage on networked computers of claim 1 wherein said file is compressed prior to said partitioning.

13. The system for distributed storage on networked computers of claim 1 wherein said fragments are retrieved simultaneously from at least two of said plurality of storage devices.

14. The system for distributed storage on networked computers of claim 1 further comprising means for making keyword searches of said data.

15. The system for distributed storage on networked computers of claim 1 further comprising means for virus scanning said data prior to distributing or retrieving.

16. The system for distributed storage on networked computers of claim 1 wherein said storage devices comprise smart disk drives.

17. The system for distributed storage on networked computers of claim 1 wherein said system comprises a document processing engine.

18. The system for distributed storage on networked computers of claim 1 further comprising means for detecting degraded documents and data.

19. The system for distributed storage on networked computers of claim 1 wherein said data is compressed and encrypted prior to being partitioned.

20. A method of storing data on a plurality of networked storage devices comprising the steps of:

instructing a distributed storage system to store said data from a source machine;

said system identifies storage machines;

said system provides the location of said storage machines to said source machine;

said source machine partitions said file into fragments;

said source machine transmits said file fragments to said storage machines;

after said system receives a request for said data from a requesting machine said storage machines stream said file fragments to said requesting machine;

said system reassembles said fragments to make said data available to said requesting machine.

21. The method of claim 20 further comprising the following steps:

said source machine compresses then encrypts said data prior to distribution; after said data is streamed to said receiving machine and reassembled, said data is decompressed and decrypted.

22. The method of claim 20 wherein said data is redundantly partitioned such that said fragments comprise overlapping data sequences that exist in a plurality of fragments.

23. The method of claim 21 further comprising the following step:

said system verifies that said receiving machine has a digital rights management key prior

to making said data available to said receiving machine.

24. The method of claim 20 wherein said system evaluates and selects storage machines based on the following factors:

(a) service level,

(b) geography,

(c) bandwidth,

(d) reliability,

(e) availability.