JP2009537918A - Distributed storage - Google Patents

Abstract

  A server for managing access to a distributed storage including a plurality of storage devices, a file service manager for receiving a request to save a file in the distributed storage, and a cluster in the distributed storage used for saving the file And an assignment manager for selecting. The allocation manager is configured to communicate with at least one other allocation manager at another server to determine which allocation manager selects the cluster to be used.

Description

The present invention relates to a distributed storage system, a distributed storage management system, and a server for managing access to the distributed storage.

BACKGROUND ART The demand for data storage for personal and commercial use is constantly increasing.

  For example, as digital recording and playback devices such as digital cameras and music players become more common, the number of individual users that need to store large amounts of data (eg, greater than 10 gigabytes) increases. Furthermore, as these devices are used, the amount of content to be generated or downloaded increases, so the amount of data to be stored also increases.

The demand for data storage is also changing.
Traditionally, data files such as digital photographs have been stored in a storage device such as a magnetic hard disk in a single location, such as on a home computer. However, users have increasingly wanted to be able to access their data from a long distance and even on the move. In addition, users may wish to share their data with other users.

  Distributed storage that includes more than one storage device and is managed using a virtual file system may provide a solution to these problems.

  Storage devices can be added to provide additional capacity. In addition, data can be stored physically different across many storage devices that can even change over time, but for users it appears that their data is stored in a single accessible "location". It is. In addition, the user may give other users, such as friends or business colleagues, access rights that can read or even update his or her files.

Conventional distributed storage systems using virtual file systems are known.
US-A-5873308 describes a system having a plurality of servers, each server managing a respective file system and at least one server operating a virtual file management system. The virtual file management system includes a management table that stores a virtual file identifier, a server name, and an actual file name for each virtual file. If a server receives a request from a client to read or modify a specified file, and the server does not hold the file, the server determines which other server stores the file, Modify the request to instruct the other server to respond directly to the client, and forward the modified request to the other server.

  This system has the disadvantage that the server also stores the data. This can limit the amount of storage that can be added to any one server. Adding additional servers may provide additional storage, but this is more expensive than simply adding additional storage devices.

  US-A-20050246393 discloses a virtual file system, a plurality of intelligent storage nodes, and a plurality of control nodes ("Distributed Object Storage Manager") for identifying and accessing an intelligent storage node storing a specified object file. A network storage system having a storage cluster with a storage cluster. Each distributed object storage manager maintains a look-up table that locates object files stored on the intelligent storage node and a state table that stores information about the overall capacity and health of the intelligent storage node. The state table is used to select an intelligent storage node to save a new object file. When the network storage system receives a request to store data in a new object file, the load balancing fabric selects one of the distributed object storage managers based on availability and forwards the request to the selected distributed object storage manager. . Instead, the distributed object storage manager selects an intelligent storage node.

  Different distributed object storage managers are responsible for storing new object files. Thus, each distributed object storage manager stores only information about the files it serves at least initially. Therefore, when the distributed object storage manager receives a request to access a given file that it does not carry, the distributed object storage manager broadcasts a request for the file to the intelligent storage node. Therefore, the intelligent storage node must be given sufficient processing capacity to handle such requests. This makes it more difficult to add storage nodes.

  The present invention seeks to provide an improved server and thus an improved distributed storage system for managing access to distributed storage.

According to a first aspect of the present invention, there is provided a server for managing access to a distributed storage including a plurality of storage devices, the server communicating with at least one other server, The server is configured to determine which part of the distributed storage to be used when storing data.

  Thus, a single server controls the storage of data, thereby helping to avoid the problem of simultaneous storage allocation and information about the data is stored in an orderly manner on a single metadata server. Help make it possible.

  The server includes a first control unit for receiving a request to save data in the distributed storage from the client, and a second control unit for selecting a portion of the distributed storage used when saving the data. The second controller may communicate with at least one other second controller to determine which of these second controllers will select the portion of the distributed storage that will be used. It is configured as follows.

  In response to a determination that another server will select the portion of the distributed storage that is to be used, this server receives and uses the identification information of the portion of the distributed storage from that other server. The distributed storage portion may be selected in response to determining that it will select the distributed storage portion itself.

  The server may be configured to select one of the storage devices in the distributed storage portion.

  The server may further include a third control unit for selecting one of the storage devices in the distributed storage portion in response to a command from the second control unit.

  The server may be configured to replicate the file in another storage device in the distributed storage portion after the file is stored in one storage device. The server may be configured to send information about the file to the metadata server after the file is saved. The server may be configured to receive a virtual file identifier corresponding to the file from the metadata server and send the virtual file identifier to the client.

  In response to a request to read or update a file, the server may be configured to send a request for information about the file to any one of the at least one metadata server.

  At least one of the storage devices may be provided by a remote storage system such as a storage access network.

  According to a second aspect of the present invention, a distributed storage management system including a plurality of access management servers and a metadata server for storing information about files stored in the distributed storage is provided.

The metadata server may be separated from the access management server.
The distributed storage management system may further include at least one additional metadata server, and each access management server sends information about the file to the metadata server, but not to the at least one additional metadata server. It is configured. Each access management server may be configured to send a request for information about the file to any of the at least one additional metadata server.
According to the third aspect of the present invention, a distributed storage system including a distributed file management system and a distributed storage including a plurality of storage devices is provided.

The storage device includes a plurality of types of storage devices.
According to a fourth aspect of the present invention, there is provided a method of operating a server for managing access to a distributed storage including a plurality of storage devices, the method communicating with at least one other server, Including determining which of these servers will select the portion of the distributed storage that will be used in storing the data. The method may include receiving a request from the client to save data in the distributed storage.

  In this method, when the first control unit receives a request to save data in the distributed storage, the second control unit communicates with at least one other second control unit, and the second control unit Determining which of the portions of the distributed storage to be used will be selected.

The method includes determining that another server will select the portion of the distributed storage that is to be used, and receiving identification information from the other server that portion of the distributed storage that is to be used. But you can. The method may include determining that the server will select the portion of the distributed storage that is to be used and selecting the portion of the distributed storage that is to be used. The method may include selecting one of the storage devices in the portion of distributed storage that will be used. The method may include storing the data in a selected portion of distributed storage.

  The method may further include replicating the file on another storage device in the distributed storage portion after the file is stored on one storage device. The method may include sending information about the file to the metadata server after the file is saved. The method may include receiving a virtual file identifier corresponding to the file from the metadata server and may include transmitting the virtual file identifier to the client.

  The method may include receiving a request to read or update a file and sending a request for information about the file to any one of the at least one metadata server.

  According to a fifth aspect of the present invention, there is provided a computer program including instructions that, when executed by a computer, cause the computer to execute the above method.

  According to a sixth aspect of the present invention, there is provided a computer readable medium for storing the above computer program.

  Embodiments of the present invention will now be described for purposes of illustration with reference to the accompanying drawings.

Detailed Description of the Embodiments of the Invention Referring to FIG. 1, a distributed storage system 1 according to the present invention is shown. Clients 2, 3, in this example desktop personal computer 2 and mobile phone handset 3, can access distributed storage system 1 via communication network 4 such as the Internet. For clarity, only two clients 2 and 3 are shown, but it will be understood that one client or more than two clients can access the system 1. The network 4 may include a plurality of wireless and wired networks including a public land mobile network (not shown) or a wireless local area network (not shown).

The distributed storage system 1 includes first, second, and third servers 5 ₁ , 5 ₂ , and 5 ₃ for managing access to the distributed storage (servers 5 ₁ , 5 ₂ , and 5 ₃ are described below). Called "File Service Provider"). The system 1 may have fewer or more file service providers than the above, for example one or two. The file service provider 5 _1, 5 _2, 5 _3, a third file service provider 5 ₃ first storage device 6 and either communicating locally, such as storage device ₆₁ in the form of a hard disk drive provided by, Alternatively, communication is performed remotely via the communication network 4 or another communication network 7 such as an intranet. For example, storage area network (storage area network; SAN) 8 is may provide a second storage apparatus 6 _2, network attached storage (network attached storage; NAS) 9 is provided a third storage apparatus 6 ₃ at best, the file server 6 ₃ may provide a fourth storage apparatus 6 _4. The number of storage devices 6 may be different. For example, SAN 8 or NAS 9 may provide more than one storage device and / or more than one SAN or NAS may be provided, each providing at least one storage device 6. Other forms of the system providing the storage device 6 may be used, such as common internet file systems (CIFS) (not shown), even if optical disk (eg DVD) drives and USB drives are used. Good. The number of storage devices 6 can change over time. For example, the storage device 6 may be added or removed. The arrangement of the storage device 6 can also change.

  As will be described in more detail later, the storage apparatus 6 collectively provides a single storage volume 26 (FIG. 4) and is arranged in a cluster 27 (FIG. 4).

The file service provider 5 communicates with the first metadata server 11 ₁ and the optional second metadata server 11 ₂ via the communication network 7 for mapping between virtual file references and actual files. . File service provider 5, first metadata server 11 ₁ from the read metadata, there to be able to write metadata. However, the file service providers 5 ₁ , 5 ₂ , and 5 ₃ can only read metadata from the second metadata server 11 ₂ . The first and second metadata servers 11 ₁ , 11 ₂ are hereinafter referred to as “active” and “passive” metadata servers 11 ₁ , 11 ₂ , respectively. The active metadata server 11 ₁ may copy the metadata to the passive metadata server 11 ₂ , for example, periodically, in response to a request or in response to a predetermined trigger. More than one passive metadata server 11 ₂ may be provided. Further, each metadata server 11 ₁ , 11 ₂ may be distributed between two or more servers.

  Still referring to FIG. 2, the file service provider 5 and the metadata server 11 are implemented by software on the respective computer systems 12. Each computer system 12 includes a processor 13, a memory 14, and an input / output (I / O) interface 15 operably connected by a bus 16. Computer system 12 may include more than one processor. The I / O interface 15 is operatively connected to a network interface 17, a storage 18 in the form of a hard disk drive, and optionally a removable storage 19. Other elements including peripheral devices such as a keyboard (not shown) and a display (not shown) may be provided temporarily or permanently.

When executed by the computer system 12, computer program code 20 ₁ , 20 ₂ that causes the computer system 12 to provide file service provider or metadata server processing is stored on the hard drive 18 for processing by the processor 13. Is loaded into the memory 14. Computer program code 20 ₁ , 20 ₂ may be stored on and removable from removable storage 19 or may be transferred through a network interface 17 from a remote source (not shown).

1 and 3, each file service provider 5 has a first service or control unit 21 for interfacing with clients 2 and 3, and a second for selecting a storage cluster 27 (FIG. 4). Service or control unit 22, a third service or control unit 23 for selecting the storage device 6 in the cluster 27 (FIG. 4), and the cluster 27 (FIG. 4) or between them. A fourth service or control unit 24 for controlling data replication and a fifth service or control unit 25 for interfacing with the metadata servers 11 ₁ and 11 ₂ are included.

The first service 21 manages requests received from clients 2 and 3 that access the file, such as writing and reading the file, as well as other components shown in FIG. The first service 21 is referred to herein as a “file service manager”.

The second service 22 selects the cluster 27 (FIG. 4) when writing data and is referred to herein as the “allocation manager”. The assignment manager 22 communicates with other assignment managers 22 as will be described in more detail later. One allocation manager 22 _M (FIG. 4) is referred to as a “master allocation manager” and is chosen to be responsible for selecting the cluster 27 (FIG. 4), while the other allocation managers 22 are master allocation managers 22 _M ( Acts as a proxy for FIG. The selected allocation manager _22M can change.

  In the writing or reading process, the third service 23 selects a certain storage device 6 and confirms its availability. Here, the third service 22 is called a “cluster manager”. The cluster manager 23 triggers to replicate the file stream in one storage device 6 to the other storage device 6. The cluster manager 23 further distributes the read processing among the storage apparatuses 6 so as to increase the processing amount. The cluster manager 23 further collects data regarding the processing amount from the client to the storage apparatus. The throughput data can be used during the write or read process to select the storage device 6 that has a faster response to the clients 2, 3.

  The fourth service 24 is referred to herein as a “replication service”. As described above, the replication service 24 controls data replication in the cluster 27 (FIG. 4) and between the clusters 27. The cluster manager 23 logs a replication transaction when any of the storage apparatuses 6 is not available.

The fifth service 25 is referred to herein as a “metadata server interface”. The metadata server interface 25 sends metadata such as information about a file to the active metadata server 11 ₁ and extracts metadata from the active or passive metadata servers 11 ₁ and 11 ₂ .

  Referring to FIG. 4, the storage device 6 provides a single storage volume or storage space 26. The storage device 6 is arranged in the cluster 27, and each storage device 6 stores the same data as another storage device 6 in the same cluster 27. That is, the storage apparatuses 6 in one cluster 27 are mirrors of each other. For clarity, only two clusters 27 are shown. In one cluster 27, the storage apparatus 6 has a different access configuration such as a network address. Each file service provider 5 can access at least one storage device 6 in each cluster 27.

The metadata servers 11 ₁ and 11 ₂ store the first, second, third, fourth, and fifth metadata tables 28, 29, 30, 31, and 32. The metadata includes information for mapping virtual file references to physical file references and the attributes of the file before filters are applied (such as file compression and virus checking). The metadata further includes other information such as a folder tree, folder contents data, and an access control list for each file and folder.

  A first table 28 (hereinafter referred to as a “virtual file system entity table”) includes a virtual entity identifier 33, a cluster identifier 34, an entity type 35 for each entity such as a file or directory, and a physical storage identifier 36. Save the path at.

  The second table 29 (hereinafter referred to as “virtual file system access control list”) stores a virtual entity identifier 37 and an access list 38 for each entity, and defines the security authority of the user, that is, the client user. The virtual file system access control list 29 lists the users who have access rights to the entity, such as read, write, create, delete, and change rights, along with a set of access rights for the entity.

  Each request for access to an entity sent by the clients 2 and 3 specifies the identification information of the user. Thus, each time a file service provider 5 receives a request for access to a given entity, it queries the virtual file system access control list 29, for example by extracting a portion of the list 29 for that entity and user. This verifies the user's access authority.

  A third table 30 (hereinafter referred to as a “virtual file system entity sharing list”) stores virtual entity identifiers 39 and access lists 40 for each entity and invites registered users to share file browsing. Specifies access rights for unregistered users (called “shared viewers”). The virtual file system access control list 30 is similar to the virtual file system access control list 29, except that access rights are typically limited to, for example, read only.

  Shared viewers are identified and authenticated differently than registered users. Identification and authentication can be performed by an external system (not shown) having a trust relationship established with the distributed storage system 1.

  The virtual file system access control list 30 may be omitted, for example, when a shared viewer is not permitted to access the distributed storage system 1.

  The fourth table 31 (hereinafter referred to as “logical cluster metadata table”) stores a (logical) cluster identifier 41 and a (physical) storage device identifier 42 for each cluster 27. The logical cluster metadata table 31 enables the cluster manager 23 to identify the storage device 6 in the cluster 27.

The fifth table 32 (hereinafter referred to as “logical cluster relation table”) stores the first cluster identifier 43, the second cluster identifier 44, and the relationship type 45 for each cluster 27, and the clusters 27 to each other. Describe the relationship between. For example, the logical cluster relation table 32 may specify that the first cluster 27 _A is a replica (ie, a mirror) of the second cluster 27 _B. Alternatively, the logical cluster relation table 32 identifies that the first cluster 27 _A is an archive for files stored in the second cluster 27 _B stored according to pre-defined criteria. May be. An archive may include files that are rarely accessed or created before a certain date and time. Therefore, the first cluster 27 _A does not need to store the files that are archived in the second cluster 27 _B. Archived files may be stored, for example, in a tape storage system.

As described in further detail below, the system 1 can be used to provide file services to clients 2, 3. Since the requests and responses are exchanged without a session, different file service providers 5 can handle different requests from the same client 2, 3. Therefore, the file service provider 5 does not need to store any information about the state of the clients 2 and 3. Instead, the clients 2, 3 can store information about their state and supply this information when sending requests to the file service provider 5. For example, the request may include authentication information required for the user to be verified against the access rights stored in the virtual file system access control list 29.

Write (create file)
The file writing process will now be described with reference to FIGS. 4, 5a and 5b.

  The client, in this example, the personal computer 2 sends a file write request 46 to the file service manager 21 of the file service provider 5 (step W1). The file service provider 5 is selected according to a routing mechanism or by a load balancer (not shown) in the network 4 (FIG. 1). File write request 46 includes file 47 to be written, including file data 48 and file attributes 49 such as file type and file size.

  The file service manager 21 transfers the cluster allocation request 50 to the allocation manager 22 (step W2). Request 50 may include a file attribute 49 that may be used to select cluster 27.

Master allocation manager 22 _M, for example the file attributes 49, and / or cluster attributes, and be based on conditions such as available space and utilization (not shown), or some other criteria such as round robin, cluster 27 is selected (step W3).

Before the cluster 27 is selected, the allocation manager 22, itself determines whether the master allocation manager 22 _M. Allocation manager 22, if a master allocation manager 22 _M, selects the cluster 27. Otherwise, the allocation manager 22 forwards the request to the master allocation manager 22 _M, the master allocation manager 22 _M is assigned a cluster 27, and returns the cluster identifier.

  The allocation manager 22 calls the cluster manager 23 with an instruction 51 so as to select the storage device 6 in the selected cluster 27 (step W4). The cluster manager 23 selects a certain storage device 6 (step W5). The cluster manager 23 makes the selection based on the highest network availability between the file service manager 21 and the storage device 6. Network availability may be determined using network status information collected by the allocation manager 22 and cluster manager 23 at a predetermined time, eg, periodically.

  The cluster manager 23 sends an instruction 52 to the storage apparatus 6 so as to create a file stream 53 having a given path in the storage apparatus 6 (step W6). The storage device 6 creates a file stream 53 (step W7). A unique file handler is associated with the file stream 53, which points to the first writable block in the file stream 53 in the storage device 6 (with a pointer P). The identification information 54 of the file handler (identifying the file stream 53) is returned to the cluster manager 23 (step W8).

  The cluster manager 23 returns the file handler identification information 54, the path 55, and the cluster identification information 56 to the allocation manager 22 (step W9). Instead, the allocation manager 22 returns this information to the file service manager 21 (step W10).

The file service manager 21 applies a write filter such as compression and / or virus check to the file data 48, and writes the filtered file data 57 to the opened file stream 53 (step W12). File stream 53 is filled with filtered file data 57. When the block is filled, the storage device advances the pointer P to point to the next available free block.

  Once the filtered file data 57 is written, the file service manager 21 sends an instruction 59 to the cluster manager 23 to close the file stream 53 (step W13). The cluster manager 23 sends an instruction 60 to the storage apparatus 6 to close the file stream 53 (step W14). The storage apparatus 6 writes an end of file (EOF) marker 61 to the file stream 52 (step W15), releases the pointer, and stores a storage system allocation table (not shown) such as a file allocation table provided by the storage apparatus 6. Update).

  The cluster manager 23 then triggers replication of the stored file stream 53 to other storage devices 6 in the same cluster 27 by sending instructions (not shown) to the replication service 24 (FIG. 2) (steps). W16).

The file service manager 21 converts the metadata into a virtual file system entity table 28, a virtual file system access control list 29, a virtual file system entity shared list table 30, and a logical cluster meter table 31 via the metadata server interface 25. When, as written as needed to the logical cluster relations table 32, and sends an instruction 62 to the metadata server 11 _1. The metadata server 11 ₁ writes the file information (step W18), and returns the virtual file identifier 63 to the file service manager 21 (step W19). The file service manager 21 sends confirmation information 64 that the file has been written to include the virtual file identifier 63 to the client 2 (step W20).

As previously mentioned, the allocation manager 22, itself determines whether the master allocation manager 22 _M. The process for achieving this will now be described in more detail.

Selection of Master Assignment Manager With reference to FIGS. 4 and 6, the assignment manager 22 receives a request 48 (FIG. 5a) to assign a cluster 27 (step S1). The assignment manager 22 checks whether the master assignment manager 22 _M is known (step S2).

If the allocation manager 22 knows the identity of the master allocation manager 22 _M, the allocation manager 22 itself determines whether the master allocation manager 22 _M (step S3). If so, the allocation manager 22 processes this request and selects the cluster 27 as described above (step S4). Otherwise, the assignment manager 22 forwards this request to the master assignment manager 22 _M (step S5). Allocation manager 22 receives a cluster identifier from the master allocation manager 22 _M.

If the assignment manager 22 does not know the identification information of the master assignment manager 22 _M , the assignment manager 22 sends a query 65 for the master identification information to the other assignment managers 22. The assignment manager 22 waits for a reply (step S7) and checks whether the query is “timed out”, ie whether a response is received before a predetermined period of time (step S8).

If the allocation manager 22 receives the reply 66 before the query times out, it saves the identification information of the master allocation manager 22 _M (step S9) and forwards the request to the master allocation manager 22 _M (step S5).

  If the assignment manager 22 does not receive a reply before the query times out, it sends a message 67 stating that it is the master (step S11). The allocation manager 22 waits for an objection (step S11) and checks whether the message 67 times out (step S12).

If no objection is received before the request times out, the assignment manager 22 records itself as the master assignment manager 22 _M (step S13), processes the request, and selects a cluster (step S4).

  However, when the objection 68 is received, the assignment manager 22 retransmits the identification query 65 to another assignment manager 22 (step S6).

  The allocation manager 22 tries to detect the query 65 and the message 67 continuously.

If it receives a query 65 or message 67 and it is the master assignment manager 22 _M , it sends a response 66 or an objection 68, respectively. The master assignment manager 22 _M may be demoted if it receives a message 67 from another assignment manager 22 or if the file service provider 5 is deactivated, eg shut down.

Write (update file)
The file update process will now be described with reference to FIGS. 4, 7a and 7b.

  The file update process is similar to the file write process described above. In short, the allocation manager 22 frees the space allocated to the old version of a file, allocates space to the newer version of the file as if it were a newly created file, Write a new version of

  The client 2 sends a file update request 69 to the file service manager 21 of the file service provider 5 (step U1). The file write request 69 includes a virtual file identifier 73 for the updated file 70 that includes the new file data 71 and the new file attributes 72 and the old file, eg, the previously saved file 47.

File service manager 21, to find out the old file to which cluster 27 is stored, sends a query 74 to the active metadata server 11 ₁ (step U2). Alternatively, the file server manager 21 may send the query 74 to the passive metadata server 11 _2. Active metadata server 11 ₁ returns the old cluster identifier 75, a path 76 in the storage apparatus 6, and the old file attributes 77 (step U3).

  The update process (steps U4 to U22) proceeds in a manner similar to steps W2 to W20 of the write process described above.

File service manager 21 forwards the cluster allocation request 78 to the allocation manager 22 (step U4), the master allocation manager 22 _M selects the cluster 27 (Step U5).

  The allocation manager 22 sends an instruction 79 to select the storage device 6 in the selected cluster 27 (step U6). The cluster manager 23 selects the storage device 6 (step U7). The cluster manager 23 sends an instruction 80 to the storage apparatus 6 so as to create a new file stream 81 having a given path in the storage apparatus 6 (step U8). The storage device 6 creates a file stream 81 (step U9) and passes a new file handler identifier 82 to the cluster manager 23 (step U10). The cluster manager 23 returns the new file stream 81, the corresponding path 83, and the cluster identifier 84 to the allocation manager 22 (step U11). The allocation manager 22 transfers this information to the file service manager 21 (step U12).

  The file service manager 21 applies a write filter to the new file data 71 (step U13), and writes the new filtered file data 85 to the opened file stream 81 (step U14).

  Once the new filtered stream 85 is written, the file service manager 21 sends an instruction 87 to the cluster manager 23 to close the file stream 81 (step U15).

  The cluster manager 23 sends an instruction 88 to close the file stream 81 to the storage device 6 (step U16). The storage apparatus 6 writes an end-of-file (EOF) marker 89 in the file stream 81, releases the pointer, and updates the storage apparatus file system allocation table (not shown).

  The cluster manager 23 sends a command (not shown) to the replication service 24 (FIG. 2) to trigger the replication of the file stream 81 to another storage device 6 in the same cluster 27 (step U18).

The file service manager 21 sends an instruction 90 to the metadata server 11 ₁ via the metadata server interface 25 to write updated metadata as necessary (step U19). The metadata server 11 ₁ writes the updated file information (step U20), and returns a new virtual file identifier 91 to the file service manager 21 (step U21). The file service manager 21 sends confirmation information 92 that the file has been written to the client 2 (step U22).

  The file service manager 21 sends an instruction 93 to the cluster manager 23 to delete the old file (step U23). The cluster manager 22 finds the storage device 6 that stores the old file stream (step U24), and sends an instruction 94 to the storage device 6 to delete the old file stream (step U25). The cluster manager 23 triggers replication of the storage device 6 (step U26).

Reading The file reading process will now be described with reference to FIGS.

  The client 2 sends a file read request 95 to the file service manager 21 of the file service provider 5 (step R1). File read request 95 includes a virtual file identifier 96 for the file, eg, previously saved file 42 or updated file 68.

File service manager 21, to find whether the file to which the cluster 27 is stored, sends a query 97 to the passive metadata server 11 ₂ (step R2). The file server manager 21 may send the query 97 to the active metadata server 11 ₁ .

Passive metadata server 11 ₂ returns a cluster identifier 98, a path 99 in the storage apparatus 6, and a file attribute 100 (step R3).

  The file service manager 21 sends a request 101 to the cluster manager 23 to select a certain storage device 6 (step R4). The cluster manager 23 selects a storage device 6 in the cluster 27 identified by the cluster identifier 98 according to the highest network availability (step R5) and sends an instruction 102 to the storage device 6 to open the file stream 103 (step R5). R6). When the access to the storage apparatus 6 fails, the cluster manager 23 refers to the logical cluster relation table 32 so as to determine whether the cluster 27 is mirrored or archived. If so, the cluster manager 23 sends a new command (not shown) to another storage device 6. The storage device 6 extracts the file data 104 from the file stream 99 and transfers it to the cluster manager 23 (step R7). The cluster manager 23 passes the file data 104 to the file service manager 21 (step R8). The file service manager 21 applies a read filter to the file data 104 (step R9), and transfers the filtered file stream 105 to the client 2 (step R10).

  As explained above, a user may grant access rights to other users called “shared viewers”. The user may grant access rights to a shared viewer, for example a link to access a file or folder in the form of a uniform resource locator (URL) identifying the file service provider 5 and the virtual file. The shared viewer can be informed of the location of the file or folder by e-mailing or text sending a message containing The shared viewer selects this link, thereby sending a request to the file service provider 5 to extract the file or folder.

  The system 1 described above can be used to provide file services regardless of the storage infrastructure. It can provide pluggable streams to handle across redundant sites for distributed file input and output across integrated storage space.

  File services may be provided to users with mobile phone handsets by mobile phone network providers, by technology integrators, and by companies.

  Since the system 1 separates the file metadata from the file data and integrates the actual file systems, that is, the storage apparatuses, they seem to be a single storage unit. This may help minimize or avoid storage device limitations such as storage capacity and storage device throughput.

  System 1 can handle file streams and does not need to divide the file stream into blocks for handling.

  It will be appreciated that many modifications can be made to the embodiments described above.

1 is a schematic diagram of a distributed storage system according to the present invention. FIG. FIG. 2 is a diagram showing a computer system for the server shown in FIG. 1. It is a block diagram of a server for managing access to a distributed storage. FIG. 2 is a block diagram of the system shown in FIG. 1. FIG. 5 is a diagram showing a file writing process performed in the system shown in FIG. 4. FIG. 5 is a diagram showing a file writing process performed in the system shown in FIG. 4. It is a processing flow figure of the method of operating an allocation manager. It is a figure which shows the file update process performed in the system shown by FIG. It is a figure which shows the file update process performed in the system shown by FIG. It is a figure which shows the file reading process performed in the system shown by FIG.

Claims (32)

  A server for managing access to a distributed storage including a plurality of storage devices, wherein the server communicates with at least one other server, and any of these servers is used for storing data. A server that is configured to determine which part of the distributed storage will be selected. A first controller for receiving a request to store data in the distributed storage from a client;
A second control unit for selecting the portion of the distributed storage used when storing the data, the second control unit communicating with at least one other second control unit; The server of claim 1, configured to determine which of these second controllers will select the portion of the distributed storage to be used.   In response to determining that another server will select the portion of the distributed storage to be used, the device is configured to receive identification information for the portion of the distributed storage from the other server. Item 3. The server according to item 1 or item 2.   3. A server according to claim 1 or claim 2, configured to select the portion of distributed storage in response to a determination that it will select the portion of distributed storage to be used. .   A server according to any preceding claim, configured to select one of the storage devices in the portion of the distributed storage.   3. Dependent on claim 2, further comprising a third controller for selecting one of the storage devices in the portion of distributed storage in response to an instruction from the second controller. The server according to claim 5 in the case.   The server according to claim 5 or 6, wherein after the file is stored in one storage device, the data is replicated in another storage device in the portion of the distributed storage. .   A server according to any preceding claim, wherein the server is configured to send information about the data to a metadata server after the data is stored.   The server of claim 8, configured to receive a virtual file identifier corresponding to the data from the metadata server.   The server of claim 9, configured to send the virtual file identifier to a client.   Any of the preceding claims, configured to send a request for information about the file to any one of at least one metadata server in response to a request to read or update a file. Server.   A server according to any preceding claim, wherein at least one of the storage devices is provided by a remote storage system. The remote storage system includes a storage access network.
2. The server according to 2. A plurality of access management servers according to any of the preceding claims;
A distributed storage management system, including a metadata server for storing information about files stored in the distributed storage.   The distributed storage management system according to claim 14, wherein the metadata server is separated from the access management server.   And further including at least one additional metadata server, wherein each of the access management servers is configured to send information about files to the metadata server, but not to the at least one additional metadata server. The distributed storage management system according to claim 14 or 15.   The distributed storage management system of claim 16, wherein each of the access management servers is configured to send a request for information about a file to any of the at least one additional metadata server. A distributed file management system according to any one of claims 14 to 17,
A distributed storage system including a distributed storage including a plurality of storage devices.   The distributed storage management system according to claim 18, wherein the storage device includes a plurality of types of storage devices. A method of operating a server for managing access to a distributed storage including a plurality of storage devices,
A method comprising communicating with at least one other server and determining which of those servers selects a portion of distributed storage that will be used in storing data.   When the first control unit receives a request to store data in the distributed storage, the second control unit communicates with at least one other second control unit, and among these second control units 21. The method of claim 20, comprising determining which one of the distributed storage to be used is selected.   Determining that another server will select the portion of the distributed storage that is to be used and receiving identification information of the portion of the distributed storage that is to be used from the other server. 22. A method according to claim 20 or claim 21.   23. The method of claims 20-22, comprising determining that the server will select a portion of distributed storage to be used and selecting a portion of the distributed storage to be used. The method according to any one of the above.   24. A method as claimed in any one of claims 20 to 23, further comprising selecting one of the storage devices in the portion of the distributed storage to be used.   25. The method of claim 24, further comprising replicating the file on another storage device in the portion of the distributed storage after the file is stored on one of the storage devices.   26. A method according to any one of claims 20 to 25 comprising sending information about the file to a metadata server after the file is saved.   27. The method of claim 26, comprising receiving a virtual file identifier corresponding to the file from the metadata server.   28. The method of claim 27, comprising sending the virtual file identifier to a client.   29. Any of claims 20 to 28, comprising receiving a request to read or update a file and sending a request for information about the file to any one of at least one metadata server. The method according to one.   29. A method according to any one of claims 20 to 28, comprising storing the data in a selected portion of the distributed storage.   A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 20 to 30.   A computer readable medium storing the computer program according to claim 31.

Images