JP2007265314A - Distributed storage system and distributed storage program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a distributed storage system with higher safety. <P>SOLUTION: In the distributed storage system, a super node 104 stores a file in a plurality of distributed disks 101 and reproduces the file by using elimination correction coding technology. Since there are a plurality of super nodes 104, they can distribute a load and since the super nodes 104 do not store management information, they are safe. A management terminal 102 stores the management information and is safe since general users can not access it. It is safe since the distributed disks are restored by using the management information from the management terminal 102 even if they are broken. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a distributed storage system and a distributed storage program.

  FIG. 14 shows a configuration diagram of a SAN (Storage Area Network) which is a typical configuration of a conventional dedicated storage system. The SAN includes a dedicated storage device 1401 that stores data, a file server 1403, a user terminal 1405, an FC SW (Fibre Channel Switch) 1402 that connects the dedicated storage device 1401 and the file server 1403 at high speed, and a file server 1403 and a user terminal 1405. The Internet 1404 is connected. When the user requests the file server 1403 from the user terminal 1405 to read or write a file, the dedicated storage device 1401 is automatically read or written.

  FIG. 15 shows a configuration of a data distributed storage system disclosed in Japanese Patent Application Laid-Open No. 2003-348065. The data distributed storage system includes a plurality of distributed storage servers 1501, user personal computers 1503, and the Internet 1502 connecting the plurality of distributed storage servers 1501 and user personal computers 1503.

  Next, the flow of data storage processing in the data distributed storage system of FIG. 15 will be described using the flowchart of FIG. Upon receiving a data storage request from the user, the user personal computer 1503 encrypts the original data file to be stored, performs error correction encoding, and generates five redundant files (step S1601). Next, the encryption key is encoded into a shared key (step S1602). Next, the redundant file and the share key are transmitted as five distributed storage pairs to the five distributed storage servers via the Internet (step S1603).

Next, the flow of data reproduction processing in the data distributed storage system of FIG. 15 will be described using the flowchart of FIG. Upon receiving a data reproduction request from the user, the user personal computer 1503 accesses the distributed storage server via the Internet and acquires a redundant file and share key pair (step S1701). If there are two shared keys, the encryption key can be restored, and if there are three redundant files, an encrypted file can be obtained by error correction decoding. Next, the encrypted file is restored by error correction decoding (step S1702). Next, the encrypted file is decrypted with the encryption key to obtain the original data file (step S1703).
JP 2003-348065 A

  In SAN, which is a typical configuration of a conventional dedicated storage system, the dedicated storage device is very expensive, the load is concentrated on the dedicated storage device, and the data is not distributed, so the dedicated storage device is damaged during a disaster. In such a case, there is a problem that all stored data may be lost.

  Further, in the data distributed storage system disclosed in Japanese Patent Application Laid-Open No. 2003-348065, the data stored when the load is concentrated on the user personal computer or the distributed storage server fails because the user personal computer performs all processing. However, there is a problem that stored data cannot be decrypted when the number of distributed storage servers more than a predetermined number is damaged. Further, in the data distributed storage system as described above, file management is performed on the user personal computer, and there is a problem in the safety of management information. In addition, when using a distributed storage server from a plurality of user computers, there was a lack of flexibility such as requesting the reproduction of the same data from a user computer different from the user computer that requested the storage of certain data.

  The present invention is for solving the above-described problems, and in a distributed storage system using erasure correction coding technology, a file storage request and a read request are received from a user, and an actual storage process and In a super node that is a computer that performs read processing, an original file to be distributed and stored, an encoded file obtained by dividing and encoding the original file, a name, an ID, and the like related to a distributed disk that stores the encoded file, etc. Management information is not stored except during storage processing and reading processing to obtain a distributed storage system with high information confidentiality, and any one of a plurality of distributed disks has failed or is abnormal. In the case of operation, all the encoded files stored on the corresponding distributed disk are all repaired or replaced on the distributed disk. To obtain a distributed storage system with higher security in terms of information concealment, reliability, maintainability, etc. Is the purpose.

The distributed storage system according to the present invention includes:
A distributed storage system having a super node connected to a plurality of distributed disks and one or a plurality of user terminals via a network and receiving a file storage request and a read request,
The super node is
When receiving the original file to be distributed and the storage request for the original file from the user terminal, a plurality of encoded files obtained by dividing and encoding the original file are generated, and the encoded file names A division encoding unit having a name including a character string for specifying the correspondence with the original file,
A distribution unit that requests storage of each of the plurality of encoded files on a plurality of different distributed disks;
After the plurality of encoded files are stored on a distributed disk, the original file, the plurality of encoded files, information that can specify the original file or the plurality of encoded files, and the plurality of encoded files are stored. A file management unit that deletes information that can identify a plurality of distributed files,
When receiving the original file name and the read request of the original file from the user terminal, it is confirmed whether or not the encoded file generated from the original file is stored in the distributed disk, A reproduction unit that reproduces the original file by obtaining and decoding a predetermined number of the encoded files from the distributed disk whose storage has been confirmed by comparing the file and the encoded file name; It is characterized by that.

The distributed storage system further includes a management terminal connected to the super node via a network,
The file management unit further generates file management information indicating correspondence between the original file, the encoded file, and the distributed disk, transmits the file management information to the management terminal, deletes the file management information that has been transmitted,
The management terminal
A file management information storage unit for storing the file management information received from the file management unit;
Based on the request for restoration of any distributed disk, the encoded file stored in the distributed disk and the original file corresponding to the encoded file are obtained from the file management information stored in the file management information storage unit. Identifying and obtaining other encoded files stored on a distributed disk other than the distributed disk necessary for decoding the original file, decoding the original file from the other encoded files, A distributed disk restoration unit that regenerates the encoded file by dividing and encoding the decoded original file, and requests the storage of the regenerated encoded file in the distributed disk; It is characterized by that.

The management terminal further includes:
Checks whether multiple distributed disks are operating normally, outputs an alarm for distributed disks that could not be confirmed to operate normally, and then started normal operation from the distributed disks that could not be confirmed to operate normally. When the notification is received, the distributed disk restoration unit includes a distributed disk state confirmation unit that instructs the distributed disk restoration unit to restore the distributed disk whose normal operation could not be confirmed.

The distributed storage system further includes a plurality of the super nodes and user terminals connected to the plurality of super nodes via a network,
The user terminal is
One arbitrary super node in normal operation is selected from the plurality of super nodes, and a file storage request and a read request are made.

Further, the distributed storage system according to the present invention includes:
A distributed storage system having a super node connected to a plurality of distributed disks and one or a plurality of user terminals via a network and receiving a file storage request, a read request, and a distributed disk restoration request,
The super node is
When receiving an original file to be distributed and a storage request for the original file from the user terminal, a division encoding unit that generates a plurality of encoded files obtained by dividing and encoding the original file;
A distribution unit that requests storage of each of the plurality of encoded files on different distributed disks;
A file management unit that generates file management information indicating correspondence between the original file, the encoded file, and the distributed disk;
A file management information storage unit for storing the file management information;
When receiving the original file name and the read request of the original file from the user terminal, the file management information identifies a distributed disk that stores the encoded file generated from the original file, and Then, a normal operation is confirmed, a predetermined number of encoded files are obtained from the confirmed distributed disk, and the playback unit that reproduces the original file and decoding request for any distributed disk are obtained by decoding. Based on the file management information, the encoded file stored in the distributed disk and the original file corresponding to the encoded file are identified from the file management information stored in the file management information storage unit, and the original file is decoded. Identify and capture other encoded files stored on a distributed disk other than the distributed disk Then, the original file is decoded from the other encoded files, the decoded original file is divided and encoded, and the encoded file is regenerated and regenerated on the distributed disk. And a distributed disk restoration unit for requesting storage of the encoded file.

The distributed storage program according to the present invention is
When receiving an original file to be distributed and a storage request for the original file from a user terminal connected via a network, a plurality of encoded files obtained by dividing and encoding the original file are generated. , A division encoding process in which the encoded file name is a name including a character string for specifying correspondence with the original file;
A delivery process for requesting storage of each of the plurality of encoded files to a plurality of different distributed disks connected via a network;
After the plurality of encoded files are stored on a distributed disk, the original file, the plurality of encoded files, information that can specify the original file or the plurality of encoded files, and the plurality of encoded files are stored. File management process to delete information that can identify multiple distributed files,
When an original file name and a request to read the original file are received from a user terminal connected via a network, whether or not the encoded file generated from the original file is stored in a distributed disk Confirming and comparing the original file with the encoded file name, obtains a predetermined number of the encoded files from the distributed disk whose storage has been confirmed, and reproduces the original file by decoding. And a playback process to be executed by a computer.

The distributed storage program according to the present invention is
When an original file to be distributed and a storage request for the original file are received from a user terminal connected via a network, a plurality of encoded files obtained by dividing and encoding the original file are generated. Division coding processing,
A delivery process for requesting storage of each of the plurality of encoded files to different distributed disks connected via a network;
A file management process for generating file management information indicating the correspondence between the original file, the encoded file, and the distributed disk;
A file management information storage process for storing the file management information;
When an original file name and a request to read the original file are received from a user terminal connected via a network, a distributed disk that stores an encoded file generated from the original file is specified by the file management information. The normal operation of the distributed disk is confirmed, a predetermined number of encoded files are obtained from the confirmed distributed disk, and decoding is performed to reproduce the original file. Based on the request to restore the distributed disk, the encoded file stored in the distributed disk and the original file corresponding to the encoded file are identified from the file management information stored in the file management information storage process. Stored on a distributed disk other than the distributed disk required to decrypt the original file Identifying and obtaining the encoded file, decoding the original file from the other encoded files, dividing the decoded original file, and encoding to regenerate the encoded file; A distributed disk restoration process for requesting storage of the regenerated encoded file to the distributed disk is executed by a computer.

  According to the present invention, a more secure distributed storage system can be obtained.

Embodiment 1 FIG.
Hereinafter, the configuration of the distributed storage system 100 according to the present embodiment will be described with reference to FIG. The distributed storage system 100 according to the present embodiment includes a plurality of distributed disks 101 that store data, a management terminal 102 that handles file management information of data stored in the distributed disk 101, a division and code for distributed storage Network connecting a plurality of supernodes 104, a plurality of user terminals 106 receiving data storage requests and data reading requests, a distributed disk 101, a management terminal 102, and a supernode 104 103, a super node 104, and a WAN 105 (Wide Area Network) connecting the user terminal 106. The user selects an arbitrary super node 104 from an arbitrary user terminal 106 and requests storage of the original file. Upon receiving the request, the super node 104 divides and encodes the original file and stores it in the distributed disk 101. The file management information relating to which distributed disk 101 the original file is stored in is stored in the management terminal 102. In addition, the user selects an arbitrary super node 104 from an arbitrary user terminal 106 and requests reading of the original file. The super node 104 that has received the request receives data related to the original file stored in the distributed disk 101. Obtaining, decoding, and transmitting to the user terminal 106. In addition, the management terminal 102 restores data on an arbitrary distributed disk 101 based on the file management information stored as described above. Since the management terminal 102 is connected to the administrator network 103 and not connected to the WAN 105, general users other than the administrator cannot access it. The administrator network 103 is preferably a LAN (Local Area Network) protected by a firewall. The WAN 105 is assumed to connect the super node 104 and the user terminals 106 arranged over a wide range, but may be constructed in a narrow range such as a LAN.

  Next, a detailed configuration of the distributed storage system 100 according to the present embodiment will be described with reference to FIG. The distribution disk 101 has an encoded data storage unit 201 that stores the divided and encoded data. The divided and encoded data will be described later with reference to FIG. The names of the divided and encoded data will be described later with reference to FIG.

  The management terminal 102 includes a distributed disk storage unit 1 (202), a file management information storage unit 203, a distributed disk state confirmation unit 204, and a distributed disk restoration unit 205. The distributed disk storage unit 1 (202) stores node names, IP addresses, and the like of the plurality of distributed disks 101 in association with each other. The file management information storage unit 203 includes an ID (original file name) that identifies an original file that is a storage target, an ID (encoded file No.) that identifies an encoded file obtained by dividing and encoding the original file, The file management information in which the ID (distributed disk node name) specifying the stored distributed disk 101 is associated is stored. The distributed disk status confirmation unit 204 confirms whether or not the distributed disk 101 is operating normally, and instructs restoration. The distributed disk restoration unit 205 restores an arbitrary distributed disk 101 based on the file management information stored in the file management information storage unit 203. The file management information stored in the file management information storage unit 203 will be described later with reference to FIG.

  The super node 104 includes a distributed disk storage unit 2 (206), a division encoding unit 207, a distribution unit 208, a file management unit 209, and a reproduction unit 210. The distributed disk storage unit 2 (206) stores node names and IP addresses of a plurality of distributed disks in association with each other. The division encoding unit 207 generates an encoded file by dividing and encoding the original file to be stored. The distribution unit 208 distributes the plurality of encoded files described above to the plurality of distributed disks 101. The file management unit 209 stores an ID (original file name) for specifying the original file to be stored, an ID (encoded file No.) for specifying the encoded file obtained by dividing the original file, and the encoded file. File management information in association with an ID (distributed disk node name) for specifying the distributed disk 101 is created and transmitted to the management terminal 102. The file management information will be described later with reference to FIG. When the playback unit 210 receives a request for playback of the original file from the user terminal 106, the playback unit 210 acquires the number of data related to the source file for which the request has been received from the distributed disk 101, and plays back the original file. .

  The user terminal 106 includes a super node storage unit 211, a storage request unit 212, and a read request unit 213. The super node storage unit 211 stores node names, IP addresses, and the like of the plurality of super nodes 104 in association with each other. The storage request unit 212 requests an arbitrary super node 104 to store the original file. The read request unit 213 requests an arbitrary super node 104 to restore the original file.

  FIG. 3 is a diagram illustrating an example of hardware resources of the distributed storage system 100. Each of the plurality of distributed disks 101, the management terminal 102, the plurality of super nodes 104, and the plurality of user terminals 106 constituting the distributed storage system 100 is configured by a personal computer as shown in FIG. 3, for example. However, the distributed disk 101 may be a storage-dedicated device.

  In FIG. 3, each of a plurality of distributed disks 101, a management terminal 102, a plurality of super nodes 104, and a plurality of user terminals 106 constituting the distributed storage system 100 includes a system unit (not shown), a CRT (Cathode Ray Tube), Hardware resources such as a display device 301 having an LCD (liquid crystal display) display screen, a keyboard 302 (K / B), a mouse 303, an FDD 304 (Flexible Disk Drive), and a CDD 305 (compact disk device) are provided. Connected with signal lines. The system unit is a personal computer, and is connected to the WAN via, for example, a LAN (local area network) or a gateway.

  In addition, each of the plurality of distributed disks 101, the management terminal 102, the plurality of super nodes 104, and the plurality of user terminals 106 constituting the distributed storage system 100 includes a CPU 311 (Central Processing Unit, a central processing unit, an arithmetic unit that executes a program) Device). The CPU 311 is connected to a ROM 313 (Read Only Memory), a RAM 314 (Random Access Memory), a communication board 315, a display device 301, a keyboard 302, a mouse 303, an FDD 304, a CDD 305, and a magnetic disk device 320 via a bus 312. Control hardware devices. Instead of the magnetic disk device 320, a recording device such as an optical disk device or a memory card reader / writer may be used.

  The RAM 314 is an example of a volatile memory. The storage media of the ROM 313, the FDD 304, the CDD 305, and the magnetic disk device 320 are an example of a nonvolatile memory.

  The communication board 315 is connected to a LAN, a WAN (Wide Area Network), or the like. The WAN may be connected via a gateway.

  The magnetic disk device 320 stores an operating system 321 (OS), a window system 322, a program group 323, and a file group 324. The programs in the program group 323 are executed by the CPU 311, the operating system 321, and the window system 322. The program group 323 stores programs that execute functions described as “˜unit”, “˜means”, and “˜processing” in the description of the present embodiment. The program is read and executed by the CPU 311. Alternatively, the file group 324 includes data, signal values, variable values, and parameters described as “˜data”, “˜information”, and “˜result” in the description of the present embodiment. It is stored as each item of “database” and “˜table”.

  In addition, the arrows in the block diagrams and flowcharts described in the description of this embodiment mainly indicate input / output of data and signals. Data and signal values are stored in a memory such as a RAM 314, a flexible disk (FD) of an FDD 304, and a CDD 305. Recording disks such as a compact disk (CD), a magnetic disk of the magnetic disk device 320, other optical disks, a mini disk (MD), and a DVD (Digital Versatile Disk). Data and signals are transmitted online via the bus 312, signal lines, cables and other transmission media.

  In addition, what is described as “˜unit”, “˜means”, and “˜processing” in the description of the present embodiment may be realized by firmware stored in the ROM 313. Alternatively, it may be realized by software alone, hardware alone, a combination of software and hardware, or a combination of firmware. Firmware and software are stored as programs in a recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, and a DVD. This program is read by the CPU 311 and executed by the CPU 311. In other words, the program causes the computer to function as “˜unit”, “˜means”, and “˜processing” described in the description of the present embodiment. Alternatively, the procedure of “˜unit”, “˜means”, and “˜processing” described in the description of the present embodiment is executed by a computer.

  FIG. 4 is a schematic diagram illustrating a division and encoding procedure in the division encoding unit 207 and a decoding procedure in the reproduction unit 210 using the erasure correction encoding technique. First, encoding will be described. The division encoding unit 207 divides the original file 401 to be stored into 100 information packets 402, further encodes (redundant) 100 information packets 402, and generates 150 encoded files 403. Is generated. Next, decoding will be described. The reproduction unit 210 reproduces a decoded file 405 having the same contents as the original file 401 by collecting and decoding an arbitrary fixed number (for example, 105) of the 150 encoded files 403 described above. it can.

  In other words, the use of the erasure correction coding technique has a feature that the original data can be reproduced even if a part of the data is lost. Since the erasure correction coding technique itself is well known, detailed description thereof is omitted. Further, in the present embodiment, the original file 401 is divided into 100 information packets 402, 150 encoded files are created, and an arbitrary fixed number (for example, 105) of encoded files 403 are decoded files 405. However, the numerical values of 100, 150, and 105 can be changed by changing the algorithm of the erasure correction coding technique.

  Furthermore, in the present embodiment, 150 encoded files created as described above are distributed and stored in a plurality of distributed disks 101, so that any part of the distributed disks 101 may fail, power off, or be maintained. Even if the network line is not operating normally due to disconnection or the like, an arbitrary fixed number of encoded files are obtained from the normally operating distributed disk 101 and decoded, and the same contents as the original file 401 are decoded. Since the file 405 can be reproduced, high reliability and continuous mobility can be obtained.

  Next, the file management information, the name of the original file, and the name of the encoded file will be described with reference to FIG. FIG. 5 is a diagram illustrating a data example of file management information generated by the file management unit 209. The file management information includes an original file name that is the name of the original file to be stored, and an encoded file No. for specifying an encoded file obtained by dividing and encoding the original file. And the distributed disk node name for identifying the distributed disk 101 that stores the encoded file. In the present embodiment, it is assumed that the original file name is unique. The encoded file No. corresponding to one original file name is also displayed. Is unique.

  In the present embodiment, the name of the encoded file includes the original file name, _ (underbar), encoded file No. Are combined. For example, the original file name is “Digitalmovie1”, and the encoded file No. Is “001”, the name of the corresponding encoded file is “Digitalmovie1_001”. That is, the name of the encoded file includes the name of the original file.

  Next, file management information stored in the file management information storage unit 203 of the management terminal 102 will be described with reference to FIG. In FIG. 5, the file management information relating to one original file generated by the file management unit 209 has been described. However, the file management information stored in the file management information storage unit 203 described in FIG. This is file management information related to all original files stored in 101. In the example of FIG. 6, for the original files Digitalmovie1, Digitalmovie2, and Digitalpicture1, the original file name and the encoded file No. for specifying the encoded file obtained by dividing the original file and encoding it. And file management information in which the distributed disk node name for specifying the distributed disk 101 storing the encoded file is associated is stored in the file management unit 209.

  Next, the flow of data storage processing in the distributed storage system 100 will be described using the flowcharts of FIGS. 7, 8, and 9. As described in FIG. 4, data storage in the distributed storage system 100 creates a plurality of encoded files obtained by dividing and encoding the original file to be stored using the erasure correction encoding technology, Each of these encoded files is stored on a different distributed disk 101.

  In FIG. 7, when the storage request unit 212 of the user terminal 106 receives a data storage request from the user, the storage request unit 212 selects any one super node 104 from the super node storage unit 211 (step S <b> 701). Note that a plurality of super nodes 104 appear to the user as one virtual super node. The storage request unit 212 of the user terminal 106 randomly selects an arbitrary super node 104. If the corresponding super node 104 is not operating normally, until the super node 104 operating normally is selected. Similarly, any one super node 104 is randomly selected. Alternatively, the load status of each super node 104 is confirmed, and the super node 104 with a low load is selected. Alternatively, any one super node 104 may be selected by other methods. Next, the storage request unit 212 of the user terminal 106 transmits the original file to be stored and a storage instruction to the selected super node 104 (step S702).

  Next, the division encoding unit 207 of the super node 104 that has received the storage instruction divides the received original file into 100, encodes, and generates 150 encoded files (step S703). Here, in the division and encoding, the erasure correction encoding technique is used as described in FIG. Further, as described in FIG. 5, the name of the encoded file includes the name of the original file.

  Next, the distribution unit 208 of the super node 104 stores the 150 encoded files generated in the division encoding unit 207 and the storage instruction in the 300 distributed disks 101 stored in the distributed disk storage unit 2 (206). Among them, each of them is transmitted to any 150 distributed disks 101 (step S704). Details of step S704 will be described later with reference to FIG.

  Next, the file management unit 209 of the super node 104 obtains information related to the correspondence between the original file and the encoded file that has been divided and encoded from the divided encoding unit 207, and the encoded file and the storage destination distributed disk 101. Is acquired from the distribution unit 208, file management information indicating the correspondence between the original file, the encoded file, and the distributed disk 101 is generated and transmitted to the file management information storage unit 203 of the management terminal 102. (Step S705). Here, the file management information is as described in FIG. Details of step S705 will be described later with reference to FIG.

  Next, the detailed processing flow of step S704 in FIG. 7 will be described with reference to FIG. The distribution unit 208 of the super node 104 makes a confirmation request as to whether all the distributed disks 101 are operating normally (step S801). In response to the confirmation request in step S801, the normally operating distributed disk 101 responds (step S802).

  On the other hand, the distribution unit 208 of the super node 104 waits for a response from the distributed disk 101 for a certain time (step S803). Then, after a certain period of time, 150 units are arbitrarily selected from the distributed disks 101 that have responded (step S804). The distribution unit 208 of the super node 104 randomly selects any 150 distributed disks 101 that have enough disk capacity to store the encoded file. Alternatively, the disk capacity of each distributed disk 101 is confirmed to be free, and 150 units are selected in order from the distributed disk 101 having the largest available capacity. Alternatively, any 150 distributed disks 101 may be selected by other methods. Next, 150 encoded files are transmitted to 150 distributed disks 101, respectively (step S805). Next, the encoded data storage unit 201 of the distributed disk 101 receives and stores the encoded file transmitted in step S805 (step S806). If there is a distributed disk 101 that cannot be normally stored in step S806, it is transmitted to another distributed disk 101 and stored.

  Next, the detailed processing flow of step S705 in FIG. 7 will be described with reference to FIG. The file management unit 209 of the super node 104 creates file management information (step S901). As described with reference to FIG. 5, the file management information includes the original file name that is the name of the original file to be stored and the encoded file No. for specifying the encoded file obtained by dividing and encoding the original file. And the distributed disk node name for identifying the distributed disk 101 that stores the encoded file. Note that the original file name and the encoded file No. Is obtained from the division encoding unit 207, and the encoded file No. And the information on the correspondence between the distributed disk node names are acquired from the distribution unit 208. Next, the file management unit 209 of the super node 104 transmits file management information to the file management information storage unit 203 of the management terminal 102 (step S902).

  Next, the file management information storage unit 203 of the management terminal 102 stores the received file management information (step S903). Next, the fact that it has been normally received is responded to the file management unit 209 of the super node 104 (step S904). Next, the file management unit 209 of the super node 104 includes file management information and the like (various information handled in the data storage processing described with reference to the flowcharts of FIGS. 7, 8, and 9). , The original file, the encoded file, the information that can specify the original file or the plurality of encoded files, the information that can specify the plurality of distributed files that store the plurality of encoded files, and the like are deleted. (Step S905). In the data storage processing flow described with reference to the flowcharts of FIGS. 7, 8, and 9, the division encoding unit 207, the distribution unit 208, and the file management unit 209 of the super node 104 include the file management information and the like described above. Are not stored in the non-volatile memory (such as the magnetic disk device 320), and the above information temporarily stored in the volatile memory (such as the RAM 314) is also explicitly displayed by the file management unit 209 in step S905. Deleted. Alternatively, the above-described information temporarily stored in the volatile memory (RAM 314) may be automatically deleted by the OS 321 when the program is terminated and the memory is released. Alternatively, the division encoding unit 207, the distribution unit 208, and the file management unit 209 of the super node 104 may store all or a part of the above-described file management information in a nonvolatile memory (such as the magnetic disk device 320). In this case, the file management unit 209 explicitly deletes the file in step S905.

  Next, the flow of data read processing in the distributed storage system 100 will be described using the flowchart of FIG. Data reading in the distributed storage system 100 refers to collecting encoded files distributed and stored in a plurality of distributed disks 101, and using an erasure correction encoding technique as described in FIG. To play.

  In FIG. 10, the read request unit 213 of the user terminal 106 selects an arbitrary super node 104 from the super node storage unit 211 when receiving a data read request from the user (step S1001). Note that a plurality of super nodes 104 appear to the user as one virtual super node. The super node 104 to be selected does not need to be the super node 104 designated at the time of storage. The read request unit 213 of the user terminal 106 randomly selects an arbitrary super node 104, and if the corresponding super node 104 is not operating normally, until the super node 104 operating normally is selected. Similarly, any one super node 104 is randomly selected. Alternatively, the load status of each super node 104 is confirmed, and the super node 104 with a low load is selected. Alternatively, any one super node 104 may be selected by other methods. Next, the read request unit 213 of the user terminal 106 transmits the original file name to be read and the read instruction to the selected super node 104 (step S1002).

  Next, the reproducing unit 210 of the super node 104 that has received the read instruction stores the encoded file generated from the original file in all the distributed disks 101 stored in the distributed disk storage unit 2 (206). A confirmation request for whether or not is made (step S1003). Here, the original file name is transmitted together with the confirmation request.

  Next, the encoded data storage unit 201 of the distributed disk 101 searches using the original file name received in Step S1003 as a key, and returns a response when the encoded file generated from the original file is stored (Step S1003). S1004). As described with reference to FIG. 5, the name of the encoded file includes the name of the original file. Therefore, the encoded data storage unit 201 of the distributed disk 101 compares the original file name received from the playback unit 210 of the super node 104 with the encoded file name, and combines the original file name, _ (underbar), and a numerical value. If there is an encoded file having the specified name, a response is returned assuming that the encoded file generated from the original file is stored.

  In step S1003, the reproduction unit 210 of the super node 104 that has received the read instruction combines the original file name, _ (underscore), and a predetermined numerical value ("001" to "150" in the present embodiment). A name is created and transmitted to all the distributed disks 101. Next, in step S1004, the encoded data storage unit 201 of the distributed disk 101 searches for the encoded file received in step S1003, and if it exists. The response may be returned assuming that the encoded file generated from the original file is stored.

  Next, the reproducing unit 210 of the super node 104 that has received the response acquires 105 encoded data from the distributed disk 101 that has responded (step S1005). Next, the encoded data is decoded, and a decoded file having the same contents as the original file is reproduced (step S1006). Here, in decoding, the erasure correction coding technique is used as described in FIG. Next, the decrypted file is transmitted to the user terminal 106 (step S1007). Next, the read request unit 213 of the user terminal 106 receives the decrypted file (step S1008). Note that the user terminal 106 may display the contents of the decrypted file to the user via the display device 301 or the like.

  In step S1003, a confirmation request is made to determine whether or not the encoded file is stored in all the distributed disks 101. In step S1004, the corresponding distributed disk 101 responds, and in step S1005, the distributed disk that has responded. 101 to 105 pieces of encoded data have been acquired, but a confirmation request is made as to whether or not the encoded file is stored for one all distributed disk 101, and if there is a response, the encoded data is acquired. May be repeated 105 times.

  Next, the flow of distributed disk restoration processing in the distributed storage system 100 will be described using the flowcharts of FIGS. 11 and 12. The distributed disk restoration in the distributed storage system 100 is to collectively restore all the encoded files stored in one arbitrary distributed disk 101.

  In FIG. 11, the distributed disk status confirmation unit 204 of the management terminal 102 makes a confirmation request as to whether all the distributed disks 101 are operating normally (step S1101). The confirmation request may be made periodically at a predetermined time.

  Next, the distributed disk 101 that has received the confirmation request responds if it is operating normally (step S1102). Next, the distributed disk status confirmation unit 204 of the management terminal 102 determines that the distributed disk 101 that has not responded for normal operation for a certain period of time is a failure and raises an alarm (step S1103). Next, the failed distribution disk 101 is replaced or repaired manually (step S1104). However, at this time, the same IP address as before the replacement or repair is taken over. Next, the distributed disk 101 notifies the management terminal 102 of the normal operation of itself and the restoration request information together with its IP address (step S1105). Next, the distributed disk restoration unit 205 of the management terminal 102 automatically performs data restoration processing on the distributed disk 101 that has been operating normally after the failure (step S1106). The data restoration process will be described later with reference to FIG. Further, before performing the data restoration process, the file management information stored in the file management information storage unit 203 is compared with the encoded file stored in the encoded data storage unit 201 of the distributed disk 101 that has normally operated after the failure. Thus, it is determined whether there is a lost encoded file from the encoded data storage unit 201 that is recorded in the file management information, and if there is a lost encoded file, the data is automatically restored. Processing may be performed.

  Next, the detailed processing flow of step S1106 of FIG. 11 will be described with reference to FIG. In order to supplement the description, the example of the file management information in FIG. 6 is used. The distributed disk restoration unit 205 of the management terminal 102 refers to the distributed disk storage unit 1 (202), and searches the distributed disk node name corresponding to the IP address that the accepted IP address is registered as the distributed disk. Then, the distributed disk node name of the distributed disk 101 that received the restoration request is recognized (step S1201). Here, for example, it is assumed that the distributed disk node name is “PC005”.

  Next, the distributed disk restoration unit 205 refers to the file management information storage unit 203, and from the file management information, the original file name corresponding to the recognized distributed disk node name and the encoded file number. Are searched (step S1202). For example, if the file management information storage unit 203 has the file management information shown in FIG. 6, the original file name corresponding to the recognized distributed disk node name “PC0005” and the encoded file No. , There are two sets, one set has the original file name “Digitalmovie1”, and the encoded file No. Is “002”. In another set, the original file name is “Digitalpicture1”, and the encoded file No. Is “001”. That is, the encoded files on the distributed disk “PC005” are “Digitalmovie1_002” and “Digitalpicture1_001”.

  Next, the distributed disk restoration unit 205 repeats the processing from step S1203 to step S1208 for each retrieved original file name. For example, the process from step S1203 to step S1209 is performed for the original file “Digitalmovie1”, and then the process from step S1203 to step S1209 is performed for the original file name “Digitalpicture1”. Hereinafter, first, the process from step S1203 to step S1209 is performed for the original file “Digitalmovie1”.

  The distributed disk restoration unit 205 stores the encoded file number. To another encoded file No. required for decoding the file. And the other encoded file No. The active distributed disk node name corresponding to is determined (step S1203). For example, since there are 150 encoded files in the present embodiment, another encoded file No. that can be used to decode the original file name “Digitalmovie 1” is used. Is obtained by removing “002”, that is, “001” and “003” to “150”.

  Next, the distributed disk restoration unit 205 reads the encoded file No. required for restoration from the active distributed disk 101. An encoded file corresponding to is acquired (step S1204). For example, in the present embodiment, since there are 105 encoded files necessary for decoding, the original file name “Digitalmovie1”, _ (underbar), encoded file No. Are encoded file names, and the encoded file No. Obtains 105 encoded files having encoded file names such as “001” or “003” to “150” from the distributed disk 101 operating normally. That is, 105 files of the encoded files “Digitalmovie1_001”, “Digitalmovie1_003” to “Digitalmovie1_150” are acquired from the distributed disk 101 that is operating normally.

  Next, the distributed disk restoration unit 205 reproduces a decoded file from the acquired encoded file (step S1205). For example, “Digitalmovie1” is reproduced.

  Next, the distributed disk restoration unit 205 divides and decodes the decoded file, and generates an encoded file to be restored (step S1206). Here, division and encoding are performed in the same procedure as in step S703. For example, by dividing and encoding “Digitalmovie1”, “Digitalmovie1 — 001”, “Digitalmovie1 — 002”,... “Digitalmovie1 — 150” is generated, and the encoded file “Digitalmov2 and others” to be restored is left out. Delete the encoded file.

  Next, the distributed disk restoration unit 205 transmits the encoded file to the distributed disk 101 that has received the restoration request (step S1207). For example, “Digitalmovie1 — 002” is transmitted to the distributed disk “PC0005”. Note that after the transmission is completed normally, the encoded file “Digitalmovie1 — 002” on the management terminal 102 is deleted.

  Next, the distributed disk restoration unit 205 checks whether the processing has been completed for all the original files searched in step S1202, and if completed, ends step S1106, and if not completed, returns to step S1203. The process up to step S1208 is repeated (step S1208). In this example, since the process for the original file name “Digitalpicture 1” has not been completed, the process returns to step S1203, and the process from step S1203 to step S1208 is performed for the original file name “Digital picture 1”.

  Next, another encoded file No. that can be used to decode the original file name “Digitalpicture1” is displayed. Is obtained by removing “001” and is “002” to “150” (step S1203). Next, the original file name “Digitalpicture 1”, _ (underscore), and the encoded file number. Are encoded file names, and the encoded file No. Acquires 105 encoded files having encoded file names such as “002” to “150” from the normally operating distributed disk 101 (step S1204). That is, 105 files of the encoded files “Digitalpicture1_002” to “Digitalpicture1_150” are acquired from the distributed disk 101 operating normally. Next, “Digitalpicture 1” is reproduced (step S1205). Next, by dividing and encoding “Digitalpicture1”, “Digitalpicture1_001”, “Digitalpicture1_002”,... “Digitalpicture1_150” is generated, and the encoded file “Digital_picture1” and the like to be restored are generated. The encoded file is deleted (step S1206). Note that after the transmission is completed normally, the encoded file “Digitalpicture1_001” on the management terminal 102 is deleted. Next, “Digitalpicture1_001” is transmitted to the distributed disk “PC0005” (step S1207). Next, since processing has been completed for all the original files “Digitalmovie1” and “Digitalpicture1” searched in step S1202, step S1106 is terminated (step S1208). In this way, all the original encoded files are restored on the distributed disk 101.

  In this embodiment, an IP address is used to specify the distributed disk 101, and the distributed disk storage unit 1 (202) and the distributed disk storage unit 2 (206) are used to associate the IP address with the distributed disk node name. However, a distributed disk node name may be used to identify the distributed disk 101, and a hosts file may be used to associate an IP address with the distributed disk node name. Here, the hosts file is a text file that describes correspondence between an IP address and a host name on a TCP / IP network, which is a well-known technique. In this case, the distributed disk node name is set to the same name as the above-described host name, and the distributed disk storage unit 1 (202) and the distributed disk storage unit 2 (206) are personally used as the distributed disk 101. It is sufficient if a host name such as a computer or a storage dedicated device is stored.

  In the distributed disk restoration processing flow described with reference to FIGS. 11 and 12, in step S1104, the failed distributed disk 101 is replaced or repaired manually. The disk node name (host name) will be the same as before replacement and repair. If the IP address for the distributed disk node name (host name) has been changed, the distributed disk node name (host) stored in the hosts file of the distributed disk 101, management terminal 102, and super node 104 is also added. Correct the corresponding IP address. In step S1105, the distributed disk 101 notifies the management terminal 102 of the normal operation of itself and the restoration request information together with the distributed disk node name (host name). In step S1201, the distributed disk restoration unit 205 of the management terminal 102 refers to the distributed disk storage unit 1 (202) and confirms that the received distributed disk node name (host name) is registered as a distributed disk. To do.

  In this embodiment, as described with reference to FIGS. 11 and 12, when the management terminal 102 is notified of the normal operation of the distributed disk 101 and the restoration request (step S1105), the distributed disk restoration unit. 205 may be configured to restore the notification-source distributed disk 101 (step S1106), but the distributed disk restoration unit 205 of the management terminal 102 is configured via an input device such as the keyboard 302 and the mouse 303 of the management terminal 102. The administrator may accept a restoration request for an arbitrary distributed disk 101 from an administrator and restore the distributed disk 101. Also in the latter case, the processing from step S1201 to step S1208 of FIG. 12 is performed in the same manner as described above for an arbitrary distributed disk 101 received from the administrator.

  In the present embodiment, the management terminal 102 may not be provided. In this case, the distributed disk restoration described with reference to FIGS. 11 and 12 is not performed. Further, for the data storage described in FIGS. 7, 8, and 9, steps S705 and FIG. 9 in FIG. 7 are not performed. That is, the file management information is not created and transmitted to the management terminal 102. Further, since the data reading described with reference to FIG. 10 is a process that does not involve the management terminal 102, it operates as described above.

  In the present embodiment, as described above, it is assumed that the original file name is unique. However, the original file name and other information (for example, user ID, terminal ID, time, etc.) are stored in the storage request unit 212 of the user terminal 106 so that the original file whose name has been duplicated in the user terminal 106 can be registered. ) To create a new unique name, the original file name may be changed to the above-mentioned unique name, and transmitted to the super node 104. Alternatively, the division encoding unit 207 of the super node 104 checks whether or not the original file name that received the storage request is stored in the file management information storage unit 203 of the management terminal 102 and has already been stored. Alternatively, an error may be returned to the storage request unit 212 of the user terminal 106. Alternatively, the division encoding unit 207 of the super node 104 checks whether or not an encoded file having a name including the original file name that has received the storage request is stored in the encoded data storage unit 201 of the distributed disk 101. If it is already stored, an error may be returned to the storage request unit 212 of the user terminal 106. In addition, when the storage request unit 212 changes the original file name, the read request unit 213 of the user terminal 106 changes the original file name that has received the read request in the same manner.

  As described above, according to the present embodiment, a storage device has recently been very expensive by using a personal computer, which has recently been increasing in speed and CPU, LAN, and a low price as a storage device. In addition, the distributed storage system 100 having the same functions and performance as the dedicated storage system can be constructed at low cost. In addition, by using the erasure correction coding technique, the original file cannot be restored even if the single distributed disk 101 is referred to, and the same information confidentiality as that of the conventional dedicated storage system can be ensured. In addition, the super node 104 includes an original file, an original file name, an original file No. Further, since management information such as the storage destination distributed disk node name is not stored, information confidentiality can be further ensured.

  In this embodiment, only the management terminal 102 stores file management information, and the file management information can be used to restore all the encoded files on any one distributed disk 101. File management information on the management terminal 102 can be safely managed by using various authentication functions, etc., and the distributed disk 101 can be easily repaired or replaced due to failure or other circumstances. The reliability and maintainability of the storage system 100 can be improved. Since the conventional data distributed storage system also uses the erasure correction encoding technology, if a certain number of encoded files can be collected even if some of the distributed storage servers are out of order or operating abnormally. There is no problem in the operation of the entire system. However, if there are too many failed or abnormally distributed distributed storage servers, the load will be concentrated on the distributed storage servers that are operating normally, resulting in a decrease in overall performance, a decrease in total storage capacity, and a certain number of data that can be decrypted. There is a problem that the encoded file cannot be secured, but in this embodiment, all the encoded files on any one distributed disk 101 that has failed or malfunctioned can be restored. Reliability and maintainability can be improved.

  Also, in this embodiment, it is confirmed whether or not the plurality of distributed disks 101 are operating normally, and the administrator is notified of the information about the distributed disks 101 whose normal operation has not been confirmed. After the operation is confirmed, restoration can be performed immediately.

  In this embodiment, the user is not aware that there are a plurality of super nodes 104 and operates as one virtual super node, but actually has a plurality of super nodes 104 and distributes the load. be able to. Further, as described above, the super node 104 includes the original file, the original file name, the original file No. Since management information such as the storage destination distributed disk node name is not stored, maintenance such as addition / deletion of the super node 104 is easy.

  As described above, in the present embodiment, it is possible to obtain the distributed storage system 100 with higher security in terms of information concealment, reliability, maintainability, and the like.

Embodiment 2. FIG.
In the present embodiment, an example in which the management terminal 102 and the super node 104 described in the first embodiment are combined into one super node 104 will be described.

  Hereinafter, the detailed configuration of the distributed storage system 100 according to the present embodiment will be described with reference to FIG. The distribution disk 101 has an encoded data storage unit 201. The super node 104 has only one configuration, and includes a distributed disk storage unit 2 (206), a division encoding unit 207, a distribution unit 208, a file management unit 209, and a reproduction unit 210. Furthermore, the super node 104 includes a file management information storage unit 203, a distributed disk state confirmation unit 204, and a distributed disk restoration unit 205. The user terminal 106 includes a super node storage unit 211, a storage request unit 212, and a read request unit 213. Since the distributed disk storage unit 1 (202) in the first embodiment stores the same information as the distributed disk storage unit 2 (206), it is not necessary in the present embodiment.

  In the first embodiment, the processing flow of data storage, data reading, and distributed disk restoration described with reference to FIGS. 7 to 12 is the same as in the second embodiment except for the following points. In this embodiment, since there is only one super node 104, the process of selecting any one super node from a plurality of super nodes is omitted. Further, the management terminal 102 in the first embodiment is read as the super node 104 in the present embodiment. Further, the distributed disk storage unit 1 (202) in the first embodiment is read as the distributed disk storage unit 2 (206).

  Hereinafter, data storage processing according to the present embodiment will be described with reference to FIGS. 7, 8, and 9. In FIG. 7, step S701 is omitted in the present embodiment (step S701). Next, the storage request unit 212 of the user terminal 106 transmits an original file to be stored and a storage instruction to the super node 104 (step S701). Next, as in the first embodiment, the original file is divided and encoded to create an encoded file (step S703). Next, as in the first embodiment, the encoded file is transmitted to the distributed disk (step S704). Next, the file management unit 209 of the super node 104 generates file management information and transmits it to the file management information storage unit 203 of the super node 104 (step S705). FIG. 8 explains the detailed operation of the above-described step S704, but this embodiment is the same as in the first embodiment, and the distributed disk 101 in which the distribution unit 208 of the super node 104 is operating. 150 are arbitrarily selected from the above, and the encoded file is transmitted, and the encoded data storage unit 201 of the distributed disk 101 stores the encoded file.

  FIG. 9 explains the detailed operation of step S704 described above. In the present embodiment, file management information is created as in the first embodiment (step S901). Next, the file management unit 209 of the super node 104 transmits file management information to the file management information storage unit 203 of the super node 104 (step S902). Next, the file management information storage unit 203 of the super node 104 stores the received file management information (step S903). Next, the file management information storage unit 203 of the super node 104 responds to the file management unit 209 of the super node 104 that it has been normally received (step S904).

  Next, the data reading process in the present embodiment will be described with reference to FIG. In this embodiment, step S1001 is omitted (step S1001). Next, the read request unit 213 of the user terminal 106 transmits an original file name to be read and a read instruction to the super node 104 (step S1002). Next, as in the first embodiment, the playback unit 210 of the super node 104 that has received the read instruction generates the source file from all the distributed disks 101 stored in the distributed disk storage unit 2 (206). A confirmation request is made as to whether or not the encoded file is stored (step S1003).

  Next, as in the first embodiment, the encoded data storage unit 201 of the distributed disk 101 returns a response when the specified encoded file is stored (step S1004). Next, from step S1004 to step 1008, as in the first embodiment, the reproducing unit 210 of the super node 104 that has received the response acquires 105 encoded data from the distributed disk 101 that has responded. (Step S1005). Next, the encoded data is decoded, and a decoded file having the same contents as the original file is reproduced (step S1006). Next, the decrypted file is transmitted to the user terminal 106 (step S1007). Next, the read request unit 213 of the user terminal 106 receives the decrypted file (step S1008).

  Next, the distributed disk restoration process according to the present embodiment will be described with reference to FIGS. 11 and 12, the detailed operation is the same as that of the first embodiment except that the management terminal 102 in the first embodiment is replaced with the super node 104.

  In FIG. 11, first, the distributed disk status confirmation unit 204 of the super node 104 issues a confirmation request to the all distributed disks 101 as to whether it is operating normally (step S1101). Next, the distributed disk 101 that has received the confirmation request responds if it is operating normally (step S1102). Next, the distributed disk status confirmation unit 204 of the super node 104 determines that the distributed disk 101 that has not responded for normal operation for a certain period of time is a failure and raises an alarm (step S1103). Next, the failed distribution disk 101 is replaced or repaired manually (step S1104). Next, the distributed disk 101 notifies the super node 104 of the normal operation of itself and the restoration request (step S1105). Next, the distributed disk restoration unit 205 of the super node 104 automatically performs data restoration processing on the distributed disk 101 that has been operating normally after the failure (step S1106).

  Next, the detailed processing flow of step S1106 of FIG. 11 will be described with reference to FIG. The distributed disk restoration unit 205 of the super node 104 refers to the distributed disk storage unit 2 (206) and recognizes the distributed disk node name of the distributed disk 101 that has received the restoration request. (Step S1201) Next, the distributed disk restoration unit 205 refers to the file management information storage unit 203, and from the file management information, the original file name corresponding to the recognized distributed disk node name and the encoded file number. Are searched (step S1202). Next, the distributed disk restoration unit 205 repeats the processing from step S1203 to step S1208 for each retrieved original file name.

  The distributed disk restoration unit 205 stores the encoded file number. To the other encoded file No. required for decoding the original file. And the other encoded file No. The active distributed disk node name corresponding to is determined (step S1203). Next, the distributed disk restoration unit 205 reads the encoded file No. required for restoration from the active distributed disk 101. An encoded file corresponding to is acquired (step S1204). Next, the distributed disk restoration unit 205 reproduces a decoded file from the acquired encoded file (step S1205).

  Next, the distributed disk restoration unit 205 divides and decodes the decoded file, and generates an encoded file to be restored (step S1206). Next, the distributed disk restoration unit 205 transmits the encoded file to the distributed disk 101 that has received the restoration request (step S1207). Next, the distributed disk restoration unit 205 checks whether the processing has been completed for all the original files searched in step S1202, and if completed, ends step S1106, and if not completed, returns to step S1203. The process up to step S1208 is repeated (step S1208).

  In the first embodiment and the present embodiment, as described in FIG. 5, the name of the encoded file includes the original file name, _ (underscore), encoded file No. Are combined. Further, as described in FIGS. 5 and 6, the file management information is used to specify the original file name that is the name of the original file to be stored and the encoded file obtained by dividing and encoding the original file. Encoded file No. And the distributed disk node name for identifying the distributed disk 101 that stores the encoded file.

  Based on such data, in the data storage process, in step S703 of FIG. 7, when the encoded file is generated, the name of the encoded file is the original file name, _ (underscore), and the encoded file. No. Are combined. In the data reading process, in step S1003 of FIG. 10, the playback unit 210 requests the all distributed disks 101 to check whether or not the encoded file generated from the original file is stored. The original file name is transmitted together with the confirmation request. In step S1004, the encoded data storage unit 201 searches using the original file name received in step S1003 as a key, and returns a response when the encoded file generated from the original file is stored.

  However, in the present embodiment, the encoded file name may be simply given a unique encoded file name in the encoded data storage unit 201 without providing the above-described restrictions on the encoded file name. In this case, the file management information described in FIG. 5 and FIG. 6 includes the original file name that is the name of the original file to be stored, and the “code” for specifying the encoded file obtained by dividing and encoding the original file. This is information in which the “encoded file name” is associated with the distributed disk node name for specifying the distributed disk 101 storing the encoded file.

  Then, on the premise of the data (encoded file name and file management information) as described above, in step S703 in FIG. 7, in the data storage process, the division encoding unit 207 of the super node 104 sets the received original file as 100. The encoded file is divided into pieces and encoded to generate 150 encoded files. The encoded data storage unit 201 assigns a unique name to the encoded file name. In step S901 in FIG. 9, the file management unit 209 of the super node 104 creates file management information. The file management information includes the original file name that is the name of the original file to be stored, and the original file. This is information in which an “encoded file name” for specifying an encoded file divided and encoded is associated with a distributed disk node name for specifying the distributed disk 101 storing the encoded file. Information relating to the correspondence between the original file name and the “encoded file name” is acquired from the division encoding unit 207, and information relating to the correspondence between the “encoded file name” and the distributed disk node name is obtained from the distribution unit. Obtained from 208.

  In the data reading process on the premise of the data (encoded file name and file management information) as described above, the file management information stored in the file management information storage unit 203 is referred to in step S1003 of FIG. Thus, it is possible to recognize the encoded file name corresponding to the original file name and the distributed disk node name corresponding to each of the aforementioned encoded files, so that the reproduction unit 210 of the super node 104 uses the file management information. Then, the distributed disk 101 that stores the encoded file generated from the original file is specified, and a normal operation confirmation request is made to the corresponding distributed disk 101. In step S1004, the encoded data storage unit 201 returns a response if it is operating normally.

  Then, assuming the data (encoded file name and file management information) as described above, the flow of the process in FIG. 12 is as follows in the data restoration process. The distributed disk restoration unit 205 of the super node 104 refers to the distributed disk storage unit 2 (206) and recognizes the distributed disk node name of the distributed disk 101 that has received the restoration request. (Step S1201) Next, the distributed disk restoration unit 205 refers to the file management information storage unit 203 and searches all the original file names and encoded file names corresponding to the recognized distributed disk node names from the file management information. (Step S1202). Next, the distributed disk restoration unit 205 repeats the processing from step S1203 to step S1208 for each retrieved original file name.

  The distributed disk restoration unit 205 converts the “encoded file name searched in step S1202” into “another encoded file name” necessary for decoding the original file and the “other encoded file name”. A corresponding active distributed disk node name is determined (step S1203). That is, the distributed disk restoration unit 205 refers to the file management information storage unit 203 and first specifies the original file name corresponding to the “encoded file name searched in step S1202”. Next, referring to the file management information storage unit 203, for each identified original file name, the encoded file name of the encoded file created by dividing and encoding the original file is specified. Next, the aforementioned “other encoded file names” are obtained by omitting “the encoded file names searched in step S1202” from all the specified encoded file names. Next, the file management information storage unit 203 is referred to and an active distributed disk node name corresponding to the “other encoded file name” is determined.

  Next, the distributed disk restoration unit 205 acquires an encoded file having “another encoded file name” necessary for restoration from the active distributed disk 101 (step S1204). Next, the distributed disk restoration unit 205 reproduces a decoded file from the acquired encoded file (step S1205).

  Next, the distributed disk restoration unit 205 divides and encodes the decoded file, and generates an encoded file to be restored (an encoded file having the “encoded file name searched in step S1202”) (step S1206). Here, division and encoding are performed in the same procedure as step S703 on the premise of the data (encoded file name and file management information) as described above. The decoded file is divided and encoded to create 150 encoded files. Among them, only the encoded file having the “encoded file name searched in step S1202” is left and the others are deleted. Next, the distributed disk restoration unit 205 transmits the encoded file created in step S1206 to the distributed disk 101 that has received the restoration request (step S1207). Next, the distributed disk restoration unit 205 confirms whether or not the processing has been completed for all the original files searched in step S1202, and if completed, ends step S1106, and if not completed, returns to step S1203. The process up to step S1208 is repeated (step S1208).

  As described above, in this embodiment, even if the configuration has only one super node and no separate management terminal, the super node 104 stores the file management information and uses the file management information. Thus, it is possible to restore all of the encoded files on any one distributed disk 101, so that it becomes easy to repair and replace the distributed disk 101 due to a failure or other circumstances. Similarly, maintainability of the distributed storage system 100 can be improved.

It is a figure which shows the structure of the distributed storage system in Embodiment 1 of this invention. It is a block diagram which shows the detailed structure of the distributed storage system in Embodiment 1 of this invention. It is a figure which shows an example of the hardware resource in Embodiment 1 of this invention. It is a schematic diagram which shows an example of the encoding and decoding of the file in Embodiment 1 of this invention. It is a figure of the example of data of the management information in Embodiment 1 of this invention. It is a figure of the example of data of the file management information storage part in Embodiment 1 of this invention. It is a flowchart which shows the detail of operation | movement of the distributed storage system which concerns on the data storage in Embodiment 1 of this invention. It is a flowchart which shows the detail of operation | movement of the distributed storage system which concerns on the data storage in Embodiment 1 of this invention. It is a flowchart which shows the detail of operation | movement of the distributed storage system which concerns on the data storage in Embodiment 1 of this invention. It is a flowchart which shows the detail of operation | movement of the distributed storage system which concerns on the data reading in Embodiment 1 of this invention. It is a flowchart which shows the detail of operation | movement of the distributed storage system which concerns on the distribution disk restoration in Embodiment 1 of this invention. It is a flowchart which shows the detail of operation | movement of the distributed storage system which concerns on the distribution disk restoration in Embodiment 1 of this invention. It is a block diagram which shows the detailed structure of the distributed storage system in Embodiment 2 of this invention. It is a figure which shows the structure of an example of the conventional exclusive storage system. It is a figure which shows the structure of the conventional data distributed storage system. It is a flowchart which shows the detail of the operation | movement regarding the data storage of the conventional data distributed storage system. It is a flowchart which shows the detail of the operation | movement in connection with the data reproduction | regeneration of the conventional data distributed storage system.

Explanation of symbols

  100 distributed storage system, 101 distributed disk, 102 management terminal, 103 administrator network, 104 super node, 105 WAN (Wide Area Network), 106 user terminal, 201 encoded data storage unit, 202 distributed disk storage unit 1, 203 file Management information storage unit, 204 Distributed disk status confirmation unit, 205 Distributed disk restoration unit, 206 Distributed disk storage unit 2, 207 Division encoding unit, 208 Distribution unit, 209 File management unit, 210 Playback unit, 211 Super node storage unit, 212 storage request unit, 213 read request unit, 301 display device, 302 K / B, 303 mouse, 304 FDD, 305 CDD, 311 CPU, 312 bus, 313 ROM, 314 RAM, 315 communication box 320, magnetic disk drive, 321 OS, 322 window system, 323 program group, 324 file group.

Claims (7)

A distributed storage system having a super node connected to a plurality of distributed disks and one or a plurality of user terminals via a network and receiving a file storage request and a read request,
The super node is
When receiving the original file to be distributed and the storage request for the original file from the user terminal, a plurality of encoded files obtained by dividing and encoding the original file are generated, and the encoded file names A division encoding unit having a name including a character string for specifying the correspondence with the original file,
A distribution unit that requests storage of each of the plurality of encoded files on a plurality of different distributed disks;
After the plurality of encoded files are stored on a distributed disk, the original file, the plurality of encoded files, information that can specify the original file or the plurality of encoded files, and the plurality of encoded files are stored. A file management unit that deletes information that can identify a plurality of distributed files,
When receiving the original file name and the read request of the original file from the user terminal, it is confirmed whether or not the encoded file generated from the original file is stored in the distributed disk, A reproduction unit that reproduces the original file by obtaining and decoding a predetermined number of the encoded files from the distributed disk whose storage has been confirmed by comparing the file and the encoded file name; Distributed storage system characterized by The distributed storage system further includes a management terminal connected to the super node via a network,
The file management unit further generates file management information indicating correspondence between the original file, the encoded file, and the distributed disk, transmits the file management information to the management terminal, deletes the file management information that has been transmitted,
The management terminal
A file management information storage unit for storing the file management information received from the file management unit;
Based on the request for restoration of any distributed disk, the encoded file stored in the distributed disk and the original file corresponding to the encoded file are obtained from the file management information stored in the file management information storage unit. Identifying and obtaining other encoded files stored on a distributed disk other than the distributed disk necessary for decoding the original file, decoding the original file from the other encoded files, A distributed disk restoration unit that regenerates the encoded file by dividing and encoding the decoded original file, and requests the storage of the regenerated encoded file in the distributed disk; The distributed storage system according to claim 1. The management terminal further includes:
Checks whether multiple distributed disks are operating normally, outputs an alarm for distributed disks that could not be confirmed to operate normally, and then started normal operation from the distributed disks that could not be confirmed to operate normally. 3. The distributed storage system according to claim 2, further comprising a distributed disk state confirmation unit that instructs the distributed disk restoration unit to restore the distributed disk that could not confirm the normal operation when the notification is received. The distributed storage system further includes a plurality of the super nodes and user terminals connected to the plurality of super nodes via a network,
The user terminal is
4. The distributed storage system according to claim 1, wherein an arbitrary one super node that is operating normally is selected from the plurality of super nodes, and a file storage request and a read request are made. . A distributed storage system having a super node connected to a plurality of distributed disks and one or a plurality of user terminals via a network and receiving a file storage request, a read request, and a distributed disk restoration request,
The super node is
When receiving an original file to be distributed and a storage request for the original file from the user terminal, a division encoding unit that generates a plurality of encoded files obtained by dividing and encoding the original file;
A distribution unit that requests storage of each of the plurality of encoded files on different distributed disks;
A file management unit that generates file management information indicating correspondence between the original file, the encoded file, and the distributed disk;
A file management information storage unit for storing the file management information;
When receiving the original file name and the read request of the original file from the user terminal, the file management information identifies a distributed disk that stores the encoded file generated from the original file, and Then, a normal operation is confirmed, a predetermined number of encoded files are obtained from the confirmed distributed disk, and the playback unit that reproduces the original file and decoding request for any distributed disk are obtained by decoding. Based on the file management information, the encoded file stored in the distributed disk and the original file corresponding to the encoded file are identified from the file management information stored in the file management information storage unit, and the original file is decoded. Identify and capture other encoded files stored on a distributed disk other than the distributed disk Then, the original file is decoded from the other encoded files, the decoded original file is divided and encoded, and the encoded file is regenerated and regenerated on the distributed disk. A distributed storage system comprising: a distributed disk restoration unit that requests storage of the encoded file. When receiving an original file to be distributed and a storage request for the original file from a user terminal connected via a network, a plurality of encoded files obtained by dividing and encoding the original file are generated. , A division encoding process in which the encoded file name is a name including a character string for specifying correspondence with the original file;
A delivery process for requesting storage of each of the plurality of encoded files to a plurality of different distributed disks connected via a network;
After the plurality of encoded files are stored on a distributed disk, the original file, the plurality of encoded files, information that can specify the original file or the plurality of encoded files, and the plurality of encoded files are stored. File management process to delete information that can identify multiple distributed files,
When an original file name and a request to read the original file are received from a user terminal connected via a network, whether or not the encoded file generated from the original file is stored in a distributed disk Confirming and comparing the original file with the encoded file name, obtains a predetermined number of the encoded files from the distributed disk whose storage has been confirmed, and reproduces the original file by decoding. A distributed storage program characterized by causing a computer to execute a reproduction process. When an original file to be distributed and a storage request for the original file are received from a user terminal connected via a network, a plurality of encoded files obtained by dividing and encoding the original file are generated. Division coding processing,
A delivery process for requesting storage of each of the plurality of encoded files to different distributed disks connected via a network;
A file management process for generating file management information indicating the correspondence between the original file, the encoded file, and the distributed disk;
A file management information storage process for storing the file management information;
When an original file name and a request to read the original file are received from a user terminal connected via a network, a distributed disk that stores an encoded file generated from the original file is specified by the file management information. The normal operation of the distributed disk is confirmed, a predetermined number of encoded files are obtained from the confirmed distributed disk, and decoding is performed to reproduce the original file. Based on the request to restore the distributed disk, the encoded file stored in the distributed disk and the original file corresponding to the encoded file are identified from the file management information stored in the file management information storage process. Stored on a distributed disk other than the distributed disk required to decrypt the original file Identifying and obtaining the encoded file, decoding the original file from the other encoded files, dividing the decoded original file, and encoding to regenerate the encoded file; A distributed storage program which causes a computer to execute a distributed disk restoration process for requesting storage of the regenerated encoded file in the distributed disk.

Images