JP2017182286A - Storage management system and storage management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce storage cost, and to reduce response time required in reading data.SOLUTION: A storage management system applies identifier to data transmitted together with a request, and transmits data, on receipt of a write request, to a storage device of a plurality of storage device, which is determined in accordance with the identifier and a predetermined rule and stores the data as master data, and a storage device different from the above storage device and storing the data as replica data of the master data. Each of the storage devices includes first storage means of determining whether to store data as master data or replica data, on the basis of an identifier and a predetermined rule, to store the data, and second storage means which is storage means of a migration destination. When the data is determined to be stored as replica data, the data is stored in the second storage means.SELECTED DRAWING: Figure 10

Description

  The present invention relates to a technology for managing storage devices.

  Storage systems that create and store replicas for the purpose of data redundancy have been proposed (Patent Documents 1 and 2). However, when only a magnetic disk is used in a storage system for storing redundant data, the magnetic disk needs to store not only master data but also replica data of the master data. It is necessary to secure a storage area. In order to reduce the storage cost for such a storage area, a method for preventing the storage device from running out of free space has been proposed (Patent Document 3).

  In addition, since the magnetic disk used in the storage system consumes power even in an idle state in which data is not accessed, the magnetic disk is expensive in terms of power consumption. Patent Document 4 describes a technology related to a storage cluster that saves power.

  Non-Patent Document 1 describes an object storage that handles data in units of objects as an example of a storage system.

Japanese Patent Laying-Open No. 2015-156071 International Publication No. 2008/120340 Japanese Patent Laid-Open No. 2003-015917 JP 2014-194803 A

Kitsuregawa Yuu, "The Age of Storage Technology Cloud and Big Data", 1st Edition, Ohm Corporation, May 15, 2015, p. 121-132

  As a method for reducing the storage cost, when a general HSM (Hierarchical Storage Management) function is used, data that is not frequently used is stored on the tape. When master data that is used infrequently is stored on a tape, if the master data is read out, it takes time to read out the data because it is read from the tape.

  In the technique related to Patent Document 1, for each data, a master node that stores the data as a master and a slave node that stores the data as a slave are set to make data redundant. However, when the HSM function is applied to the technology related to Patent Document 1, data that is used infrequently moves to the tape in both the master node and the slave node, and the data that has been moved to the tape is read. Takes time. Similarly, in the technique related to Patent Document 3, data that is used infrequently moves to the tape, and it takes time to read the data moved to the tape.

  Further, in the case of applying a storage that stores the data as a master and a storage that stores the data as a slave to the technology related to Patent Document 2, a disk storage that stores primary data and a secondary data are stored. The tape storage is configured in the same device. In this case, the same data exists in the same device, and the storage cost increases.

  Further, in the technique related to Patent Document 4, when data is accessed to an idle node, processing is performed after returning from the idle state, and thus it takes a long time to respond.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for reducing the storage cost and shortening the response time required for reading data.

  A storage management system according to an aspect of the present invention includes a client terminal, a proxy server, and a plurality of storage devices, and the proxy server assigns an identifier to data transmitted together with a request from the client terminal. When the request is a write request, a storage device that stores the data as master data, which is determined according to the identifier and a predetermined rule, among the plurality of storage devices, and a storage device different from the storage device A storage device that stores the data as replica data of the master data, and each of the plurality of storage devices has its own proxy server. Store data sent from as master data Determination means for determining whether to store as replica data based on the identifier and the predetermined rule, a first storage means for storing the data transmitted from the proxy server, a migration destination A second storage unit that is a storage unit, and when the determination unit determines to store the data transmitted from the proxy server as replica data, the determination unit stores the data in the second storage unit.

  A storage management method according to an aspect of the present invention is a storage management method for a storage management system including a client terminal, a proxy server, and a plurality of storage devices, and the proxy server receives a request from the client terminal. An identifier is assigned to the transmitted data, and when the request is a write request, a storage device that stores the data as master data determined according to the identifier and a predetermined rule among the plurality of storage devices; The storage device is different from the storage device, and the data is transmitted to a storage device that stores the data as replica data of the master data, and each of the plurality of storage devices is transmitted from the proxy server. A first storage for storing the processed data And a second storage unit that is a storage unit of a migration destination, and the identifier and the predetermined data indicate whether the device itself stores data transmitted from the proxy server as master data or replica data If it is determined to store the data transmitted from the proxy server as replica data, the data is stored in the second storage means.

  According to the present invention, the storage cost can be reduced, and the response time required for reading data can be shortened.

It is a figure for demonstrating the outline | summary of the storage management system which concerns on 1st Embodiment of this invention. It is a block diagram which shows an example of a structure of the proxy server which concerns on 1st Embodiment of this invention. It is a figure for demonstrating an access rule. 1 is a block diagram illustrating an example of a configuration of a storage device according to a first embodiment of the present invention. It is a flowchart which shows an example of operation | movement of the storage management system which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the storage management system which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the storage management system which concerns on 1st Embodiment of this invention. It is a block diagram which shows an example of a structure of the storage management system which concerns on 2nd Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the storage management system which concerns on 2nd Embodiment of this invention. It is a block diagram which shows an example of a structure of the storage management system which concerns on 3rd Embodiment of this invention. It is a block diagram which shows an example of the hardware constitutions of the information processing apparatus which implement | achieves the assistance apparatus which concerns on each embodiment of this invention. It is a flowchart explaining the operation | movement of the migration process using HSM (Hierarchical Storage Management) relevant to this invention.

  Hereinafter, the present invention will be described in detail with reference to the drawings as appropriate.

<First Embodiment>
FIG. 1 is a block diagram for explaining an overview of the storage management system 1 according to the first embodiment of the present invention. The storage management system 1 includes a client terminal 10, a proxy server 20, a storage device 30, and a storage device 40. The client terminal 10 and the proxy server 20 are connected to each other via a wired or wireless network. The proxy server 20, the storage device 30, and the storage device 40 are connected to each other by, for example, an IP (Internet Protocol) network. The number of storage devices may be two or more. Note that the number of client terminals may be one or more.

  The client terminal 10 sends a request including operation information indicating an operation (writing or reading) to data, data specifying data or data to be operated, and terminal information specifying the client terminal 10 to the proxy server. 20 to send. A plurality of client terminals 10 may be provided.

  When the operation indicated by the operation information is writing, the request includes operation information, an object (data) to be operated, and terminal information. When the operation indicated by the operation information is a read operation, the request includes operation information, information for specifying an object (data) to be operated, and terminal information. The information for specifying data to be operated may be an object name, for example. Hereinafter, data is also referred to as an object.

  The proxy server 20 receives a request from the client terminal 10 and performs processing using the received request. The proxy server 20 assigns an object ID that is an ID (Identifier) that identifies the object to the object included in the received request. Then, the proxy server 20 multiplexes the object to which the object ID is allocated (post-allocation object) and transmits the multiplexed object to the plurality of storage devices (30, 40). Thereby, when the operation information included in the request indicates writing, the proxy server 20 can provide redundancy to the data (object) to be written. In addition, object ID, the member which the proxy server 20 has, and the specific process which the proxy server 20 performs are mentioned later.

  The storage device 30 and the storage device 40 receive the allocated object from the proxy server 20. The storage device 30 and the storage device 40 convert the received object ID of the allocated object into a file ID. The file ID will be described later. Then, the storage device 30 and the storage device 40 write data or read data using the file ID and the allocated object. Hereinafter, members of the proxy server 20, the storage device 30, and the storage device 40 will be described with reference to FIGS.

  FIG. 2 is a block diagram showing the storage management system 1 according to the first embodiment of the present invention, and in particular an example of a detailed configuration of the proxy server 20. The proxy server 20 includes an allocation unit 21, a calculation unit 22, a change unit 23, and a server storage unit 24.

  For example, the allocating unit 21 receives a request from the client terminal 10, and when the received request is a write, the allocating unit 21 allocates an object ID that identifies the object to the object included in the request. Further, when the request is read, the assigning unit 21 displays the object ID assigned to the object specified by the information (for example, the object name) specifying the operation target data included in the request as the information. Associate with. Hereinafter, description will be made using information for specifying data to be operated as an object name. Then, the assigning unit 21 sets the object to which the object ID is assigned or the object name associated with the object ID as the assigned object, and transmits the assigned object to the calculating unit 22. When the request is read, the post-assignment object may include only the object ID. The object ID assigned here is a unique character string, for example, a GUID (Global Unique Identifier). Further, the assigning unit 21 associates the object name with the object ID and stores them in the server storage unit 24 in order to manage which object ID is assigned to which object. Therefore, the same object ID is assigned to the same object.

  The calculating unit 22 receives the allocated object from the allocating unit 21 and calculates a hash value based on the hash function using the received object ID of the allocated object. The hash value is a fixed length value without regularity. Then, the calculation unit 22 refers to the access rule stored in the server storage unit 24 based on the hash value, and determines a storage device that transmits the post-allocation object. Thereafter, the calculation unit 22 transmits the allocated object to the determined storage device. The access rule will be described later.

  Further, the calculation unit 22 receives data from the storage device 30, for example. Then, the received data is transmitted to the client terminal 10 as a response to the request transmitted from the client terminal 10. For example, the calculation unit 22 may assign an object ID to the data received from the storage device 30 so that the client terminal 10 can recognize which request is a response.

  FIG. 3 is a diagram for explaining the access rule. As shown in FIG. 3, the access rule includes a hash value that identifies data, a location where data is stored as master data (storage location of master data), and a location where data is stored as replica data (storage location of replica data). Are associated with each other. Hereinafter, data stored as master data is referred to as master data, and data stored as replica data is referred to as replica data.

  Further, in each column of the master data storage location and the replica data storage location, storage number information that is information for specifying a storage device is stored. For this reason, the calculation unit 22 can uniquely determine the storage apparatus that transmits the allocated object from the plurality of storage apparatuses by referring to the access rule based on the hash value of the data. Further, in the storage management system 1, consistent hashing may be used as an algorithm for transmitting the allocated object to a plurality of storage apparatuses. The access rule may be set by the user using the consistent hashing.

  Specifically, when the hash value is “hash 3” and the request is a write, the calculation unit 22 “master device storage location” 40 “storage device 40” and replica data storage location “storage device 30”. And transmits the allocated object to the determined storage apparatus 30 and storage apparatus 40.

  When the request is read with the hash value “hash 2”, the calculation unit 22 determines to transmit the allocated object to the “storage device 30” that is the storage location of the master data, and the determined storage device 30 Send the assigned object to.

  The changing unit 23 appropriately monitors the status of the storage device 30 and the storage device 40 via the storage device 30 and the processing unit 33 included in the storage device 40, which will be described later, and the server storage unit according to the result of the migration process described later. The access rule stored in 24 is changed. The access rule may be changed by the user.

  For example, data may be biased to either the storage device 30 or the storage device 40. The change unit 23 monitors the storage device 30 and the storage device 40. If the data is biased to either the storage device 30 or the storage device 40, the change unit 23 balances with the free capacity or the data capacity of the storage device 30 and the storage device 40. Change the access rules so that

  Then, the change unit 23 transmits the changed access rule to the storage device 30 and the storage device 40. For example, when the access rule stored in the server storage unit 24 is changed by the user, the changing unit 23 transmits the changed accelerator rule to the storage device 30 and the storage device 40. That is, the changing unit 23 performs file synchronization so that the access rules of the proxy server 20 and the access rules of the storage device 30 and the storage device 40 have the same content.

  FIG. 4 is a block diagram showing an example of the configuration of the storage device 30 and the storage device 40 according to the first embodiment of the present invention. The storage device 30 includes a management unit 31, a storage storage unit 32, a processing unit 33, a disk 34, a tape 35, and a determination unit 36.

  The management unit 31 receives the allocated object from the proxy server 20 and converts the received object ID of the allocated object into a file ID. The file ID is, for example, an ID for specifying data when the processing unit 33 writes or reads data stored in the disk 34 or the tape 35 in units of files. For example, the file ID is described in a URI (Uniform Resource Identifier) format including a path and a directory. For example, when software called “Swift” is used to construct an object storage, the object ID is included in the file ID, and therefore the object ID can be uniquely obtained from the file ID. Then, the management unit 31 transmits the file ID and the allocated object to the processing unit 33. The management unit 31 may store the file ID and the object ID in the storage storage unit 32 in association with each other.

  In addition, the management unit 31 receives data from the processing unit 33 and transmits the received data to the calculation unit 22 of the proxy server 20. Further, the management unit 31 receives the changed access rule from the proxy server 20 and stores the received access rule in the storage storage unit 32.

  The management unit 31 acquires an access rule and information indicating the own device from the storage storage unit 32. Then, the management unit 31 transmits the acquired access rule and information indicating the own device to the determination unit 36. For example, the storage storage unit 32 of the storage device 30 stores information indicating that the own device is the storage device 30 as information indicating the own device.

  The processing unit 33 receives the file ID and the allocated object from the management unit 31, and writes or writes the data included in the allocated object to the disk 34 and the tape 35 based on the received file ID. Data indicated by the object name included in the assigned object is read. Further, the processing unit 33 has, for example, a virtual file system function. Specifically, the processing unit 33 has a function of making the disk 34 and the tape 35 appear virtually to the upper side as the same storage device.

  The processing unit 33 manages the data stored in the disk 34 or the tape 35 for each file based on the file ID. Hereinafter, a file ID and data related to the file ID are also referred to as a file. The processing unit 33 associates information (file information) related to data such as the position (position information) where the file is stored on the disk or tape, the frequency of access to the data, and the file ID for each file as a file information table. It is stored in its own storage unit (not shown). The processing unit 33 writes the file on the disk 34 or the tape 35.

  Specifically, when the allocated object includes an object ID and an object, the processing unit 33 writes a file related to the object on the disk 34. Further, when the allocated object includes an object ID and an object name, the processing unit 33 reads a file related to the object name from the disk 34.

  Thereafter, the processing unit 33 transmits data included in the read file to the management unit 31. Then, the management unit 31 receives data from the processing unit 33 and transmits the received data to the client terminal 10 via the proxy server 20.

  The determination unit 36 refers to the file information table of the processing unit 33 periodically and acquires a file ID. Then, the determination unit 36 acquires the object ID based on the acquired file ID. Then, the determination unit 36 calculates a hash value based on the acquired object ID. Thereafter, the determination unit 36 determines whether or not the file indicated by the acquired file ID is replica data, using the access rule acquired from the management unit 31, information indicating the own device, and the calculated hash value. . If the file is replica data, the determination unit 36 moves the file to the tape 35. The determination unit 36 has a migration service function, for example. When the determination unit 36 moves the file to the tape 35, the determination unit 36 instructs the processing unit 33 to update the position information included in the file information of the file.

  When receiving the allocated object from the proxy server 20, the management unit 31 does not know whether the data included in the allocated object is replica data or master data. Further, the processing unit 33 also does not know whether the data included in the allocated object received from the management unit 31 is replica data or master data. The determination unit 36 can determine whether the data is replica data or master data for the first time by performing the determination described above.

  Therefore, the storage device 30 and the storage device 40 can add the data transmitted from the proxy server 20 to the master data according to the access rule without adding information that allows the proxy server 20 to determine whether the master data or the replica data is included in the data. Or replica data.

  FIG. 5 is a flowchart showing an example of the operation of the storage management system 1 according to the first embodiment of the present invention. Specifically, an example of the write processing operation of the storage management system 1 will be described. The dashed arrows between the processing of the proxy server 20 and the processing of the storage device 30 and between the processing of the proxy server 20 and the processing of the storage device 40 indicate the flow of information.

  First, the client terminal 10 transmits a request to the proxy server 20 (step S101). The assigning unit 21 receives a request from the client terminal 10 (step S102), and assigns an object ID to an object included in the received request (step S103). Then, the assigning unit 21 transmits the object (assigned object) to which the object ID is assigned to the calculating unit 22.

  Thereafter, the calculation unit 22 receives the allocated object from the allocation unit 21, and calculates a hash value based on the hash function using the object ID included in the received allocated object (step S104). Further, the calculation unit 22 refers to the access rule stored in the server storage unit 24 using the calculated hash value, and determines a storage device to which the post-allocation object is transmitted (step S105). Then, the calculation unit 22 transmits the allocated object to the storage device determined in step S105 (in this example, the storage device 30 and the storage device 40) (step S106).

  Then, the management unit 31 of each storage apparatus receives the allocated object from the proxy server 20 (step S107, step S109). Thereafter, the management unit 31 converts the received object ID of the post-assignment object into a file ID (step S108, step S110). Then, the management unit 31 transmits the file ID and the allocated object to the processing unit 33. Thereafter, the processing unit 33 receives the file ID and the allocated object from the management unit 31.

  Thereafter, the processing unit 33 performs processing based on the file ID and the assigned object. In this example, since it is a writing process, the processing unit 33 writes the file to the disk 34 based on the file ID (steps S111 and S112). Further, after step S <b> 111 or step S <b> 112 is completed, the storage apparatus 30 may transmit ACK (ACKnowledgement) to the client terminal 10 via the proxy server 20.

  FIG. 6 is a flowchart showing an example of the operation of the storage management system 1 according to the first embodiment of the present invention. Specifically, an example of the read processing operation of the storage management system 1 will be described. A dashed arrow between the processing of the proxy server 20 and the processing of the storage device 30 indicates the information flow. The following processing will be described on the assumption that the storage device 30 stores master data to be read as an example.

  First, the client terminal 10 transmits a request to the proxy server 20 (step S201). The assigning unit 21 receives the request from the client terminal 10 (step S202), and associates the object ID assigned to the object indicated by the object name included in the received request with the object name (step S203). Then, the assignment unit 21 transmits the object name (object after assignment) associated with the object ID to the calculation unit 22.

  Thereafter, the calculation unit 22 receives the allocated object from the allocation unit 21, and calculates a hash value based on the hash function using the object ID included in the received allocated object (step S204).

  Further, the calculation unit 22 refers to the access rule stored in the server storage unit 24 using the calculated hash value, and determines that the storage device that transmits the allocated object is, for example, the storage device 30 (step S30). S205). Then, the calculation unit 22 transmits the allocated object to the storage device 30 determined in step S205 (step S206).

  The management unit 31 of the storage device 30 receives the allocated object from the proxy server 20 (step S207). Thereafter, the management unit 31 converts the received object ID of the allocated object into a file ID (step S208), and transmits the file ID and the allocated object to the processing unit 33.

  Next, the processing unit 33 receives the file ID and the allocated object from the management unit 31. Then, the processing unit 33 performs processing based on the received file ID and the assigned object. In this example, since it is a reading process, the processing unit 33 reads a file from the disk 34 based on the file ID (step S209).

  Thereafter, the processing unit 33 transmits the read file data to the management unit 31. And the management part 31 receives a file from the process part 33, and transmits only the data of the received file to the proxy server 20 as a response of the allocated object transmitted from the proxy server 20 (step S210).

  Next, the proxy server 20 receives data from the storage apparatus 30 storing the master data as a response to the allocated object (step S211), and transmits the received data to the client terminal 10 as a response to the request (step S211). S212). Then, the client terminal 10 receives data from the proxy server 20 (step S213).

  FIG. 7 is a flowchart showing an example of migration processing performed by the storage device 30 and the storage device 40 according to the first embodiment of the present invention.

  First, the determination unit 36 acquires a file ID from the file information table of the processing unit 33 (step S301). Next, the determination unit 36 acquires an object ID based on the file ID of the acquired file information table (step S302), and calculates a hash value using the extracted object ID (step S303). Thereafter, the determination unit 36 acquires an access rule and information indicating the own device from the storage storage unit 32 via the management unit 31 (step 304).

  Then, the determination unit 36 determines whether the data included in the file indicated by the acquired file ID is replica data using the access rule, the information indicating the own device, and the hash value (step S305). . If the data included in the file indicated by the file ID is replica data (YES in step S305), determination unit 36 moves the file to tape 35 (step S306). On the other hand, when the data included in the file indicated by the file ID is master data (NO in step S305), determination unit 36 determines whether all files have been determined (step S307). If determination unit 36 has not determined all the files (NO in step S307), it repeats steps S301 to S306 for the next file. If determination unit 36 determines all files in disk 34 (YES in step S307), the process ends. For example, the determination unit 36 may perform the above-described migration process when a file is written on the disk 34.

  As described above, the storage management system 1 uses not only a disk but also a tape for storing data. The tape drive does not consume power in the idle state. Therefore, the storage management system 1 can reduce power consumption compared to the case where data is stored only on a disk.

  Incidentally, as an example of migration processing, HSM (Hierarchical Storage Management) technology may be used. Hereinafter, migration processing using the HSM technology will be described with reference to the flowchart of FIG.

  As shown in FIG. 12, file information of a file stored on the disk is acquired (step S601), and it is determined whether the file information satisfies a predetermined condition (step S602). If the file information satisfies a predetermined condition (YES in step S602), specifically, if the file has not been accessed for a predetermined time or longer, the file of the file information is transferred from the disk to the tape. (Step S603). Then, a flag indicating that the migration process has been performed is set in the file information of the file (step S604).

  If the file information does not satisfy a predetermined condition (NO in step S602), the process proceeds to step S605. Then, the processes in steps S601 to S604 are repeated until all files are checked.

  In the migration process described above, for example, when access to a file has not been performed for a predetermined time or longer, the file associated with the file information is transferred from the disk to the tape. Therefore, for example, when a master data file is not accessed for a predetermined time or more, the file is transferred from the disk to the tape. As a result, for example, when the file is read from the client terminal, the file is read from the tape. That is, since it takes time to read the file, it takes time until the file is transmitted to the client terminal.

  In contrast, the storage management system 1 moves only the replica data to the tape according to the access rule. Thereby, for example, when reading is performed from the client terminal 10, it is possible to read from the disk instead of from the tape. Therefore, the storage management system 1 can shorten the response time compared to the migration processing using the above-described HSM technology.

Second Embodiment
FIG. 8 is a block diagram showing an example of the configuration of the storage management system 2 according to the second embodiment of the present invention. In FIG. 8, the storage management system 2 includes a client terminal 10, a proxy server 20, a storage device 50, and a storage device 60. The client terminal 10 and the proxy server 20 are connected to each other via a wired or wireless network. Hereinafter, the configuration of the storage device 50 and the storage device 60 will be described, but the client terminal 10 and the proxy server 20 have the same configuration as in the first embodiment, and thus the description thereof will be omitted. The storage device 50 and the storage device 60 include a management unit 51, a determination unit 52, a storage storage unit 53, a processing unit 54, a disk 55, and a tape 56.

  The management unit 51 receives the assigned object from the proxy server 20. Further, the management unit 51 converts the object ID included in the received post-allocation object into a file ID. Then, the management unit 51 transmits the file ID and the allocated object to the determination unit 52. In addition, the management unit 51 stores the file ID and the object ID in the storage storage unit 53 in association with each other.

  The determination unit 52 receives the file ID and the allocated object from the management unit 51. Further, the determination unit 52 acquires the object ID from the storage storage unit 53 based on the file ID. Then, the determination unit 52 calculates a hash value based on the acquired object ID.

  In addition, the determination unit 52 extracts the access rule stored in the storage storage unit 53 and information indicating the own device. Then, the determination unit 52 determines whether the data included in the file is replica data based on the calculated hash value, the access rule, and the information indicating the device itself.

  When the data included in the file is replica data, the determination unit 52 instructs the processing unit 54 to perform an operation (write in this example) on the file. That is, an instruction to write a file to the tape 56 and the file are transmitted.

  On the other hand, when the data included in the file is master data, the determination unit 52 instructs the processing unit 54 where to operate the file. That is, an instruction to write a file to the disk 55 and the file are transmitted.

  The processing unit 54 receives from the determination unit 52 an instruction to write a file and a file to the disk 55 or an instruction to write a file to the tape 56. The processing unit 54 associates the file ID with the file information for each file, and stores the file ID in a storage unit (not shown) as a file information table.

  Then, the processing unit 54 writes the file to the disk 55 or the tape 56 based on the instruction to write the file to the disk 55 or the instruction to write the file to the tape 56. Further, the processing unit 54 may be included in the determination unit 52.

  FIG. 9 is a flowchart showing an example of write processing in the storage device 50 and the storage device 60 according to the second embodiment.

  First, the management unit 51 receives an assigned object from the proxy server 20 (step S401). Further, the management unit 51 converts the object ID included in the received post-allocation object into a file ID (step S402). Then, the management unit 51 transmits the file ID and the allocated object to the determination unit 52.

  Next, the determination unit 52 receives the file ID and the allocated object from the management unit 51. Further, the determination unit 52 acquires the object ID from the storage storage unit 53 based on the file ID. Then, the determination unit 52 calculates a hash value based on the acquired object ID (step S403). Further, the determination unit 52 acquires the access rules stored in the storage storage unit 53 and information indicating the own device (step S404).

  Then, the determination unit 52 refers to the extracted access rule based on the calculated hash value and information indicating the own device, and determines whether the data included in the file is replica data (step S405). When the data included in the file is replica data (YES in step S405), determination unit 52 transmits an instruction to write the file to tape 56 and the file to processing unit 54. The processing unit 54 receives an instruction to write a file on the tape 56 and the file from the determination unit 52.

  Then, the processing unit 54 writes a file on the tape 56 based on the received instruction to write on the tape 56 (step S406).

  On the other hand, when the data included in the file is master data (NO in step S405), determination unit 52 transmits an instruction to write the file to disk 55 and the file to processing unit 54. Then, the processing unit 54 receives an instruction to write a file to the disk 55 and the file from the determination unit 52.

  Then, based on the instruction to write the received file to the disk 55, the processing unit 54 writes the file to the disk 55 (step S407).

  As described above, when writing the file, the storage management system 2 determines whether or not the data included in the file is replica data. If the data included in the file is replica data, the file is written on the tape 56. . As a result, since the replica data is not written to the disk 55, the storage management system 2 can efficiently use the capacity of the disk 55.

  Each storage device determines whether the data included in the file is replica data based on the access rule. Therefore, the proxy server 20 does not need to add information that can determine master data or replica data to the data included in the request.

  In addition, the storage management system 2 writes master data to the disk 55. Therefore, the storage management system 2 always reads data from the disk 55. Compared to the case where data is read from the tape 56, the storage management system 2 can shorten the response time required for reading data.

<Third Embodiment>
FIG. 10 is a block diagram showing an example of the configuration of the storage management system 3 according to the third embodiment of the present invention. The storage management system 2 includes a client terminal 90, a proxy server 70, and storage devices 80 (80-1, 80-2,..., 80-N (N: natural number)). In the present embodiment, the plurality of storage devices are collectively referred to as a storage device 80. The client terminal 90 transmits data to the proxy server 70 via a wired or wireless network. Further, the number of storage devices is not limited to one and may be one or more.

  The proxy server 70 includes an allocation unit 71 and a calculation unit 72. The assigning unit 71 assigns an identifier to data transmitted from the client terminal 90 together with the request.

  When the request is a write request, the calculation unit 72 is a storage device that is determined according to the identifier and a predetermined rule, and is a storage device that is different from the storage device that stores data as master data. Then, the data is transmitted to the storage apparatus that stores the data as replica data of the master data.

  The storage device 80 includes a first storage unit 81, a determination unit 82, and a second storage unit 83. The first storage unit 81 stores data transmitted from the proxy server 70.

  The determination unit 82 determines whether the device itself stores data transmitted from the proxy server 70 as master data or replica data based on the identifier and a predetermined rule.

  If the determination unit 82 determines to store the data transmitted from the proxy server as replica data, the determination unit 82 stores the data in the second storage unit 83 which is a migration destination storage unit.

When the data is written, the storage management system 3 determines whether or not the data is replica data. If the data is replica data, the storage management system 3 stores the data in the second storage unit 83 (for example, tape). Write to. The storage management system 3 writes the master data to be read into the first storage unit 81 (for example, a disk) that is not the migration destination. For example, as compared with the case where master data to be read is stored in the second storage unit 83, the storage management system 3 can shorten the response time required for reading the data.

  In each embodiment of the present invention, each component of each system represents a functional unit block. Part or all of each component of each system is realized by an arbitrary combination of an information processing apparatus 900 and a program as shown in FIG. 11, for example. The information processing apparatus 900 includes the following configuration as an example.

CPU (Central Processing Unit) 901
ROM (Read Only Memory) 902
-RAM (Random Access Memory) 903
A program 904 loaded into the RAM 903
A storage device 905 that stores the program 904
A drive device 907 that reads / writes data from / to the recording medium 906
A communication interface 908 connected to the communication network 909
・ Input / output interface 910 for inputting / outputting data
-Bus 911 connecting each component
Each component of each system in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. A program 904 that realizes the function of each component of each system is stored in advance in the storage device 905 or the RAM 903, for example, and is read out by the CPU 901 as necessary. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in the recording medium 906 in advance, and the drive device 907 may read the program and supply it to the CPU 901.

  There are various modifications to the method of realizing each system. For example, each system may be realized by an arbitrary combination of an information processing apparatus 900 and a program that are different for each component. A plurality of constituent elements included in each system may be realized by an arbitrary combination of one information processing apparatus 900 and a program.

  Also, some or all of the components of each system are realized by other general-purpose or dedicated circuits, processors, and the like, or combinations thereof. These may be configured by a single chip or may be configured by a plurality of chips connected via a bus.

  Part or all of each component of each system may be realized by a combination of the above-described circuit and the like and a program.

  When some or all of the components of each system are realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributedly arranged. Also good. For example, the information processing apparatus, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system and a cloud computing system.

  As mentioned above, although this indication was explained with reference to an embodiment, this indication is not limited to the above-mentioned embodiment. Various changes that can be understood by those skilled in the art can be made to the configurations and details of the present disclosure within the scope of the present disclosure.

A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited to the following.
(Appendix 1)
Including a client terminal, a proxy server, and a plurality of storage devices,
The proxy server includes an assigning unit that assigns an identifier to data transmitted together with a request from the client terminal;
When the request is a write request, a storage device that stores the data as master data, which is determined according to the identifier and a predetermined rule, among the plurality of storage devices, and a storage device that is different from the storage device. And a calculation unit that transmits the data to a storage device that stores the data as replica data of the master data,
Each of the plurality of storage devices is
A determination means for performing determination processing for determining whether the device itself stores data transmitted from the proxy server as master data or replica data based on the identifier and the predetermined rule;
First storage means for storing the data transmitted from the proxy server;
A second storage means that is a storage destination storage means,
When the determination unit determines to store the data transmitted from the proxy server as replica data, the storage management system stores the data in the second storage unit.
(Appendix 2)
Each of the plurality of storage devices is
When the data is received from the proxy server, the data storage device further comprises management means for storing the data in the first storage means,
The determination unit determines whether the device stores data stored in the first storage unit as master data or replica data, and determines that the data is stored as replica data. The storage management system according to appendix 1, wherein the storage management system is moved from the first storage unit to the second storage unit.
(Appendix 3)
Each of the plurality of storage devices further comprises management means for receiving the data from the proxy server,
The determination unit performs the determination process when the management unit receives the data from the proxy server, and stores the data in the first storage unit or the second storage unit based on a result of the determination process. The storage management system according to appendix 1.
(Appendix 4)
The management means converts the identifier into a file identifier that is an identifier for specifying the data in file units,
The storage management system according to appendix 2 or 3, wherein the data is stored in the first storage unit or the second storage unit in file units.
(Appendix 5)
In the predetermined rule, a storage device that stores the data as master data and a storage device that stores the data as replica data of the master data are set.
The proxy server further includes a changing unit that changes the predetermined rule according to a result of the determination process,
The storage management system according to any one of appendices 1 to 4, wherein the changing unit transmits the changed predetermined rule to each of the plurality of storage apparatuses.
(Appendix 6)
The calculation means calculates a hash value using the identifier, refers to the predetermined rule based on the calculated hash value, and stores the data assigned the identifier as master data; and the master Determine the storage device that stores the data as replica data of the data,
The storage management according to any one of appendices 1 to 5, wherein the determination unit calculates a hash value using the identifier, refers to the predetermined rule based on the calculated hash value, and performs the determination process. system.
(Appendix 7)
When the request is a read request, information specifying data to be read is transmitted together with the request,
The assigning means associates an identifier assigned to the data to be read with information identifying the data to be read;
The calculation means transmits the identifier to a storage device that stores, as master data, the data to be read, determined according to the identifier and the predetermined rule, among the plurality of storage devices,
The determination unit reads the data to be read from the first storage unit based on the identifier, and transmits the read data to be read to the calculation unit.
The storage management system according to any one of appendices 1 to 6, wherein the calculation unit transmits the data to be read to the client terminal as a response to the read request.
(Appendix 8)
A storage management method for a storage management system including a client terminal, a proxy server, and a plurality of storage devices,
The proxy server assigns an identifier to data transmitted with a request from the client terminal,
When the request is a write request, among the plurality of storage devices, a storage device that stores data assigned with the identifier, determined according to the identifier and a predetermined rule, as master data, and the storage device Is a different storage device, and transmits the data to the storage device storing the data assigned the identifier as replica data of the master data,
Each of the plurality of storage devices is
First storage means for storing the data transmitted from the proxy server;
A second storage means that is a storage destination storage means,
Based on the identifier and the predetermined rule, a determination process is performed to determine whether the own device stores data transmitted from the proxy server as master data or replica data, and is transmitted from the proxy server. A storage management method for storing data in the second storage means when it is determined that data is stored as replica data.
(Appendix 9)
Each of the plurality of storage devices receives the data from the proxy server, stores the data in the first storage means,
When the own device determines whether the data stored in the first storage means is stored as master data or replica data, and determines that the data is stored as replica data, the data is stored in the first storage Item 9. The storage management method according to appendix 8, wherein the storage management method migrates from the means to the second storage means.
(Appendix 10)
Each of the plurality of storage devices receives the data from the proxy server, performs the determination process, and stores the data in the first storage unit or the second storage unit based on the result of the determination process The storage management method according to appendix 8.
(Appendix 11)
Each of the plurality of storage devices converts the identifier into a file identifier that is an identifier for specifying the data in file units,
The storage management method according to appendix 9 or 10, wherein the data is stored in file units in the first storage unit or the second storage unit.
(Appendix 12)
In the predetermined rule, a storage device that stores data assigned the identifier as master data, and a storage device that stores data assigned the identifier as replica data of the master data are set,
12. The proxy server according to any one of appendices 8 to 11, wherein the proxy server changes the predetermined rule according to a result of the determination process, and transmits the changed predetermined rule to each of the plurality of storage devices. Storage management method.
(Appendix 13)
The proxy server calculates a hash value using the identifier, refers to the predetermined rule based on the calculated hash value, and stores the data to which the identifier is assigned as master data, and the data And a storage device for storing the master data as replica data of the master data,
Each of the plurality of storage devices calculates a hash value using the identifier, and refers to the predetermined rule based on the hash value calculated by the own device, and performs the determination process according to any one of appendices 8 to 12 The storage management method according to one.
(Appendix 14)
When the request is a read request, information specifying data to be read is transmitted together with the request,
The proxy server associates an identifier assigned to the data to be read with information specifying the data to be read, and determines the identifier according to the identifier and the predetermined rule among the plurality of storage devices Is sent to the storage device that stores the data to be read as master data,
Each of the plurality of storage devices reads the data to be read from the first storage unit based on the identifier, and transmits the read data to be read to the client terminal as a response to the read request. 14. The storage management method according to any one of appendices 8 to 13.

DESCRIPTION OF SYMBOLS 1 Storage management system 2 Storage management system 10 Client terminal 20 Proxy server 21 Allocation part 22 Calculation part 23 Change part 24 Server storage part 30 Storage apparatus 31 Management part 32 Storage storage part 33 Processing part 34 Disk 35 Tape 36 Determination part 40 Storage apparatus DESCRIPTION OF SYMBOLS 50 Storage apparatus 51 Management part 52 Judgment part 53 Storage storage part 54 Processing part 55 Disk 56 Tape 60 Storage apparatus 70 Proxy server 71 Allocation part 72 Calculation part 80 Storage apparatus 81 1st memory | storage part 82 Judgment part 83 2nd memory | storage part 90 Client Terminal 900 Information processing apparatus 901 CPU
902 ROM
903 RAM
904 Program 905 Storage device 906 Recording medium 907 Drive device 908 Communication interface 909 Communication network 910 Input / output interface 911 Bus

Claims (10)

Including a client terminal, a proxy server, and a plurality of storage devices,
The proxy server includes an assigning unit that assigns an identifier to data transmitted together with a request from the client terminal;
When the request is a write request, a storage device that stores the data as master data, which is determined according to the identifier and a predetermined rule, among the plurality of storage devices, and a storage device that is different from the storage device. And a calculation unit that transmits the data to a storage device that stores the data as replica data of the master data,
Each of the plurality of storage devices is
A determination means for performing determination processing for determining whether the device itself stores data transmitted from the proxy server as master data or replica data based on the identifier and the predetermined rule;
First storage means for storing the data transmitted from the proxy server;
A second storage means that is a storage destination storage means,
When the determination unit determines to store the data transmitted from the proxy server as replica data, the storage management system stores the data in the second storage unit. Each of the plurality of storage devices is
When the data is received from the proxy server, the data storage device further comprises management means for storing the data in the first storage means,
The determination unit determines whether the device stores data stored in the first storage unit as master data or replica data, and determines that the data is stored as replica data. The storage management system according to claim 1, wherein the storage management system is moved from the first storage means to the second storage means. Each of the plurality of storage devices further comprises management means for receiving the data from the proxy server,
The determination unit performs the determination process when the management unit receives the data from the proxy server, and stores the data in the first storage unit or the second storage unit based on a result of the determination process. The storage management system according to claim 1. The management means converts the identifier into a file identifier that is an identifier for specifying the data in file units,
The storage management system according to claim 2 or 3, wherein the data is stored in the first storage unit or the second storage unit in file units. In the predetermined rule, a storage device that stores the data as master data and a storage device that stores the data as replica data of the master data are set.
The proxy server further includes a changing unit that changes the predetermined rule according to a result of the determination process,
The storage management system according to any one of claims 1 to 4, wherein the changing unit transmits the changed predetermined rule to each of the plurality of storage apparatuses. The calculation means calculates a hash value using the identifier, refers to the predetermined rule based on the calculated hash value, and stores the data assigned the identifier as master data; and the master Determine the storage device that stores the data as replica data of the data,
The storage according to claim 1, wherein the determination unit calculates a hash value using the identifier, refers to the predetermined rule based on the calculated hash value, and performs the determination process. Management system. When the request is a read request, information specifying data to be read is transmitted together with the request,
The assigning means associates an identifier assigned to the data to be read with information identifying the data to be read;
The calculation means transmits the identifier to a storage device that stores, as master data, the data to be read, determined according to the identifier and the predetermined rule, among the plurality of storage devices,
The determination unit reads the data to be read from the first storage unit based on the identifier, and transmits the read data to be read to the calculation unit.
The storage management system according to any one of claims 1 to 6, wherein the calculation unit transmits the data to be read to the client terminal as a response to the read request. A storage management method for a storage management system including a client terminal, a proxy server, and a plurality of storage devices,
The proxy server assigns an identifier to data transmitted with a request from the client terminal,
When the request is a write request, a storage device that stores the data as master data, which is determined according to the identifier and a predetermined rule, among the plurality of storage devices, and a storage device that is different from the storage device. And sending the data to a storage device storing the data as replica data of the master data,
Each of the plurality of storage devices is
First storage means for storing the data transmitted from the proxy server;
A second storage means that is a storage destination storage means,
Based on the identifier and the predetermined rule, a determination process is performed to determine whether the own device stores data transmitted from the proxy server as master data or replica data, and is transmitted from the proxy server. A storage management method for storing data in the second storage means when it is determined that data is stored as replica data. Each of the plurality of storage devices receives the data from the proxy server, stores the data in the first storage means,
When the own device determines whether the data stored in the first storage means is stored as master data or replica data, and determines that the data is stored as replica data, the data is stored in the first storage The storage management method according to claim 8, wherein the storage management method migrates from a means to the second storage means.   Each of the plurality of storage devices receives the data from the proxy server, performs the determination process, and stores the data in the first storage unit or the second storage unit based on the result of the determination process The storage management method according to claim 8.

Images