JP5673232B2 - Storage system - Google Patents

Description

  The present invention relates to a storage system, and more particularly to a storage system having a duplicate storage elimination function.

  Normally, a file server caches data in a memory in order to reduce access to a hard disk drive having a low access speed. And the capacity of non-volatile storage such as hard disk drives continues to expand year by year, but the capacity of memory used as a cache has not caught up. However, in recent years, as a large-capacity SSD has become widespread, storage using this SSD as a cache has appeared. For this reason, when dealing with a large-capacity cache, how to efficiently use the cache is a problem.

  For example, in distributed node type network storage, emphasis is placed on distributing the load, and sufficient consideration has not been given to improving the efficiency of the cache area. In the future, when the number of nodes in the distributed node storage increases, it is considered that the efficiency of the cache across the nodes becomes important.

  As an example of the above-described distributed node type network storage, in recent years, there is a deduplication system disclosed in Japanese Patent Laid-Open No. 2004-151405 that controls not to repeatedly store data having the same content. Such a deduplication system constitutes a content address storage (CAS) device in which a physical storage location is virtualized, and controls the hard disk drive not to have the same data by deduplication technology, It is controlled so that any file can be accessed from the node. At this time, the deduplication file system efficiently uses the hard disk drive by not holding multiple pieces of the same data, and the cache on the memory is controlled not to hold multiple pieces of the same data within one node. Yes.

JP 2010-79886 A

  However, in the content address storage system, which node processes the file depends on the access from the client, and efficiency is not improved by load distribution in the entire system. On the other hand, there is a problem that by distributing the processing to a plurality of nodes, the waste that the same cache data exists in the plurality of nodes increases as shown in FIG.

  Therefore, an object of the present invention is to suppress cache waste while suppressing an increase in load on a specific node device in a storage system including a plurality of node devices, which is the above-described problem.

In order to achieve the above object, a storage system according to one aspect of the present invention provides:
Multiple storage devices,
Each divided data obtained by dividing the file is distributed and stored in the plurality of storage devices using a content address that is specific to the data content of the divided data and specifies a storage position of the divided data in the storage device, and Deduplication processing for referencing the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the storage device A plurality of storage processing devices to perform,
The storage processing device includes:
In addition to having a cache memory that is used when performing predetermined processing on files,
The content address indicating the storage location of the divided data constituting the specific file managed by the own device set in advance is transmitted to the other storage processing device, and the divided data specified by the content address is transmitted. The cache memory status information indicating the storage status of the cache memory in the other storage processing device is requested to the other storage processing device, and the identification is performed based on the cache memory status information acquired from the other storage processing device. An owner node selection unit for selecting the storage processing device for managing the file of
Take the configuration.

In addition, the storage processing device according to another aspect of the present invention is
Each divided data obtained by dividing the file is distributed and stored in a plurality of storage devices using content addresses that are specific to the data content of the divided data and specify the storage position of the divided data in the storage device, and the storage Storage for performing deduplication processing for referring to the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the apparatus A processing device comprising:
In addition to having a cache memory that is used when performing predetermined processing on files,
The content address indicating the storage position of the divided data constituting the specific file managed by the own device set in advance is transmitted to another storage processing device, and the divided data specified by the content address is transmitted. Request cache memory status information indicating the storage status of the cache memory in another storage processing device to the other storage processing device, and based on the cache memory status information acquired from the other storage processing device, the specific memory An owner node selection unit for selecting a storage processing device for managing files;
Take the configuration.

Moreover, the program which is the other form of this invention is:
Each divided data obtained by dividing the file is distributed and stored in a plurality of storage devices using content addresses that are specific to the data content of the divided data and specify the storage position of the divided data in the storage device, and the storage Performing deduplication processing to refer to the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the device; In a storage processing device having a cache memory used when performing predetermined processing on a file,
The content address indicating the storage position of the divided data constituting the specific file managed by the own device set in advance is transmitted to another storage processing device, and the divided data specified by the content address is transmitted. Request cache memory status information indicating the storage status of the cache memory in another storage processing device to the other storage processing device, and based on the cache memory status information acquired from the other storage processing device, the specific memory This is a program for realizing an owner node selection unit that selects a storage processing device that manages files.

In addition, an information processing method according to another aspect of the present invention includes:
Multiple storage devices,
Each divided data obtained by dividing the file is distributed and stored in the plurality of storage devices using a content address that is specific to the data content of the divided data and specifies a storage position of the divided data in the storage device, and Deduplication processing for referencing the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the storage device An information processing method by a storage system including a plurality of storage processing devices including a cache memory used when performing predetermined processing on a file,
The storage processing device transmits the content address representing the storage location of the divided data constituting the specific file managed by the device itself set in advance to the other storage processing device, and specified by the content address The cache memory status information obtained from the other storage processing device by requesting the other storage processing device for cache memory status information indicating the storage status of the divided data to the cache memory in the other storage processing device Selecting the storage processing device to manage the specific file based on
Take the configuration.

  With the configuration as described above, the present invention makes it possible to efficiently use a cache while suppressing an increase in the load on a specific node device in a storage system including a plurality of node devices.

It is a figure which shows the mode of the data processed within the accelerator node which comprises a storage system. It is a figure which shows the mode of the data processed within the accelerator node which comprises a storage system. It is a functional block diagram which shows the structure of the accelerator node which comprises a storage system. It is a figure explaining operation | movement of the owner control function part disclosed in FIG. It is a figure explaining operation | movement of the owner control function part disclosed in FIG. It is a figure explaining operation | movement of the owner control function part disclosed in FIG. It is a figure explaining the other example of operation | movement of the owner control function part disclosed in FIG. It is a functional block diagram which shows the structure of the storage system in attachment 1 of this invention.

  A first embodiment of the present invention will be described with reference to FIGS. 1 to 3 are diagrams for explaining the configuration of the storage system, and FIGS. 4 to 7 are diagrams for explaining the operation.

  FIG. 1 shows the overall configuration of the storage system in this embodiment. As shown in this figure, the storage system includes a load balancer 301 connected to an NFS client 302 that requests file writing and reading, and a plurality of nodes 303 (storage processing devices that perform file storage processing and reading processing). Node A, node B), and a large-capacity nonvolatile storage 306 comprising a plurality of storage devices for storing files.

  In this embodiment, a network storage system (Network File System: NFS) will be described as an example of the storage system. However, the storage system is not limited to NFS.

  The load balancer 301 (distribution device) distributes processing requests for the file such as file writing and reading from the NFS client 302 to an arbitrary node 303 based on the file identifier (FH) of the file. At this time, the load balancer 301 stores an FH / destination table (management designation table) that designates a node that manages the file for each file, and designates a file to be processed according to the FH / destination table. To the designated node. However, if the file related to the processing request is a new file, the file is transferred to a node determined with reference to the load of each node 303, the network bandwidth connected to each node 303, and the like.

  Then, as will be described later, when there is an FH / destination table update request for changing the designation of a node that manages a file from the node 303, the load balancer 301 responds to the update request with the FH / Update the contents of the destination table.

  In the present embodiment, a load balancer is described as an example of a device that controls the transfer destination of a file related to a processing request, but the device that performs such control may be another device. For example, any of the nodes 303 may function as a load balancer.

  Each of the nodes 303 includes an NFS server function unit 304 and a deduplication FS 305. In response to a file storage request from the NFS client 302 transferred from the load balancer 301, the node 303 stores the file in the large-capacity nonvolatile storage 306. It has a function of performing deduplication and storing. In the figure, only two nodes 303 are shown, but more nodes 303 are actually provided. However, the number of nodes 303 only needs to be plural, and is not limited to the above number.

  In the deduplication processing by the deduplication FS 305, specifically, first, a file to be stored is divided into divided data having a predetermined capacity. Then, the storage position in the large-capacity nonvolatile storage 306 is specified by a unique content address set according to the content of the divided data, and the divided data is further distributed and stored redundantly in the storage position. Note that the distribution and redundancy processing when storing data in the large-capacity nonvolatile storage 306 is an existing technology, and thus detailed description thereof is omitted.

  At this time, the content address is specific to the data content of the divided data. Therefore, when storing other divided data having the same content as the divided data, the content address of the already stored divided data is changed to another data. By referencing as divided data, it is not necessary to store the other divided data. When reading the other divided data, the same content data can be read by referring to the content address and reading the data stored in the reference destination. In this way, the deduplication FS 305 realizes deduplication processing that eliminates duplicate storage of files.

  The file data is managed by CA information 309 in which the file and each content address of each divided data obtained by dividing the file are associated. For example, as shown in FIG. 2, the CA information 309 includes a forward link that traces the content address (CA link: CA1, etc.) of each divided data constituting the file from the file, and a reverse direction that traces the file from the content address. Have a link. As for CA information 309, CA information 309 of a file managed by the own node is stored in each node. The content address is common to each node.

  Each node 303 is equipped with a cache memory 307 that caches data used by each node 303 for file processing. Each node 303 does not have the same data in the cache memory in the same node, but is controlled by the cache information management unit so as to eliminate duplication.

  Each node 303 includes an owner control function unit 313. The owner control storage unit 313 is information that specifies a node that manages a specific file, that is, information on which node the file is processed by. Owner information 314 stored therein. The owner information 314 is the same information as the FH / destination table stored in the load balancer 301 as described above. Furthermore, as shown in FIG. 3, the owner control function unit 313 performs processing for changing a node that manages a specific file, as shown in FIG. 3, a non-duplicate data number calculation unit 315, an owner node selection unit 306, and an owner movement unit. 317. The configurations and operations of these units 315, 316, and 317 will be described in detail below.

  Here, the non-overlapping data number calculation unit 315, the owner node selection unit 316, and the owner movement unit 317 included in the owner control function unit 313 operate based on the following three operations (1) to (3). However, the conditions under which the above units 315 to 317 operate are not limited to the following cases.

(1) NFS processing When a WRITE request is received from the NFS client 302 and the cache for the file of the node managing the file is updated, the owner control function unit 313 operates to process the owner node, that is, the file. Select a power node. Further, the owner is moved as necessary, and thereafter, the load balancer 301 transfers the NFS to an appropriate node.

(2) Cache eviction When the cache free space runs out and the old cache is evicted from the memory, there is a possibility that the number of duplicate data in the cache in the node so far has changed. Therefore, the owner node is reselected. That is, the content address of the evicted cache is confirmed, and all files linked from the content address are listed. “Calculation of the number of non-overlapping data”, “Selection of owner node”, and “Move owner” are performed for all the listed files.

(3) Timer The network bandwidth may change over time. For this reason, the owner node is periodically reselected for a file for which the movement of the owner is abandoned during a high load. Periodic owner node reselection is performed for all files. The load may increase while reselecting the owner node. Therefore, a period (for example, midnight, etc.) is provided in advance, and a file that does not end within the period is given up reselection. However, remember how far the owner node has been reselected, and continue from the next time.

  Next, as an example, the operation of the owner control function unit 313 starting from NFS WRITE processing will be described. Note that the processing in the “calculation of the number of non-overlapping data”, “owner node selection unit”, and “owner movement unit” described below does not change even when READ processing, cache eviction, and timer are used as starting points.

1. First, when the NFS client 302 writes a file, it sends a WRITE request to the load balancer 301.
2. The load balancer selects an appropriate node 303 from its own FH / destination table and transfers the WRITE request.
3. The node 303 to which the WRITE request has been transferred performs normal write processing.
4). When file data is accumulated to some extent by WRITE, “calculation of the number of non-duplicate data” is performed by the non-duplicate data number calculation unit 315 shown below.

“Calculation of the number of non-overlapping data” (Figure 4)
S1. Among the CA information 309 described above, a CA list that is a group of content addresses referred to by the file requested for WRITE is acquired from the file information table 401 in which each content address of each divided data constituting the file is stored. As shown in FIG. 2, the file information table 401 stores information associating a file with a content address.
S2. For each content address constituting the CA list, the reference number 402 in the own node is confirmed.

S3. When the reference number of the content address is “1”, the non-duplicate data variable 404 is incremented and the value is increased by “1”.
S4. When the reference number of the content address is “2” or more, the file that refers to the content address whose reference number is “2” or more by following the list 403 from the content address to the file in the CA information 309 described above. Check whether “one” or “two or more” exists.
S5. When only one file refers to the content address, the non-duplicate data variable 404 is incremented and the value is increased by “1”.

  When the confirmation is completed for all the content addresses of the file related to the WRITE request, “owner node selection” is performed by the owner node selection unit 316.

"Owner node selection" (Figure 5)
S11. As described above, it is confirmed whether the value of the non-duplicate data variable 501 counted by the non-duplicate data count calculation unit 315 exceeds a preset threshold value. If the value does not exceed the threshold value, the process is terminated.
S12. When the non-duplicate data number variable 501 exceeds the threshold, the usage rate of the network bandwidth 502 is confirmed.
S13. The CA list which is stored in the own node 303 and which is the content address group of the file managed by the own node 303 is broadcasted to all other nodes. For example, in the node A (303) of FIG. 2, when the “file 3” managed by the node A is the target file, the content addresses CA8 and CA9 of the divided data constituting the “file 3” are set to other values. Transfer to node. Then, another node is inquired and requested for the storage status (cache memory status information) of the data corresponding to the content address to be transferred to the cache memory of the other node.

S14. The owner control function unit of the other node that has received the CA list checks how many content addresses that are the same as the content address included in the CA list exist in the cache memory of the other node. That is, by checking the existence of the same content address as the content address included in the CA list, the number of the same divided data stored in the cache memories of the node that transmitted the CA list and the received node is checked. . Then, the other node confirms the number of duplicate data and the usage rate of the network bandwidth at that time, and returns it to the node that has transmitted the CA list.
S15. A node that receives the number of duplicate data and the network bandwidth from another node moves the node that manages the file for a file in which many pieces of duplicate data are stored in the cache memory of the other node. For example, another node having the largest amount of divided data of a specific file and held in the cache memory is selected, and the other node is changed to manage the specific file. At this time, it is desirable that the network bandwidth is more than a predetermined value. Thereafter, the owner moving unit 317 performs processing for changing a node that actually manages the file.

“Owner Movement” (Figure 6)
S21. As described above, when there is a change in a node that manages a certain file, first, owner information 601 that represents the relationship between the file stored in the own node and the management node is updated.
S22. Then, the other node is notified that the owner, which is the management node of a file, has changed. Specifically, the file information, which is CA information 309 indicating the correspondence between the file managed by the own node and the content address, is transmitted to and stored in the selected other node as the management destination ( (See the arrow shown at the bottom of FIG. 2).

S23. As described above, the FH / destination table 602 on the load balancer 301 that associates the node selected as a node for managing a file with the file is updated.
S24. The cache 603 already stored in the own node is flushed. That is, the data in the cache 603 is deleted.
S25. Compare the free area size of the cache between the own node and the other node, and if the other node's free area size is equal or smaller, not only move in one direction but also move (replace) in the opposite direction to keep the cache even Do. That is, the management of the file stored in the cache of the other node that is the management destination is moved to the own node. Specifically, the file information representing the correspondence between the file managed by the other node and the content address is moved from the other node to the own node and stored in the own node. Updates owner information and FH / destination table for specifying a node for managing the file.

  Thereafter, the load balancer 301 transfers the file related to the processing request to the designated node with reference to the FH / destination table.

  As described above, according to the storage system of the present embodiment, duplication of data stored in the cache memory can be suppressed at each node that performs data processing on the storage, and the cache memory can be used efficiently. Accordingly, since more data is stored in the cache, the speed of NFS access can be increased. In particular, when multiple clients read and write multiple files at the same time, or when the cache is not used up by one file, the cache is used up by multiple files (for example, cache is 4GByte, file size is 100MByte, etc.) It is effective for.

  In the distributed node type storage system, the cache size increases in proportion to the number of nodes added. However, in the above-described storage system according to the present invention, the increased cache is effectively used to ensure the throughput proportional to the number of nodes. This makes system design easier.

  Here, in the above, the node that processes the file is determined using the content address. However, the node that processes the file may be determined by using other information about the file as an auxiliary. . For example, the load is reduced by determining a node in advance based on metadata such as a file name.

  Hereinafter, a method for selecting an owner node using metadata will be described with reference to FIG. 7 together with the method using the content address described above. Note that the metadata information may not be known at the time of WRITE, so not only at the time of WRITE but also at CREATE (file creation) and LOOKUP (file search) Make a selection.

"Selecting the owner node by pre-setting" (Figure 7)
S31. A conditional expression is acquired from setting information 701 set in advance.
S32. Information on the path name is acquired from the cache 702. If the path name is not cached, exit.
S33. If the path name is compared with the conditional expression and they match, “owner movement” is performed on the node set in the condition. For example, if the conditional expression is “kernel-2.6.18-*. X86_64.rpm”, “*” is interpreted as an arbitrary character string, and “kernel-2.6.18-release1.x86_64.rpm” matches. , “Kernel-2.6.18-release2.ia64.rpm” is determined not to match.

  In the above description, a node that processes each file is distributed by the load balancer. However, a node constituting the distributed file system may play the role of the load balancer. That is, in FIG. 3, it is possible for the node A to have an FH / destination table and to transfer a request from the NFS client to the node B as necessary. In addition, a form in which a client directly accesses a node using network control such as ARP without using a specific node is also possible.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The outline of the configuration of the storage system in the present invention will be described below with reference to FIG. However, the present invention is not limited to the following configuration.

(Appendix 1)
A plurality of storage devices 10;
Each divided data obtained by dividing the file is distributed and stored in the plurality of storage devices using a content address that is specific to the data content of the divided data and specifies a storage position of the divided data in the storage device, and Deduplication processing for referencing the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the storage device A plurality of storage processing devices 20 to perform,
The storage processing device 20
A cache memory 21 used for performing predetermined processing on a file is provided,
The content address indicating the storage location of the divided data constituting the specific file managed by the own device set in advance is transmitted to the other storage processing device, and the divided data specified by the content address is transmitted. The cache memory status information indicating the storage status of the cache memory in the other storage processing device is requested to the other storage processing device, and the identification is performed based on the cache memory status information acquired from the other storage processing device. An owner node selection unit 22 for selecting the storage processing device that manages the files of
Storage system.

(Appendix 2)
The storage system according to attachment 1, wherein
The owner node selection unit of the storage processing device is configured such that, based on the cache memory status information acquired from the other storage processing device, the divided data constituting the specific file managed by the own device is the other storage processing. Selecting the other storage processing device contained most in the cache memory of the device as a storage processing device for managing the specific file;
Storage system.

(Appendix 3)
The storage system according to appendix 1 or 2,
The storage processing device includes:
While storing a file information table indicating the correspondence between the divided data constituting the specific file managed by the own device and the content address of the divided data,
An owner moving unit that transmits and stores the file information table of the specific file to the other storage processing device selected as the storage processing device that manages the specific file by the owner node selection unit;
Storage system.

(Appendix 4)
The storage system according to attachment 3, wherein
The owner moving unit of the storage processing device erases data in the cache memory of the own device,
Storage system.

(Appendix 5)
The storage system according to appendix 3 or 4,
A distribution device that stores a management designation table that designates a specific file managed by the storage processing device, and distributes a processing request for the specific file to the storage processing device designated by the specific file based on the management designation table With
The owner moving unit of the storage processing device updates the management designation table so that the other storage processing device selected by the owner node selection unit manages the specific file.
Storage system.

(Appendix 6)
The storage system according to any one of appendices 3 to 5,
The owner moving unit included in the storage processing device has a specific management managed by the other storage processing device from the other storage processing device that has transmitted the file information table of the specific file managed by the own device. Receiving and storing the file information table of the specific file so that the device manages the file;
Storage system.

(Appendix 7)
The storage system according to appendix 6, wherein
The owner moving unit of the storage processing device has a free capacity of the cache memory provided in the other storage processing device that has transmitted the file information table of the specific file managed by the own device. The file information table of the specific file is managed so that the specific device manages the specific file managed by the other storage processing device from the other storage processing device when the cache memory is less than the free capacity of the cache memory Receive and remember,
Storage system.

(Appendix 8)
The storage system according to any one of appendices 1 to 7,
The storage processing device includes a non-duplicate data number calculation unit that counts the number of content addresses whose reference number of the content address of the divided data constituting the specific file managed by the device is “1”. ,
The owner node selection unit included in the storage processing device is configured such that, when the value counted by the non-duplicate data number calculation unit exceeds a preset threshold, the divided data constituting a specific file managed by the own device Sending the content address to the other storage processing device to request the cache memory status information.
Storage system.

(Appendix 9)
The storage system according to attachment 8, wherein
In addition to the content address where the reference number of the content address of the divided data constituting the specific file managed by the device is “1”, the non-overlapping data number calculation unit of the storage processing device includes: Counting the number of the content addresses whose reference number of the content address is “2” or more but the number of files to be referred to is “1”;
Storage system.

(Appendix 10)
Each divided data obtained by dividing the file is distributed and stored in a plurality of storage devices using content addresses that are specific to the data content of the divided data and specify the storage position of the divided data in the storage device, and the storage Storage for performing deduplication processing for referring to the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the apparatus A processing device comprising:
In addition to having a cache memory that is used when performing predetermined processing on files,
The content address indicating the storage position of the divided data constituting the specific file managed by the own device set in advance is transmitted to another storage processing device, and the divided data specified by the content address is transmitted. Request cache memory status information indicating the storage status of the cache memory in another storage processing device to the other storage processing device, and based on the cache memory status information acquired from the other storage processing device, the specific memory An owner node selection unit for selecting a storage processing device for managing files;
Storage processing device.

(Appendix 11)
The storage processing device according to attachment 10, wherein
The owner node selection unit, based on the cache memory status information acquired from the other storage processing device, has the divided data constituting the specific file managed by the own device most in the cache memory of the other storage processing device. The other storage processing device that is included in a large number is selected as a storage processing device that manages the specific file.
Storage processing device.

(Appendix 12)
Each divided data obtained by dividing the file is distributed and stored in a plurality of storage devices using content addresses that are specific to the data content of the divided data and specify the storage position of the divided data in the storage device, and the storage Performing deduplication processing to refer to the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the device; In a storage processing device having a cache memory used when performing predetermined processing on a file,
The content address indicating the storage position of the divided data constituting the specific file managed by the own device set in advance is transmitted to another storage processing device, and the divided data specified by the content address is transmitted. Request cache memory status information indicating the storage status of the cache memory in another storage processing device to the other storage processing device, and based on the cache memory status information acquired from the other storage processing device, the specific memory A program for realizing an owner node selection unit that selects a storage processing device that manages files.

(Appendix 13)
The program according to attachment 12, wherein
The owner node selection unit, based on the cache memory status information acquired from the other storage processing device, has the divided data constituting the specific file managed by the own device most in the cache memory of the other storage processing device. The other storage processing device that is included in a large number is selected as a storage processing device that manages the specific file.
program.

(Appendix 14)
Multiple storage devices,
Each divided data obtained by dividing the file is distributed and stored in the plurality of storage devices using a content address that is specific to the data content of the divided data and specifies a storage position of the divided data in the storage device, and Deduplication processing for referencing the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the storage device An information processing method by a storage system including a plurality of storage processing devices including a cache memory used when performing predetermined processing on a file,
The storage processing device transmits the content address representing the storage location of the divided data constituting the specific file managed by the device itself set in advance to the other storage processing device, and specified by the content address The cache memory status information obtained from the other storage processing device by requesting the other storage processing device for cache memory status information indicating the storage status of the divided data to the cache memory in the other storage processing device Selecting the storage processing device to manage the specific file based on
Information processing method.

(Appendix 15)
The information processing method according to attachment 14, wherein
Based on the cache memory status information acquired by the storage processing device from the other storage processing device, the divided data constituting the specific file managed by the storage device is the most in the cache memory of the other storage processing device. The other storage processing device that is included in a large number is selected as a storage processing device that manages the specific file.
Information processing method.

301 Load Balancer 302 NFS Client 303 Node 304 NFS Server Function Unit 305 Deduplication FS
306 Large-capacity nonvolatile storage 307 Cache 309 CA information 310 Setting information 311 Network bandwidth information 312 Timer device 313 Owner control function unit 314 Owner information 315 Non-duplicate data number calculation unit 316 Owner node selection unit 317 Owner migration unit 10 Storage device 20 Storage processing device 21 Cache memory 22 Owner node selector

Claims (10)

Multiple storage devices,
Each divided data obtained by dividing the file is distributed and stored in the plurality of storage devices using a content address that is specific to the data content of the divided data and specifies a storage position of the divided data in the storage device, and Deduplication processing for referencing the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the storage device A plurality of storage processing devices to perform,
The storage processing device includes:
In addition to having a cache memory that is used when performing predetermined processing on files,
The content address indicating the storage position of the divided data constituting the specific file managed by the own device is transmitted to the other storage processing device, and the other storage of the divided data specified by the content address is performed. Requests cache memory status information indicating the storage status of the cache memory in the processing device to the other storage processing device, and manages the specific file based on the cache memory status information acquired from the other storage processing device An owner node selection unit for selecting the storage processing device to be
Storage system. The storage system according to claim 1,
The owner node selection unit of the storage processing device is configured such that, based on the cache memory status information acquired from the other storage processing device, the divided data constituting the specific file managed by the own device is the other storage processing. Selecting the other storage processing device contained most in the cache memory of the device as a storage processing device for managing the specific file;
Storage system. The storage system according to claim 1 or 2,
The storage processing device includes:
While storing a file information table indicating the correspondence between the divided data constituting the specific file managed by the own device and the content address of the divided data,
An owner moving unit that transmits and stores the file information table of the specific file to the other storage processing device selected as the storage processing device that manages the specific file by the owner node selection unit;
Storage system. The storage system according to claim 3,
The owner moving unit of the storage processing device erases data in the cache memory of the own device,
Storage system. The storage system according to claim 3 or 4,
A distribution device that stores a management designation table that designates a specific file managed by the storage processing device, and distributes a processing request for the specific file to the storage processing device designated by the specific file based on the management designation table With
The owner moving unit of the storage processing device updates the management designation table so that the other storage processing device selected by the owner node selection unit manages the specific file.
Storage system. The storage system according to any one of claims 3 to 5,
The owner moving unit included in the storage processing device has a specific management managed by the other storage processing device from the other storage processing device that has transmitted the file information table of the specific file managed by the own device. Receiving and storing the file information table of the specific file so that the device manages the file;
Storage system. The storage system according to claim 6,
The owner moving unit of the storage processing device has a free capacity of the cache memory provided in the other storage processing device that has transmitted the file information table of the specific file managed by the own device. The file information table of the specific file is managed so that the specific device manages the specific file managed by the other storage processing device from the other storage processing device when the cache memory is less than the free capacity of the cache memory Receive and remember,
Storage system. The storage system according to any one of claims 1 to 7,
The storage processing device includes a non-duplicate data number calculation unit that counts the number of content addresses whose reference number of the content address of the divided data constituting the specific file managed by the device is “1”. ,
The owner node selection unit included in the storage processing device is configured such that, when the value counted by the non-duplicate data number calculation unit exceeds a preset threshold, the divided data constituting a specific file managed by the own device Sending the content address to the other storage processing device to request the cache memory status information.
Storage system. The storage system according to claim 8, wherein
In addition to the content address where the reference number of the content address of the divided data constituting the specific file managed by the device is “1”, the non-overlapping data number calculation unit of the storage processing device includes: Counting the number of the content addresses whose reference number of the content address is “2” or more but the number of files to be referred to is “1”;
Storage system. Multiple storage devices,
Each divided data obtained by dividing the file is distributed and stored in the plurality of storage devices using a content address that is specific to the data content of the divided data and specifies a storage position of the divided data in the storage device, and Deduplication processing for referencing the content address of the already stored divided data as the other divided data when storing other divided data having the same content as the divided data already stored in the storage device An information processing method by a storage system including a plurality of storage processing devices including a cache memory used when performing predetermined processing on a file,
The storage processing device transmits the content address representing the storage location of the divided data constituting a specific file managed by the device itself to the other storage processing device, and the division specified by the content address Request cache memory status information indicating the storage status of the data in the cache memory in the other storage processing device to the other storage processing device, and based on the cache memory status information acquired from the other storage processing device, Selecting the storage processing device that manages the specific file;
Information processing method.

Images