JPWO2012117658A1 - Storage system - Google Patents

Abstract

The storage system according to the present invention distributes data to a plurality of storage devices and stores a plurality of storage processing devices for performing deduplication processing, and assigns a data flow consisting of a group of data to any one of the storage processing devices. A switch unit that sets the data flow to flow to the allocated storage processing device, a flow characteristic detection unit that detects a predetermined characteristic of the data flow for each data flow, and a predetermined characteristic of the storage processing device A device characteristic detection unit that detects each storage processing device, and the switch unit includes a data flow characteristic detected by the flow characteristic detection unit, and a storage processing device characteristic detected by the device characteristic detection unit. Based on the above, the storage processing device to which the data flow is assigned is determined.

Description

  The present invention relates to a storage system, and particularly relates to a storage system including a plurality of storage devices.

  In recent years, with the development and spread of computers, various types of information have been converted into digital data. As a device for storing such digital data, there are storage devices such as a magnetic tape and a magnetic disk. Since the data to be stored increases day by day and becomes enormous, a large-capacity storage system is required. In addition, reliability is required while reducing the cost of the storage device. In addition to this, it is necessary that data can be easily retrieved later. As a result, there is a demand for a storage system that can automatically increase storage capacity and performance, eliminate duplicate storage, reduce storage costs, and have high redundancy.

  In response to this situation, in recent years, content address storage systems have been developed. In this content address storage system, data is distributed and stored in a plurality of storage devices, and the storage location where the data is stored is specified by a unique content address specified according to the content of the data. Specifically, in the content address storage system, predetermined data is divided into a plurality of fragments, and a fragment that becomes redundant data is further added, and the plurality of fragments are respectively stored in a plurality of storage devices.

  Then, by designating the content address later, it is possible to read out the data stored at the storage location specified by the content address, that is, the fragment, and restore the predetermined data before the division from the plurality of fragments.

  The content address is generated to be unique according to the content of data, for example, a hash value of data is used. For this reason, if it is duplicate data, the data of the same content can be acquired by referring to the data at the same storage position. Therefore, it is not necessary to store the duplicate data separately, and duplicate recording can be eliminated and the data capacity can be reduced.

  Here, a storage system that stores a large amount of data, such as the content address storage system described above, includes a plurality of information processing apparatuses. As described above, in a system including a plurality of information processing apparatuses, load distribution among the information processing apparatuses is required. As a general load balancing technique, there is a round robin method. An example of a system that performs load distribution is disclosed in Patent Document 1.

  In the storage system disclosed in Patent Document 1, main data and sub data having the same contents are stored, and management information for managing the arrangement status of the main data and sub data in the storage device is stored. In addition, the latest load information of each storage device is continuously collected. As the load information, for example, the CPU load, the number of received access requests, the network usage rate, and the like are collected. Based on the management information and the collected load information, the roles of the main data and the sub data are changed between a set of data. That is, by changing the storage device that is the access destination for the main data and the sub data, the load distribution of the storage device that stores the data is performed without moving the data.

Special table 2008-136075 gazette

  However, the load distribution method using round robin and the load distribution method disclosed in Patent Document 1 have a problem that efficient load distribution cannot be performed. This is because in a storage system that performs deduplication, performance and functions required for data storage processing differ depending on the characteristics of stored data. For example, when overlapping data is stored, data is not actually stored, so there is no need to perform data compression processing, and performance can be improved in this respect. On the other hand, when data that has already been compressed / encrypted by backup software or the like is written, deduplication or compression processing is difficult to function effectively and performance is degraded. Furthermore, in order to improve the duplication rate, it may be necessary to separate the marker information added by the backup software. However, if there is no function to perform such processing, the deduplication efficiency will decrease. sell.

  Further, since data storage processing is performed by a plurality of information processing apparatuses, efficient load distribution becomes difficult even when the performance and functions of the information processing apparatuses are different. For example, there are an expansion card that processes compression processing and Hash calculation in place of the CPU, and an expansion card equipped with an SSD that can process small I / O at high speed. However, these expansion cards are expensive. It is expensive to install in this information processing apparatus. Further, the number of expansion cards that can be mounted on one information processing apparatus is limited by the maximum number of slots of the apparatus, and various cards cannot be mounted on one apparatus.

  Furthermore, when considering a storage device as a backup device in consideration of long-term use, a mixture of multiple generation storage devices occurs. Then, the performance of the CPU, memory, etc. mounted on the old generation storage device is inferior to that of the new generation storage device, so the performance of the entire system is lowered.

  Therefore, an object of the present invention is to achieve efficient load distribution in the storage system, which is the problem described above.

In order to achieve the above object, a storage system according to one aspect of the present invention provides:
Multiple storage devices,
Data that is already stored in the storage device when data is distributed and stored in the plurality of storage devices and other data having the same content as the data that is already stored in the storage device is stored. A plurality of storage processing devices for performing deduplication processing to refer to as other data,
A switch unit that assigns a data flow composed of a group of data to any one of the storage processing devices and sets the data flow to flow to the assigned storage processing device;
A flow characteristic detection unit for detecting a predetermined characteristic of the data flow for each data flow;
A device characteristic detection unit for detecting a predetermined characteristic of the storage processing device for each storage processing device;
The storage unit assigns the data flow based on the characteristics of the data flow detected by the flow characteristic detector and the characteristics of the storage processing device detected by the apparatus characteristic detector. Determine the equipment,
Take the configuration.

Moreover, the switch control apparatus which is the other form of this invention is the following.
When distributing data to a plurality of storage devices and storing other data having the same contents as the data already stored in the storage device, the data already stored in the storage device is stored. A switch control device connected to a switch unit that sets a data flow consisting of a group of data to flow through any one of the plurality of storage processing devices that perform deduplication processing to be referred to as the other data. There,
Based on a predetermined characteristic of the data flow detected for each of the data flows and a predetermined characteristic of the storage processing device detected for each of the storage processing devices, the data flow is stored in the data flow. A flow setting unit that assigns a processing device and sets the switch unit so that the data flow flows to the allocated storage processing device;
Take the configuration.

Moreover, the program which is the other form of this invention is:
When distributing data to a plurality of storage devices and storing other data having the same contents as the data already stored in the storage device, the data already stored in the storage device is stored. A switch control device connected to a switch unit configured to set a data flow consisting of a group of data to flow through one of the storage processing devices among a plurality of storage processing devices that perform deduplication processing referred to as the other data. ,
Based on a predetermined characteristic of the data flow detected for each of the data flows and a predetermined characteristic of the storage processing device detected for each of the storage processing devices, the data flow is stored in the data flow. A flow setting unit for allocating a processing device and setting the switch unit so that the data flow flows to the allocated storage processing device;
It is a program for realizing.
Take the configuration.

In addition, a flow control method according to another embodiment of the present invention is as follows.
Multiple storage devices,
Data that is already stored in the storage device when data is distributed and stored in the plurality of storage devices and other data having the same content as the data that is already stored in the storage device is stored. A plurality of storage processing devices for performing deduplication processing to refer to as other data,
A flow control method by a storage system comprising: a switch unit that assigns a data flow consisting of a group of data to any one of the storage processing devices and sets the data flow to flow to the assigned storage processing device. ,
Detecting a predetermined characteristic of the data flow for each data flow, and detecting a predetermined characteristic of the storage processing apparatus for each storage processing apparatus;
The switch unit determines the storage processing device to which the data flow is assigned based on the detected characteristic of the data flow and the detected characteristic of the storage processing device;
Take the configuration.

  By configuring as described above, the present invention can achieve efficient load distribution in the storage system.

1 is a block diagram showing an outline of a configuration of a storage system in the present invention. It is a functional block diagram which shows the structure of the accelerator node disclosed in FIG. 1 and a PFS control apparatus. It is a figure which shows a mode when storing the data in the storage system disclosed in FIG. It is a figure which shows a mode when storing the data in the storage system disclosed in FIG. 2 is a flowchart showing an operation in the storage system disclosed in FIG. 1. 2 is a flowchart showing an operation in the storage system disclosed in FIG. 1. 2 is a flowchart showing an operation in the storage system disclosed in FIG. 1. It is a block diagram which shows the structure of the storage system in attachment 1 of this invention.

  A first embodiment of the present invention will be described with reference to FIGS. 1 and 2 are diagrams for explaining the configuration of the storage system, and FIGS. 3 to 7 are diagrams for explaining the operation of the storage system.

[Constitution]
As shown in FIG. 1, the storage system includes a plurality of accelerators 100, 110, 120, a PFS control device 300, a CAS (Content-Addressable Storage) 400, a PFS (Programmable Flow Switch) 500, And a plurality of clients 200, 210, 220, 230 for storing data are connected to the storage system. Hereinafter, each configuration will be described in detail.

  The client 200 or the like is an information processing apparatus equipped with backup software. Then, the client 200 or the like transmits, to the storage system, a data flow composed of a group of data stored in the own device by processing of backup software or by an operation of the operator. .

  The PFS 500 has a function of routing or redirecting data transmitted from the client 200 or the like in units of flows. That is, the PFS 500 can set a route to the accelerator node 100 or the like of a data flow that is a group of data transmitted from the client 200 or the like, or can change the accelerator node 100 or the like that is a transmission destination. Note that the setting and changing of the data flow path by the PFS 500 are controlled by the PFS control device 300 as described later. Therefore, the PFS 500 and the PFS control device 300 function as a switch unit that assigns a data flow to one of the accelerator nodes 100 and sets the data flow to flow to the assigned accelerator node.

  The accelerator node 100 and the like and the CAS 400 divide and make redundant and store the data in a distributed manner, and store the storage location where the data is stored by a unique content address set according to the content of the stored data. The specified content address storage system is configured. According to this content address storage system, when storing other data having the same content as the data already stored in the CAS 400, it is possible to refer to the already stored data as other data using the content address. The other data need not be stored. When other data is read, the same data can be read by reading the stored data with reference to the content address. In this way, the storage system realizes deduplication processing that eliminates duplicate storage of data.

  The CAS 400 plays a role of storing data, and includes a plurality of storage nodes (storage devices). The accelerator node 100 and the like have a role of performing data storage processing, that is, deduplication processing, on the CAS 400.

  Next, the accelerator node 100 and the like will be further described in detail. Although a plurality of accelerator nodes 100, 110, and 120 are provided, it is assumed that all accelerator nodes have a configuration of an accelerator node denoted by reference numeral 100 described below. Further, the number of accelerator nodes is not limited to the number shown in FIG.

  Since the accelerator node 100 or the like is an information processing apparatus that serves as an entrance to the CAS 400, the client 200 or the like looks like NAS (Network Attached Storage). The storage system including the accelerator node 100 and the like and the CAS 400 has a GNS (Global Name Space) function, and any accelerator node 100, 110, 120 can access the same file system.

  As shown in FIG. 2, the accelerator node 100 or the like includes a CAS processing unit 620. The CAS processing unit 620 cooperates with the CAS 400 to manage files using content addresses, so that the same contents are obtained. The de-duplication processing is performed in which the data is not stored in the CAS 400 in duplicate. Here, the state of deduplication processing when data is stored by the accelerator node 100 and the CAS 400 will be described with reference to FIGS.

  First, when the storage system receives an input of data A (data flow) to be stored (see arrow Y1 in FIG. 4), the storage system stores the data A in a predetermined capacity as indicated by arrow Y2 in FIGS. The data is divided into block data D (for example, 64 KB). Based on the data contents of the block data D, a unique hash value H representing the data contents is calculated (arrow Y3 in FIG. 4). For example, the hash value H is calculated from the data content of the block data D using a preset hash function.

  Subsequently, the storage system uses the hash value H of the block data D to check whether the block data D is already stored in the CAS 400. Specifically, first, for the already stored block data D, as will be described later, a content address CA representing a storage position including the hash value H is registered in the content address management table MFI. Therefore, when the hash value H of the block data D calculated before storage does not exist in the content address management table MFI, it can be determined that the block data D having the same content is not yet stored. On the other hand, when the hash value H exists in the content address management table MFI, it can be determined that the block data D having the same content is already stored (arrow Y4 in FIG. 4).

  Subsequently, the storage system compresses the block data D determined that the same block data D is not yet stored in accordance with a preset compression rule, and a plurality of predetermined data are indicated as indicated by an arrow Y5 in FIG. Is divided into fragment data of the capacity of. For example, as shown by symbols D1 to D9 in FIG. 3, the data is divided into nine fragment data (divided data 11). Further, the storage system generates redundant data so that the original block data can be restored even if some of the divided fragment data is missing, and adds it to the divided fragment data 11. For example, three pieces of fragment data (redundant data 12) are added as indicated by reference numerals D10 to D12 in FIG. As a result, a data set 10 composed of 12 pieces of fragment data composed of nine divided data 11 and three redundant data 12 is generated.

  Subsequently, the storage system stores each fragment data constituting the generated data set in a distributed manner in each storage area formed in the storage apparatus constituting the CAS 400. For example, as shown in FIG. 3, when 12 pieces of fragment data D1 to D12 are generated, one piece of each piece of fragment data D1 to D12 is stored in each data storage file formed in each of the 12 storage areas. Each is stored (see arrow Y6 in FIG. 4).

  Further, the storage system sets the content address CA indicating the storage position of the fragment data D1 to D12 stored in the storage device as described above, that is, the storage position of the block data D restored by the fragment data D1 to D12. Generate and manage. Specifically, a part of the hash value H (short hash) calculated based on the contents of the stored block data D (for example, the top 8B (bytes) of the hash value H), information indicating the logical storage position, Are combined to generate a content address CA (arrow Y7 in FIG. 4). The deduplication system 10 associates the identification information such as the file name of the storage target data with the content address CA and manages it in the file system, and also creates a new entry for the generated content address CA in the content address management table MFI. to add.

  Further, the storage system, when the hash value H of the block data D of the data to be stored already exists in the content address management table MFI, that is, when the block data D having the same content is already stored. The content address CA including the hash value that matches the hash value H of the block data D before storage is acquired from the content address management table MFI. Then, the content address CA is set as a content address CA representing the storage destination of the block data D of the storage target data. As a result, the already stored data referred to by the content address CA is referred to as the block data D requested to be stored, and the block data D itself related to the storage request needs to be stored redundantly. Disappear. That is, it is excluded that data having the same contents are stored repeatedly.

  As described above, the storage system including the accelerator node 10 and the CAS 400 and the storage system divides the data to be stored into small block sizes, and compares the block with a block that has already been stored. Only blocks are compressed and saved. If it is determined that a block having the same content already exists, information indicating the position where the block is stored (content address CA) can be recorded, so that duplicate recording of data can be eliminated and actually stored. Data amount to be reduced. As described above, the deduplication processing of compressing and storing data while performing deduplication of data is mainly performed by the CAS processing unit 620 (FIG. 2) constructed by incorporating a program into the CPU of the accelerator node 100. ).

  Further, as shown in FIG. 2, the accelerator node 100 and the like are equipped with various dedicated cards 630 (dedicated devices) that execute specific processes that are a part of processes required when storing data. doing. Thereby, the accelerator node 100 or the like can cause the various dedicated cards 630 that are hardware to execute a part of the data storage processing without using a CPU mounted on the accelerator node 100 or the like. As a result, the load on the accelerator node 100 and the like can be suppressed, and the processing speed can be increased.

  For example, an accelerator node indicated by reference numeral 100 in FIG. 1 is equipped with a “compression card” that performs data compression processing before data is stored in the CAS 400. The accelerator node denoted by reference numeral 110 is equipped with an “SSD card” equipped with a solid state drive (SSD) that is faster in data input / output than an auxiliary storage device such as a hard disk drive installed in the CAS 400. By using this “SSD card”, the processing when handling a large amount of data with a small file size becomes faster. Furthermore, the accelerator node denoted by reference numeral 120 includes a “Hash calculation card” that is a device that executes a process of calculating the hash value H based on the contents of the block data D that is a part of the deduplication process described above. .

  However, all the accelerator nodes 100 and the like do not have to be equipped with the dedicated card 630 as described above. Note that the device performance such as the CPU processing performance and memory transfer performance of the accelerator node 100 and the like are not necessarily the same.

  Further, as shown in FIG. 2, the accelerator node 100 or the like includes a flow reception unit 600, a session movement processing unit 610, a flow characteristic determination unit 640, an AN characteristic determination, which are constructed by incorporating a program in the equipped CPU. Unit 650 and a flow characteristic detection unit 660.

  The flow receiving unit 600 receives a data flow sent from the client 200 or the like, and passes the contents of the same data flow to the flow characteristic detecting unit 600 or the CAS processing unit 620. In addition, the flow reception unit 600 stops or receives a flow according to a request from the session movement processing unit 610.

  The session movement processing unit 610 performs a process of requesting the flow reception unit 600 to stop / start the data flow or moving information on a session with the client to another accelerator node (AN).

  The flow characteristic detection unit 660 includes units 661 to 665 that detect various predetermined characteristics of the data flow for each data flow and perform processing to pass to the flow characteristic determination unit 640. For example, the duplication rate calculation unit 661 calculates the duplication rate indicating the degree of duplication of the data in the data flow with the data stored in the CAS 400 based on the deduplication processing by the CAS processing unit 620. Detect as a characteristic. Based on the deduplication processing by the CAS processing unit 620, the compression rate calculation unit 662 detects a compression rate representing the degree of compression before and after storage of data in the data flow in the CAS 400 as a characteristic of this data flow. The compression detection unit 663 refers to the marker information included in the data flow by backup software or the like installed in the client 200, and determines whether or not the data flow has already been compressed before being input to the accelerator node 100 or the like. Are detected as characteristics of this data flow. The encryption detection unit 664 refers to marker information included in the data flow by backup software or the like installed in the client 200, and detects whether the data flow is encrypted as a characteristic of the data flow. . The file size detection unit 665 detects the size of the file included in the data flow as a characteristic of the data flow. Note that the characteristics of the data flow detected by the flow characteristic unit detection unit 660 are not limited to those described above.

  The flow characteristic determination unit 640 sends information representing the characteristics of each data flow detected by the flow characteristic detection unit 660 to the PFS control apparatus 300. At this time, the flow characteristic discriminating unit 640 uses, as information for specifying the data flow, the “client IP address” that is information for identifying the client 200 or the like that has transmitted the data flow, and the data flow duplication rate and compression described above. Characteristic information such as the rate, the presence / absence of compression, the presence / absence of encryption, and the file size is sent to the PFS control device 300.

  The AN characteristic discriminating unit 650 (apparatus characteristic detecting unit) collects information representing the characteristics of the accelerator node 100 and the like for each accelerator node 100 and transmits the information to the PFS control apparatus. For example, the AN characteristic discriminating unit 650 collects information such as the performance of the mounted CPU and memory, the load status, and the type of the dedicated card installed as the characteristics of the accelerator node 100 and the like. Then, the AN characteristic determination unit 650 sends these pieces of information to the PFS control apparatus 300 together with information for identifying the accelerator node 100 and the like. Note that the characteristics of the accelerator node 100 and the like collected by the AN characteristic determination unit 650 are not limited to those described above.

  It should be noted that the above-described data flow characteristic and accelerator node characteristic detection processing and transmission to the PFS control device 300 are always performed at regular intervals by all accelerator nodes 100 and the like.

  Next, the PFS control device 300 (switch control device) that functions as a switch unit in cooperation with the PFS 500 as described above will be described. As shown in FIG. 2, the PFS control device 300 includes a flow setting unit 330 that is constructed by incorporating a program into an equipped arithmetic device. The PFS control device 300 includes an AN characteristic information database (DB) 310 and a flow characteristic information database (DB) 320 formed in the equipped storage device.

  The flow setting unit 330 stores data flow characteristic information sent from each accelerator node 100 or the like in the flow characteristic information DB 320, and stores characteristic information of the accelerator node 100 or the like in the AN characteristic information DB 310. Each characteristic information is updated every time it is transmitted from each accelerator node 100 or the like.

  The flow setting unit 330 then assigns an accelerator to each data flow based on the characteristic information of the data flow stored in the flow characteristic information DB 320 and the characteristic information of the accelerator node 100 or the like stored in the AN characteristic information DB 310. Node 100 and the like are determined. Then, control is performed to newly set or switch the path to the PFS 500 so that the data flow flows to the accelerator node 100 or the like assigned to the data flow.

  Specifically, the flow setting unit 330 first assigns the accelerator node 100 or the like equipped with the dedicated card 630 corresponding to the data flow characteristic to the data flow. For example, when the data flow overlap rate is higher than a predetermined value, an accelerator node equipped with a Hash calculation card is assigned to the data flow. If the compression rate of the data flow is higher than a predetermined value, an accelerator node equipped with a compression card is assigned to the data flow. When the size of the file in the data flow is smaller than a predetermined value, an accelerator node equipped with an SSD card is assigned to the data flow. However, if the data flow is encrypted by backup software on the client 200 side or the like, or already compressed, an accelerator node equipped with a Hash calculation card or a compression card is not assigned.

  In addition, when there is no accelerator node 100 or the like on which each dedicated card 630 assigned to the data flow is installed as described above, the flow setting unit 300 loads the accelerator node 100 or the like stored in the AN characteristic information DB 310. Depending on the situation, an accelerator node 100 having a low load is assigned to the data flow.

  Note that, as described above, the flow setting unit 300 performs processing for assigning an accelerator node to a data flow and performing routing and redirection of the PFS 500 at regular intervals or whenever each characteristic information is updated.

[Operation]
Next, the operation of the above-described storage system will be described with reference to the flowcharts of FIGS.

  The same IP address is set in advance for the accelerator nodes 100, 110, and 120. As a result, the clients 200, 210, 220, and 230 appear to access one device in terms of IP. However, as will be described later, the accelerator node 100 or the like that is actually accessed is determined by the routing / redirection of the PFS 500.

  First, the PFS control device 300 detects that the client 200 or the like is connected to the PFS 500 (step A1). Next, the PFS control device 300 acquires the load status of each accelerator node 100 or the like from the AN characteristic information DB 310 (step A2), and selects the one with the lowest load from the accelerator nodes 100 or the like (step A3). Then, the PFS 500 is set by the flow setting unit 330 so that the flow from the client is transmitted to the selected accelerator node (step A4). In this way, the accelerator node that the client accesses first is determined.

  The accelerator node receives a flow from the client by the flow reception unit 600. The flow reception unit 600 causes the same data flow to flow in a flow characteristic detection path and a normal processing path for storing data that has been deduplicated. In other words, the flow reception unit 600 causes a flow characteristic detection path to flow to the flow characteristic detection unit 660 and a process path for storing data to flow to the CAS processing unit 620.

  When the flow characteristic detection unit 660 of the accelerator node receives the data flow from the flow reception unit 600, the data flow such as the data flow duplication rate, compression rate, presence / absence of compression / encryption by backup software, file size detection, etc. Detection processing of each characteristic is performed. The flow characteristic detection unit 660 passes the detected data flow characteristic to the flow characteristic determination unit 640. Then, the flow characteristic determination unit 640 creates flow characteristic information that associates the IP address of the client that has transmitted the data flow based on the detected information, and sends the flow characteristic information to the PFS control apparatus 300.

  In addition, the CAS processing unit 620 that has received the path of the processing for performing normal data storage performs data storage processing that eliminates duplicate storage on the CAS 400 as described with reference to FIGS. At this time, when the dedicated card 630 that performs processing instead of the CPU is mounted in the accelerator node, the CAS processing unit 620 leaves the portion that can be processed by the mounted dedicated card 630 to the dedicated card 630. In this way, the data flow data is finally stored in the CAS 400.

  The AN characteristic determination unit 650 of the accelerator node 100 or the like collects processing performance and load status of the accelerator node 100 or the like, and information on the installed dedicated card, and sends the collected information to the PFS control apparatus 300.

  Thereafter, when the characteristics of the data flow change or the load on the accelerator node increases, the PFS control device 300 redirects the data flow using the flow setting unit 330 for load distribution.

  First, the flow setting unit 330 of the PFS control device 300 detects data flow characteristic changes and accelerator node load changes using information in the AN characteristic information DB 310 and the flow characteristic information DB 320 (step B1). Then, the flow setting unit 330 determines the allocation of the accelerator node corresponding to the characteristics of the data flow, and sets the data flow to flow to the accelerator node. Specifically, the flow setting unit 330 notifies the session movement processing unit 610 of the accelerator node that is the movement source of the data flow of the accelerator node that is the movement destination and issues a redirect instruction (step B2).

  The session movement processing unit 610 requests the flow reception unit 600 to stop the data flow (step B3). Also, the session movement processing unit 610 transmits the session information of the data flow to the session movement processing unit 610 of the movement destination accelerator node (step B4). The session movement processing unit 610 of the accelerator node that has received the session information reconstructs the session based on the session information (step B5). When the reconfiguration of the session is completed, the PFS control device 300 sets data flow redirection using the flow setting unit 330 (step B6).

  Next, an example of a method for assigning data flows to accelerator nodes by the PFS control device 300 described above will be described with reference to FIG.

  First, the presence of marker information included in the data flow is checked, and the client 200 or the like uses the backup software to check whether the data flow is transmitted for data storage (step C1). When the client 200 or the like uses backup software (Yes in Step C1), it is checked whether the data flow has already been compressed by the backup software with reference to the marker information added to the data flow by the backup software. (Step C2). If compression is performed by the backup software (Yes in step C2), it is highly possible that compression is not possible any more, so it is determined that compression does not work, and then it is confirmed whether there is duplication (step C4). .

  Subsequently, if the data flow duplication rate is higher than the predetermined value in step C4, the data of the data flow may be duplicated in the future (Yes in step C4), so the Hash calculation card is mounted. Priority allocation is performed to the existing accelerator node (step C8). On the other hand, when the duplication rate is equal to or less than the predetermined value (No in step C4), priority allocation is performed according to the load status of the accelerator node (step C9).

  If it is determined in step C2 that the data flow has already been compressed with the backup software and compression is not effective any more (No in step C2), the marker information included in the data flow is referred to. Then, it is determined whether the data flow is encrypted by the backup software (step C5).

  If the data flow is encrypted (Yes in Step C5), it is considered that neither compression nor deduplication is effective, and therefore allocation is determined according to the load status of the accelerator node (Step C10). On the other hand, if the data flow is not encrypted (No in step C5), it is confirmed from the data flow duplication rate whether the data of the data flow is duplicated (step C6). If they overlap (Yes in step C6), priority allocation is performed to the accelerator node on which the Hash calculation card is mounted (step C11). If they do not appear to overlap, it is confirmed whether compression is effective for the data flow (step C7).

  Subsequently, in step C7, if the compression ratio of the data flow is higher than the predetermined value, it can be determined that the compression is effective (Yes in step C7), so priority allocation to the accelerator node on which the compression card is mounted is performed. Perform (Step C12). On the other hand, if the compression rate of the data flow is equal to or less than a predetermined value and compression does not work (No in step C7), priority allocation is performed according to the load status of the accelerator node (step C13).

  If it is determined in step C1 that the client 200 or the like is not using backup software (No in step C1), the size of the file written next in the data stream is confirmed (step C3). ). If it is determined that the file size is equal to or smaller than the predetermined value (No in step C3), it can be determined that a small file is being written. For this reason, priority allocation is performed to the accelerator node on which the SSD card for increasing the small file processing performance is mounted (step C14). On the other hand, if the file size is larger than the predetermined value (Yes in step C3), it is confirmed from the data flow duplication rate whether the data of the data flow is duplicated (step C6). The subsequent contents are the same as described above.

  Here, when an accelerator node to be assigned to a data flow is determined, a plurality of accelerator nodes may be selected as candidates. In this case, referring to the load status and processing performance of the accelerator node, it is also possible to select an accelerator node having the most load and processing performance from among a plurality of selected accelerator nodes. Further, when there is no accelerator node equipped with a specific dedicated card to be allocated to the data flow, the allocation is performed using the load status of the accelerator node as a criterion.

  As described above, according to the storage system of the present invention, the data flow duplication rate and compression rate, the information added by the backup software, the processing performance and load status of the accelerator node are combined, and the flow using PFS is based on that combination. Can be distributed to accelerator nodes appropriate for the data flow. Therefore, efficient load distribution can be realized while speeding up data storage processing as a whole system.

  In addition, even if new and old generation accelerator nodes coexist in the storage system, the load can be distributed according to the performance of each, so that effective load distribution can be realized in a storage device for long-term use. it can.

  Furthermore, since load distribution is performed using information that can be detected by the accelerator node and PFS, the load can be distributed in units of flows without the client being aware of the data characteristics.

  It should be noted that information on the files to be backed up from the clients 200, 210, 220, and 230 (the name of the backup software, whether there is compression / encryption by the backup software, the size of the file to be backed up, etc.) And the PFS control device 300 may determine load distribution based on the information. That is, the PFS control device 300 may detect data flow information and assign an accelerator node according to the data flow characteristics.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The outline of the storage system in the present invention will be described below with reference to FIG. However, the present invention is not limited to the following configuration.

(Appendix 1)
A plurality of storage apparatuses 1000;
Data that is already stored in the storage device when data is distributed and stored in the plurality of storage devices and other data having the same content as the data that is already stored in the storage device is stored. A plurality of storage processing devices 1100 for performing deduplication processing for referring to the data as the other data;
A switch unit 1200 that assigns a data flow consisting of a group of data to any one of the storage processing devices and sets the data flow to flow to the assigned storage processing device;
A flow characteristic detection unit 1110 for detecting a predetermined characteristic of the data flow for each data flow;
A device characteristic detection unit 1120 for detecting a predetermined characteristic of the storage processing device for each storage processing device;
The switch unit 1200 assigns the data flow based on the characteristics of the data flow detected by the flow characteristic detection unit and the characteristics of the storage processing device detected by the device characteristic detection unit. Determine the processing equipment,
Storage system.

(Appendix 2)
The storage system according to attachment 1, wherein
The device characteristic detection unit detects a dedicated device that executes a specific process for storing the data flow mounted in the storage processing device in the storage device as a characteristic of the storage processing device,
The switch unit assigns the storage processing device equipped with the dedicated device corresponding to the data flow characteristic detected by the flow characteristic detection unit to the data flow.
Storage system.

(Appendix 3)
The storage system according to appendix 2,
The flow characteristic detection unit detects a degree of duplication representing a degree of duplication of data in the data flow with data stored in the storage device as the characteristic of the data flow,
The device characteristic detection unit detects a dedicated device that executes the deduplication process mounted on the storage processing device as a characteristic of the storage processing device,
The switch unit includes the storage processing device equipped with a dedicated device that executes the deduplication processing in the data flow when the degree of duplication of the data flow detected by the flow characteristic detection unit is higher than a predetermined value. assign,
Storage system.

(Appendix 4)
The storage system according to attachment 3, wherein
The flow characteristic detection unit detects whether the data flow is encrypted as the characteristic of the data flow,
The switch unit is when the data flow is not encrypted by the flow characteristic detection unit, and when the degree of duplication of the data flow detected by the flow characteristic detection unit is higher than a predetermined value, Assigning the storage processing device equipped with a dedicated device for executing the deduplication processing to the data flow;
Storage system.

(Appendix 5)
The storage system according to any one of appendices 2 to 4,
The storage processing device compresses and stores data in the storage device,
The flow characteristic detection unit detects the degree of compression of data in the data flow before and after storage in the storage device as the characteristic of the data flow,
The device characteristic detection unit detects a dedicated device that executes a compression process mounted on the storage processing device as a characteristic of the storage processing device,
When the degree of compression of the data flow detected by the flow characteristic detection unit is higher than a predetermined value, the switch unit allocates the storage processing device equipped with a dedicated device that executes the compression processing to the data flow ,
Storage system.

(Appendix 6)
The storage system according to appendix 5,
The flow characteristic detection unit detects, as the data flow characteristic, whether or not the data flow is compressed before being input to the storage processing device,
The switch unit is a case where the data flow is not compressed by the flow characteristic detection unit before being input to the storage processing device, and the degree of compression of the data flow detected by the flow characteristic detection unit Assigning the storage processing device equipped with a dedicated device for executing the compression processing to the data flow when the value is higher than a predetermined value,
Storage system.

(Appendix 7)
The storage system according to any one of appendices 2 to 6,
The flow characteristic detection unit detects the size of a file in the data flow as the characteristic of the data flow,
The device characteristic detection unit is a dedicated device equipped with a high-speed storage device that is faster in data input / output installed in the storage processing device than the auxiliary storage device installed in the storage device, as the characteristics of the storage processing device. Detect
When the file size of the data flow detected by the flow characteristic detection unit is smaller than a predetermined value, the switch unit includes the storage processing device equipped with a dedicated device equipped with the high-speed storage device for the data flow. assign,
Storage system.

(Appendix 8)
The storage system according to any one of appendices 2 to 7,
The device characteristic detection unit detects a load status of the storage processing device as a characteristic of the storage processing device,
The switch unit detects a load detected by the flow characteristic detection unit in the data flow when there is no storage processing device equipped with a dedicated device corresponding to the data flow characteristic detected by the flow characteristic detection unit. Assign other said storage processing device according to the situation,
Storage system.

(Appendix 9)
When distributing data to a plurality of storage devices and storing other data having the same contents as the data already stored in the storage device, the data already stored in the storage device is stored. A switch control device connected to a switch unit that sets a data flow consisting of a group of data to flow through any one of the plurality of storage processing devices that perform deduplication processing to be referred to as the other data. There,
Based on a predetermined characteristic of the data flow detected for each of the data flows and a predetermined characteristic of the storage processing device detected for each of the storage processing devices, the data flow is stored in the data flow. A flow setting unit that assigns a processing device and sets the switch unit so that the data flow flows to the allocated storage processing device;
Switch control device.

(Appendix 10)
The switch control device according to attachment 9, wherein
As a characteristic of the storage processing device, when a dedicated device that executes a specific process for storing the data flow mounted in the storage processing device in the storage device is detected,
The flow setting unit assigns the storage processing device equipped with the dedicated device corresponding to the detected characteristic of the data flow to the data flow;
Switch control device.

(Appendix 11)
When distributing data to a plurality of storage devices and storing other data having the same contents as the data already stored in the storage device, the data already stored in the storage device is stored. A switch control device connected to a switch unit configured to set a data flow consisting of a group of data to flow through one of the storage processing devices among a plurality of storage processing devices that perform deduplication processing referred to as the other data. ,
Based on a predetermined characteristic of the data flow detected for each of the data flows and a predetermined characteristic of the storage processing device detected for each of the storage processing devices, the data flow is stored in the data flow. A flow setting unit for allocating a processing device and setting the switch unit so that the data flow flows to the allocated storage processing device;
A program to realize

(Appendix 12)
The program according to attachment 11, wherein
As a characteristic of the storage processing device, when a dedicated device that executes a specific process for storing the data flow mounted in the storage processing device in the storage device is detected,
The flow setting unit assigns the storage processing device equipped with the dedicated device corresponding to the detected characteristic of the data flow to the data flow;
program.

(Appendix 13)
Multiple storage devices,
Data that is already stored in the storage device when data is distributed and stored in the plurality of storage devices and other data having the same content as the data that is already stored in the storage device is stored. A plurality of storage processing devices for performing deduplication processing to refer to as other data,
A flow control method by a storage system comprising: a switch unit that assigns a data flow consisting of a group of data to any one of the storage processing devices and sets the data flow to flow to the assigned storage processing device. ,
Detecting a predetermined characteristic of the data flow for each data flow, and detecting a predetermined characteristic of the storage processing apparatus for each storage processing apparatus;
The switch unit determines the storage processing device to which the data flow is assigned based on the detected characteristic of the data flow and the detected characteristic of the storage processing device;
Flow control method.

(Appendix 14)
The flow control method according to attachment 13, wherein
As a characteristic of the storage processing device, a dedicated device that executes a specific process for storing the data flow mounted in the storage processing device in the storage device is detected,
The switch unit assigns the storage processing device equipped with the dedicated device corresponding to the detected characteristic of the data flow to the data flow.
Flow control method.

  Note that the program described above is stored in a storage device or recorded on a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

  Although the present invention has been described with reference to the above embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  In addition, this invention enjoys the benefit of the priority claim based on the patent application of Japanese Patent Application No. 2011-41864 for which it applied for a patent in Japan on February 28, 2011, and was described in the said patent application. The contents are all included in this specification.

100, 110, 120 Accelerator node (AN)
200, 210, 220, 230 Client 300 PFS control device 310 AN characteristic information DB
320 Flow characteristic information DB
330 flow setting unit 400 CAS
500 PFS
600 Flow reception unit 610 Session movement processing unit 620 CAS processing unit 630 Dedicated card 640 Flow characteristic determination unit 650 AN characteristic determination unit 660 Flow characteristic detection unit 661 Duplicate rate calculation unit 662 Compression rate calculation unit 663 Compression detection unit 664 Encryption detection unit 665 File size detection unit 1000 Storage device 1100 Storage processing device 1110 Flow characteristic detection unit 1120 Device characteristic detection unit 1200 Switch unit

Claims (10)

Multiple storage devices,
Data that is already stored in the storage device when data is distributed and stored in the plurality of storage devices and other data having the same content as the data that is already stored in the storage device is stored. A plurality of storage processing devices for performing deduplication processing to refer to as other data,
A switch unit that assigns a data flow composed of a group of data to any one of the storage processing devices and sets the data flow to flow to the assigned storage processing device;
A flow characteristic detection unit for detecting a predetermined characteristic of the data flow for each data flow;
A device characteristic detection unit for detecting a predetermined characteristic of the storage processing device for each storage processing device;
The storage unit assigns the data flow based on the characteristics of the data flow detected by the flow characteristic detector and the characteristics of the storage processing device detected by the apparatus characteristic detector. Determine the equipment,
Storage system. The storage system according to claim 1,
The device characteristic detection unit detects a dedicated device that executes a specific process for storing the data flow mounted in the storage processing device in the storage device as a characteristic of the storage processing device,
The switch unit assigns the storage processing device equipped with the dedicated device corresponding to the data flow characteristic detected by the flow characteristic detection unit to the data flow.
Storage system. The storage system according to claim 2,
The flow characteristic detection unit detects a degree of duplication representing a degree of duplication of data in the data flow with data stored in the storage device as the characteristic of the data flow,
The device characteristic detection unit detects a dedicated device that executes the deduplication process mounted on the storage processing device as a characteristic of the storage processing device,
The switch unit includes the storage processing device equipped with a dedicated device that executes the deduplication processing in the data flow when the degree of duplication of the data flow detected by the flow characteristic detection unit is higher than a predetermined value. assign,
Storage system. The storage system according to claim 3,
The flow characteristic detection unit detects whether the data flow is encrypted as the characteristic of the data flow,
The switch unit is when the data flow is not encrypted by the flow characteristic detection unit, and when the degree of duplication of the data flow detected by the flow characteristic detection unit is higher than a predetermined value, Assigning the storage processing device equipped with a dedicated device for executing the deduplication processing to the data flow;
Storage system. The storage system according to any one of claims 2 to 4,
The storage processing device compresses and stores data in the storage device,
The flow characteristic detection unit detects the degree of compression of data in the data flow before and after storage in the storage device as the characteristic of the data flow,
The device characteristic detection unit detects a dedicated device that executes a compression process mounted on the storage processing device as a characteristic of the storage processing device,
When the degree of compression of the data flow detected by the flow characteristic detection unit is higher than a predetermined value, the switch unit allocates the storage processing device equipped with a dedicated device that executes the compression processing to the data flow ,
Storage system. The storage system according to claim 5,
The flow characteristic detection unit detects, as the data flow characteristic, whether or not the data flow is compressed before being input to the storage processing device,
The switch unit is a case where the data flow is not compressed by the flow characteristic detection unit before being input to the storage processing device, and the degree of compression of the data flow detected by the flow characteristic detection unit Assigning the storage processing device equipped with a dedicated device for executing the compression processing to the data flow when the value is higher than a predetermined value,
Storage system. The storage system according to any one of claims 2 to 6,
The flow characteristic detection unit detects the size of a file in the data flow as the characteristic of the data flow,
The device characteristic detection unit is a dedicated device equipped with a high-speed storage device that is faster in data input / output installed in the storage processing device than the auxiliary storage device installed in the storage device, as the characteristics of the storage processing device. Detect
When the file size of the data flow detected by the flow characteristic detection unit is smaller than a predetermined value, the switch unit includes the storage processing device equipped with a dedicated device equipped with the high-speed storage device for the data flow. assign,
Storage system. The storage system according to any one of claims 2 to 7,
The device characteristic detection unit detects a load status of the storage processing device as a characteristic of the storage processing device,
The switch unit detects a load detected by the flow characteristic detection unit in the data flow when there is no storage processing device equipped with a dedicated device corresponding to the data flow characteristic detected by the flow characteristic detection unit. Assign other said storage processing device according to the situation,
Storage system. When distributing data to a plurality of storage devices and storing other data having the same contents as the data already stored in the storage device, the data already stored in the storage device is stored. A switch control device connected to a switch unit that sets a data flow consisting of a group of data to flow through any one of the plurality of storage processing devices that perform deduplication processing to be referred to as the other data. There,
Based on a predetermined characteristic of the data flow detected for each of the data flows and a predetermined characteristic of the storage processing device detected for each of the storage processing devices, the data flow is stored in the data flow. A flow setting unit that assigns a processing device and sets the switch unit so that the data flow flows to the allocated storage processing device;
Switch control device. Multiple storage devices,
Data that is already stored in the storage device when data is distributed and stored in the plurality of storage devices and other data having the same content as the data that is already stored in the storage device is stored. A plurality of storage processing devices for performing deduplication processing to refer to as other data,
A flow control method by a storage system comprising: a switch unit that assigns a data flow consisting of a group of data to any one of the storage processing devices and sets the data flow to flow to the assigned storage processing device. ,
Detecting a predetermined characteristic of the data flow for each data flow, and detecting a predetermined characteristic of the storage processing apparatus for each storage processing apparatus;
The switch unit determines the storage processing device to which the data flow is assigned based on the detected characteristic of the data flow and the detected characteristic of the storage processing device;
Flow control method.

Images