JP6281333B2 - Storage system - Google Patents

Description

  The present invention relates to a storage system, and more particularly to a storage system that eliminates redundant storage of data having the same content.

  In recent years, with the development and spread of computers, various types of information have been converted into digital data. As a device for storing such digital data, there are storage devices such as a magnetic tape and a magnetic disk. Since the data to be stored increases day by day and becomes enormous, a large-capacity storage system is required. In addition, reliability is required while reducing the cost of the storage device. In addition to this, it is necessary that data can be easily retrieved later. As a result, there is a demand for a storage system that can automatically increase storage capacity and performance, eliminate duplicate storage, reduce storage costs, and have high redundancy.

  In response to this situation, in recent years, content address storage systems have been developed. In this content address storage system, data is distributed and stored in a plurality of storage devices, and the storage location where the data is stored is specified by a unique content address specified according to the content of the data. Further, in the content address storage system, predetermined data is divided into a plurality of fragments, and a fragment that becomes redundant data is further added, and the plurality of fragments are stored in a plurality of storage devices, respectively. There are also things.

  Then, in the content address storage system as described above, by designating the content address later, the data stored in the storage location specified by the content address, that is, the fragment is read out, and a plurality of fragments before the division are read. Predetermined data can be restored.

  The content address is generated based on a value generated to be unique according to the data content, for example, a hash value of the data. For this reason, if it is duplicate data, the data of the same content can be acquired by referring to the data at the same storage position. Therefore, there is no need to store duplicate data separately, and duplicate recording can be eliminated to reduce the data capacity.

  In particular, in a storage system having a function of eliminating duplicate storage as described above, data to be written such as a file is divided into a plurality of block data of a predetermined capacity, compressed, and written to a storage device. In this way, by eliminating duplicate storage in units of block data obtained by dividing a file, the duplication rate increases and the data capacity is reduced.

JP 2013-47933 A

  Here, Patent Document 1 proposes to improve the duplicate storage exclusion ratio by controlling the position of stored data in a storage system having a volume of a plurality of hierarchies. On the other hand, as a storage medium constituting the storage system, a low-speed device such as an HDD (Hard disk drive) that can be realized at low cost while reading and writing is low, or an SSD (Solid State Drive) that is high in cost but high in reading and writing. Such a high-speed device is used. In this case, if the storage destination of the stored data is determined considering only the improvement of the duplicate storage exclusion rate, the storage data may be stored in a device inappropriate for the usage and attributes of the stored data. Then, problems such as processing delay due to storage in a low-speed device, reduction in utilization efficiency of the high-speed device, and increase in data movement processing between devices occur. As a result, there arises a problem that the performance of the storage system is degraded.

  Therefore, an object of the present invention is to provide a storage system that can solve the problem of performance degradation in a storage system having a deduplication function.

A storage system according to an aspect of the present invention
A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage control unit used as the duplicate data with reference to other data that is present,
The storage location determination unit determines a storage location of the duplicate data that is determined to be related to the non-redundant data according to a preset criterion, and determines a storage location of the non-redundant data based on the determination result ,
The configuration is as follows.

The program which is the other form of this invention is:
In the information processing device,
A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device And a data storage control unit that is used as the duplicate data with reference to other data,
The storage location determination unit determines a storage location of the duplicate data that is determined to be related to the non-redundant data according to a preset criterion, and determines a storage location of the non-redundant data based on the determination result ,
It takes the structure of realizing.

A data storage method according to another aspect of the present invention includes:
Determine whether the data to be stored is in a duplicate state already stored in the storage device,
Determine the storage destination of non-duplicate data that is the storage target data that is not duplicated,
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage method for performing data storage processing to be used as the duplicate data with reference to other data,
Determining the storage location of the duplicate data that is determined to be related to the non-duplicate data according to a preset criterion, and determining the storage location of the non-duplication data based on the determination result;
The configuration is as follows.

  With the configuration as described above, the present invention can improve performance in a storage system having a deduplication function.

1 is a block diagram showing an outline of the configuration of a storage system in Embodiment 1 of the present invention. 1 is a functional block diagram showing a configuration of a storage system in Embodiment 1 of the present invention. FIG. 3 is a diagram illustrating an example of data stored in the storage system disclosed in FIG. 2. FIG. 3 is a diagram illustrating an example of data stored in the storage system disclosed in FIG. 2. FIG. 3 is an explanatory diagram for explaining a state of data writing processing in the storage system disclosed in FIG. 2. FIG. 3 is an explanatory diagram illustrating a state of data write processing in the storage system disclosed in FIG. 2. FIG. 3 is an explanatory diagram illustrating an outline of a data writing process in the storage system disclosed in FIG. 2. 3 is a flowchart showing an operation of data write processing in the storage system disclosed in FIG. 2. 3 is a flowchart showing an operation of data write processing in the storage system disclosed in FIG. 2. 3 is a flowchart showing an operation of data write processing in the storage system disclosed in FIG. 2. 3 is a flowchart showing an operation of data write processing in the storage system disclosed in FIG. 2. It is a block diagram which shows the structure of the storage system in attachment 1 of this invention.

<Embodiment 1>
A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing an outline of the configuration of the storage system, and FIG. 2 is a functional block diagram showing the configuration of the storage system. 3 to 4 are diagrams illustrating an example of data stored in the storage system. 5 to 7 are explanatory diagrams for explaining the state of data write processing in the storage system. 8 to 11 are flowcharts showing the operation of the storage system.

[Constitution]
As shown in FIG. 1, the storage system 1 of the present invention has a configuration in which a plurality of server computers are connected. Specifically, the storage system 1 includes an access node 2 that is a server computer that controls storage and reproduction operations in the storage system 1 itself, and a storage node 3 that is a server computer including a storage device that stores data. Yes. Note that the number of access nodes 2 and the number of storage nodes 3 are not limited to those shown in FIG. 2, and more nodes 2 and 3 may be connected. Further, the storage system 1 of the present invention may be composed of one computer.

  Here, the storage system 1 in this embodiment will be described as being configured by a deduplication controller 20 and a hierarchical storage system 30 as shown in FIG. The deduplication controller 20 is mainly configured by the access node 2 described above, and the hierarchical storage system 30 is mainly configured by the storage node 3 described above. The deduplication controller 20 and the tiered storage system 30 are not necessarily limited to being configured by the access node 2 and the storage node 3.

  As shown in FIG. 2, the deduplication controller 20 includes a deduplication processing unit 21, a data management unit 22, a NAS interface 23, and a virtual file system, which are constructed by incorporating a program into the equipped arithmetic device. 24, a data I / OI / F 25, and a non-overlapping block storage position determination unit 26. In addition, the deduplication controller 20 has a determination selection policy 27 stored in the equipped storage device.

  The NAS interface 23 is an interface that accepts a connection using a NAS protocol (CIFS, NFS, etc.). Here, the storage system 1 itself is NAS (Network Attached Storage), and accepts data to be stored from other backup servers and archive servers.

  The deduplication processing unit 21 (duplication determination unit) divides input data into fixed-length or variable-length blocks, and determines whether each block is already stored with reference to the data management unit 22. Specifically, the deduplication processing unit 21 determines whether the block is a duplicate block in a duplicate state that has already been stored, or a non-overlap block in a non-duplicate state that has not yet been stored. And

  The data management unit 22 manages the storage destination address of each block in a hash table that is referred to based on the hash value of each block.

  The virtual file system 24 provides requested data by associating data inputted / outputted by the NAS protocol with block-unit data actually stored.

  The non-redundant data storage destination device determination unit 26 (storage destination determination unit) determines in which device of the hierarchical storage system 30 a non-redundant block in a non-overlapping state that does not exist in the storage is stored, and is hierarchical Request to the storage system 30.

  The data I / O I / F 25 is an interface for transferring data to the hierarchical storage system 30. For example, the interface is realized by FC (Fibre Channel), iSCSI or the like.

  The determination selection policy 27 is a table that defines a method for determining the storage destination of non-overlapping blocks. This policy can be rewritten from the outside.

  As shown in FIG. 2, the hierarchical storage system 30 includes a data I / OI / F 31, an existing data storage destination acquisition I / F 32, which are constructed by incorporating a program into the equipped arithmetic device. A new data storage destination request reception I / F 33 and a hierarchization management unit 34 are provided. The hierarchization management unit 34 further includes a data storage I / O management unit 35, a data storage destination request unit 36, and a data storage destination change unit 37.

  The hierarchical storage system 30 includes devices 41, 42, and 43, which are a plurality of storage devices with different performance for storing data. For example, the device is a storage device in which the access speed to the storage area is divided into high speed, medium speed, and low speed according to a predetermined standard.

  The data I / O I / F 31 is an interface that exchanges data with the deduplication controller 20.

  The hierarchization management unit 34 (data storage destination control unit) has a function of managing data that is distributed and stored in a plurality of devices and moving the data based on the access frequency. Specifically, the data storage I / O management unit 35 that constitutes the hierarchical management unit 34, when an I / O request is received from the host, the address from the host and the device that is actually stored. Converts to address and passes data. For example, when a new data write request is received, an inquiry is made to the “data storage destination request” as to whether or not a storage destination device is specified for the data to be written. In addition, it has a function of returning the number of the storage destination device of already written data.

  The data storage destination request unit 36 constituting the hierarchization management unit 34 receives a request for designating a storage destination device from the deduplication controller 20 side. The data storage destination changing unit 37 constituting the hierarchization management unit 34 periodically moves the storage destination of the stored data based on the access frequency or the like.

  The existing data storage destination device acquisition I / F 32 receives the address and offset from the deduplication controller 20, and stores the device numbers (1 to T in order from the highest one) in which the data is stored in the data storage I / O management unit 35. ) And return the value.

  The new data storage destination request reception I / F unit 33 receives the address, offset, and storage device number from the deduplication controller 20 and stores them in the data storage destination request unit 36.

  Here, the deduplication controller 20 and the hierarchical storage system 30 which are the storage systems 1 in this embodiment have a function of eliminating duplicate storage of data. Such a duplicate storage elimination function is realized mainly by the duplicate elimination processing unit 21, the data management unit 22, and the data storage I / O management unit 35. Details of the function will be described with reference to FIGS. To do. Hereinafter, the deduplication controller 20 and the tiered storage system 30 will not be distinguished from each other and will be described as operations by the storage system 1. Note that the method for eliminating duplicate memory is not limited to the method described below.

  First, as indicated by an arrow Y1 in FIGS. 5 and 6, the storage system 1 receives an input of a file A requested to be written. Then, as indicated by an arrow Y2 in FIGS. 5 and 6, the file A is divided into predetermined capacity (for example, 64 KB) or variable length block data D.

  Subsequently, based on the data content of the divided block data D, a unique hash value H representing the data content is calculated (arrow Y3 in FIG. 6). For example, the hash value H is calculated from the data content of the block data D using a preset hash function.

  Subsequently, using the hash value H of the block data D of the file A, it is checked whether or not the block data D is already stored. Specifically, first, the block data D that has already been stored has its hash value H and a content address CA representing the storage position associated with each other and registered in an MFI (Main Fragment Index) file. Therefore, when the hash value H of the block data D calculated before the storage exists in the MFI file, it can be determined that the block data D having the same contents is already stored (arrow Y4 in FIG. 6). ). In this case, the content address CA associated with the hash value H in the MFI that matches the hash value H of the block data D before storage is acquired from the MFI file. Then, this content address CA is returned as the content address CA of the block data D requested to be written.

  The already stored data referred to by the returned content address CA is used as the block data D requested to be written. That is, it is assumed that the block data D requested to be written is stored by designating an area referred to by the returned content address CA as a storage destination of the block data D requested to be written. This eliminates the need to store the block data D relating to the write request in a device that is actually a storage device.

  When it is determined that the block data D related to the write request is a non-overlapping block that has not yet been stored, the block data D related to the write request is written as follows. First, the block data D related to the write request is compressed and divided into a plurality of fragment data of a predetermined capacity as indicated by an arrow Y5 in FIG. For example, as shown by symbols D1 to D9 in FIG. 5, the data is divided into nine fragment data (divided data 91). Further, even if some of the divided fragment data is missing, redundant data is generated so that the original block data can be restored and added to the divided fragment data 91. For example, three pieces of fragment data (redundant data 92) are added as indicated by reference numerals D10 to D12 in FIG. As a result, a data set 90 composed of 12 fragment data composed of nine divided data 91 and three redundant data 92 is generated.

  Subsequently, each fragment data constituting the data set generated as described above is distributed and stored in each storage area formed in the storage device. For example, as shown in FIG. 5, when 12 pieces of fragment data D1 to D12 are generated, each piece of fragment data D1 to D12 is stored in a data storage file formed in each of a plurality of storage devices. (See arrow Y6 in FIG. 6).

  Subsequently, the storage system 1 generates a content address CA indicating the storage position of the fragment data D1 to D12 stored as described above, that is, the storage position of the block data D restored by the fragment data D1 to D12. Manage. Specifically, a part of the hash value H (short hash) calculated based on the contents of the stored block data D (for example, the top 8B (bytes) of the hash value H), information indicating the logical storage position, Are combined to generate a content address CA. Then, this content address CA is returned to the file system in the storage system 1 (arrow Y7 in FIG. 6). Then, the storage system 1 manages identification information such as the file name of the backup target data and the content address CA in association with the file system.

  Further, each storage node 3 manages the content address CA of the block data D and the hash value H of the block data D in an MFI file (arrow 8 in FIG. 6). As described above, the content address CA is stored in the device in association with the information specifying the file, the hash value H, and the like.

  As described above, in the storage system 1 according to the present embodiment, the duplicate block has the same data contents and refers to other data stored in the device, so that the storage process is performed without actually storing the data. Perform deduplication processing. Non-overlapping blocks are divided into fragments to make them redundant, and are distributed and stored in a plurality of storage devices. However, the storage system 1 in the present invention is not necessarily limited to eliminating duplicate storage by the above-described method, and storage processing may be performed by other methods.

  Here, in the present invention, when the block data D related to the write request is a non-overlapping block that has not yet been stored as described above, processing for determining a storage destination device is performed as follows. The operation will be described mainly with reference to the explanatory diagram of FIG. 7 and the flowcharts of FIGS. 8 shows the operation in the entire storage system, and FIGS. 9 to 10 show the operation when determining the storage destination of the block.

  First, the operation of the deduplication controller 20 will be described.

  When a file is input to the deduplication controller 20, the file is temporarily stored in the buffer area (step S1 in FIG. 8). Then, the deduplication processing unit 21 divides the block into fixed lengths from the beginning of the file, and determines whether each block has already been stored in the storage (step S2 in FIG. 8). At this time, duplication determination is performed from the top block of the file as described above. Note that block division may be performed with a variable length.

  As a result of the duplication determination, the duplicate block refers to the existing block without storing the actual data, and if it is not the end of the file, it proceeds to the duplication judgment of the next block (step S11 in FIG. 9: No, FIG. 10). Step S28: No, see step S29).

  On the other hand, as a result of the duplication determination, the non-overlapping block is stored in the device after determining the storage destination device. Hereinafter, processing for determining a device that is a storage destination of non-overlapping blocks will be described in detail.

  Specifically, when determining the storage destination of the non-overlapping block, first, as shown in FIG. 7A, it is referred to whether or not the file attribute is defined in the determination selection policy 27 (FIG. 8). Step S3). Then, as the policy for determining the storage destination of the non-overlapping blocks constituting the file, the “present method” set in the present invention is used, or the “specified policy” already set in the hierarchical storage system 30 is used. It is determined whether or not. Note that the “prescribed policy” is a policy that determines the storage destination of non-overlapping blocks, which is set as an initial setting, or is set based on access frequency, free space, or the like.

  The determination selection policy 27 is set as shown in FIG. 3, for example. In FIG. 3, “attribute” means an attribute of a file written from the top, and defines a determination method and a determination criterion parameter when the attribute corresponds to this attribute. Among these, the “determination method” indicates a method for determining a storage destination of a non-overlapping block that becomes new data based on a file attribute at the stage when the file is created.

  In the example of FIG. 3, when the file extension is “.XXX”, the process proceeds to the next process in order to determine the storage destination of the non-overlapping block by “this method” (after step S4 in FIG. 8). FIG. 9 and subsequent steps S12). If not satisfied, the storage destination is determined using the “default policy”, and the non-overlapping block is stored (step S6 in FIG. 8).

  In addition, as described above, the reason for defining the policy for determining the storage location based on the “attribute” of the file is that it is better to be able to access it at high speed, it is better to save it at low speed, but it is necessary depending on the type of file. This is because the characteristics are different. For example, differential data (non-overlapping blocks) between OS (Operating System) kernels and application minor versions is expected to be accessed at high speed in the same way as stored data (overlapping blocks) in other peripheral areas. On the other hand, data files and the like may be allowed to be always stored in a low-speed but inexpensive device.

  Subsequently, when determining the storage destination of non-overlapping blocks from the “attribute” of the file by “this method”, the storage destination of non-overlapping blocks is determined as follows, and storage as shown in FIG. The previous list is created (step S4 in FIG. 8). Hereinafter, the operation when the storage destination of the non-overlapping block is determined by the “present method” will be described mainly with reference to FIGS. 9 to 10.

  First, for a non-overlapping block (step S11 in FIG. 9: Yes), a determination criterion parameter (M, N, P) corresponding to the “attribute” of the file is acquired from the determination selection policy 27 (step S12 in FIG. 9). . Here, M is a value that specifies the number of peripheral blocks of non-overlapping blocks. N is a value that designates the ratio of the number of overlapping blocks in the preceding and following M blocks, which is determined when using this method. P is a value used to determine whether or not P% or more of the overlapping blocks of the preceding and succeeding M blocks are stored in a specific storage destination device.

  Subsequently, it is determined whether or not the rate (duplication rate) at which the duplicate block B2 located in the predetermined range before and after the non-redundant block B1 already exists in the storage is equal to or greater than the threshold value. Specifically, as shown in FIG. 7B, first, the number of overlapping blocks B2 in the M blocks before and after the non-overlapping block B1 is counted (step S13 in FIG. 9). Then, the ratio of the number of duplicate blocks B2 is obtained out of 2M which is the total number of M blocks before and after the non-overlapping block B1, and it is checked whether the ratio exceeds N% (threshold) (step in FIG. 9). S14). If the overlapping block ratio does not exceed N% (step S14 in FIG. 9: No), the device set by the prescribed policy is determined as the storage destination of the non-overlapping block B1 (step S25 in FIG. 10).

  On the other hand, when the ratio of overlapping blocks exceeds N% (step S14 in FIG. 9: Yes), as shown in FIG. 7C, storage of the overlapping blocks among the M blocks before and after the non-overlapping blocks. The previous distribution is determined (step S15 in FIG. 9). Specifically, the number of the current storage destination device is acquired for all overlapping blocks B2 of the preceding and following M blocks, and the number for each device is counted.

  Here, when the duplicated block storage position determination unit 26 inputs an address / offset for each duplicated block to the existing data storage destination I / F 32 of the hierarchical storage system 30, the device number (1 ~ T). Device numbers 1 to T (T = number of devices) are assigned to the devices 41 to 43 of the hierarchical storage system 30 in descending order of access performance. That is, the device with device number 1 is the fastest device, and the device number T is the slowest device.

  Then, it is checked whether or not the duplicate block among the M blocks before and after the non-duplicate block is stored in a specific device at P% (threshold) or more (step S16 in FIG. 9). When P% of the duplicate block is not stored in a specific device (step S16 in FIG. 9: No), the device set by the prescribed policy is determined as the storage destination of the non-duplicate block B1 (FIG. 10). Step S25).

  On the other hand, when P% or more of the duplicate block is stored in a specific device (step S16: Yes in FIG. 9), as shown in FIG. Based on the stored storage destination and the determination selection policy 27, the storage destination of the non-overlapping block is finally determined. Specifically, from the device number in which P% or more of the duplicate blocks are stored, the device is a high-speed device (the device number <= T / 2) or a low-speed device (the device number> T / 2) is determined (step S21 in FIG. 10). Then, referring to the storage location when the storage location of the duplicate block is a high-speed device or the storage location when it is a low-speed device, which is defined in the judgment selection policy 27, the specified storage location Based on this, the storage position of the non-overlapping block is determined.

  For example, in the example of the determination selection policy 27 in FIG. 3, in the case of “file having a file size of ZZZ bytes or more” and “P% or more of overlapping blocks of preceding and following M blocks are high-speed devices” (step S21 in FIG. 10). : Yes), the setting of the high-speed device is used (step S22 in FIG. 10). In this case, a device in which P% or more of the duplicate blocks are stored is determined as a non-duplicate block storage destination (step S24 in FIG. 10: Yes, step S25). On the other hand, if “P% or more of the overlapping blocks of the preceding and following M blocks is a low speed device” (step S21 in FIG. 10: No), the setting of the low speed device is used (step S23 in FIG. 10). In this case, a device based on the default policy is determined as the storage destination (step S24 in FIG. 10: No, step S26).

  Subsequently, when the storage destination of the non-overlapping block is determined as a storage destination by the above-described processing (step S25 in FIG. 10), the information is stored as follows. It adds to the list | wrist requested | required of the hierarchical storage system 30 (step S27 of FIG. 10). For example, as shown in FIG. 4, an address, an offset, and a device number of the storage destination are added to the list as information on the storage destination of the non-overlapping block (step S27).

  The above processing is executed up to the end block of the file (step S28 in FIG. 10). When the end of the file is reached, the deduplication controller 20 sends a list of requests to the data storage destination request reception I / F 33 of the hierarchical storage system 30 (step S5 in FIG. 8). Then, the non-overlapping blocks are written to the hierarchical storage system 30 through the data I / O I / Fs 25 and 31 (step S6 in FIG. 8).

  Next, the operation of the hierarchical storage system 30 will be described in detail with reference to the flowchart of FIG.

  First, when a data write request is received in the data I / OI / F 31, the data storage destination request is referred to whether or not a storage destination device is designated from the deduplication controller 20 for the write data (FIG. 11). Step S31). When the storage destination device is designated by the request from the deduplication controller 20 (step S31 in FIG. 11: Yes), the storage destination in which the request has been made is acquired (step S32 in FIG. 11). For example, the storage position of the non-overlapping block is acquired from the address of the list shown in FIG.

  Then, the non-overlapping block is written in the acquired storage position in the device specified by the list, and the address conversion table is updated (step S33 in FIG. 11). When the storage destination device is not specified by the request (step S31 in FIG. 11: No), the address conversion table is updated by writing to the device specified by the default policy of the hierarchical storage system 30 (FIG. 11). Step S34).

  In the above storage system according to the present invention, non-overlapping blocks constituting a file can be stored in the same device in which the related overlapping blocks belonging to the same file are stored. Can be stored on any device. As a result, it is possible to suppress a processing delay and a decrease in utilization efficiency of the storage device, and to suppress a data movement process between the storage devices, thereby improving the performance of the storage system.

  Here, in the above description, when determining the storage destination of a non-overlapping block, the storage destination of a predetermined range of overlapping blocks before and after the non-overlapping block is referred to. It is not limited to using. The storage location of the non-overlapping block may be determined with reference to the storage location of the overlapping block determined to be related to the non-overlapping block by other criteria.

  In the storage system of the present invention, a device constituting another storage device may be used instead of the above-described hierarchical storage system 30. For example, a combination of a CPU cache, memory, and HDD may be used as a combination of devices capable of storing a plurality of data with different access speeds. However, the storage device as the storage destination does not necessarily have to have different access speeds, and may be any storage device having the same performance or different performance.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The outline of the configuration of the storage system (see FIG. 12), program, and data storage method in the present invention will be described below. However, the present invention is not limited to the following configuration.

(Appendix 1)
A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage control unit used as the duplicate data with reference to other data that is present,
The storage location determination unit determines a storage location of the duplicate data that is determined to be related to the non-redundant data according to a preset criterion, and determines a storage location of the non-redundant data based on the determination result ,
Storage system.

(Appendix 2)
The storage system according to attachment 1, wherein
The non-duplicate data and the duplicate data constituting the storage target data are data obtained by dividing the data to be stored into a plurality of blocks,
The storage destination determining unit determines a storage destination of the duplicate data located in a predetermined range before and after the position of the non-duplicate data before division, and determines a storage destination of the non-duplicate data based on the determination result To
Storage system.

(Appendix 3)
The storage system according to appendix 2,
The storage location determination unit determines the storage location of the duplicate data when the ratio of the duplicate data to all blocks located in a predetermined range before and after the position of the non-duplicate data before the division is equal to or greater than a threshold value. Based on the determination result, the storage location of the non-duplicate data is determined.
Storage system.

(Appendix 4)
The storage system according to appendix 2 or 3,
The storage destination determination unit determines a ratio of the duplicate data stored in a predetermined range before and after the position of the non-duplicate data before the division, and stores the non-duplicate data based on the determination result. Determine the destination,
Storage system.

(Appendix 5)
The storage system according to appendix 4, wherein
The storage destination determination unit identifies the performance of the storage device that is the storage destination in which the ratio of each of the duplicate data storage destinations located in a predetermined range before and after the position of the non-duplicate data before the division is greater than or equal to a threshold value Then, the performance of the specified storage device is used as the determination result, and the storage destination of the non-duplicate data is determined based on the determination result.
Storage system.

(Appendix 6)
The storage system according to appendix 5,
The storage destination determination unit determines whether the access speed to the storage area of the storage device is high according to a predetermined standard as the performance of the storage device that is the storage destination in which the ratio of the duplicate data for each storage destination is equal to or greater than a threshold value. Determining whether or not the performance of the specified storage device is the determination result, and determining the storage location of the non-duplicate data based on the determination result;
Storage system.

(Appendix 7)
The storage system according to any one of appendices 1 to 6,
A prescription policy in which a storage destination of the non-duplicate data is set in advance and a determination result policy in which a storage destination of the non-duplicate data is set in accordance with the determination result are stored,
The storage destination determination unit determines whether to use the specified policy or the determination result policy based on the determination result, and determines a storage destination of the non-duplicate data based on the determined policy.
Storage system.

(Appendix 8)
The storage system according to appendix 7,
The storage location determination unit
Decide whether to use the prescribed policy according to the attribute of the data to be stored,
When it is determined to use the specified policy according to the attribute of the data to be stored, the storage destination of the non-duplicate data is determined based on the specified policy,
When it is determined not to use the specified policy according to the attribute of the data to be stored, it is determined based on the determination result whether to use the specified policy or the determination result policy, and the determined policy To determine a storage location for the non-duplicate data,
Storage system.

  According to the above invention, first, it is determined whether the storage target data is already stored in the storage device (whether it is duplicate data) or not yet stored (non-duplication data), and the duplicate data is already stored in the storage device. The storage process is completed by referring to the existing data stored. On the other hand, for non-duplicate data, the storage location of the duplicate data related to the non-duplication data is determined, the storage location of the non-duplication data is determined based on the determination result, and the storage device at the determined storage location is determined. Store non-duplicate data.

  At this time, for example, as duplicate data related to non-redundant data, a block located in a predetermined range before and after the data arrangement before division of the block which is non-redundant data is used. And, among the blocks located in the predetermined range before and after the non-duplicate data, the storage location of the non-duplication data is determined according to the ratio of the duplicate data related to the non-duplication data and the ratio for each storage destination of the duplicate data. decide. For example, the storage destination of non-redundant data is determined according to the access performance of the storage device that is the storage destination of the duplicate data. Further, it is determined whether the storage destination is set according to the prescribed policy or the storage destination is determined according to the determination result according to the attribute of the data to be stored and the determination result.

  Thus, in the present invention, non-duplicate data can be stored in a storage device that is the same as or related to the storage device in which the related duplicate data is stored, and the appropriate storage device according to the use and attribute of the data. Can be stored. As a result, it is possible to suppress a processing delay and a decrease in utilization efficiency of the storage device, and to suppress a data movement process between the storage devices, thereby improving the performance of the storage system.

(Appendix 9)
In the information processing device,
A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device And a data storage control unit that is used as the duplicate data with reference to other data,
The storage location determination unit determines a storage location of the duplicate data that is determined to be related to the non-redundant data according to a preset criterion, and determines a storage location of the non-redundant data based on the determination result ,
A program to make things happen.

(Appendix 10)
Determine whether the data to be stored is in a duplicate state already stored in the storage device,
Determine the storage destination of non-duplicate data that is the storage target data that is not duplicated,
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage method for performing data storage processing to be used as the duplicate data with reference to other data,
Determining the storage location of the duplicate data that is determined to be related to the non-duplicate data according to a preset criterion, and determining the storage location of the non-duplication data based on the determination result;
Data storage method.

(Appendix 11)
The data storage method according to appendix 10, wherein
The non-duplicate data and the duplicate data constituting the storage target data are data obtained by dividing the data to be stored into a plurality of blocks,
Determining the storage destination of the duplicate data located in a predetermined range before and after the position of the non-redundant data before the division, and determining the storage destination of the non-redundant data based on the determination result;
Data storage method.

(Appendix 12)
The data storage method according to attachment 11, wherein
When the ratio of the duplicate data to all the blocks located in a predetermined range before and after the position of the non-duplicate data before the division is equal to or greater than a threshold, the storage location of the duplicate data is determined, and based on the determination result Determining a storage location of the non-duplicate data;
Data storage method.

(Appendix 13)
The data storage method according to appendix 11 or 12,
Determining a ratio for each storage destination of the duplicate data located in a predetermined range before and after the position of the non-duplicate data before the division, and determining a storage destination of the non-duplication data based on the determination result;
Data storage method.

(Appendix 14)
A data storage method according to attachment 13, wherein
The performance of the storage device that is the storage destination in which the ratio for each storage destination of the duplicate data that is located in a predetermined range before and after the position of the non-duplicate data before the division is equal to or greater than a threshold is specified, and the specified storage device As the determination result, the storage destination of the non-duplicate data is determined based on the determination result.
Data storage method.

(Appendix 15)
The data storage method according to attachment 14, wherein
As a performance of the storage device that is the storage destination in which the ratio for each storage destination of the duplicate data is equal to or greater than a threshold value, specify whether the access speed to the storage area of the storage device is high speed according to a predetermined criterion, The performance of the identified storage device is used as the determination result, and the storage destination of the non-duplicate data is determined based on the determination result.
Data storage method.

(Appendix 16)
The data storage method according to any one of appendices 10 to 15,
Based on the determination result, it is determined whether to use a prescribed policy in which the storage destination of the non-duplicate data is set in advance or a determination result policy in which the storage destination of the non-duplication data is set in accordance with the determination result, Based on the determined policy, a storage location of the non-duplicate data is determined.
Data storage method.

(Appendix 17)
The data storage method according to appendix 16, wherein
Decide whether to use the prescribed policy according to the attribute of the data to be stored,
When it is determined to use the specified policy according to the attribute of the data to be stored, the storage destination of the non-duplicate data is determined based on the specified policy,
When it is determined not to use the specified policy according to the attribute of the data to be stored, it is determined based on the determination result whether to use the specified policy or the determination result policy, and the determined policy To determine a storage location for the non-duplicate data,
Data storage method.

  Note that the above-described program is stored in a storage device or recorded on a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

  Although the present invention has been described with reference to the above-described embodiment and the like, the present invention is not limited to the above-described embodiment. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Storage system 2 Access node 3 Storage node 20 Deduplication controller 21 Deduplication processing part 22 Data management part 23 NAS interface 24 Virtual file system 25 Data I / OI / F
26 Non-overlapping block storage position determination unit 27 Determination selection policy 30 Hierarchical storage system 31 Data I / OI / F
32 Existing data storage location acquisition I / F
33 New data storage location request I / F
34 Hierarchical management unit 35 Data storage I / F management unit 36 Data storage destination request unit 37 Data storage destination change unit 41 High speed device 42 Medium speed device 43 Low speed device 100 Storage system 101 Duplicate determination unit 102 Storage destination determination unit 103 Data storage Control unit 110 storage device

Claims (12)

A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage control unit used as the duplicate data with reference to other data that is present,
The non-duplicate data and the duplicate data constituting the storage target data are data obtained by dividing the data to be stored into a plurality of blocks,
The storage destination determining unit determines a storage destination of the duplicate data located in a predetermined range before and after the position of the non-duplicate data before division, and determines a storage destination of the non-duplicate data based on the determination result To
Storage system. The storage system according to claim 1 ,
The storage location determination unit determines the storage location of the duplicate data when the ratio of the duplicate data to all blocks located in a predetermined range before and after the position of the non-duplicate data before the division is equal to or greater than a threshold value. Based on the determination result, the storage location of the non-duplicate data is determined.
Storage system. The storage system according to claim 1 or 2 ,
The storage destination determination unit determines a ratio of the duplicate data stored in a predetermined range before and after the position of the non-duplicate data before the division, and stores the non-duplicate data based on the determination result. Determine the destination,
Storage system. The storage system according to claim 3 ,
The storage destination determination unit identifies the performance of the storage device that is the storage destination in which the ratio of each of the duplicate data storage destinations located in a predetermined range before and after the position of the non-duplicate data before the division is greater than or equal to a threshold value Then, the performance of the specified storage device is used as the determination result, and the storage destination of the non-duplicate data is determined based on the determination result.
Storage system. The storage system according to claim 4 ,
The storage destination determination unit determines whether the access speed to the storage area of the storage device is high according to a predetermined standard as the performance of the storage device that is the storage destination in which the ratio of the duplicate data for each storage destination is equal to or greater than a threshold value. Determining whether or not the performance of the specified storage device is the determination result, and determining the storage location of the non-duplicate data based on the determination result;
Storage system. The storage system according to any one of claims 1 to 5 ,
A prescription policy in which a storage destination of the non-duplicate data is set in advance and a determination result policy in which a storage destination of the non-duplicate data is set in accordance with the determination result are stored,
The storage destination determination unit determines whether to use the specified policy or the determination result policy based on the determination result, and determines a storage destination of the non-duplicate data based on the determined policy.
Storage system. The storage system according to claim 6 ,
The storage location determination unit
First, determine whether to use the prescribed policy based on the attribute of the data to be stored,
When it is determined to use the specified policy based on the attribute of the data to be stored, the storage location of the non-duplicate data is determined based on the specified policy,
When it is determined not to use the specified policy based on the attribute of the data to be stored, it is determined based on the determination result whether to use the specified policy or the determination result policy. To determine a storage location for the non-duplicate data,
Storage system. A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage control unit used as the duplicate data with reference to other data that is present,
The storage destination determination portion, the determined storage destination of the duplication data is judged to be related by a predetermined reference and non-overlapping data, based on the determination result, determines a storage destination of the non-duplicate data ,
Further, a prescribed policy in which the storage destination of the non-duplicate data is set in advance, and a determination result policy in which the storage destination of the non-duplicate data is set in accordance with the determination result are stored,
The storage destination determination unit determines whether to use the specified policy or the determination result policy based on the determination result, and determines a storage destination of the non-duplicate data based on the determined policy.
Storage system. In the information processing device,
A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device And a data storage control unit that is used as the duplicate data with reference to other data,
The non-duplicate data and the duplicate data constituting the storage target data are data obtained by dividing the data to be stored into a plurality of blocks,
The storage destination determining unit determines a storage destination of the duplicate data located in a predetermined range before and after the position of the non-duplicate data before division, and determines a storage destination of the non-duplicate data based on the determination result To
A program to make things happen. In the information processing device,
A duplication determination unit that determines whether or not the storage target data is in a duplication state that is already stored in the storage device;
A storage destination determination unit that determines a storage destination of non-duplicate data that is the storage target data that is not duplicated;
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device And a data storage control unit that is used as the duplicate data with reference to other data,
The storage destination determination portion, the determined storage destination of the duplication data is judged to be related by a predetermined reference and non-overlapping data, based on the determination result, determines a storage destination of the non-duplicate data ,
Further, the information processing apparatus stores a regulation policy in which a storage destination of the non-duplicate data is set in advance, and a determination result policy in which the storage destination of the non-duplicate data is set according to the determination result,
The storage destination determination unit determines whether to use the specified policy or the determination result policy based on the determination result, and determines a storage destination of the non-duplicate data based on the determined policy.
A program to make things happen. Determine whether the data to be stored is in a duplicate state already stored in the storage device,
Determine the storage destination of non-duplicate data that is the storage target data that is not duplicated,
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage method for performing data storage processing to be used as the duplicate data with reference to other data,
The non-duplicate data and the duplicate data constituting the storage target data are data obtained by dividing the data to be stored into a plurality of blocks,
Determining the storage destination of the duplicate data located in a predetermined range before and after the position of the non-redundant data before the division, and determining the storage destination of the non-redundant data based on the determination result;
Data storage method. Determine whether the data to be stored is in a duplicate state already stored in the storage device,
Determine the storage destination of non-duplicate data that is the storage target data that is not duplicated,
The non-duplicate data is stored in the storage device that is the determined storage destination, and the duplicate data that is the storage target data is the same data content as the duplicate data and is already stored in the storage device A data storage method for performing data storage processing to be used as the duplicate data with reference to other data,
Determine the storage location of the duplicate data determined to be related to the non-duplicate data according to a preset criterion, and determine the storage location of the non-duplication data based on the determination result ,
Furthermore, a prescription policy in which the storage destination of the non-duplicate data is set in advance and a determination result policy in which the storage destination of the non-duplicate data is set according to the determination result are stored,
Determining whether to use the specified policy or the determination result policy based on the determination result, and determining a storage destination of the non-duplicate data based on the determined policy;
Data storage method.

Images