JP2017123070A - Control device, storage device, data management program, and data management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently store information on redundancy removal in a memory.SOLUTION: A control device 1a comprises: a first determination part 13 for determining whether or not first data are redundantly written in both first and second virtual volumes; a second determination part 16 for, when it is determined that the first data are not redundantly written by the first determination part 13, determining whether or not the first data are written in the first or second virtual volume by using individual data information indicating the data written in the first or the second virtual volume; and an individual data registration part 17 for, when it is determined that the first data are not written in any of the first and second virtual volumes by the second determination part 16, registering information on the first data in the individual data information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a control device, a storage device, a data management program, and a data management method.

  When a plurality of pieces of data having the same contents (may be referred to as “blocks”) are included as a function of the storage device, a plurality of pieces of data having the same contents are collectively stored as one. Technology is known.

  In the deduplication technique, for example, a hash value management table is used to store what block has been written. The hash value management table includes a hash value calculated from the value of the written block and an index indicating a physical location where the written block is actually written (may be referred to as a “write physical address”). And may include.

  The hash value management table is referenced every time a block is written. Therefore, information regarding the hash value management table may be stored in a memory that can operate at high speed.

  However, when a new block is written, the number of entries in the hash value management table increases, which may cause a memory capacity shortage.

  Therefore, it is conceivable to detect as many overlapping blocks as possible while limiting the size of the information related to the hash value management table to a size that can be stored in the memory. In other words, it is possible not to store all hash values, but to store, for example, the relationship between hash values and physical addresses of non-overlapping blocks in the memory.

  As one technique for reducing the size of the hash value management table, a technique for managing the hash value management table using LRU (Least Recently Used) is known. In the technique using LRU, when the hash value management table is referred to when writing a block, the priority of the entry that has been hit may be set to the highest. Then, when a block having a new hash value is written, the content of the entry with the lowest priority may be rewritten with the content of the new hash value.

JP 2010-79391 A JP 2010-182302 A

  However, in the technique of managing using the hash value management table using LRU, when a large number of blocks having new hash values are temporarily written, entries of blocks that are used for a long period of time are stored. There is a risk of being erased. Further, there is a possibility that the entry of the hash value management table is occupied by a block that is specifically used by a specific application. For example, in an application for editing a moving image, there is a possibility that the hash value management table is occupied by a block of the moving image because a large number of edited moving image blocks are written.

  In one aspect, the present invention aims to efficiently store information related to deduplication in a memory.

  Therefore, the control device is a control device provided in a storage device that expands the first virtual volume and the second virtual volume, and the first data spans the first and second virtual volumes. A first determination unit that determines whether the first data has been written in duplicate, and the first determination unit when the first determination unit determines that the first data has not been written in duplicate A second determination unit configured to determine whether the first data is written to the first or second virtual volume using individual data information indicating data written to the virtual volume; and the second determination. Information about the first data in the individual data information when it is determined that the first data has not been written in any of the first and second virtual volumes. Comprising the individual data registration unit for registering, the.

  According to the disclosed control device, it is possible to efficiently store information related to deduplication in the memory.

1 is a diagram schematically illustrating a configuration of a storage system as an example of an embodiment. It is a figure which illustrates typically the original file before applying a duplication removal technique. It is a figure which illustrates typically the file after applying a duplication removal technique. It is a figure which shows the 1st example of a hash value management table. It is a figure which shows the 1st example of a virtual address space table. It is a figure which shows the 1st example of a management table. It is a figure which shows the 2nd example of a hash value management table. It is a figure which shows the 2nd example of a virtual address space table. It is a figure which shows the 2nd example of a management table. It is a figure which shows typically the function of the storage system as an example of embodiment. It is a figure which shows typically the function of the duplication removal part of the 1st step as an example of embodiment. It is a figure which shows typically the function of the duplication removal part of the 2nd step as an example of embodiment. It is a flowchart explaining the duplication removal operation | movement of the 1st step as an example of embodiment. It is a flowchart explaining the duplication removal operation | movement of the 2nd step as an example of embodiment. It is a flowchart explaining the 1st example of the production | generation operation | movement of the 1st hash value management table as an example of embodiment. It is a flowchart explaining the 2nd example of the production | generation operation | movement of the 1st hash value management table as an example of embodiment. It is a figure explaining the usage example of the hash value management table as a related technique. It is a figure explaining the usage example of the hash value management table as an example of embodiment.

  Hereinafter, an embodiment will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude application of various modifications and techniques not explicitly described in the embodiment. That is, the present embodiment can be implemented with various modifications without departing from the spirit of the present embodiment.

  Each figure is not intended to include only the components shown in the figure, and may include other functions.

  Hereinafter, in the drawings, the same reference numerals indicate the same parts, and the description thereof is omitted.

[A] Example of Embodiment [A-1] System Configuration FIG. 1 is a diagram schematically illustrating a configuration of a storage system 100 as an example of an embodiment.

  As illustrated in FIG. 1, the storage system 100 includes a storage device 1 and a plurality (three in the illustrated example) of server devices 4.

  The server apparatus 4 is a computer having a server function (may be referred to as “information processing apparatus”).

  The storage device 1 is a device that includes a storage device 30 to be described later and provides a storage area to the server device 4. For example, data is distributed or redundantly distributed to the storage device 30 using RAID (Redundant Arrays of Inexpensive Disks). Save it in a state.

  The storage device 1 may store the file in the storage device 30 in a state in which duplication of data having the same content included in the file is removed by using a deduplication technique. Further, as will be described later with reference to FIG. 10 and the like, the storage apparatus 1 may expand a plurality of virtual volumes 30a and store files in the expanded virtual volumes 30a.

  In this specification, “file” refers to a set of a plurality of “data”. “Data” may also be referred to as “block”.

  As illustrated in FIG. 1, the storage device 1 includes a CPU (Central Processing Unit) 10, a memory 20, and a storage device 30.

  For example, the CPU 10 and the memory 20 are provided in a CM (Controller Module) 1a. “CM” may be referred to as a “control device”.

  The CM 1a is a control device that performs various controls, and performs various controls in accordance with storage access requests from the server device 4 (may also be referred to as “access control signals” or “host I / O”).

  In the example shown in FIG. 1, the storage apparatus 1 includes one CM 1a. However, the present invention is not limited to this. The number of CMs 1a provided in the storage device 1 can be variously changed.

  The storage device 30 is, for example, a device that stores data in a readable / writable manner, and for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) may be used. The storage device 30 may store information related to a first hash value management table, a second hash value management table, a virtual address space table, and a management table, which will be described later with reference to FIGS. 11 and 12.

  In the example shown in FIG. 1, the storage apparatus 1 shows one storage device 30 for convenience, but the present invention is not limited to this. The number of storage devices 30 provided in the storage device 1 can be variously changed.

  The storage device 1 as an example of the embodiment illustratively uses a large-capacity storage device 30 (may be referred to as a “volume”) by dividing it for each use. The unit into which the volume is divided is called LU (Logical Unit).

  The memory 20 is, for example, a storage device including a ROM (Read Only Memory) and a RAM (Random Access Memory). A program such as BIOS (Basic Input / Output System) may be written in the ROM of the memory 20. The software program in the memory 20 may be appropriately read and executed by the CPU 10. The RAM of the memory 20 may be used as a primary recording memory or a working memory.

  The CPU 10 is, for example, a processing device that performs various controls and operations, and implements various functions by executing an OS (Operating System) and a program stored in each of the memories 20. That is, the CPU 10 includes the common data registration unit 11, the first data input unit 12, the first determination unit 13, the communication unit 14, the second data input unit 15, the second determination unit 16, the individual data registration unit 17, and the writing unit 18. May function as.

  The common data registration unit 11, the first data input unit 12, the first determination unit 13, the communication unit 14, the second data input unit 15, the second determination unit 16, the individual data registration unit 17, and the writing unit 18 are used. Examples of the program for realizing the functions are flexible disk, CD (CD-ROM, CD-R, CD-RW, etc.), DVD (DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, HD DVD, etc.), Blu-ray disc, magnetic disc, optical disc, magneto-optical disc, etc. may be provided in a form recorded on a computer-readable recording medium. The computer (CPU 10 in this embodiment) may read the program from the above-described recording medium via a reading device (not shown), transfer the program to an internal recording device or an external recording device, and use it. Further, the program may be recorded in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided to the computer via a communication path from the storage device.

  Functions as the common data registration unit 11, the first data input unit 12, the first determination unit 13, the communication unit 14, the second data input unit 15, the second determination unit 16, the individual data registration unit 17, and the writing unit 18 are realized. In doing so, a program stored in the internal storage device (memory 20 in this embodiment) may be executed by a computer (CPU 10 in this embodiment). Further, the computer may read and execute the program recorded on the recording medium.

  Details of functions of the common data registration unit 11, the first data input unit 12, the first determination unit 13, the communication unit 14, the second data input unit 15, the second determination unit 16, the individual data registration unit 17, and the writing unit 18 Prior to the description, the deduplication technique will be described with reference to FIGS.

  The deduplication technique described with reference to FIGS. 2 to 9 may be realized by the storage system 100 as an example of the embodiment.

  FIG. 2 is a diagram schematically illustrating an original file before applying the deduplication technique, and FIG. 3 is a diagram schematically illustrating a file after the deduplication technique is applied.

  The deduplication technique is a technique for combining blocks having the same contents into one when a plurality of files having the same contents are included in a plurality of files.

  As shown in FIG. 2, consider an example in which files A and B as original files are written in a storage device. File A contains blue, red, purple, yellow and green image data blocks, and file B contains green, brown, red, purple and yellow image data blocks. The data size of each block is 4 KB (kilobytes), and the total data size of file A and file B is 40 KB.

  When deduplication is applied to file A and file B, an index is assigned to each block.

  In the example shown in FIG. 3, indexes 01, 02, 03, 04, and 05 are assigned to the blocks of blue, red, purple, yellow, and green image data in file A, respectively. In addition, indexes 05, 06, 02, 03, and 04 are assigned to the green, brown, red, purple, and yellow image data blocks of file B, respectively. In other words, in the file A and the file B, the same index is assigned to the blocks of the same color image data.

  The storage device 1 stores information on a management table as shown in FIG. 3 in order to manage the files from which duplicates have been removed. In the management table, an index and a pointer are associated with each other.

  The storage apparatus 1 can refer to the block included in the file via the pointer by referring to the index in the management table. In the example shown in FIG. 3, pointers indicating blocks of image data of blue, red, purple, yellow, green, and brown are associated with indexes 01, 02, 03, 04, 05, and 06, respectively. Yes.

  4 is a diagram showing a first example of a hash value management table, FIG. 5 is a diagram showing a first example of a virtual address space table, and FIG. 6 is a diagram showing a first example of a management table. is there.

  The hash value management table, virtual address space table, and management table when the file A shown in FIG. 1 is written from the virtual address 0x0000 of the storage device 30 are as shown in FIGS.

  In the hash value management table, a hash value and a write physical address (may be referred to as “page”) are associated. The hash value is information that is created based on the contents of the block and uniquely identifies the block. The physical address indicates the position where the block is actually written in the storage device 30.

  In the example shown in FIG. 4, in the hash value management table, hash values blue, red, purple, yellow, and green are associated with blocks of blue, red, purple, yellow, and green image data of file A, respectively. ing. In the example shown in FIG. 4, the hash value management table includes write physical addresses 01, 02, 03, 04, and 05 for blocks of blue, red, purple, yellow, and green image data of file A. Each is associated.

  In the virtual address space table, a virtual address and a physical address are associated with each other. The virtual address indicates a position where a block is written in a virtual volume 30a (described later with reference to FIG. 10) developed by the storage device 1.

  In the example shown in FIG. 5, the write physical addresses 01, 02, 03, 04, and 05 are associated with the virtual addresses 0x0000, 0x1000, 0x2000, 0x3000, and 0x4000, respectively.

  FIG. 7 is a diagram showing a second example of the hash value management table, FIG. 8 is a diagram showing a second example of the virtual address space table, and FIG. 9 is a diagram showing a second example of the management table. is there.

  The hash value management table and the virtual address space table management table after the file B shown in FIG. 2 is written from the virtual address 0x6000 of the storage device 30 from the state shown in FIGS. As shown in FIG.

  The hash value management table is referenced every time a block is written. Therefore, information regarding the hash value management table may be stored in a memory that can operate at high speed.

  However, when a new block is written, the number of entries in the hash value management table increases, which may cause a memory capacity shortage.

  Therefore, it is conceivable to detect as many overlapping blocks as possible while limiting the size of the information related to the hash value management table to a size that can be stored in the memory. In other words, instead of storing all hash values, for example, it is conceivable that the relationship between the hash value of a non-overlapping block and the write physical address is not stored in the memory.

  As one technique for reducing the size of a hash value management table, a technique for managing a hash value management table using an LRU is known. In the technique using LRU, when the hash value management table is referred to when writing a block, the priority of the entry that has been hit may be set to the highest. Then, when a block having a new hash value is written, the content of the entry with the lowest priority may be rewritten with the content of the new hash value.

  However, in the technique of managing using the hash value management table using LRU, when a large number of blocks having new hash values are temporarily written, entries of blocks that are used for a long period of time are stored. There is a risk of being erased. Further, there is a possibility that the entry of the hash value management table is occupied by a block that is specifically used by a specific application.

  FIG. 10 is a diagram schematically illustrating functions of the storage system 100 as an example of the embodiment.

  As illustrated in FIG. 10, the storage apparatus 1 may expand a plurality (n + 1 in the illustrated example) of virtual volumes 30a (LUN0 to LUNn). “LUN” is an abbreviation for Logical Unit Number. The plurality of virtual volumes 30 a may be expanded by using the storage area of the storage device 30.

  Data written to the volume may have a characteristic for each LU. For example, it is assumed that LUN0 is assigned for application A that handles character documents and LUN1 is assigned for application B that handles images. In this case, a large amount of character document data is recorded in LUN0, and a large amount of image data is recorded in LUN1, and duplication is unlikely to occur between character document data and image data.

  However, some data is recorded regardless of the application. For example, data written by a file system or OS may be used in common among a plurality of applications and stored in the same manner in a plurality of virtual volumes 30a corresponding to each application.

  That is, the data stored in the storage device 1 includes data that overlaps between LUNs and data that is unique to the LUNs. The storage system 100 as an example of the embodiment reduces the size of the hash value management table by separating functions for duplication management between data overlapping between LUNs and data unique to the LUNs. Achieve a high deduplication rate.

  Therefore, the storage apparatus 1 as an example of the embodiment functions as a first-stage deduplication unit 10a and a plurality of (n + 1 in the illustrated example) second-stage deduplication unit 10b as illustrated in FIG. To do. A plurality of second-stage deduplication units 10b may be provided corresponding to each of the plurality of virtual volumes 30a.

  The first-stage deduplication unit 10a illustratively performs deduplication across LUNs. The first-stage deduplication unit 10a may function as the common data registration unit 11, the first data input unit 12, the first determination unit 13, and the communication unit 14 illustrated in FIG.

  For example, the second-stage deduplication unit 10b performs deduplication for each LUN. The second-stage duplicate elimination unit 10b may function as the second data input unit 15, the second determination unit 16, the individual data registration unit 17, and the writing unit 18 illustrated in FIG.

  FIG. 11 is a diagram schematically illustrating the function of the first-stage deduplication unit 10a as an example of the embodiment.

  In the example illustrated in FIG. 11, some of the functions included in the first-stage duplicate removal unit 10 a are illustrated.

  The common data registration unit 11 (not shown in FIG. 11) illustratively creates a first hash value management table. The common data registration unit 11 may create a first hash value management table in advance before the storage apparatus 1 is operated. Further, the common data registration unit 11 may store information on the created first hash value management table in a predetermined area of the storage device 30, for example.

  The first hash value management table exemplarily shows information related to data that is written redundantly across a plurality of virtual volumes 30a. In other words, the first hash value management table may indicate hash values of data used by the OS that controls the overall operation of the storage apparatus 1. In other words, the first hash value management table may indicate the hash value of data that is used redundantly regardless of the application executed by the storage device 1.

  A hash value and a write physical address may be associated with the first hash value management table. In the example illustrated in FIG. 11, write physical addresses luna0-01 and luna2-05 are associated with the hash values blue and red.

  The common data registration unit 11 may operate the storage system 100 for a predetermined time in advance to collect information about data written between LUNs. Then, based on the collected information, the common data registration unit 11 may create a first hash value management table in which a predetermined number (for example, 100,000) of information related to data with a higher write count is registered.

  The common data registration unit 11 may collect duplicately used data in the first hash value management table in a state where the storage system 100 is operated without starting an application. For example, the common data registration unit 11 may register data written by a process executed as an operating environment default of the OS, file system, and storage system 100 in the first hash value management table.

  In other words, the common data registration unit 11 may register information on data to be written redundantly in the first hash value management table in a state where an application to be executed by the storage device 1 is not activated. Thereby, the common data registration unit 11 may generate the first hash value management table.

  Further, the common data registration unit 11 may collect data used redundantly in the first hash value management table in a state where the application is started and the storage system 100 is operated. For example, the common data registration unit 11 may register data used in common among a plurality of applications in the first hash value management table in addition to data written by the OS or the like. The common data registration unit 11 specifies the data used in common among the plurality of applications by inquiring of the data written in each virtual volume 30a to the plurality of second-stage deduplication units 10b. Good. Here, “data written by the OS or the like” is, for example, data written by a process executed by the operating environment default of the OS, file system, or storage system 100.

  In other words, the common data registration unit 11 uses the first hash value to store information about data that is written redundantly across the plurality of virtual volumes 30a in a state where the application executed by the storage device 1 is operating. It may be registered in the management table. Thereby, the common data registration unit 11 may generate the first hash value management table.

  The first data input unit 12 exemplarily receives input data (in other words, “write data”) transmitted from the server device 4. In other words, the first data input unit 12 may accept input of a write address (in other words, “LUN and address”) and data transmitted from the server device 4.

  The first data input unit 12 divides the input data into blocks of a predetermined size (hereinafter sometimes referred to as “input blocks”), and calculates each input block and the write address of the input block. It's okay. Then, the first data input unit 12 may input a set of an input block and a write address to the first determination unit 13.

  For example, the first determination unit 13 determines whether or not the input block is written over the plurality of virtual volumes 30a. In other words, the first determination unit 13 may determine whether the input block is a block used by the OS or a block used by an individual application. In other words, the first determination unit 13 may calculate a hash value of the input block, search the first hash value management table, and check whether or not an entry with the same hash value is registered.

  The first determination unit 13 may determine whether or not the input block is written over the plurality of virtual volumes 30a using the first hash value management table. The first determination unit 13 may determine that the block registered in the first hash value management table has been written over a plurality of virtual volumes 30a. In addition, the first determination unit 13 may determine that a block that is not registered in the first hash value management table has not been written over a plurality of virtual volumes 30a.

  For example, the communication unit 14 inputs information related to the input block to the second-stage deduplication unit 10b based on the determination result by the first determination unit 13.

  Specifically, the communication unit 14 may read the write physical address of the entry for the input block when information about the input block is registered in the first hash value management table. Then, the communication unit 14 sets the combination of the virtual address of the input block and the write physical address to the second-stage deduplication unit 10b associated with the LUN of the virtual address determined from the information such as the header of the input block. May be notified.

  In addition, when the information related to the input block is not registered in the first hash value management table, the communication unit 14 checks the virtual address from the information such as the header of the input block and specifies the LUN in which the input block is to be written. It's okay. And the communication part 14 may notify the virtual address and content of an input block with respect to the duplicate removal part 10b of the 2nd step | paragraph matched with the specified LUN.

  FIG. 12 is a diagram schematically illustrating the function of the second-stage deduplication unit 10b as an example of the embodiment.

  In the example illustrated in FIG. 12, some of the functions included in the second-stage duplicate removal unit 10 b are illustrated.

  The information related to the second hash value management table shown in FIG. 12 is illustratively stored, for example, in a predetermined area of the storage device 30 for each of the plurality of second-stage duplicate removal units 10b. The second hash value management table is a table for managing duplication of data by closing to the LUN corresponding to the second-stage duplicate removal unit 10b, and a predetermined number (for example, 100,000) entries managed by the LRU. May be included.

  A hash value and a write physical address may be associated with the second hash value management table. In the example shown in FIG. 12, a write physical address 06 is associated with the hash value brown.

  The second data input unit 15 exemplarily receives information related to the input block input from the first-stage deduplication unit 10a.

  Specifically, when the input block is not written redundantly across the plurality of virtual volumes 30a, the second data input unit 15 sends the virtual address of the input block from the first-stage deduplication unit 10a. And a set of contents may be received. Further, when the input block is written redundantly across the plurality of virtual volumes 30a, the second data input unit 15 receives the virtual address and write physical of the input block from the first-stage duplicate removal unit 10a. You may receive a pair with an address.

  When the second data input unit 15 receives the set of the virtual address and the content of the input block, the second data input unit 15 inputs the set of the received virtual address and the content of the input block to the second determination unit 16. Good. In addition, when the second data input unit 15 receives the set of the virtual address and the write physical address of the input block, the second data input unit 15 sends the set of the virtual address and the write physical address of the received input block to the write unit 18. May be entered.

  For example, the second determination unit 16 determines duplication of input blocks in the corresponding virtual volume 30a using the second hash value management table.

  Specifically, the second determination unit 16 may read the write physical address from the second hash value management table when information about the input block is already registered in the second hash value management table. Then, the second determination unit 16 may input a set of the virtual address of the input block and the read write physical address to the writing unit 18.

  When the information regarding the input block is not already registered in the second hash value management table, the second determination unit 16 inputs the set of the virtual address and content of the input block to the individual data registration unit 17. It's okay.

  The individual data registration unit 17 (not shown in FIG. 12) exemplarily registers information about the input block in the second hash value management table.

  Specifically, when the information about the input block is not registered in the second hash value management table, the individual data registration unit 17 has the highest LRU priority among the plurality of entries in the second hash value management table. An entry with a low may be selected. The individual data registration unit 17 may write the hash value of the input block to the selected entry (in other words, overwrite), and input the set of the virtual address and content of the input block to the writing unit 18. .

  For example, the writing unit 18 writes the input block to the corresponding virtual volume 30a.

  Specifically, when the writing unit 18 receives a set of the virtual address and the write physical address of the input block, the writing unit 18 associates the write physical address of the input block with the corresponding virtual address in the virtual address space table. You can register. Thereby, the value of the virtual address of the input block becomes the content of the block recorded at the write physical address.

  When the writing unit 18 receives a set of the virtual address and content of the input block, the writing unit 18 may write the content of the received input block into the storage device 30. The writing unit 18 may add one new entry to the management table and register the write physical address for the input block in the added entry. Further, the writing unit 18 may register the writing physical address in association with the corresponding virtual address in the virtual address space table.

  The individual data registration unit 17 (not shown in FIG. 12) registers the write physical address registered in the management table by the write unit 18 in the write physical address column of the entry for the input block of the second hash value management table. You can do it.

  When the set of the virtual address and content of the input block is received, the value of the input block becomes the content of the block recorded at the write physical address by the functions of the writing unit 18 and the individual data registration unit 17.

[A-2] Operation The first-stage deduplication operation (steps S1 to S5) as an example of the embodiment configured as described above will be described with reference to the flowchart shown in FIG.

  The first data input unit 12 receives the write data transmitted from the server device 4 (step S1).

  The first determination unit 13 calculates a hash value of the write data (Step S2).

  The first determination unit 13 determines whether the hash value of the write data is registered in the first hash value management table (step S3).

  When the hash value of the write data is not registered in the first hash value management table (see No route in step S3), the communication unit 14 specifies the virtual volume 30a to be written from the virtual address of the write data. Then, the communication unit 14 notifies the second-stage duplicate removal unit 10b corresponding to the identified virtual volume 30a of the virtual address and content of the write data (step S4), and the process ends.

  On the other hand, when the hash value of the write data is registered in the first hash value management table (see the Yes route in step S3), the communication unit 14 performs the corresponding second-stage deduplication unit 10b. A set of the virtual address and write physical address of the write data is notified (step S5). Then, the process ends.

  Next, the second-stage deduplication operation (steps S11 to S16) as an example of the embodiment will be described with reference to the flowchart shown in FIG.

  The second data input unit 15 receives the write data input from the first-stage deduplication unit 10a (step S11).

  The second determination unit 16 calculates a hash value of the write data (Step S12).

  The second determination unit 16 determines whether the hash value of the write data is registered in the second hash value management table (step S13).

  If the hash value of the write data is registered in the second hash value management table (see the Yes route in step S13), the process ends.

  On the other hand, when the hash value of the write data is not registered in the second hash value management table (see No route in step S13), the individual data registration unit 17 updates the second hash value management table. Specifically, the individual data registration unit 17 rewrites the content of the entry that is least used in the second hash value management table with the hash value of the write data (step S14).

  The writing unit 18 writes write data into the storage device 30. The writing unit 18 registers the virtual address of the write data in the management table, and creates an index of the write data (step S15).

  The individual data registration unit 17 rewrites the contents of the write physical address in the corresponding entry of the second hash value management table with the created index (step S16). Then, the process ends.

  Next, a first example of the first hash value management table creation operation (steps S21 to S29) as an example of the embodiment will be described with reference to the flowchart shown in FIG. In FIG. 15, the first hash value management table creation operation in a state where the storage system 100 is operated without activating an application will be described.

  The common data registration unit 11 creates a working hash value management table having the same number of entries as the first hash value management table (step S21). In the state of step S21, each entry in the first hash value management table and the created work hash value management table is empty.

  The common data registration unit 11 operates the storage system 100 without starting the application (step S22).

  The first data input unit 12 receives the write data transmitted from the server device 4 (step S23).

  The first determination unit 13 calculates a hash value of the write data (step S24).

  The first determination unit 13 determines whether the hash value of the write data is registered in the first hash value management table (step S25).

  If the hash value of the write data is not registered in the first hash value management table (see No route in step S25), the common data registration unit 11 updates the first hash value management table. Specifically, the common data registration unit 11 rewrites the least used entry in the first hash value management table with the hash value of the write data (step S26). Then, the process returns to step S23.

  On the other hand, when the hash value of the write data is registered in the first hash value management table (see the Yes route in step S25), the common data registration unit 11 refers to the work hash value management table. If the hash value of the write data is not yet registered in the work hash value management table, the common data registration unit 11 registers the hash value of the write data in the work hash value management table (step S27).

  The common data registration unit 11 determines whether or not hash values are registered in all entries of the work hash value management table (step S28).

  If hash values are not registered in all entries (see No route in step S28), the process returns to step S23.

  On the other hand, when hash values are registered in all entries (see the Yes route in step S28), the common data registration unit 11 copies the contents of the working hash value management table to the first hash value management table ( In other words, overwrite). The common data registration unit 11 discards the work hash value management table (step S29). Then, the process ends.

  Next, a second example of the first hash value management table creation operation (steps S31 to S39) as an example of the embodiment will be described with reference to the flowchart shown in FIG. In FIG. 16, the creation operation of the first hash value management table in a state where the application is started and the storage system 100 is operated will be described.

  The common data registration unit 11 creates a working hash value management table having the same number of entries as the first hash value management table (step S31). In the state of step S31, each entry in the first hash value management table and the created work hash value management table is empty.

  The common data registration unit 11 activates the application and operates the storage system 100 (step S32).

  The first data input unit 12 receives the write data transmitted from the server device 4 (step S33).

  The first determination unit 13 calculates a hash value of the write data (step S34).

  The communication unit 14 specifies the virtual volume 30a in which the write data is to be written, from the virtual address of the write data. Then, the communication unit 14 notifies the second-stage deduplication unit 10b corresponding to the identified virtual volume 30a of the virtual address of the write data and the contents of the block (step S35).

  Whether the first determination unit 13 has a common entry in the plurality of second hash value management tables for the hash value of the write data based on the determination result by the second determination unit 16 of the plurality of second-stage deduplication units 10b. Is determined (step S36).

  If there is no common entry for the hash value of the write data in the plurality of second hash value management tables (see No route in step S36), the process returns to step S33.

  On the other hand, if there is a common entry in the plurality of second hash value management tables for the hash value of the write data (see the Yes route in step S36), the common data registration unit 11 refers to the work hash value management table. To do. If the hash value of the write data is not yet registered in the work hash value management table, the common data registration unit 11 registers the hash value of the write data in the work hash value management table (step S37).

  The common data registration unit 11 determines whether or not hash values are registered in all entries of the work hash value management table (step S38).

  If hash values are not registered in all entries (see No route in step S38), the process returns to step S33.

  On the other hand, when hash values are registered in all entries (see the Yes route in step S38), the common data registration unit 11 copies the contents of the work hash value management table to the first hash value management table ( In other words, overwrite). Further, the common data registration unit 11 discards the work hash value management table (step S39). Then, the process ends.

  Hereinafter, a usage example of the hash value management table as the related technique will be compared with a usage example of the hash value management table as an example of the embodiment with reference to FIGS. 17 and 18.

  FIG. 17 is a diagram for explaining a usage example of a hash value management table as a related technique, and FIG. 18 is a diagram for explaining a usage example of a hash value management table as an example of an embodiment.

  Consider a case in which two applications A and B are executed in the storage apparatus 1 and applications A and B use LUN0 and LUN1, respectively.

  Assume that data written by the application A is C-0, A-0, A-1, C-1, A-0, C-1,. Further, data written by the application B is in the order of time C-1, B-0, B-1, B-2, B-3, C-0, B-3, B-4, C-1,. Suppose that Note that data starting with A is data used only by application A, data starting with B is data used only by application B, and data starting with C is used by both application A and application B. Data.

  It is assumed that data written by the application A and the application B are mixed and written to the storage device 30 in the following order as shown in FIGS.

  (1) C-0, (2) C-1, (3) A-0, (4) A-1, (5) B-0, (6) B-1, (7) B-2, ( 8) B-3, (9) C-1, (10) A-0, (11) C-0, (12) B-3, (13) C-1, (14) B-4, (15 ) C-1, ...

  In the example shown in FIG. 17, one hash value management table used in the related art is shown. The hash value management table shown in FIG. 17 has six entries, and all entries are empty in the initial state (reference numeral A1).

  In the actual storage apparatus 1, the hash value management table may have hundreds of thousands or more entries, but in this example, the number of entries in the hash value management table is reduced for the sake of explanation.

  The hash value management table in the related technology shown in FIG. 17 may be used by the storage apparatus 1 as an example of the embodiment.

  Hereinafter, with reference to FIG. 17, a change in the contents of the hash value management table when the data (1) to (10) is sequentially written in the storage device 30 is simulated. If new data written when new data is written to the storage device 30 is not registered in the hash value management table, the content of the least used entry is discarded and the content is discarded. New data may be registered in the new entry.

  First, the data of (1) to (6) are written in the six entries of the hash value management table in order (reference A2). In other words, (1) C-0, (2) C-1, (3) A-0, (4) A-1, (5) B-0 and (6) B-1 are in order, It is written in six entries of the hash value management table.

  When the data (7) and (8) are written, the data C-0 and C-1 that are least used in the hash value management table are overwritten with the data B-2 and B-3, respectively ( Reference A3).

  When the data (9) is written, the data A-0 that is least used in the hash value management table is discarded, and the data C-1 is registered (reference A4).

  When the data of (10) is written, the data A-1 that is least used in the hash value management table is discarded and the data A-0 is registered (reference A5).

  As described above, in the usage example of the hash value management table in the related technology, the data C-1 and A-1 are written twice in the data writing of the above (1) to (10). , Data duplication is not found.

  In the example illustrated in FIG. 18, one first hash value management table and two second hash value management tables used in the example of the embodiment are illustrated. Of the two second hash value management tables, one corresponds to LUN0 and the other corresponds to LUN1. Hereinafter, the second hash value management table corresponding to LUN0 may be referred to as “LUN0 second hash value management table”, and the second hash value management table corresponding to LUN1 may be referred to as “LUN1 second hash value management table”. is there. Each of the first hash value management table and the second hash value management table shown in FIG. 18 has two entries.

  In the actual storage device 1, the first hash value management table and the second hash value management table may have, for example, hundreds of thousands or more entries, but in this example, the hash value management table is used for explanation. The number of entries is reduced.

  Hereinafter, with reference to FIG. 18, a change in the contents of the hash value management table when the data (1) to (10) is sequentially written in the storage device 30 is simulated. If new data written when new data is written to the storage device 30 is not registered in the first or second hash value management table, the content of the least used entry is discarded. New data may be registered in the entry whose contents are discarded.

  First, in the initial state, data C-0 and C-1 are registered in the first hash value management table, and the two second hash value management tables are empty (reference numeral B1).

  Even if the data of (1) and (2) are written, since the data C-0 and C-1 are registered in the first hash value management table, the contents of any of the second hash value management tables are represented by the code B1. No change from the state.

  When the data of (3) and (4) is written, the first hash value management table is referred to, but the data A-0 and A-1 are not registered. Therefore, the second hash value management table for LUN0 is referred to, but since data A-0 and A-1 are not registered, data A-0 and A-1 are registered in the second hash value management table for LUN0. (Reference B2).

  When the data of (5) and (6) are written, the first hash value management table is referred to, but the data B-0 and B-1 are not registered. Therefore, the second hash value management table for LUN1 is referred to, but since data B-0 and B-1 are not registered, data B-0 and B-1 are registered in the second hash value management table for LUN1. (Reference B3).

  When the data of (7) and (8) is written, the first hash value management table is referenced, but the data B-2 and B-3 are not registered. Therefore, the LUN1 second hash value management table is referred to, but since the data B-2 and B-3 are not registered, the data B-0 and B-1 are discarded, and the LUN1 second hash value is stored. Data B-2 and B-3 are registered in the management table (reference B4).

  Even if the data of (9) is written, the data C-1 is registered in the first hash value management table (in other words, duplication has been found in the first hash value management table). The content of the second hash value management table does not change from the state of the code B4.

  When the data (10) is written, the first hash value management table is referred to, but the data A-0 is not registered. Therefore, since the second hash value management table for LUN0 is referred to and data A-0 is registered (in other words, duplication of the second hash value management table can be found), any second hash value is stored. The contents of the management table also do not change from the state of B4.

  As described above, in the usage example of the hash value management table in the example of the embodiment, the data of (1) to (10) is written, and C-1 and A-1 are written twice, respectively. I was able to discover.

  For example, in an application for writing a moving image or an image, a large amount of non-overlapping data is written as in the above application B, and therefore the situation shown by reference numerals B1 to B4 in FIG. 18 may frequently occur.

  For the discovery of duplicate data as described above between the usage example of the hash value management table as the related technique shown in FIG. 17 and the usage example of the hash value management table as an example of the embodiment shown in FIG. As an example of the cause of the difference, the following may be considered.

  First, because application B has written a large amount of data that does not overlap, the hash value management table as a related technology has rewritten the entries registered in the hash value management table so far. That is.

  Second, in a hash value management table as a related technique, writing of all applications is managed by one hash value management table. Therefore, the entry registered by the writing of the application A is erased due to the writing of the application B.

  For example, the first cause is that the duplication of the data C-1 cannot be found in the hash value management table as the related technology, and the second cause is the data of the data A-1 in the hash value management table as the related technology. This is the reason why duplicates could not be found.

  By using the first and second hash value management tables as an example of the embodiment, it is possible to prevent data entries used between LUNs from being deleted from the hash value management table. Further, it is possible to prevent an entry in the hash value management table of data used in a certain LUN from being erased due to the influence of data used in another LUN.

[A-3] Summary According to the storage device 1 as an example of the embodiment described above, for example, the following effects can be obtained.

  The first determination unit 13 determines whether input data is written redundantly over a plurality of virtual volumes 30a. In addition, when the first determination unit 13 determines that the input data has not been written in duplicate, the second determination unit 16 uses the second hash value management table to correspond to the virtual volume 30a to which the input data corresponds. Is written in The individual data registration unit 17 registers information about the input data in the second hash value management table when the second determination unit 16 determines that the input data has not been written in the corresponding virtual volume 30a. .

  As a result, information related to deduplication can be efficiently stored in the memory.

  The writing unit 18 writes the input data related to the information registered by the individual data registration unit 17 to the corresponding virtual volume 30a.

  The writing unit 18 does not perform the determination by the second determination unit 16 and the registration by the individual data registration unit 17 when the input data is written redundantly across the plurality of virtual volumes 30a. Are written into the corresponding virtual volume 30a.

  As a result, necessary data can be stored while deduplication is being performed, so that the efficiency of the storage apparatus 1 can be improved.

  The first determination unit 13 performs determination using the first hash value management table.

  Thereby, by using the two hash value management tables of the first hash value management table and the second hash value management table, it is possible to efficiently manage the information of overlapping blocks according to the characteristics of the blocks. . A high deduplication rate can be achieved in a predetermined area size.

  Data that is written redundantly across a plurality of virtual volumes 30a is data used by the OS. The data written to any of the plurality of virtual volumes 30a is data used by individual applications.

  Thereby, efficient deduplication can be performed according to the purpose of use of data.

  The common data registration unit 11 generates a first hash value management table by registering information about data to be written redundantly in the first hash value management table when the application is not activated.

  The common data registration unit 11 registers the information about the data that is written over the plurality of virtual volumes 30a in the first hash value management table in a state where the application is activated, thereby Generate a hash value management table.

  As a result, information about data commonly used in the storage system 100 can be appropriately registered in the first hash value management table.

[B] Others The disclosed technology is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present embodiment. Each structure and each process of this embodiment can be selected as needed, or may be combined suitably.

[C] Appendix (Appendix 1)
A control device provided in a storage device for expanding a first virtual volume and a second virtual volume,
A first determination unit that determines whether or not first data is written redundantly across the first and second virtual volumes;
When it is determined by the first determination unit that the first data has not been written in duplicate, using the individual data information indicating the data written in the first or second virtual volume, A second determination unit that determines whether the first data is written to the first or second virtual volume;
When the second determination unit determines that the first data has not been written in any of the first and second virtual volumes, information about the first data is added to the individual data information. An individual data registration unit to be registered;
A control device comprising:

(Appendix 2)
A writing unit for writing the first data related to the information registered by the individual data registration unit to the first or second virtual volume;
The control device according to attachment 1.

(Appendix 3)
The storage apparatus further expands a third virtual volume different from the first and second virtual volumes,
When it is determined by the first determination unit that the first data has been written over the first and second virtual volumes, the determination by the second determination unit and the individual A writing unit that writes the first data to the third virtual volume that is the storage destination of the first data without performing the registration by the data registration unit;
The control device according to attachment 1.

(Appendix 4)
The first determination unit performs the determination using common data information indicating data that is written redundantly across the first and second virtual volumes.
The control device according to any one of appendices 1 to 3.

(Appendix 5)
The data written redundantly across the first and second virtual volumes is data used by an operating system that controls the overall operation of the storage device,
The data written to one of the first and second virtual volumes is data used by an individual application operating in the storage device.
The control device according to appendix 4.

(Appendix 6)
A common data registration unit that generates the common data information by registering, in the common data information, information about data that is written in duplicate in a state where the application is not activated;
The control device according to appendix 5.

(Appendix 7)
The common data information is generated by registering, in the common data information, information about data that has been written over the first and second virtual volumes while the application is running. A common data registration unit
The control device according to appendix 5.

(Appendix 8)
A storage device that expands a first virtual volume and a second virtual volume,
A first determination unit that determines whether or not first data is written redundantly across the first and second virtual volumes;
When it is determined by the first determination unit that the first data has not been written in duplicate, using the individual data information indicating the data written in the first or second virtual volume, A second determination unit that determines whether the first data is written to the first or second virtual volume;
When the second determination unit determines that the first data has not been written in any of the first and second virtual volumes, information about the first data is added to the individual data information. An individual data registration unit to be registered;
A storage apparatus.

(Appendix 9)
A writing unit for writing the first data related to the information registered by the individual data registration unit to the first or second virtual volume;
The storage apparatus according to appendix 8.

(Appendix 10)
The storage apparatus further expands a third virtual volume different from the first and second virtual volumes,
When it is determined by the first determination unit that the first data has been written over the first and second virtual volumes, the determination by the second determination unit and the individual A writing unit that writes the first data to the third virtual volume that is the storage destination of the first data without performing the registration by the data registration unit;
The storage apparatus according to appendix 8.

(Appendix 11)
The first determination unit performs the determination using common data information indicating data that is written redundantly across the first and second virtual volumes.
The storage device according to any one of appendices 8 to 10.

(Appendix 12)
The data that is written over the first and second virtual volumes is data used by an operating system that controls the overall operation of the storage device,
The data written to one of the first and second virtual volumes is data used by an individual application operating in the storage device.
The storage device according to attachment 11.

(Appendix 13)
A common data registration unit that generates the common data information by registering, in the common data information, information about data that is written in duplicate in a state where the application is not activated;
The storage device according to attachment 12.

(Appendix 14)
The common data information is generated by registering, in the common data information, information about data that has been written over the first and second virtual volumes while the application is running. A common data registration unit
The storage device according to attachment 12.

(Appendix 15)
In a computer provided in a storage device that expands the first virtual volume and the second virtual volume,
Determining whether first data is written redundantly across the first and second virtual volumes;
When it is determined that the first data is not written redundantly, the first data is stored using the individual data information indicating the data written to the first or second virtual volume. Determine whether it is written to the first or second virtual volume;
Registering information about the first data in the individual data information when it is determined that the first data has not been written to any of the first and second virtual volumes;
Data management program that executes processing.

(Appendix 16)
Writing the first data relating to the registered information to the first or second virtual volume;
The data management program according to appendix 15, which causes the computer to execute processing.

(Appendix 17)
The storage apparatus further expands a third virtual volume different from the first and second virtual volumes,
If the first data has been written over the first and second virtual volumes, the determination using the individual data information and the registration to the individual data information are performed. Without writing the first data to the third virtual volume that is the storage destination of the first data,
The data management program according to appendix 15, which causes the computer to execute processing.

(Appendix 18)
The determination is performed using common data information indicating data that has been written over the first and second virtual volumes.
The data management program according to any one of appendices 15 to 17, which causes the computer to execute processing.

(Appendix 19)
The data written redundantly across the first and second virtual volumes is data used by an operating system that controls the overall operation of the storage device,
The data written to one of the first and second virtual volumes is data used by an individual application operating in the storage device.
The data management program according to appendix 18.

(Appendix 20)
In the state where the application is not activated, the common data information is generated by registering in the common data information information about data to be written redundantly.
The data management program according to appendix 19, which causes the computer to execute processing.

(Appendix 21)
The common data information is generated by registering, in the common data information, information about data that has been written over the first and second virtual volumes while the application is running. To
The data management program according to appendix 19, which causes the computer to execute processing.

(Appendix 22)
In the storage device that expands the first virtual volume and the second virtual volume,
Determining whether first data is written redundantly across the first and second virtual volumes;
When it is determined that the first data is not written redundantly, the first data is stored using the individual data information indicating the data written to the first or second virtual volume. Determine whether it is written to the first or second virtual volume;
Registering information about the first data in the individual data information when it is determined that the first data has not been written to any of the first and second virtual volumes;
Data management method.

(Appendix 23)
Writing the first data relating to the registered information to the first or second virtual volume;
The data management method according to attachment 22.

(Appendix 24)
The storage apparatus further expands a third virtual volume different from the first and second virtual volumes,
If the first data has been written over the first and second virtual volumes, the determination using the individual data information and the registration to the individual data information are performed. Without writing the first data to the third virtual volume that is the storage destination of the first data,
The data management method according to attachment 22.

(Appendix 25)
The determination is performed using common data information indicating data that has been written over the first and second virtual volumes.
The data management method according to any one of appendices 22 to 24.

(Appendix 26)
The data written redundantly across the first and second virtual volumes is data used by an operating system that controls the overall operation of the storage device,
The data written to one of the first and second virtual volumes is data used by an individual application operating in the storage device.
The data management method according to attachment 25.

(Appendix 27)
In the state where the application is not activated, the common data information is generated by registering in the common data information information about data to be written redundantly.
The data management method according to attachment 26.

(Appendix 28)
The common data information is generated by registering, in the common data information, information about data that has been written over the first and second virtual volumes while the application is running. To
The data management method according to attachment 26.

DESCRIPTION OF SYMBOLS 100 Storage system 1 Storage apparatus 1a Control apparatus 10 CPU
DESCRIPTION OF SYMBOLS 10a 1st stage duplication removal part 10b 2nd stage duplication removal part 11 Common data registration part 12 1st data input part 13 1st determination part 14 Communication part 15 2nd data input part 16 2nd determination part 17 Individual data registration Unit 18 writing unit 20 memory 30 storage device 30a virtual volume 4 server device

Claims (10)

A control device provided in a storage device for expanding a first virtual volume and a second virtual volume,
A first determination unit that determines whether or not first data is written redundantly across the first and second virtual volumes;
When it is determined by the first determination unit that the first data has not been written in duplicate, using the individual data information indicating the data written in the first or second virtual volume, A second determination unit that determines whether the first data is written to the first or second virtual volume;
When the second determination unit determines that the first data has not been written in any of the first and second virtual volumes, information about the first data is added to the individual data information. An individual data registration unit to be registered;
A control device comprising: A writing unit for writing the first data related to the information registered by the individual data registration unit to the first or second virtual volume;
The control device according to claim 1. The storage apparatus further expands a third virtual volume different from the first and second virtual volumes,
When it is determined by the first determination unit that the first data has been written over the first and second virtual volumes, the determination by the second determination unit and the individual A writing unit that writes the first data to the third virtual volume that is the storage destination of the first data without performing the registration by the data registration unit;
The control device according to claim 1. The first determination unit performs the determination using common data information indicating data that is written redundantly across the first and second virtual volumes.
The control device according to claim 1. The data written redundantly across the first and second virtual volumes is data used by an operating system that controls the overall operation of the storage device,
The data written to one of the first and second virtual volumes is data used by an individual application operating in the storage device.
The control device according to claim 4. A common data registration unit that generates the common data information by registering, in the common data information, information about data that is written in duplicate in a state where the application is not activated;
The control device according to claim 5. The common data information is generated by registering, in the common data information, information about data that has been written over the first and second virtual volumes while the application is running. A common data registration unit
The control device according to claim 5. A storage device that expands a first virtual volume and a second virtual volume,
A first determination unit that determines whether or not first data is written redundantly across the first and second virtual volumes;
When it is determined by the first determination unit that the first data has not been written in duplicate, using the individual data information indicating the data written in the first or second virtual volume, A second determination unit that determines whether the first data is written to the first or second virtual volume;
When the second determination unit determines that the first data has not been written in any of the first and second virtual volumes, information about the first data is added to the individual data information. An individual data registration unit to be registered;
A storage apparatus. In a computer provided in a storage device that expands the first virtual volume and the second virtual volume,
Determining whether first data is written redundantly across the first and second virtual volumes;
When it is determined that the first data is not written redundantly, the first data is stored using the individual data information indicating the data written to the first or second virtual volume. Determine whether it is written to the first or second virtual volume;
Registering information about the first data in the individual data information when it is determined that the first data has not been written to any of the first and second virtual volumes;
Data management program that executes processing. In the storage device that expands the first virtual volume and the second virtual volume,
Determining whether first data is written redundantly across the first and second virtual volumes;
When it is determined that the first data is not written redundantly, the first data is stored using the individual data information indicating the data written to the first or second virtual volume. Determine whether it is written to the first or second virtual volume;
Registering information about the first data in the individual data information when it is determined that the first data has not been written to any of the first and second virtual volumes;
Data management method.

Images