JP2018185760A - Storage device for eliminating duplication of data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a storage device capable of holding an entry of a hash table in a data duplication elimination technique in a cache memory of a constant amount, allowing retrieval at a high speed, and preventing the shortage of an amount of the cache memory.SOLUTION: A storage device comprises a volatile memory that divides data designated by a host computer into a plurality of chunk, generates each hush value on the basis of the data of each chunk, and holds a hush table in which an identifier and the hush value of the chunk written into a storage medium are stored in association with each other. When the chunk is written into the storage medium, it is searched whether the hush value matched to the hush value generated on the basis of the data of the chunk has been recorded into the hush table. If the hush value is not recorded, the hush value is written into the hush table. If the hush value has been recorded, the writing of the hush value is suppressed.SELECTED DRAWING: Figure 7

Description

  Embodiments described herein relate generally to a storage apparatus for eliminating data duplication.

In recent years, not only has the capacity of a storage device increased to store big data used in an information processing system, but also a storage device that effectively uses a limited storage capacity has been demanded. For example, when storing data, the data is divided into a certain size, each hash value is calculated, and if there are duplicate values on the data table storing the hash value, the corresponding data is not written Data deduplication technology is attracting attention.

Patent No. 5444506

The data deduplication technique requires a search in a hash table because of its operation principle. The hash table increases as the amount of stored data increases. To improve the data deduplication processing speed, it is important to shorten the hash table search time. For example, if a hash table is arranged on the memory, the search can be performed at high speed. However, since the required memory amount increases as the data amount increases, the cost increases and the capacity becomes insufficient depending on the data amount. Further, if a hash table entity such as a virtual memory is held in a secondary storage device such as an HDD, the cost can be reduced and the upper limit of the data amount can be improved, but the search takes time.

The storage apparatus according to the embodiment includes a chunk dividing unit that divides data designated by the host computer into a plurality of chunks, a hash generation unit that generates each hash value based on the data of each of the plurality of chunks, and a recording A volatile memory that holds a hash table in which identifiers of chunks written on the medium and hash values of the chunks are stored in association with each other, and when the first chunk is written to the recording medium, Hash value that matches the hash value generated by the hash generation unit based on the chunk data,
A control unit that searches whether or not a hash is already recorded in the hash table, writes a first chunk if no matching hash is recorded, and suppresses writing if a matching hash is recorded And comprising.

1 is a block diagram of a storage system according to an embodiment. The block diagram which shows the function structure of a disk array control apparatus. The figure which shows an example of the relationship between the data designated by the write request of a host computer, and a chunk. The figure which shows the data structure example of a hash table. The figure which shows the data structure example of a metatable. The figure which shows the data structure example of a chunk table. The figure which shows the procedure of the data write process performed when a disk array control apparatus receives a data write request. The figure which shows the procedure of the data write process performed when a disk array control apparatus receives a data read request. The figure which shows the procedure of the aging period process for sorting the entry of a hash table.

  Hereinafter, embodiments for carrying out the invention will be described.

FIG. 1 is a block diagram of a storage system according to the embodiment. The storage system of this embodiment includes a host computer 100 (hereinafter referred to as host 100) and a storage apparatus 200. The storage apparatus 200 is connected to the host 1 via the network 110.
00 is connected.

The host 100 is a physical computer such as a server or a client PC. Host 1
In 00, an application program for accessing data in the storage apparatus 200 operates. The host 100 uses the storage device 200 as an external storage device via the network 110 according to the application program. The type of network is SAN (Storage Area Network) or Ethernet (registered trademark).

The storage device 200 includes a disk array control device 210 and a RAID disk array 2
30.

The disk array control device 210 is connected to the RAID disk array 230 via the disk interface bus 220. The interface type of the disk interface bus 220 is SCSI, FC, SAS (Serial Attached SCSI), or SAT.
A (Serial AT Attachment).

The disk array controller 210 includes a host interface (host I / F) 211,
Disk interface (disk I / F) 212, cache memory 213, cache controller 214, FROM 215, local memory 216, and CPU 21
7, a chip set 218, and an internal bus 219.

The disk array control device 210 is connected to the host 100 that uses the storage device 200 as an external storage device via the network 110 via the host I / F 211. The interface type of the host I / F 211 is FC (Fibre Channel) or iSCSI
(Internet SCSI).

The host I / F 211 controls data transfer (data transfer protocol) with the host 100.

The disk I / F 212 is an interface for transmitting / receiving a data write / read request and a response to the drive of the RAID disk array 230 from the host 100. Also,
The cache controller 214 makes a data write / read request to the cache memory 213.

The cache memory 213 is used as a cache for a write / read request from the host 100. In this embodiment, the hash table 213a shown mainly in FIG. 2 is stored. Note that the cache memory 213 may be configured to have redundancy such as mirroring.

The cache controller 214 performs data read / write processing to the cache memory 213 in accordance with instructions from the disk array control program operating on the CPU 217. Z

The FROM 215 is a rewritable nonvolatile memory for storing a disk array control program executed by the CPU 217. In this embodiment, the disk array control program stored in the FROM 215 is copied to the local memory 216 in the first process when the disk control device is activated.

The local memory 216 is used to store a disk array control program copied from the FROM 215. In addition, a part of the local memory 216 is stored in the CPU 21.
7 is used as a work area used for operation.

The CPU 217 controls each unit in the disk array control device 210 according to the program code of the disk array device control program stored in the local memory 216. Also,
The CPU 217 operates the disk array control program stored on the local memory 216 via the chip set 218 to control the entire disk array control apparatus 210.

The chip set 218 is a bridge circuit that couples the CPU 217 and its peripheral circuits to the internal bus 219.

The internal bus 219 is a general-purpose bus, for example, a PCI-express bus. Host I / F21
1, disk I / F 212, cache controller 214 and chipset 218
Interconnected by an internal bus 219. Also, FROM 215, local memory 21
6. The CPU 217 is connected to the internal bus 219 via the chipset 218.

The RAID disk array 230 is configured using a plurality of hard disk drives (HDD). Instead of the RAID disk array 230, for example, a RAID array configured using a plurality of HDDs and a plurality of SSDs (solid state drives) or a RAI configured using only a plurality of SSDs
A D array may be applied. Also, a simple set of storage drives (drive array) having no RAID configuration may be applied. RAID disk array 230
Are managed by a disk array control program. Subsequent RAID disk array 23
Functions using 0 are also controlled by the disk array control program. Further, in the present embodiment, the RAID disk array 230 stores the meta table 230a and the chunk table 230b shown in FIG. Instead of the RAID disk array 230,
A flash memory or the like can also be used.

In this embodiment, the disk array control device 210 is provided in a storage device including the RAID disk array 230. However, the disk array control device 210 may be provided independently of the storage device 200. In this case, the disk array control device 210 may be built in the host 100. Further, the function of the disk array control device 210 may be realized by using a part of the function of the operating system (OS) that the host 100 has.

Further, the disk array control device 210 may be provided in a card that is used by being installed in a card slot of the host 100. Further, a part of the disk array control device 210 may be built in the host 100, and the rest of the disk array control device 210 may be provided in the card.

FIG. 2 is a block diagram showing a functional configuration of the disk array control apparatus 210 shown in FIG. The disk array control apparatus 210 includes a dividing unit 300, a duplication management unit 301, a duplication determination unit 302, a hash generation unit 303, and an access controller 304. Dividing unit 30
The functions of 0, the duplication management unit 301, the duplication determination unit 302, and the hash generation unit 303 will be described later. The access controller 304 controls reading and writing of data with respect to the RAID disk array 230. These functional elements 300 to 304 are software modules realized by the CPU 217 of the disk array control apparatus shown in FIG. 1 executing a control program. A part or all of the functional elements 300 to 304 may be realized by a hardware module.

The local memory 216 includes a control program area 216a, a table area 216b, and a work area 216c. The control program area 216a is used to store at least a part of a control program executed by the CPU 217. As described above, this control program is stored in advance in the FROM 215, and at least a part of the control program is loaded from the FROM 215 to the control program area 216a of the local memory 216 when the disk array control device 210 is activated.

The table area 216b is used to store at least a part of various tables stored in the disk array control device 210. The work area 216c is used to store temporary data used when the CPU 217 executes the control program.

The RAID disk array 230 stores a meta table 230a and a chunk table b (details will be described later). That is, the RAID disk array 230 includes storage areas for storing the meta table 230a and the chunk table 230b, respectively.

The cache memory 213 stores a hash table 213a (details will be described later). That is, the cache memory 213 includes a storage area in which the hash table 213a is stored. The storage capacity of the hash table 213a is assumed to be a fixed length.

In a typical storage system, the storage capacity initially assigned to the logical volume (
For example, the operation starts with a minimum unit storage capacity. A logical volume refers to a storage area that is recognized by a host as a logical storage drive. For example, a storage area (physical area) of the RAID disk array 230 is appropriately assigned to the logical volume every 4K (kilo) bytes. This type of storage system has a function of flexibly increasing the storage capacity of the logical volume according to the situation after the start of operation. The same applies to the storage system in this embodiment.

FIG. 3 shows an example of the relationship between the data specified by the write request from the host 100 and the chunk. The dividing unit 300 first divides the data shown in FIG. 3 into data chunks having a fixed length, for example, 4 kilobytes (4 Kbytes). A chunk of data every 4 Kbytes is called a chunk. The duplication management unit 301 and the duplication determination unit 302 search for and manage duplication for each chunk. In the example of FIG. 3, the size of the data F is 4NK bytes. In this case, the data F is divided into N chunks # 0 to # N-1. Duplicate management unit 301 uses chunk # 0.
Also, a chunk number that is a unique identification number is assigned to # N-1. In the present embodiment, the chunk number is represented by 8 bytes.

Here, an outline of deduplication at the time of data writing applied in the present embodiment will be described.
First, it is assumed that the host 100 requests the disk array control apparatus 210 of the storage apparatus 200 to write data including chunk A. Further, it is assumed that the hash value of chunk A is H (A). The duplication judgment unit 302 uses the hash value H (A
) Are sequentially compared with the hash values registered in the hash table. Based on the comparison result, the duplication determination unit 302 determines whether a chunk having the same contents as the chunk A (hereinafter referred to as chunk A) is stored in the RAID disk array 230.

First, the hash table 213a stored in the cache memory 213 will be described with reference to FIG. The hash table has a set of entries associated with chunk numbers. Each entry in the hash table includes a hash value, a reference bit, and a hit counter related to the chunk to which the chunk number registered in the entry is assigned.

As described above, the hash value is a value calculated using a predetermined hash function based on the data of the chunk to which the associated chunk number is assigned. The reference bit is set to bit “1” when the chunk indicated by the associated chunk number is accessed within the period of the aging cycle process described later, and the associated chunk has been accessed. This is flag information indicating this. The reference bit is set to bit “0” after the aging cycle process described later is completed, and the flag information is cleared. The hit counter is used to determine the priority order of hash entries during the aging cycle process described later. The hash table 213a is used by the duplication determination unit 302 to determine whether a chunk having the same content as the chunk to be written to the RAID disk array 230 has already been written.

If a hash value that matches the hash value H (A) (that is, the hash value H (A)) is registered in the hash table 213a, the duplication determination unit 302 stores the chunk A in the RAID disk array 230. It is determined that That is, the duplication determination unit 302 determines that the chunk A is duplicated. On the other hand, if the hash value H (A) is not registered in the hash table 213a, the duplication determination unit 302 determines that the chunk A does not overlap.

When it is determined that the chunk A does not overlap, the access controller 304 stores the physical address indicated by the physical address assigned to the logical address specified by the write request.
An instruction to write chunk A is issued. At this time, the duplication management unit 301 registers hash information including a pair of the hash value H (A) and the chunk number of the chunk A in the hash table 213a.

On the other hand, when it is determined that the chunk A is duplicated, the access controller 304
Writing of the chunk A to the RAID disk array 230 is suppressed. In this case, the duplication management unit 301 assigns (that is, maps) a physical address in which chunk A has already been written to the RAID disk array 230 to a logical address designated by a write request from the host 100.

FIG. 5 shows an example of the data structure of the meta table 230a. The meta table 230a has each area (4K byte area) obtained by dividing the logical volume into 4K (kilo) bytes.
Used to manage chunks written in The meta table 230a is a set of entries associated with logical addresses indicating the respective 4 Kbyte areas of the logical volume. Each entry of the meta table 230a is used to register a chunk number of a chunk stored in a 4 Kbyte area indicated by a logical address associated with the entry. Therefore, the duplication management unit 301 can identify the chunk and data stored in the area specified by the target logical address by referring to the meta table 230a.

FIG. 6 shows an example of the data structure of the chunk table 230b. Chunk table 230b
Has a set of entries associated with chunk numbers. Chunk table 230b
Each entry is used to register chunk information related to a chunk having a chunk number associated with the entry. Each entry of the chunk table 230b is composed of a chunk number, a physical address, and a duplicate number. The physical address is a physical address in which a chunk to which the associated chunk number is assigned is recorded. The duplication number indicates how many logical addresses the chunks to which the associated chunk numbers are assigned are associated with each other. When the duplication number is 1, it indicates that the chunk assigned with the associated chunk number is written only in the 4 Kbyte area indicated by one logical address in the logical volume. In this case, the corresponding chunk is not duplicated. When the duplication number is 2, it indicates that although one chunk is actually written in the RAID disk array 230 by deduplication, the chunk is associated with two logical addresses. That is, as viewed from the host 100, chunks having the same contents are 2 in the logical volume.
This indicates that data is written in the 4 Kbyte area indicated by one logical address.

(Operation during write processing)
Next, an operation at the time of a write process executed when the disk array control device 210 of the storage apparatus 200 receives a data write request from the host 100 in this embodiment will be described with reference to FIG.

Now, it is assumed that a data write request designating data write is sent from the host 100 to the storage apparatus 200 via the network 110. Assume that the disk array control device 210 of the storage device 200 has received a data write request from the host 100.

First, the dividing unit 300 of the disk array control device 210 divides the data designated by the data write request, for example, every 4 Kbytes. Thereby, the dividing unit 300 divides the designated data into a plurality of chunks having a size of 4 Kbytes (S1). That is, the duplication management unit 301 acquires a plurality of chunks constituting the data from the designated data.
When the size of the designated data is 4 Kbytes or less, the dividing unit 300 acquires the data itself as one chunk. Here, the chunk size need not be a fixed length, and the chunk size may be a variable length.

The duplication management unit 301 sets the number of acquired chunks as a variable N (S2). here,
The acquired N chunks are denoted as chunks C_1 to C_N.

Then, the duplication management part 301 sets the variable n used for designating one of the N chunks acquired at step S1 to an initial value 1 (S3). Subsequently, the hash generation unit 303 calculates a hash value Hn (C_n) of the nth chunk C_n (S4). That is, since the initial value n = 1, the hash value H1 (C_1) of the first chunk C_1 is calculated.
The hash value is calculated using a hash function called “SHA-256”, for example. Further, in the present embodiment, the hash values H1 (C_1) to H1 (C_N) are each represented by 32 bytes.

Next, the duplication management unit 301 executes a hash value search process for searching the hash table 213a for a hash value that matches the hash value Hn (C_n) of the n-th chunk C_n calculated in step S4. (S5). Based on the result of the hash value search process, it is determined whether or not a hash value matching the hash value Hn (C_n) of the selected chunk C_n exists in the hash table 213a (S6).

Based on the result of determination by the overlap determination unit 302, the overlap management unit 301 performs step S
7 or Step S14.

A case where there is no hash value that matches the hash value Hn (C_n) of the selected chunk C_n will be described (No in S6). In this case, the duplication management unit 301 determines that no chunk having the same content as the chunk C_n is stored.

In step S7, the duplication management unit 301 assigns a chunk number CNC_n to the chunk C_n. Next, the access controller 304 writes the chunk C_n to a free storage area in the RAID disk array 230 (S8).

Subsequently, the duplication management unit 301 acquires a logical address indicating the position of the selected chunk C_n in the logical volume. This logical address is obtained by adding 4K to the logical address specified by the data write request from the host 100. Duplicate management unit 30
1 further registers the chunk number CNC_n assigned to the selected chunk C_n in the entry of the meta table 230a associated with the acquired logical address (S9).
).

Further, the duplication management unit 301 registers the physical address PC_n of the storage area in the RAID disk array 230 in which the chunk C_n is written in the entry of the chunk number CNC_n of the chunk table 230b associated with the chunk number CNC_n of the chunk C_n. To do. Furthermore, the duplication management unit 301 uses the chunk number CN in the chunk table 230b.
1 is set to the overlapping number of entries corresponding to C_n (S10).

Subsequently, the duplication management unit 301 registers the chunk number CNC_n and the hash value Hn (C−n) of the chunk C_n in the hash table 213a. At this time, hash table 2
If there are few empty entries in 13a, the entry at the bottom of the hash table 213a (that is, the entry with the smallest value stored in the hit counter) is deleted, and the entry of chunk C_n is registered (details will be described later). The duplication management unit 301 also has a hash table 213.
In a, the reference bit of the entry corresponding to the chunk number CNC_n is set to bit “1” (S11).

Next, it is determined whether the variable n exceeds the number of chunks N. If variable n is the number of chunks N
If it does not exceed (No in S12), the duplication management unit 301 sets the variable n to 1 in order to process writing of the next chunk included in the data specified by the data write request from the host 100. Increment (S13) and return to step S4. On the other hand, if the variable n exceeds the number of chunks N (Yes in S12), the duplication management unit 301
It is determined that writing of all the chunks included in the data designated by the data write request from the host 100 has been completed. In this case, the data writing process ends.

Next, an operation when there is a hash value that matches the hash value Hn (C_n) of the chunk C_n, that is, when the duplication determination unit 302 determines Yes in step S6 will be described. The duplication management unit 301 has already confirmed that the chunk having the same content as the selected chunk C_n
It is determined that the data is stored in the RAID disk array 230. Here, RAID disk array 2
Chunk C is already stored in 30 and has the same content as Chunk C_n.
Indicated as _x. At this time, the chunk number CNC_x of the chunk C_x is used as the chunk number of the chunk C_n.

In this case, in order to exclude a plurality of chunks having the same contents from being stored in the RAID disk array 230, the access controller 304 selects the selected chunk C_n.
The operation of writing to the RAID disk array 230 is suppressed (S14).

Subsequently, the duplication management unit 301 updates the entry information of the chunk table 230b associated with the chunk number CNC_x of the selected chunk C_n (S15). Specifically, the overlapping number of entries is increased by one.

Next, the duplication management unit 301 has the same contents as the chunk C_n in the entry associated with the logical address to which the chunk C_n of the meta table 230a is to be written, and the chunk number of the chunk C_x that has already been written to the RAID disk array 230. CNC_x is registered (S16).

Subsequently, the duplication management unit 301 updates the entry information of the hash table 213a associated with the chunk number CNC_x of the chunk C_n (S17). If bit “0” is stored in the reference bit column, it is updated to bit “1” and a flag is set. Already bit “1”
Is stored and the flag is set, it is left as it is. Thereafter, the process proceeds to step S12.

(Operation during read processing)
Next, an operation at the time of read processing executed when the disk array control device 210 of the storage apparatus 200 receives a data read request from the host 100 in this embodiment will be described with reference to FIG.

The access controller 304 reads the data from the disk array 230 based on the top logical address and the data length (number of sectors) of the data specified in the data read request received from the host 100, and the work area 216c of the local memory 216. (S20). When reading this data, the physical address where each chunk constituting the read data is recorded can be obtained as follows. That is, the logical address of each chunk constituting the read data is specified by adding the number of sectors (8 sectors = 4 KB) constituting the chunk sequentially to the top logical address. By referring to the meta table 230a based on the specified logical address, the chunk number of the chunk recorded at the logical address can be specified. Further, referring to the chunk table 230b based on the specified chunk number, the physical address where each chunk is recorded can be specified.

Next, the access controller 304 reads the data stored in the read buffer of the local memory 216 and transfers it to the host 100 (S21).

After transferring the data, the dividing unit 300 of the disk array control device 210 divides the transferred data into a plurality of chunks (S22). The duplication management unit 301 sets the number of acquired chunks as a variable N (S23).

Subsequently, the duplication management unit 301 sets a variable n used to designate one of the N chunks acquired in step S23 to an initial value 1 (S24). Next, the hash generation unit 3
03 calculates the hash value, Hn (C_n), of the chunk C_n (S25). That is, since the initial value n = 1, the hash value H1 (C_1) of the first chunk C_1 is calculated.

Subsequently, the duplication management unit 301 determines the chunk C_n whose hash value has been calculated in step S25.
A hash value search process is executed to search the hash table 213a for a hash value that matches the hash value Hn (C_n) (S26). Based on the result of the hash value search process, it is determined whether or not a hash value matching the hash value Hn (C_n) of the selected chunk C_n exists in the hash table 213a (S27).

Based on the result of determination by the overlap determination unit 302, the overlap management unit 301 performs step S
Go to step 28 or step S29.

When there is no hash value that matches the hash value Hn (C_n) of the chunk C_n (
No in S27), the duplication management unit 301 determines the chunk number CNC_n corresponding to the chunk C_n.
Is determined not to be held in the hash table 213a. Subsequently, the duplication management unit 301 registers the hash value Hn (C_n) of the chunk C_n in the hash table 213a in association with the chunk number CNC_n (S28). At this time, if there are few empty entries in the hash table 213a, the entry at the bottom of the hash table 213a (that is, the entry with the smallest value stored in the hit counter) is deleted, and the entry of the chunk C_n is registered (for details) Will be described later). Also, set the reference bit of the corresponding entry to “
Set to 1 ”.

If there is a hash value that matches the hash value Hn (C_n) of the chunk C_n and the duplication determination unit 302 determines Yes in step S27, the duplication management unit 301 determines that the hash value is Hn (C_n). Update the entry (S29)
. The duplication management unit 301 sets the reference bit “1” to the entry of the hash table 213a whose hash value is Hn (C_n).

The duplication management unit 301 determines whether the variable n exceeds the number of chunks N set in step S23 (S30). If the variable n does not exceed the number of chunks N (No in S30), the duplication management unit 301 sets the variable n to process the next chunk included in the data designated by the host 100. Increment by 1 (S31) and return to step S25. On the other hand, if the variable n exceeds the number of chunks N (Yes in S30), the duplication management unit 301 determines that the processing of all chunks has been completed.

Thus, in the present embodiment, registration of a new hash table entry is performed not only at the time of write processing but also at the time of completion of read processing. As a result, deduplication is more efficiently realized even when entries corresponding to all chunks written in the RAID disk array 230 cannot be registered because the storage capacity of the hash table is a fixed value. it can.

That is, when a certain chunk is written, it is already written because the hash table has a fixed length even though it has already been written to the same RAID disk array 230 as that chunk. A case where an entry corresponding to a chunk is not registered in the hash table is assumed. In this case, since no entry is registered in the hash table, deduplication cannot be performed. However, duplication can be eliminated by registering entries in the hash table even during read processing. For example, host 10
For the purpose of 0 reading data from the RAID disk array 230, it is assumed that the data is copied with a different data name, or the read data is updated and rewritten. In such a case, by registering the entry in the hash table even during the read process, it is possible to avoid a situation in which deduplication cannot be performed because no entry is registered in the hash table.

(Replace hash entry)
Since the storage capacity of the hash table 213a is a fixed length, a countermeasure when there are no more empty entries will be described. When there are no more empty entries, select the entry with the smallest hit counter value, delete the recorded contents of the chunk number, hash value, reference bit, and bit counter recorded in that entry, and newly Chunk number to register,
Replaces by recording the hash value, reference bit, and bit counter value. In order to smoothly proceed with the replacement process, the entries recorded in the hash table 213a are periodically sorted in advance based on the value of the hit counter. Such regular sorting processing is called aging cycle processing, and the details thereof will be described with reference to FIG. Note that the periodic processing is performed at an arbitrary time period using, for example, a timer function provided by the OS.

First, the duplication management unit 301 selects the top entry of the hash table 213a (
S32). In this embodiment, for example, the entry at the top is selected. Subsequently, the duplication management unit 301 right shifts the hit counter by 1 bit (divides by 2) (S33). Assume that the hit counter of this embodiment is formed of 8 bits. For example, “10000000 (hexadecimal number: 80)” is stored in the hit counter of the entry. By shifting 1 bit to the right, the value of the hit counter becomes “01000000 (hexadecimal number: 40)”.

Next, the duplication management unit 301 determines whether the reference bit “1” of the entry is set (
S34). For example, after step S33, the value of the hit counter of the entry is “0100”.
Consider the case of “0000 (hexadecimal number: 40)”. When the reference bit “1” is set, the most significant bit is set (S35), and the value of the hit counter is “11000000 (hexadecimal number: C0).
) ”. When the reference bit is “0”, the value of the hit counter is “01000000 (16
(Advance number: 40) ", and the process proceeds to S36. In this way, the value of the hit counter of the most recently accessed (referenced) chunk is increased by performing the processing of step S34 or step S35 after the processing of step S33. On the other hand, the hit counter value of a chunk that has not been accessed (referenced) most recently is decreased. By changing the value of the hit counter in this way, even if the number of accesses is small, the most recently accessed chunk has a high priority because the value of the hit counter is set large.

Subsequently, the duplication management unit 301 determines whether all entries have been processed (S36). If it is determined that all entries have not been processed (No in S36), the next entry is selected (S37), and the process returns to step S33. If it is determined that all entries have been processed (No in S36), all entries are sorted and moved to a position where the value of the hit counter is in ascending order (S38), and the aging cycle process ends.

By performing this aging cycle process, the priority of the entries in the hash table 213a is given. If there is no free space in the entry according to the priority, the lowest entry in the hash table 213a (that is, the entry with the smallest hit counter value) is selected, and the chunk recorded in the entry is selected. The recorded contents of the number, hash value, reference bit, and bit counter are deleted and replaced by recording the newly registered chunk number, hash value, reference bit, and bit counter value.

Since the hash table 213a exists only on the cache memory 213, it is volatilized when the apparatus is stopped. However, when the apparatus is shut down, the hash table 213a is saved in a nonvolatile medium such as an HDD or an SSD and restored when the apparatus is activated.

As described above, in this embodiment, only a certain amount of hash entries are held in the cache memory 213, and the hash entries are managed by a hit counter weighted based on the number of accesses and the access interval. A hash entry is also registered during the read process. As a result, a deduplication function with a high deduplication rate can be realized at high speed.

As mentioned above, although embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 100 ... Host 110 ... Network 200 ... Storage apparatus 210 ... Disk array control apparatus 211 ... Host I / F
212 ... Disk I / F
213 ... Cache memory 213a ... Hash table 214 ... Cache controller 215 ... FROM
216 ... Local memory 216a ... Control program area 216b ... Table area 216c ... Work area 217 ... CPU
218 ... Chip set 220 ... Disk interface bus 230 ... RAID disk array 230a ... Meta table 230b ... Chunk table 300 ... Dividing unit 301 ... Duplicate managing unit 302 ... Duplicate determining unit 303 ... Hash generating unit 304 ... Access controller

Claims (2)

In a storage device having a recording medium for recording data,
A chunk division unit that divides data designated by a data write request from a host computer into a plurality of chunks;
A hash generation unit that generates a hash value of each of the plurality of chunks based on data of each of the plurality of chunks;
A volatile memory that holds a hash table in which the identifier of the chunk written in the recording medium and the hash value of the chunk are stored in association with each other;
Whether or not a hash value that matches the hash value generated by the hash generation unit based on the data of the first chunk has already been recorded in the hash table when writing the first chunk to the recording medium If the matching hash is not recorded in the hash table, the first chunk is written to the recording medium, and if the matching hash is recorded in the hash table, A control unit for suppressing writing of the first chunk to the recording medium;
A storage device comprising: When the control unit reads data designated by a read request from a host computer from the recording medium,
For a plurality of chunks constituting the data, it is searched whether or not a hash value that matches a hash value generated based on each chunk is already registered in the hash table, and the matching hash value is the hash table. If not recorded in the hash table, the chunk identifier and hash value are registered in the hash table.
The storage apparatus according to claim 1.

Images