JP2017146919A - Control device, storage system, control method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a storage system and the like that make a data reading process efficient.SOLUTION: A control device of a storage system comprises: a storing control unit for performing control so as to store chunks into which writing object data to a storage device is divided in the storage device or in storage means by adding priority to the chunks; a determination unit for determining whether the chunks related to the reading object data is stored in the storage means on the basis of identification information transmitted by the storage device; and a reading control unit for constructing the reading object data by using the chunks stored in the storage means and storage device on the basis of the determination result.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a control device, a storage system, a control method, and a program.

  In a storage system, a deduplication function is being used. By using the de-duplication function, it is possible to increase the efficiency of the storage area required in the storage system and the communication network bandwidth required for writing to the storage system.

  Patent Document 1 provides a deduplication function that can be implemented in the actual operation of a storage system, and a deduplication method for stored data that can reduce the time required to restore the original data from the data after deduplication processing. Is described.

  Patent Document 2 describes an autonomous distributed deduplication file system in which the degree of duplication of the same data is moderately controlled, and both the elimination of excessive duplication and parallel access can be realized.

  Patent Document 3 describes a storage system capable of improving performance while realizing deduplication of stored data. The storage system described in Patent Literature 3 switches nodes that mainly execute deduplication processing according to the importance score value. Therefore, the storage system described in Patent Literature 3 can reduce the amount of communication between the accelerator node and the storage node while suppressing the processing load on the accelerator node.

  Patent Document 4 describes a storage system and the like. In the technique described in Patent Document 4, a specific management unit that is not sufficiently utilized for page allocation is released from allocation to a virtual volume. In the technique described in Patent Document 4, the allocated page belonging to the specific management unit is moved to another management unit, thereby writing the storage area of the specific management unit to another virtual volume from the host computer. Provide access.

  Patent Document 5 describes a cache storage program and the like. In the technique described in Patent Document 5, the communication device stores data in the cache storage area in consideration of the data duplication characteristics, and thus the cache storage area can be efficiently hit.

International Publication No. 2013/051129 JP 2014-160311 A JP 2011-175421 A Special table 2012-533099 gazette JP, 2015-201050, A

  When data is read from the storage system using the deduplication function, all data is read from the storage. Therefore, even in a storage system using the deduplication function, a large network bandwidth is required when stored data is read. That is, with the techniques described in the above-mentioned patent documents, no special consideration is given to the efficiency of processing when data is read.

  The present invention has been made to solve the above-described problems, and has as its main object to provide a storage system and the like that make data read processing more efficient.

  The control device according to an aspect of the present invention includes a storage control unit that controls to store a chunk into which data to be written to the storage device is divided in a storage device or a storage unit with a priority, and a transmission from the storage device A determination unit that determines whether a chunk related to the read target data is stored in the storage unit based on the identification information to be read, and a read target data using the chunk stored in the storage unit and the storage device based on the determination result Read control means for constructing.

A storage system according to an aspect of the present invention includes a control device, a reply unit that transmits identification information identifying a chunk related to read target data to the control device based on a request from the control device, and a chunk related to read target data. A storage device including read control means for controlling to transmit a chunk not stored in the storage means of the control device to the control device.

  In the control method according to one aspect of the present invention, the chunk in which the data to be written to the storage device is divided is controlled to be stored in the storage device with the storage device or priority, and the identification information transmitted from the storage device Based on the above, it is determined whether the chunk related to the read target data is stored in the storage unit, and based on the determination result, the read target data is constructed using the chunk stored in the storage unit and the storage device.

  The program according to an aspect of the present invention is a program for controlling a computer to store a chunk into which data to be written to the storage device is divided into a storage device or a storage unit with a priority, and to transmit the chunk from the storage device Processing for determining whether a chunk related to the read target data is stored in the storage unit based on the identification information to be read, and based on the determination result, the read target data is stored using the chunk stored in the storage unit and the storage device. The process to build is executed.

  According to the present invention, it is possible to provide a storage system or the like that improves the efficiency of data read processing.

It is a figure which shows the structure of the control apparatus in the 1st Embodiment of this invention, a storage apparatus, and a storage system. It is an example which shows a structure in case the storage system in 1st Embodiment of this invention has a some control apparatus or storage apparatus. It is a figure which shows an example of the fundamental operation | movement of the data writing in the storage system in the 1st Embodiment of this invention. It is a figure which shows an example of the fundamental operation | movement of the data reading in the storage system in the 1st Embodiment of this invention. It is a figure which shows an example of a basic operation | movement when backup data is stored in the storage system in the 1st Embodiment of this invention. It is a figure which shows the operation example of the writing of the data in the storage system in the 1st Embodiment of this invention. It is a figure which shows the operation example of the reading of the data in the storage system in the 1st Embodiment of this invention. It is a figure which shows the operation example in case backup data is stored in the storage system in the 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement regarding writing of the storage system etc. in the 1st Embodiment of this invention. It is a flowchart which shows an example of the operation | movement regarding reading of the storage system etc. in the 1st Embodiment of this invention. It is a figure which shows an example of the information processing apparatus which implement | achieves the control apparatus etc. in embodiment of this invention.

  Embodiments of the present invention will be described with reference to the accompanying drawings. In the embodiment of the present invention, each component of each device such as a control device to be described later represents a functional unit block. Part or all of each component of each device is realized by an arbitrary combination of an information processing device 500 and a program as shown in FIG. 11, for example. The information processing apparatus 500 includes the following configuration as an example.

CPU (Central Processing Unit) 501
ROM (Read Only Memory) 502
-RAM (Random Access Memory) 503
A program 504 loaded into the RAM 503
A storage device 505 for storing the program 504
A drive device 507 for reading / writing the recording medium 506
Communication interface 508 connected to the communication network 509
An input / output interface 510 for inputting / outputting data
-Bus 511 connecting each component
Each component of each device in each embodiment is realized by the CPU 501 acquiring and executing a program 504 that realizes these functions. The program 504 that realizes the function of each component of each device is stored in advance in the storage device 505 or the RAM 503, for example, and is read by the CPU 501 as necessary. Note that the program 504 may be supplied to the CPU 501 via the communication network 509 or may be stored in the recording medium 506 in advance, and the drive device 507 may read the program and supply it to the CPU 501.

  There are various modifications to the method of realizing each device. For example, each device may be realized by an arbitrary combination of the information processing device 500 and a program that are separately provided for each component. A plurality of constituent elements included in each device may be realized by an arbitrary combination of one information processing device 500 and a program.

  In addition, some or all of each component of each device is realized by a general-purpose or dedicated circuit including a processor or the like or a combination thereof. These may be configured by a single chip or may be configured by a plurality of chips connected via a bus. Part or all of each component of each device may be realized by a combination of the above-described circuit and the like and a program.

(First embodiment)
Subsequently, a first embodiment of the present invention will be described. FIG. 1 is a diagram illustrating a configuration of a control device, a storage device, and a storage system according to the first embodiment of the present invention. FIG. 2 is an example showing a configuration in the case where the storage system according to the first embodiment of the present invention has a plurality of control devices or storage devices. FIG. 3 is a diagram showing an example of a basic operation of data writing in the storage system according to the first embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a basic operation of reading data in the storage system according to the first embodiment of the present invention. FIG. 5 is a diagram showing an example of a basic operation when backup data is stored in the storage system according to the first embodiment of the present invention. FIG. 6 is a diagram showing an operation example of data writing in the storage system according to the first embodiment of the present invention. FIG. 7 is a diagram showing an operation example of reading data in the storage system according to the first embodiment of the present invention. FIG. 8 is a diagram showing an operation example when backup data is stored in the storage system according to the first embodiment of the present invention. FIG. 9 is a flowchart showing an example of an operation related to writing in the storage system or the like in the first embodiment of the present invention. FIG. 10 is a flowchart showing an example of an operation related to reading in the storage system or the like in the first embodiment of the present invention. FIG. 11 is a diagram illustrating an example of an information processing device that implements a control device or the like according to an embodiment of the present invention.

  It is assumed that the storage system in this embodiment has a general deduplication function. That is, when data is stored in the storage, data to be written (hereinafter referred to as “write target data”) is divided into units called chunks. Further, information for identifying a chunk such as a hash value is obtained for the chunk. Each chunk may have a predetermined size (that is, a fixed length), or may have a different size (that is, a variable length). Based on the hash value, it is determined whether the chunk is already stored in the storage device. As a result of the determination, data that is not stored in the storage device is stored in the storage.

  First, the outline of the storage stem in the first embodiment of the present invention will be described.

  The storage system in this embodiment has a general deduplication function. The storage system also includes a server provided with a control device and a storage device. In the control device, a storage unit that can be accessed at a higher speed than the storage device is prepared as a local storage unit.

  FIGS. 3 and 4 are diagrams respectively showing data writing or reading processing when a storage unit is provided in a control device of a storage system having a general deduplication function. In the example illustrated in FIG. 3, it is assumed that the storage system includes a server including a control device and a storage device, similarly to the storage system in the present embodiment.

  As described above, when data is written to a storage system having a general deduplication function, write target data is divided into units called chunks. In the example shown in FIG. 3 or 4, the write target data is divided into five chunks from chunk A to chunk E. In the example illustrated in FIG. 3, it is assumed that three chunks A, C, and D have been stored in the storage device in advance. In addition, it is assumed that chunks A and C are stored in the local storage unit in advance. In this case, as an example of the operation, the two chunks B and E that are not stored in the storage device are stored in the local storage unit together with the transfer from the server to the storage device.

  When data is read from the storage system in this embodiment, chunks stored in the local storage unit are used. That is, in the control device, the hash value of the chunk to be read is inquired of the storage device. Subsequently, it is confirmed whether the chunk exists in the local storage unit using the hash value. As a result, when it is confirmed that the chunk to be read is stored in the local storage unit, the chunk is read from the local storage unit.

  In the example shown in FIG. 4, it is assumed that four chunks of chunks A, B, C, and E have already been stored in the local storage unit provided in the server. The read target data is composed of five chunks from chunks A to E. In this case, the chunks A, B, C, and E are read from the local storage unit when the read target data is constructed. Chunk D is transferred from the storage device. That is, in the storage system according to the present embodiment, the amount of data transferred from the storage apparatus when reading data is reduced as compared with a storage system having a general deduplication function.

  That is, when the control device includes the storage unit, the storage system according to the present embodiment makes data read processing more efficient.

  However, the capacity of the storage unit prepared for the server that implements the control apparatus may be smaller than the storage capacity of the storage apparatus. When there is no free space in the storage unit, a chunk stored in advance in the storage unit may be deleted when a new chunk is stored. Then, when deleting chunks stored in the storage unit in advance, it is considered that there is room for further improvement in the method for determining chunks to be deleted.

FIG. 5 shows an example of chunks stored in the storage unit prepared for the server when data is repeatedly written to the storage device. In the example shown in FIG. 5, the data that is repeatedly written is backup data. In this example, so-called full backup is performed in the first writing. That is, all the data to be backed up is written. In subsequent writing, so-called incremental backup is performed. That is, only the data whose contents are added or changed among the data to be backed up is written as the backup data. In addition, the storage unit can store six chunks. When there is no free space, it is assumed that the chunks stored in the earliest time (in other words, the oldest) are deleted in order. In the example shown in FIG. 5, in the first full up-back, the write target data is divided into, for example, chunks A to E. The five chunks are stored in a storage unit prepared in the storage device and the control device.

Subsequently, at the time of incremental backup, for example, a portion corresponding to the data chunk D in the full backup is updated according to the addition or change of the contents.
At the time of the first incremental backup, the chunk corresponding to the chunk D is updated to the chunk F. The chunk F is stored in a storage unit prepared in the server. In this case, since the storage unit is empty, the chunk F is stored in the storage unit without deleting other chunks.

  At the time of the second incremental backup, the chunk corresponding to the chunk F is updated to the chunk G. The chunk G is stored in the storage unit 140 prepared in the server. However, since there is no free space in the storage unit 140, one of the chunks stored in the storage unit 140 is deleted as the chunk G is stored in the storage unit 140. In the example shown in FIG. 5, chunk A, which is one of the chunks stored at the earliest time, is deleted.

  Similarly, in the third and subsequent incremental backups, the chunks corresponding to the original chunk D are sequentially updated. The updated chunk is stored in the storage unit 140 in the same manner as the chunk F or G. As the updated chunks are stored, the chunks stored in the storage unit 140 are deleted in the above order. In the example shown in FIG. 5, the chunks stored at the time of full backup are deleted sequentially. After the sixth incremental backup, six chunks F, G, H, I, J, and K are stored in the storage unit.

  Assume that the backup data is restored after six incremental backups. FIG. 6 shows an example of a chunk read procedure when restoration is performed. In the example shown in FIGS. 5 and 6, the backup data is composed of five chunks A, B, C, K, and E. Therefore, among the chunks stored in the storage unit, the chunk used for restoration is one of the chunks K. Accordingly, the chunks A, B, C, and E are read from the storage device, and the chunk K is read from the storage unit, so that backup data is constructed.

  In the example illustrated in FIG. 5 or 6, the amount of data transferred from the storage apparatus is reduced by using one of the chunks stored in the storage unit. That is, the data reading process is made efficient. However, among the chunks stored in the storage unit, chunks F to J are chunks that are not used for the restoration of the backup data described above. That is, it is considered that the efficiency of the data reading process can be further improved by changing the selection of the chunk to be deleted from the storage unit. In the example shown in FIG. 5 or 6, if chunks A, B, C, and E are stored in the storage unit, the backup data can be restored without reading them from the storage device.

  Therefore, in the storage system according to the present embodiment, a control device provided in a server or the like includes a storage unit, and further gives priority to each of the chunks stored in the storage unit 1. That is, when there is no free space in the storage unit, control is performed so that chunks with high priority remain in the storage unit and chunks with low priority are deleted from the storage unit.

  FIG. 7 shows an example of chunks stored in the storage unit prepared for the server when data is repeatedly written to the storage device, similarly to the example shown in FIG. As in the example shown in FIG. 5, it is assumed that data to be repeatedly written is backup data. Similarly to the example shown in FIG. 5, it is assumed that a full backup is performed at the first writing and an incremental backup is performed at the subsequent writing. In addition, the chunk stored in the storage unit 140 at the time of full backup has a higher priority than others, and the chunk stored in the storage unit 140 at the time of incremental backup has a lower priority than others. Assume that priority is set.

  In the example shown in FIG. 7, in the first full backup and incremental backup, the chunks stored in the storage unit are the same as those in FIG. In this case, among the chunks stored in the storage unit, chunks A, B, C, D, and E are given higher priority than others, and chunk F has a lower priority than others. Is granted.

  In the second incremental backup, chunk G is stored in the storage unit. However, since there is no free space in the storage unit, one of the chunks stored in the storage unit is deleted. In this case, the chunk to be deleted is determined according to the priority.

  In the example shown in FIG. 7, as described above, low priority is given to the chunk F stored in the first incremental backup. Therefore, at the time of the second incremental backup, chunk F is deleted and chunk G is stored in the storage unit prepared in the server.

  Similarly, when writing is performed with subsequent incremental backup, a low priority is given to the chunk stored in the incremental backup. Therefore, when a chunk to the storage unit is stored, it is deleted and a new chunk is stored.

  Also in the example illustrated in FIG. 7, it is assumed that backup data is restored after 6 incremental backups have been performed. FIG. 8 shows an example of a chunk reading procedure when restoration is performed in the example shown in FIG. In the example shown in FIGS. 7 and 8, the backup data is composed of five chunks of chunks A, B, C, K, and E. These chunks are all stored in the storage unit. Therefore, the backup data is constructed using only the chunks stored in the storage unit. That is, in the examples shown in FIGS. 7 and 8, backup data is constructed without transferring data from the storage device to the server provided with the control device.

  As described above, the priority is given to the chunks stored in the storage unit 140, so that the storage system according to the present embodiment can further improve the efficiency of the data reading process.

  Next, an overview of the configuration of the storage system in the first embodiment of the present invention will be described.

  As shown in FIG. 1, the storage system 10 according to the first embodiment of the present invention includes a control device 11 and a storage device 12. The control device 11 includes at least a storage control unit 110, a determination unit 120, and a read control unit 130 that are included in a region surrounded by a one-dot chain line in the drawing. In addition, the control device 11 includes a storage unit 140 and a hash table 150. In the present embodiment, it is assumed that the control device 11 includes a dividing unit 160 and a calculating unit 170 in addition to a general server that accesses a storage device having a deduplication function.

  The storage control unit 110 performs control so that a chunk related to data to be written to the storage device 12 is prioritized and stored in the storage unit 140. Further, the storage control unit 110 performs control so that chunks related to data to be written to the storage apparatus 12 are stored in the storage apparatus 12. In the present embodiment, a chunk indicates a data unit when data to be written to the storage device 12 is divided into an arbitrary number. Whether the determination unit 120 stores in the storage unit 140 a chunk related to data to be read (hereinafter referred to as “read target data”) based on identification information such as a hash value transmitted from the storage device 12. Determine. Based on the determination result by the determination unit 120, the read control unit 130 constructs read target data using the chunks stored in the storage unit 150 and the storage device 12. The storage unit 140 stores the chunks divided by the dividing unit 160. The storage unit 140 may be referred to as a “local storage unit” in the following description. The hash table 150 stores hash values for the chunks stored in the storage unit 140. The dividing unit 160 divides the data to be written to the storage device 12 into an arbitrary number of chunks. The calculating unit 170 calculates a hash value for each of the chunks generated by the dividing unit 160. In the present embodiment, the hash value is used as identification information for distinguishing each chunk.

  The storage device 12 includes a storage unit 210, a hash table 220, a storage unit 230, a reply unit 240, a transfer unit 250, and a duplication determination unit 260. The storage unit 210 stores data divided into chunks. The hash table 220 stores a hash value for the chunk stored in the storage unit 210. The duplication determination unit 260 determines whether a chunk related to the hash value is stored in the storage unit 210 based on the hash value calculated by the calculation unit 170 of the control device 11. The storage unit 230 stores the chunk sent from the storage control unit 110 of the control device 11 in the storage unit 210. The reply unit 240 transmits a hash value for identifying a chunk related to the read target data to the determination unit 120 of the control device 11 based on the request of the control device 11. The transfer unit 250 performs control so that chunks that are not stored in the storage unit 140 of the control device 11 among the chunks related to the read target data are acquired from the storage unit 210 and transmitted to the control device 11.

  In the present embodiment, the control device 11 is provided in a server or the like that writes data to or reads data from the storage device 12. That is, the control device 100 is realized as a program that operates on a server such as the information processing device 500 shown in FIG. In this case, the server has a configuration like the information processing apparatus 500 shown in FIG. 8 described above, for example. However, the control apparatus 100 may be implement | achieved in another aspect.

  In the present embodiment, the storage system 10 is a storage that performs backup of data, for example. That is, the server provided with the control device 11 is, for example, a backup server that backs up data. The storage device 12 is a backup storage, for example. However, the storage system 10 in this embodiment may be used for other purposes such as a file server.

  Note that the storage system 10 illustrated in FIG. 1 includes one control device 11 and one storage device 12, respectively. However, the number of control devices 11 or storage devices 12 is not particularly limited. That is, as in the example illustrated in FIG. 2, the storage system 10 may include the control devices 11-1 to 11-N and the storage devices 12-1 to 12-N. In this case, each of the control devices 11-1 to 11-N may use any one of the storage devices 12-1 to 12-N as a target of data writing or reading.

  Next, details of each component of the storage system 10 in the first embodiment of the present invention will be described. First, details of each component of the control device 11 included in the storage system 10 will be described.

  The storage control unit 110 controls the chunk to be stored in the storage unit 140 or the storage device 12 that is a local storage unit. The storage control unit 110 controls to store in the storage unit 210 the chunk that has been determined not to be stored in the storage unit 210 by the duplication determination unit 260 of the storage device 12. This control is the same as the control in a general server that accesses a storage apparatus having a general deduplication function. At the same time, the storage control unit 110 stores a chunk related to data to be written to the storage device 12 in the storage unit 140. As described above as an outline, the chunk stored in the storage unit 140 is used when data related to the chunk is read data.

  The data stored in the storage unit 140 by the storage control unit 110 is not particularly limited, and may be a chunk related to data to be written to the storage device 12. That is, the storage control unit 110 may store in the storage unit 140 the chunk that the duplication determination unit 260 of the storage device 12 has determined to be unstored in the storage unit 210. Alternatively, even when the duplication determination unit 260 determines that the chunk is already stored in the storage unit 210, the storage control unit 110 determines that the chunk is not stored in the storage unit 140. May be stored in the storage unit 140.

  When the chunk is stored in the storage unit 140, a time corresponding to the configuration of the storage unit 140 or the like is required. However, in general, the time is shorter than the time required when the chunk is stored in the storage unit 210 of the storage device 12. Therefore, the storage control unit 110 stores the chunks that are not stored in the storage unit 210 in the storage unit 140, so that the time required for storage in the storage unit 140 is substantially concealed.

  Further, the storage control unit 110 manages the chunks stored in the storage unit 140 based on the priority assigned to each chunk.

  For example, when the storage control unit 110 stores a chunk, the storage unit 140 may not have a free space for storing the chunk. In this case, the storage control unit 110 deletes a part of the chunk stored in the storage unit 140, and secures an area for storing a new chunk. The storage control unit 110 determines a chunk to be deleted based on the priority assigned to each of the chunks stored in the storage unit 140. In this case, the storage control unit 110 does not delete the chunk with the relatively high priority but leaves it in the storage unit 140 and deletes the chunk with the relatively low priority from the storage unit 140. Thus, the storage unit 140 is managed.

  The high priority given to the chunk by the storage control unit 110 is determined by various methods. As an example, the storage control unit 110 determines the high priority according to the type of write target data that is the source of the chunk stored in the storage unit 140.

  As described above, for example, when the write target data is backup data, the storage control unit 110 determines the high priority according to whether the data is a chunk related to full backup or a chunk related to incremental backup. To do. That is, the storage control unit 110 gives a relatively high priority to the chunk related to the full backup and a relatively low priority to the chunk related to the incremental backup. Data chunks for incremental backups are often less needed in a shorter period of time than data chunks for full backups. Therefore, the storage control unit 110 assigns such priorities, thereby making it possible to improve the efficiency of the data reading process.

  The storage control unit 110 gives higher priority to chunks that are expected to be read based on the type of data to be written that is the source of chunks stored in the storage unit 140 than others. May be.

As another method, the storage control unit 110 may determine the high priority based on the determination result in the duplication determination unit 260 of the storage device 12.
In this case, for example, the storage control unit 110 gives a relatively low priority to a chunk that is determined to be unstored and is newly stored in the storage unit 140. When the chunk determined to be stored in the storage device 12 by the duplication determination unit 260 is stored in the storage unit 140, the chunk is updated so that the priority of the chunk is relatively high. When the storage control unit 110 assigns such priorities, it is possible to increase the efficiency of the reading process with respect to data that can be read, for example.

  In this case, the high priority given to each chunk may be the reverse of the above-described example. For example, the storage control unit 110 may determine that the storage control unit 110 is not stored and give a relatively low priority to a chunk that is newly stored in the storage unit 140.

  The storage control unit 110 may change the priority of the chunks stored in the storage unit 140 according to the time since storage or the like. For example, the storage control unit 110 may update the priority of the chunk stored in the storage unit 140 so as to decrease the priority set at the time of writing to the storage unit 140 every predetermined period. .

  In addition, the storage control unit 110 may determine the high priority given to the chunk by combining the above-described methods. For example, the storage control unit 110 may change the priority for each chunk and update the assigned priority according to the time since storage.

  In this example, when the data is backup data, the storage control unit 110 assigns a priority of “8” to a chunk related to full backup and a priority of “1” to a chunk related to incremental backup. Store in the storage unit 140. Then, the storage control unit 110 updates the priority given to each chunk so as to decrease by one per day.

  In this example, when a full backup is executed once a week, when the backup data is restored, the chunk related to the latest full backup or incremental backup is preferentially stored in the storage unit 140. Become. This is because the priority of the chunk related to the full backup executed immediately before is smaller than the priority of the chunk related to the incremental backup relative to the latest full backup.

  Therefore, restoration using chunks stored in the storage unit 140 can be performed, and as a result, the data read processing is made efficient.

  The storage control unit 110 is not limited to the case where there is no free space in the storage unit 140, but is stored when the free space in the storage unit 140 satisfies a predetermined condition, for example, the free space becomes a predetermined size. The chunk stored in the unit 140 may be deleted.

  Further, when storing the chunk in the storage unit 140, the storage control unit 110 also stores a hash value corresponding to the chunk in the hash table 150. In the present embodiment, the method and format for obtaining the hash value in the storage system 10 are not particularly limited. In other words, the hash value may be information that allows each of the chunks stored in the storage unit 140 or 210 to be identified.

  The determination unit 120 determines whether a chunk related to the read target data is stored in the storage unit 140. The determination unit 120 determines whether the chunk is stored in the storage unit 140 based on the hash value transmitted from the storage device 12.

  The determination unit 120 inquires of the storage apparatus 12 for a hash value related to the read target data. In this case, the reply unit 240 included in the storage device 12 returns a hash value for each of the chunks constituting the read target data to the determination unit 120. Whether the chunk is stored in the storage unit 140 by comparing the hash value returned from the reply unit 240 with each hash value stored in the hash table 150 included in the control device 11. Determine. That is, when any of the hash values stored in the hash table 150 matches the hash value returned from the reply unit 240, the determination unit 120 stores the chunk associated with the hash value in the storage unit 140. It is determined that

  The read control unit 130 constructs read target data based on the determination result by the determination unit 120. The read control unit 130 reads the chunk from the storage unit 140 when the determination unit 120 determines that the chunk related to the read target data exists in the storage unit 140. When it is determined that the chunk related to the data to be read does not exist in the storage unit 140, the read control unit 130 acquires the chunk via the transfer unit 250 of the storage device 12.

  When the read control unit 130 acquires all the chunks related to the read target data, the read control unit 130 constructs the read target data. The constructed read target data is appropriately transmitted to the request source of the data.

  As described above, the read control unit 130 uses the chunk stored in the storage unit 140 when constructing the read target data, thereby reducing the amount of data transferred from the storage device 12 to the control device 11.

  The storage unit 140 stores the chunks divided by the dividing unit 160. As described above, the chunk stored in the storage unit 140 is used when reading data from the storage system 10.

  It is preferable that the storage unit 140 can be accessed at high speed from each component of the control device 11 as compared with the storage device 12. As an example, the storage unit 140 is preferably implemented using a storage device built in a server that implements the control device 11. As a storage device built in the server, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like is used. The storage unit 140 may be connected to the control device 11 via a communication network. For example, the storage unit 140 may be a storage device such as a NAS (Network Attached Storage) connected via a communication network having a wider bandwidth than the storage device 12. In this case, it is preferable that the communication network that connects the storage unit 140 to the control device 11 has, for example, a higher transfer speed and a wider bandwidth than the communication network that connects the control device 11 and the storage device 12.

  The capacity of the storage unit 140 may be smaller than that of the storage unit 210 of the storage apparatus 12. That is, the storage unit 140 may be configured to store a part of chunks stored in the storage device 12.

  The hash table 150 stores hash values for the chunks stored in the storage unit 140. In the present embodiment, for example, a part of the storage unit 140 is used as the hash table 150.

  The dividing unit 160 and the calculating unit 170 are elements that are generally provided in a storage system having a deduplication function. That is, the dividing unit 160 divides data to be written to the storage device 12 into an arbitrary number of chunks. As described above, the dividing unit 160 may divide the chunks so that the chunks have the same size, or may divide the chunks so that the sizes of the chunks are different.

  Further, the calculation unit 170 calculates a hash value for each of the chunks generated by the dividing unit 160. The calculation unit 170 transmits the calculated hash value to the storage apparatus 12 and inquires whether the chunk corresponding to the hash value has been stored in the storage apparatus 12. In the storage apparatus 12, it is determined whether the chunk corresponding to the hash value has been stored in the storage apparatus 12 based on the hash value.

  Next, details of each component of the storage apparatus 12 included in the storage system 10 will be described.

  The storage unit 210 stores the data divided into chunks as the storage device 12. The storage unit 210 is realized using a nonvolatile storage device such as an HDD or an SSD. The storage unit 210 may have a RAID (Redundant Arrays of Inexpensive Disks) configuration.

  The hash table 220 stores a hash value for each chunk stored in the storage unit 210.

  The duplication determination unit 260 determines whether or not the chunk is stored in the storage unit 210. Similar to a general storage system having a deduplication function, the duplication determination unit 260 stores a chunk corresponding to the hash value in the storage unit 210 based on the hash value calculated by the calculation unit 170 of the control device 11. Determine whether it is stored. That is, the duplication determination unit 260 acquires the hash value calculated by the calculation unit 170 of the control device 11 and compares the hash value with each of the hash values stored in the hash table 220. Then, the duplication determination unit 260 determines that the chunk corresponding to the hash value is stored in the storage unit 210 when the hash value matches one of the hash values stored in the hash table 220. The duplication determination unit 260 determines that the chunk corresponding to the hash value is not stored in the storage unit 210 when both the hash value and the hash value stored in the hash table 220 match.

  The storage unit 230 stores the chunk transmitted from the storage control unit 110 of the control device 11 in the storage unit 210. The storage unit 230 acquires a hash value corresponding to the chunk together with the chunk and stores it in the hash table 220.

  Based on the inquiry from the determination unit 120 of the control device 11, the reply unit 240 transmits the hash value for the chunk related to the read target data to the determination unit 120 of the control device 11. The determination unit 120 determines whether a chunk corresponding to the hash value is stored in the storage unit 140 using the hash value acquired from the reply unit 240.

  The transfer unit 250 controls to acquire a chunk related to the read target data from the storage unit 210 and transmit the chunk to the control device 11. The read control unit 130 of the control device 11 acquires a chunk from the storage device 12 when the chunk related to the read target data does not exist in the storage unit 140. That is, the transfer unit 250 performs control so that chunks that are not stored in the storage unit 140 of the control device 11 are transmitted to the control device 11.

  Next, with reference to the flowcharts shown in FIGS. 9 and 10, details of the operation of the storage system 10 or each of its components in the first embodiment of the present invention will be described.

  First, with reference to the flowchart shown in FIG. 9, the details of the write operation of mainly the control device 11 of the storage system 10 in the first embodiment of the present invention will be described.

  First, the dividing unit 160 of the control device 11 divides the write target data into chunks (step S101). Next, the calculation unit 170 calculates a hash value for the chunk obtained in step S101 (step S102).

  Next, the calculation unit 170 transmits the hash value obtained in step S102 to the storage apparatus 12 and inquires whether the chunk corresponding to the hash value has been stored in the storage apparatus 12 (step S103). In the storage apparatus 12, the duplication determination unit 260 determines whether the chunk corresponding to the hash value has already been stored in the storage apparatus 12.

  When the duplication determination unit 260 determines that the chunk is not stored in the storage apparatus 12 (step S104: No), the storage control unit 110 of the control apparatus 11 stores the chunk in the storage apparatus 12. That is, the storage control unit 110 transfers the chunk to the storage device 12 (step S105). In the storage apparatus 12, when the storage unit 230 acquires the chunk, the acquired chunk is stored in the storage unit 210. The storage unit 230 acquires a hash value corresponding to the chunk together with the chunk and stores it in the hash table 220.

  If the duplication determination unit 260 determines that the chunk has been stored in the storage device 12 (step S104: Yes), the storage control unit 110 continues without transferring the chunk to the storage device 12. Then, the process of step S106 is performed. The operations after step S106 may be performed only when it is determined in step S104 that the chunk is not stored in the storage apparatus.

  Next, the storage control unit 110 determines whether or not to store the above-described chunk in the storage unit 140 that is a local storage unit (step S106).

  When storing the chunk in the storage unit 140 (step S106: Yes), the storage control unit 110 checks whether the storage unit 140 has free space (step S107).

  If there is no free space in the storage unit 140 (step S107: No), the storage control unit 110 deletes a part of the chunk stored in the storage unit 140 to ensure the free space in the storage unit 140 ( Step S108). In step S108, the storage control unit 110 determines the chunk to be deleted based on the priority assigned to each chunk stored in the storage unit 140.

  When the free space of the storage unit 140 is secured in the process of step S108, or when it is determined in the process of step S107 that the storage unit 140 has free space, the storage control unit 110 performs the process of step S109. Process. That is, the storage control unit 110 gives priority to the chunk and stores it in the storage unit 140. Subsequently, the storage control unit 110 stores the hash value corresponding to the chunk stored in the storage unit 140 in step S109 in the hash table (step S110).

  Note that the storage control unit 110 may appropriately change the priority assigned to the chunk stored in the storage unit 140 in accordance with the processing in step S109.

  In step S106, when the storage control unit 110 does not store the chunk in the storage unit 140 (step S106: No), the storage control unit 110 ends the writing process.

  In the process of step S101, the write target data is generally divided into a plurality of chunks. In this case, the control device 11 or the like performs the processing from step S102 on each of the plurality of chunks.

  Next, with reference to a flowchart shown in FIG. 10, details of the read operation of the control device 11 of the storage system 10 in the first embodiment of the present invention will be described.

  First, the determination unit 120 inquires of the storage apparatus 12 for a hash value corresponding to a chunk related to read target data (step S201). In this case, the reply unit 240 of the storage apparatus 12 transmits the hash value for the chunk related to the read target data to the determination unit 120 based on the inquiry from the determination unit 120.

  Next, the determination unit 120 determines whether the chunk related to the read target data is stored in the storage unit 140 which is a local storage unit, using the hash value acquired in step S201 (step S202). That is, the determination unit 120 compares the hash value returned from the reply unit 240 with each of the hash values stored in the hash table 150, and whether the chunk related to the read target data is stored in the storage unit 140. Determine.

  The determination unit 120 determines that the chunk exists in the storage unit 140 when the hash value acquired in step S202 matches any of the hash values stored in the hash table 150 (step S202: Yes). In this case, the read control unit 130 reads the chunk from the storage unit 140 (step S203).

  The determination unit 120 determines that the chunk does not exist in the storage unit 140 when the hash value acquired in step S202 does not match any of the hash values stored in the hash table 150 (step S202: No). The read control unit 130 reads a chunk from the storage device 12 (step S204). The read control unit 130 acquires a chunk via the transfer unit 250 of the storage device 12.

  When all the chunks related to the read target data are read, the read control unit 130 combines the chunks to construct the read target data (step S205). The read control unit 130 appropriately transmits the constructed data to the request source of the data, and ends the data read process.

  The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. The configurations in the embodiments can be combined with each other without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Storage system 11 Control apparatus 12 Storage apparatus 110 Storage control part 120 Judgment part 130 Reading control part 140 Storage part 150 Hash table 160 Dividing part 170 Calculation part 210 Storage part 220 Hash table 230 Storage part 240 Reply part 250 Transfer part 260 Duplicate judgment Part 500 Information processing device 501 CPU
502 ROM
503 RAM
504 Program 505 Storage device 506 Recording medium 507 Drive device 508 Communication interface 509 Communication network 510 Input / output interface 511 Bus

Claims (12)

A storage control means for controlling the chunk into which the data to be written to the storage device is divided to store the chunk in the storage means or with a priority;
A determination unit that determines whether the chunk related to the read target data is stored in the storage unit based on the identification information transmitted from the storage device;
A control apparatus comprising: a read control unit that constructs the read target data using the chunk stored in the storage unit and the storage device based on the determination result.   The control device according to claim 1, wherein the storage control unit determines the priority of the chunk stored in the storage unit based on a type of the data to be written.   The control device according to claim 2, wherein when the write target data is backup data, the storage control unit assigns a higher priority to the chunk related to full backup data than other chunks.   4. The control device according to claim 2, wherein, when the write target data is backup data, the storage control unit assigns a lower priority to the chunk related to incremental backup data than other chunks. 5.   5. The control device according to claim 1, wherein the storage control unit determines a priority for the chunk based on a time from when the chunk is stored in the storage unit.   The storage unit determines the priority of the chunk stored in the storage unit based on a determination result of whether or not the chunk is stored in the storage device in the storage device. The control device according to claim 5.   The control according to any one of claims 1 to 6, wherein the storage control unit deletes the chunk having the low priority from the storage unit when a free capacity of the storage unit satisfies a predetermined condition. apparatus.   The control device according to any one of claims 1 to 7, wherein the read control unit reads the data from the storage unit when the storage unit stores the read target data.   The control device according to claim 1, comprising the storage unit. A control device according to any one of claims 1 to 8;
Based on the request from the control device, return means for transmitting identification information for identifying a chunk related to the data to be read to the control device;
Read control means for controlling the chunk that is not stored in the storage means of the control device among the chunks related to the read target data to be transmitted to the control device;
A storage system comprising: a storage system comprising: Control the storage device or the chunk that is divided into the data to be written to the storage device, to store it in the storage means with the storage device or priority,
Based on the identification information transmitted from the storage device, it is determined whether the chunk related to the data to be read is stored in the storage means, and based on the determination result, the storage means and the storage device stored in the storage device A control method for constructing the read target data using chunks. On the computer,
Processing for controlling the storage device or the storage unit to store the chunk in which the data to be written to the storage device is divided, with a priority,
A process of determining whether the chunk related to the read target data is stored in the storage unit based on the identification information transmitted from the storage device;
A program for executing processing for constructing the read target data using the chunk stored in the storage unit and the storage device based on the determination result.

Images