JP5526824B2 - Storage system - Google Patents

Description

  The present invention relates to a storage system, and more particularly to a content address type storage system that manages a storage location according to the content of stored data.

  In recent years, with the development and spread of computers, various types of information have been converted into digital data. As a device for storing such digital data, there are storage devices such as a magnetic tape and a magnetic disk. Since the data to be stored increases day by day and becomes enormous, a large-capacity storage system is required. In addition, reliability is required while reducing the cost of the storage device. In addition to this, it is necessary that data can be easily retrieved later. As a result, there is a demand for a storage system that can automatically increase storage capacity and performance, eliminate duplicate storage, reduce storage costs, and have high redundancy.

  In response to such a situation, in recent years, for example, as shown in Patent Document 1, data is distributed and stored in a plurality of storage devices, and a specific content address specified according to the content of the data is used. A content address storage system for specifying a storage location where the data is stored has been developed.

  FIG. 1 shows an example of the configuration of a content address storage system. As shown in this figure, the storage system 100 has a plurality of data storage units 110 and a content address list 120, a deduplication function unit 101, a distributed redundant arrangement function unit 102, and a data movement function unit 103. And a data deletion function unit 104. Each storage system 100 having such a configuration operates as follows.

  The deduplication function unit 101 performs processing as shown in the flowchart of FIG. First, when data storage is started (step S101), the written data is chunked and finely divided into blocks (step S102). Then, the hash value of each block is calculated (step S103), and based on the hash value, it is compared with an already stored block, and it is determined whether this block needs to be stored (step S104). At this time, if the same hash value exists, it is determined that such a block is duplicated (step S104: No), and the duplicated block is not stored. In this case, the CA of the already stored block that is duplicated is acquired (step S106), and the actual block can be accessed based on this CA.

  On the other hand, if the same hash value does not exist, it is determined that the block does not overlap (step S104: Yes), and the unique block that does not overlap is newly stored (step S105). . When the block is stored, the CA (Content Address) of the block is obtained (step S106), and the actual block can be accessed based on the CA. As will be described later, the CA is composed of block hash values and information indicating logical storage positions. Then, the above process is performed on all blocks (step S107).

  As shown in FIG. 3, the distributed redundant arrangement function unit 102 performs a process of storing a block determined to be unique by the deduplication function unit 101 (step S111). First, the block is further divided into a specified number (step S112). These further divided blocks are called fragments. Then, a redundant fragment is created based on the fragment (step S113), and the fragment and the redundant fragment are distributed and arranged (step S114). At this time, the CA of the block is generated using the fragment storage position and the hash value (step S115).

  As a result, even when a disk failure or the like occurs, the original block can be restored because the redundant fragment exists. For example, when designating divided fragment: redundant fragment = 9: 3, the unique block is divided into nine, and 12 fragments including three redundant fragments are further distributed and arranged. Then, the original block can be restored if up to 3 out of 12 fragments are lost due to a disk failure or the like. Note that the ratio between the number of fragment fragments and the number of redundant fragments can be selected according to the type of data.

  The data movement function unit 103 moves a block. For example, when it is necessary to move a block due to the convenience of the storage system (for example, a distributed redundant arrangement function provided in the storage system), the data movement function unit 103 moves the data.

  Here, an example when data A and data B are stored will be described with reference to FIG. First, data A and data B are divided into blocks a, b, and c and blocks c, d, and e, respectively, by chunking. At this time, only one block c is stored and referenced from data A and data B by the deduplication function of the storage system 100. Each unique block is divided into fragments a1 to a9 (see symbol A10), and redundant fragments a10 to a12 are added (see symbol A20). These fragments a1 to a12 are distributed and stored in each data storage unit 110, respectively.

  Here, FIG. 5 shows an example of the components of the content address described above. As shown in this figure, the content address (CA) is determined from the hash value of the block and the logical storage position of the block. Note that the logical storage position is a representative physical position of a fragment constituting a block. For example, the physical storage position of the fragment a1 in the data A. That is, it is assumed that the other fragments a2 to a12 are stored in conjunction with the fragment a1, and that the other fragments a2 to a12 can be referred to if the logical storage position of the fragment a1 is known.

  In the content address storage system as described above, some special processing is required when deleting physical data. That is, since the block actually stored may be shared by a plurality of data, the block cannot be easily deleted at the time of data deletion. For this reason, when deleting a block, it is confirmed which block is used for all data, and an unused block is deleted. This operation is performed by the data deletion function unit 104 according to the flow shown in FIG.

  Specifically, the storage system 100 has a list in which “flag” is associated with each CA used as shown in FIG. Then, as shown in FIG. 7, first, flags for all content addresses (CA) in this list are initialized to 0 (step S121). Thereafter, the data stored without being deleted is read (step S122), and the CA is confirmed (step S123). The flag is checked for each CA (step S124), and if the flag is 0, it is changed to 1 (step S126). If the flag is 1, do nothing. Then, after confirming the CA of all data (step S125), only the block corresponding to the CA with the flag 0 is deleted (step S127).

  FIG. 8 shows an example of data deletion. Here, as shown on the left side of FIG. 8, a case is considered where only data B is deleted after data A and data B are stored. At this time, CAs corresponding to the blocks a, b, c, d, and e are respectively CA1, CA2, CA3, CA4, and CA5. First, the flags of CA1 to CA5 are all 0. When data stored at the present time without deletion, that is, data A is read, the CA1, CA2, and CA3 flags corresponding to blocks a, b, and c constituting the data A are set to 1, respectively. And the CA5 flag remains at 0. After that, as shown on the right side of FIG. 8, the blocks corresponding to CA4 and CA5 whose flags are 0, that is, blocks d and e are deleted.

JP 2005-235171 A

However, when the storage system as described above has the following configuration, there is a problem that the same block is stored in a plurality of forms, and the data capacity to be stored increases.
(1) When the ratio of the number of divided fragments to the number of redundant fragments is changed depending on the type of data in the same storage system.
(2) The stored block needs to be moved by the distributed redundant arrangement function of the storage system.

  The case (1) will be described in detail with reference to FIG. First, consider a case where data A and data B are stored. At this time, when the redundancy set in advance for data B is equal to or lower than the redundancy set for data A (for example, the number of redundant fragments), as shown in FIG. May share block c of data A. The data redundancy is specified by the importance of data set in advance by the user in accordance with, for example, the type of data and the storage location of the data.

  However, when the data B is stored with a higher redundancy than the data A, the block c cannot be shared because the redundancy cannot be satisfied if the block c is shared. Therefore, as shown in FIG. 9, it is necessary to newly store the block c '. In such a case, the block c and the block c ′ are stored despite the same data contents.

  The flow of the above process is shown in FIG. First, as described above, when data storage is started (step S131), the written data is chunked and finely divided into blocks (step S132). Then, the hash value of each block is calculated (step S133), and based on the hash value, it is compared with a block that has already been stored, and it is determined whether this block needs to be stored (step S134). At this time, if it is determined that the same hash value exists and the block is already stored (step S134: Yes), the redundancy (duplication degree) of the stored block and the redundancy of the block to be written are determined. The degrees are compared (step S135). If the redundancy of the block to be written is higher than that of the same stored block (step S136: Yes), the block to be written is newly stored (step S137). Then, similarly to the above, the CA (Content Address) of the stored block is acquired (step S138).

  On the other hand, if the redundancy of the block to be written is the same or lower than that of the same stored block (step S136: No), the block to be written is not newly written, and the stored same block is replaced. Reference is made (step S138). Then, the above processing is performed on all the blocks (step S139).

  The case of moving the block (2) described above will be described with reference to FIGS. First, when the data A is stored, the storage position of the block c is moved due to the convenience of the storage system (for example, the distributed redundant arrangement function provided in the storage system) (step S141). However, since the data A continues to refer to the block c, the block c cannot be erased unless the reference destination of the data A is changed from the block c to the block c ′. Therefore, as shown in FIG. 11, the block c and the block c ′ exist despite the same block (steps S142 and S143).

  As described above, in the content address storage system, there is a problem that a plurality of the same blocks are stored, and the data storage efficiency is lowered.

  Therefore, an object of the present invention is to provide a storage system that solves the above-mentioned problem, ie, a decrease in data storage efficiency.

In order to achieve such an object, a storage apparatus according to an aspect of the present invention provides:
A data storage control unit for storing the storage target data in the storage device, and generating and storing address data indicating a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device.
When the data storage control unit separately stores other storage target data having the same contents as the storage target data already stored in the storage device in the storage device, the storage target data stored in the storage device Is stored in association with other address data representing the storage position of the other storage target data.
Further, the data deletion processing unit deletes the storage target data stored in the storage position represented by one of the address data among the address data associated with each other from the storage device.
The configuration is as follows.

A storage system according to another embodiment of the present invention
A storage device;
A data storage control unit for storing the storage target data in the storage device, and generating and storing address data indicating a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device.
When the data storage control unit separately stores other storage target data having the same contents as the storage target data already stored in the storage device in the storage device, the storage target data stored in the storage device Is stored in association with other address data representing the storage position of the other storage target data.
Further, the data deletion processing unit deletes the storage target data stored in the storage position represented by one of the address data among the address data associated with each other from the storage device.
The configuration is as follows.

Moreover, the program which is the other form of this invention is:
In the information processing device,
A data storage control unit for storing the storage target data in the storage device, and generating and storing address data indicating a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device.
When the data storage control unit separately stores other storage target data having the same contents as the storage target data already stored in the storage device in the storage device, the storage target data stored in the storage device Is stored in association with other address data representing the storage position of the other storage target data.
Further, the data deletion processing unit deletes the storage target data stored in the storage position represented by one of the address data among the address data associated with each other from the storage device.
The configuration is as follows.

In addition, a data processing method according to another aspect of the present invention includes:
The storage target data is stored in the storage device, and address data representing the storage position of the storage target data is generated and stored, and other storage target data having the same contents as the storage target data already stored in the storage device Is stored separately in the storage device, the address data indicating the storage position of the storage target data already stored is associated with other address data indicating the storage position of the other storage target data, and stored.
When the storage object data stored in the storage device is deleted from the storage device, it is stored in the storage location represented by one of the address data among the address data associated with each other. Deleting the storage target data from the storage device,
The configuration is as follows.

  With the configuration as described above, the present invention can improve data storage efficiency while maintaining the reliability of stored data.

It is a block diagram which shows the structure of the storage system relevant to this invention. 2 is a flowchart showing an operation at the time of data storage in the storage system disclosed in FIG. 1. 2 is a flowchart showing an operation at the time of data storage in the storage system disclosed in FIG. 1. It is explanatory drawing which shows the mode at the time of the data storage in the storage system disclosed in FIG. It is a figure which shows the data structure of a content address. It is a figure which shows the data structure of the CA list | wrist memorize | stored in the storage system disclosed in FIG. 3 is a flowchart showing an operation at the time of data deletion in the storage system disclosed in FIG. 1. FIG. 2 is an explanatory diagram showing a state when data is deleted in the storage system disclosed in FIG. 1. It is explanatory drawing which shows the mode at the time of the data storage in the storage system disclosed in FIG. 2 is a flowchart showing an operation at the time of data storage in the storage system disclosed in FIG. 1. FIG. 2 is an explanatory diagram showing a state during data movement in the storage system disclosed in FIG. 1. 2 is a flowchart showing an operation at the time of data movement in the storage system disclosed in FIG. 1. 1 is a block diagram showing a configuration of a storage system in Embodiment 1 of the present invention. It is a figure which shows the data structure of the storage information memorize | stored in the storage system disclosed in FIG. It is a figure which shows the data structure of the CA list | wrist memorize | stored in the storage system disclosed in FIG. 14 is a flowchart showing an operation at the time of data storage in the storage system disclosed in FIG. It is explanatory drawing which shows the mode at the time of the data storage in the storage system disclosed in FIG. FIG. 18 is a diagram showing data contents of stored information when data is stored as shown in FIG. 17 in the storage system disclosed in FIG. 13. FIG. 18 is a diagram showing data contents of a CA list when data is stored as shown in FIG. 17 in the storage system disclosed in FIG. 13. 14 is a flowchart showing an operation at the time of data movement in the storage system disclosed in FIG. It is explanatory drawing which shows the mode at the time of data movement in the storage system disclosed in FIG. 14 is a flowchart showing an operation at the time of data deletion in the storage system disclosed in FIG. It is explanatory drawing which shows the data content of the CA list at the time of the data deletion in the storage system disclosed in FIG. It is explanatory drawing which shows the data content of the CA list at the time of the data deletion in the storage system disclosed in FIG. It is explanatory drawing which shows the data content of the storage information at the time of data deletion in the storage system disclosed in FIG. It is explanatory drawing which shows the mode at the time of data movement in the storage system disclosed in FIG. It is a block diagram which shows the outline of a structure of the storage system in attachment 1 in this invention.

<Embodiment 1>
A first embodiment of the present invention will be described with reference to FIGS. FIG. 13 is a block diagram showing the configuration of the storage system. 14 to 15 are diagrams showing the structure of data stored in the storage system. 16 to 26 are explanatory diagrams showing the operation of the storage system.

[Constitution]
The storage system 1 according to the present invention is a content address storage system. Basically, data is distributed and stored in a plurality of storage devices, and a specific content address specified according to the content of the data is used. The storage location where the data is stored is specified.

  And as shown in FIG. 13, the storage system 1 in this embodiment is comprised with the server computer provided with the arithmetic unit and the memory | storage device. In addition, the storage system 1 has a plurality of data storage units 10 in an equipped storage device, and stores a content address list (CA list) 20 and storage information 30. In addition, the storage system 1 is constructed by incorporating a program into the equipped arithmetic device, the deduplication function unit 11, the distributed redundant arrangement function unit 12, the data movement function unit 13, and the data deletion function unit 14. And. Hereinafter, each of these components will be described in detail.

  The storage system in the present invention is not necessarily limited to the configuration shown in FIG. The storage system may be configured by connecting a plurality of computers, or may be configured by a single computer. Further, the storage device in which the data storage unit 10 is formed or the CA list 20 or the storage information 30 is stored may be externally equipped.

  The deduplication function unit 11 and the distributed redundant arrangement function unit 12 cooperate to have a function (data storage control unit) for distributing and storing data to be stored in a plurality of data storage units 10. For example, the data to be stored is chunked and divided into small blocks, and the hash value of each block is calculated. Then, based on the hash value, when the same hash value exists as compared with the already stored block, it is determined that the block is duplicated, and the duplicate block Is not stored. In this case, an actual block can be accessed by acquiring a content address CA (Content Address) of a duplicated block that has already been stored and referring to the storage location of the CA.

  On the other hand, if the same hash value does not exist, it is determined that such a block is not duplicated, and the unique block that is not duplicated is newly stored. A content address CA (Content Address) is obtained, and an actual block can be accessed based on this CA. The CA is composed of a hash value of a block and a logical storage position.

  In the deduplication function unit 11 and the distributed redundant arrangement function unit 12, when storing data as described above, the identification information for identifying each stored data and the CA are associated with each other in the storage information 30. Remember. FIG. 14 shows an example of the stored information 30. As shown in this figure, for example, when data 1 is divided into three blocks, identification information for identifying each block and the CA of each block are stored in association with each other. Thus, when data 1 is read, data 1 can be read by reading the real block from the storage position referenced by the CA associated with the identification information of the block into which the data 1 is divided.

  Further, in the deduplication function unit 11 and the distributed redundant arrangement function unit 12 in the present embodiment, when storing data as described above, the CA representing the storage position of each block is separately stored in the CA list 20. It has a function. Specifically, as shown in FIG. 15, the CA list 20 is provided with an “updated CA” field and a “flag” field in association with each CA. The data stored in each column will be described later.

  Furthermore, even if the same block is already stored in the data storage unit 10 when the data is stored in the deduplication function unit 11 and the distributed redundancy arrangement function unit 12 in the present embodiment as described above. The block having the same data content may be separately stored in the data storage unit 10. For example, as described above, the redundancy (importance) set in advance for a block having the same content to be newly stored is greater than the redundancy (importance) set for an already stored block. If it is higher, the newly stored block is stored in the data storage unit 10 separately. At this time, in the CA list 20, the newly stored CA of the block is stored in the “updated CA” column associated with the CA of the already stored block having the same contents.

  The data redundancy (importance) described above is a value corresponding to the number of redundant fragments added when data is fragmented, for example, depending on the type of data and the storage location of the data. Thus, the value is preset by the user. However, the importance of data is not necessarily limited to being determined by the redundancy of data.

  The data movement function unit 13 has a function (data storage control unit) for moving data already stored in the data storage unit 10 to another storage position. When the data movement function unit 13 moves the block, the block having the same data content is separately copied and stored in the movement destination while leaving the movement source block. At this time, in the CA list 20, the data movement function unit 13 replaces the CA of the newly stored block of the movement destination with the “updated CA” associated with the CA of the movement source block already stored with the same contents. In the "" column.

  The data deletion function unit 14 (data deletion processing unit) deletes the block stored in the data storage unit 10. In particular, in the present embodiment, as described above, when there are blocks having the same data contents, only the most effective block is left, and one is deleted to improve data storage efficiency. Let

  Specifically, at the time of data deletion, the data deletion function unit 14 reads the “update CA” column associated with the CA (CAa) of the remaining block that is not the deletion target from the CA list 20. If CA (CAb) is described in the “update CA” column, the CA (CAa) associated with the identification information of the remaining block in the stored information 30 is replaced with the CA in the “update CA” column. Rewrite to (CAb). As a result, the reference destination of the remaining block is changed to the CA (CAb) described in the “update CA” column, and the actual block stored in the storage position represented by the CA is referred to. Therefore, since the real block stored in the storage position represented by CA (CAa) of the block that should remain is not referenced by any block, the real block is deleted from the data storage unit 10. can do. Detailed functions of the data deletion function unit 14 will be described when the following operation is described.

  The above-described data deletion function unit 14 operates to delete one of the same stored blocks, but the case where a plurality of the same blocks are stored is limited to the above two patterns. It is not something. In other words, the data deletion function unit 14 is not limited to operating on the same stored block when a block having different redundancy (importance) is stored or when the block is moved, for any reason. This also operates on the same block stored in plural.

[Operation]
Next, the operation of the above-described storage system 1 will be described with reference to FIGS. First, the operation for storing data A and data B will be described with reference to the flowchart of FIG. 16 and FIGS. 17 to 19. Here, an example when data A and data B are stored will be described.

  First, it is assumed that data storage of data A is started (step S1). Then, as shown in FIG. 17, the written data A is chunked and finely divided into blocks a, b, and c (step S2). Then, the hash value of each block is calculated (step S3), and based on the hash value, it is compared with a block that has already been stored, and it is determined whether this block needs to be stored (step S4). In addition, the process from step S4 to step S6 is performed with respect to all the blocks a, b, and c (step S7).

  At this time, if the same hash value as that of the blocks a, b, and c constituting the data A does not exist, it is determined that the blocks do not overlap (No in step S4), and the blocks do not overlap. A unique block is newly stored (step S5). When the block is stored, the CA (Content Address) of the block is obtained (step S6), and this CA is stored in the storage information 30 as shown in FIG. 18 in association with the identification information of each block. To do. Here, CA1 is stored in block a, CA2 in block b, and CA3 in block c, respectively. Further, as shown in FIG. 19, each CA is stored in a CA list, and an “updated CA” field and a “flag” field are associated with each CA.

  Subsequently, it is assumed that data storage of data B is started (step S1). Then, as shown in FIG. 17, the written data B is chunked and finely divided into blocks c, d, and e (step S2). Then, the hash value of each block is calculated (step S3), and based on the hash value, it is compared with a block that has already been stored, and it is determined whether this block needs to be stored (step S4).

  At this time, if the hash value of the block c ′ of the data B exists and has the same data content as the block c of the data A that has already been stored, it is determined that such a block is duplicated (step S4: Yes). In this case, the redundancy (importance) of the overlapping blocks is compared (step S8). For example, the redundancy in the already stored data is stored in the storage information 30 in association with the data or block, and the redundancy in the newly stored data is determined in advance according to the type of data and the storage destination. Is set. If the redundancy of the block c ′ of the data B is higher than the redundancy of the block c of the already stored data A (step S9: Yes), the block c and the block c ′ are the same data. Although it is the contents, the block c ′ is separately stored as shown in FIG. 17 (step S10).

  Then, as described above, the CA of the stored block is acquired (step S11), and this CA is associated with the block identification information and stored in the storage information 30 as shown in FIG. Here, CA3 ′ is associated with the block c ′. CA3 ′ is stored in the CA list 20 as shown in FIG. 19, and this CA is provided with an “updated CA” field and a “flag” field. Further, at this time, as shown in FIG. 19, the CA list 20 includes a CA of the block c ′ in the “updated CA” column of the CA 1 which is the CA of the block c having the same contents as the newly stored block c ′. Is stored (step S12).

  If the redundancy of the block to be newly written is the same or lower than that of the stored same block (step S9: No), the block to be written is not newly written and the same stored block The CA is referred to (step S6).

  Then, the same processing as described above is performed for the other blocks d and e constituting the data B (step S7: No). In this case, since the same block is not stored (step S4: No), it is stored as it is (step S5). When the block is stored, the CA of the block is obtained (step S6), and this CA is associated with the identification information of each block and stored in the storage information 30 as shown in FIG. Here, CA4 is associated with block d, and CA5 is associated with block e. Further, as shown in FIG. 19, each CA is stored in a CA list, and each CA is provided with an “updated CA” field and a “flag” field.

  As described above, when data A and data B are stored with different redundancy, blocks a, b, c, c ′, d, and e are stored in the data storage unit 10 as shown in FIG. Are CA1, CA2, CA3, CA3 ′, CA4, and CA5, respectively. In the stored information 30, as shown in FIG. 18, each CA is associated with each block, and in the CA list 20, CA3 ′ is described in the “update CA” column of CA3 as shown in FIG. Is done.

  Next, the operation when the block c of data A is moved will be described with reference to the flowchart of FIG. 20 and FIG. First, when the data A is stored, the storage position of the block c is moved due to the convenience of the storage system (step S21). At this time, since the data A continues to refer to the block c, the block c cannot be erased unless the reference destination of the data A is changed from the block c to the block c ′. Therefore, as shown in FIG. 21, a copy of the block c is stored as a block c 'in another storage location in spite of the same block (step S22).

  Thereafter, the CA of the newly stored block c 'is acquired (step S23), and the "updated CA" column of CA3 of the block c that is the movement source in the CA list 20 is confirmed (step S24). If another CA exists in the “updated CA” column, nothing is done (step S24: Yes). On the other hand, if no information exists in the “update CA” column (step S24: No), the CA3 ′ of the block c ′ to be moved is described in the “updated CA” column (step S25).

  The reason why “updated CA” is not overwritten at the time of data movement is that, if overwritten, CA information with high redundancy may be deleted. By not overwriting, even if the same block is moved several times, the only block remaining after the area release is the first moved block. When the data is stored with a high redundancy, it is overwritten, so that only the block stored with the highest redundancy remains for the same block.

  Next, the operation when deleting data will be described with reference to the flowchart of FIG. 22 and FIGS. Here, as described above, a case will be described in which data B is deleted from the state where data A and data B are stored as shown in FIG.

  First, flags associated with all CAs in the CA list 20 are initialized to 0 (initial value) (step S31). As a result, as shown in FIG. 23, the flags CA1, CA2, CA3, CA3 ', CA4, and CA5 are all 0.

  Thereafter, the data stored at the present time, that is, the data A different from the data B to be deleted is read (step S32), and the CA (CAa) of the data is acquired (step S33). Here, the CA of each block a, b, c of data A is acquired one by one from the storage information 30 shown in FIG.

  Then, in the CA list 20, it is confirmed whether or not another CA is described in the “update CA” column of the acquired CA (CAa) (step S34). At this time, if another CA (CAb) is described in the “update CA” column of the acquired CA (CAa) (step S34: Yes), the CA (CAa) acquired in the storage information 30 is associated with the CA (CAa). The identification information of the existing block is changed so as to be associated with another CA (CAb) in the “update CA” field (step S35). That is, the reference destination of the block referring to the acquired CA (CAa) is changed from the CA (CAa) stored in the “update CA” column to another CA (CAb).

  Further, in the CA list 20, “flag” of another changed CA (CAb) is checked, and if the value is 0 (initial value) (step S36: flag = 0), the value of the “flag” Is changed to 1 (another value) (step S38). On the other hand, as described above, the value of “flag” in the CA list 20 of the CA (CAa) changed in the storage information 30 remains 0 (initial value). If the value of “flag” of another changed CA (CAb) is 1 (step S36: flag = 1), the value of “flag” remains at 1.

  Specifically, for example, when CA3 of block c of data A is acquired, as shown in FIG. 23, since “update CA” of CA3 acquired in CA list 20 has a description of CA3 ′, FIG. As shown in FIG. 5, the reference destination of the block c referred to by CA3 in the storage information 30 is changed from CA3 to CA3 ′. That is, as shown in FIG. 26, CA representing the storage position where the real block of block A of data A is stored is changed to CA3 ′. Further, as shown in FIG. 24, in the CA list 20, the value of “flag” of CA3 ′ is changed from 0 to 1. Note that the value of “flag” in CA3 remains 0.

  And the process mentioned above is performed also with respect to the other block of the data A currently stored (step S37: No). Here, the CA of each block a and b of the data A is acquired (step S33), and it is confirmed whether or not another CA is described in the “update CA” column of the acquired CA (step S34). For blocks a and b, since there is no description of other CAs in the “update CA” column of the acquired CA (CA1, CA2) (step S34: No), the “flag” of the CA (CA1, CA2) is simply set. If it is confirmed that the value is 0 (initial value) (step S36: flag = 0), the value is changed to 1 as shown in FIG. 24 (step S38).

  After the above processing is completed for the CAs of all the data blocks that are not to be deleted (step S37: Yes), the CA list 20 is examined and associated with the CA whose "flag" is 0 (initial value). The deleted block is deleted (step S39). Here, as shown in FIG. 24, since the “flag” of CA3, CA4, and CA5 is 0, the blocks c, d, and e at the storage location referred to by CA3, CA4, and CA5 are deleted.

  Thus, as shown in FIG. 26, a plurality of blocks c and c ′ stored in spite of the same block are replaced with a block c with low redundancy while leaving only the most effective block c ′ with high redundancy. Can be deleted. At this time, the reference of the data A to the deleted block c is changed to the block c ′. Accordingly, it is possible to improve the data storage efficiency by suppressing the duplicate storage of the storage target data having the same contents while maintaining the reliability of the data.

  Note that the above-described data deletion processing is not only performed when one block is deleted when the same block with different redundancy is stored, but also when the block is moved and a plurality of the same block is stored, This is also effective when a plurality of the same blocks are stored for the reason.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The outline of the configuration of the storage apparatus 50 according to the present invention will be described below with reference to FIG. The configuration of the storage system, program, and data processing method in the present invention will be described. However, the present invention is not limited to the following configuration.

(Appendix 1)
A data storage control unit 51 for storing the storage target data in the storage device 60 and generating and storing address data representing the storage position of the storage target data;
A data deletion processing unit 52 that deletes the storage target data stored in the storage device 60 from the storage device 60;
When the data storage control unit 51 separately stores in the storage device 60 other storage target data having the same content as the storage target data already stored in the storage device 60, the storage target already stored Stores the address data representing the storage location of the data in association with other address data representing the storage location of the other storage target data,
The data deletion processing unit 52 deletes the storage target data stored in the storage position represented by one of the address data among the address data associated with each other from the storage device 60.
Storage device.

(Appendix 2)
The storage device according to attachment 1, wherein
The data storage control unit stores address data representing a storage position of the storage target data in association with identification information for identifying the storage target data;
The data deletion processing unit corresponds to the identification information associated with the one address data among the address data associated with each other, and the other address data instead of the one address data. And deleting the storage target data stored in the storage position represented by the one address data from the storage device,
Storage device.

  According to the above invention, first, in the storage device, when storing the storage target data in the storage device, the address data indicating the storage position of the storage target data is set, and the address data and the identification information of the storage target data are set. Are stored in association with each other. At this time, address data is stored in a list. Thus, when the storage target data that has already been stored is read, the storage target data can be read by reading the data at the storage position represented by the address data. Further, when storing other storage target data having the same contents as the storage target data that has already been stored, the storage destination of the other storage target data represents the storage position of the storage target data that has already been stored. By setting the address data, it is not necessary to store the same data separately. Therefore, redundant storage of data can be eliminated.

  On the other hand, in some cases, it may be necessary to store the storage target data having the same contents in the storage device in duplicate. In such a case, the address data indicating the storage location of the storage target data that has already been stored is associated with the address data that refers to the storage location of other storage target data of the same content that is stored later, and stored in the address list. Keep it.

  After that, when deleting the storage target data stored in the storage device, it is checked whether other address data is associated with the address data stored in the list. In the case of association, only the storage target data stored in the storage position represented by any one of the address data is deleted from the storage device. As a result, one of the storage target data having the same contents remains in the storage device, so that data duplication can be eliminated while maintaining data reliability.

  In particular, when deleting data, the identification information of the storage target data associated with one address data is associated with the other address data instead of the one address data, and then is represented by the one address data. Delete the storage target data at the storage location. As a result, one storage target data specified by the identification information associated with one address data is stored in the storage position represented by the other address data, and the other storage target having the same contents is stored. Data will be referenced. Then, since one storage object data stored in the storage position represented by one address data is no longer referred to, the data can be deleted.

  As described above, at the time of data deletion, even when data with the same content is stored redundantly, the storage location of other storage target data with the same content is referred to for the reference destination of the actual data of the storage target data that is not deleted. By changing and setting the address data, appropriate data can be left in the storage device. Thereby, while maintaining the reliability of the stored data, it is possible to suppress the duplicate storage of the storage target data having the same contents, and it is possible to improve the data storage efficiency.

(Appendix 3)
The storage device according to attachment 2, wherein
When deleting the storage target data, the data deletion processing unit associates the other address data with the one address data representing a storage position of the storage target data different from the storage target data to be deleted The one address data associated with the identification information of the different storage target data is changed to the other address data and associated with the identification information, and the one address data Deleting the storage target data stored in the storage location represented from the storage device;
Storage device.

  According to the above invention, at the time of data deletion, one of the address data associated with the identification information of the storage target data not to be deleted among the address data associated with each other is stored. Change to the other address data shown. Thereafter, the storage target data stored in the storage position represented by the other address data is left without being deleted, and the storage target data stored in the storage position represented by the one address data is deleted. . Thereby, the actual data of the storage target data to be deleted can be replaced as storage target data that is not deleted, and appropriate data can be left in the storage device. Then, instead of deleting the actual data of the storage target data that is not scheduled to be deleted, it is possible to suppress duplicate storage of the storage target data having the same contents.

(Appendix 4)
The storage device according to attachment 3, wherein
The data deletion processing unit sets, as an initial value, a flag associated with the address data indicating the storage position of all the storage target data when deleting the storage target data, and the storage target data to be deleted When the other address data is associated with the one address data associated with the identification information of the storage target data different from the storage target data, the storage target data is associated with the identification information of the different storage target data. The one address data is changed to the other address data and associated with the identification information, the flag associated with the changed other address data is set to another value, and the initial value is The storage target data stored in the storage position represented by the address data associated with the flag is stored in the storage device. To remove from the device,
Storage device.

(Appendix 5)
The storage device according to appendix 4, wherein
The data deletion processing unit sets, as an initial value, a flag associated with the address data indicating the storage position of all the storage target data when deleting the storage target data, and the storage target data to be deleted The flag associated with the address data associated with the identification information of the storage target data different from the other is set to another value, and the one of the ones associated with the identification information of the different storage target data When the other address data is associated with the address data, the flag associated with the one address data is not set to another value and remains at the initial value.
Storage device.

(Appendix 6)
The storage device according to any one of appendices 1 to 5,
The data storage control unit is configured to store other storage target data having the same content as the storage target data already stored with respect to the importance set in the storage target data already stored in the storage device. When the importance set in advance is high, the other storage target data is separately stored in the storage device, and the address data indicating the storage location of the already stored storage target data is stored in the other storage Associating and storing other address data indicating the storage position of the target data,
Storage device.

(Appendix 7)
The storage device according to any one of appendices 1 to 6,
The data storage control unit, when moving the storage target data already stored in the storage device to another storage position, stores other storage target data that is a copy of the storage target data already stored. Separately storing in the storage device, and storing the address data indicating the storage position of the storage target data already stored in association with other address data indicating the storage position of the other storage target data,
Storage device.

(Appendix 8)
A storage device;
A data storage control unit that stores storage target data in a storage device, and generates and stores address data representing a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device;
The data storage control unit stores the already stored storage target data when separately storing in the storage device other storage target data having the same contents as the storage target data already stored in the storage device. Store the address data representing the position in association with other address data representing the storage position of the other storage target data,
The data deletion processing unit deletes the storage target data stored in the storage position represented by one of the address data among the address data associated with each other from the storage device.
Storage system.

(Appendix 9)
The storage system according to attachment 8, wherein
The data storage control unit stores address data representing a storage position of the storage target data in association with identification information for identifying the storage target data;
The data deletion processing unit corresponds to the identification information associated with the one address data among the address data associated with each other, and the other address data instead of the one address data. And deleting the storage target data stored in the storage position represented by the one address data from the storage device,
Storage system.

(Appendix 10)
In the information processing device,
A data storage control unit that stores storage target data in a storage device, and generates and stores address data representing a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device, and
The data storage control unit stores the already stored storage target data when separately storing in the storage device other storage target data having the same contents as the storage target data already stored in the storage device. Store the address data representing the position in association with other address data representing the storage position of the other storage target data,
The data deletion processing unit deletes the storage target data stored in the storage position represented by one of the address data among the address data associated with each other from the storage device.
program.

(Appendix 11)
The program according to attachment 10, wherein
The data storage control unit stores address data representing a storage position of the storage target data in association with identification information for identifying the storage target data;
The data deletion processing unit corresponds to the identification information associated with the one address data among the address data associated with each other, and the other address data instead of the one address data. And deleting the storage target data stored in the storage position represented by the one address data from the storage device,
program.

(Appendix 12)
Storing storage target data in a storage device, generating and storing address data representing a storage position of the storage target data, and other storage target data having the same contents as the storage target data already stored in the storage device Separately stored in the storage device, the address data indicating the storage position of the storage target data already stored is associated with other address data indicating the storage position of the other storage target data, and stored.
When the storage target data stored in the storage device is deleted from the storage device, it is stored in a storage position represented by one of the address data associated with each other. Deleting the storage target data from the storage device,
Data processing method.

(Appendix 13)
A data processing method according to attachment 12, wherein
When storing the storage target data in a storage device, address data representing a storage position of the storage target data is stored in association with identification information for identifying the storage target data;
When the storage target data is deleted, the identification information associated with the one address data among the address data associated with each other is replaced with the one address data instead of the one address data. Associating data and deleting the storage target data stored in the storage position represented by the one address data from the storage device;
Data processing method.

DESCRIPTION OF SYMBOLS 1 Storage system 10 Data storage part 11 Deduplication function part 12 Distributed redundant arrangement function part 13 Data movement function part 14 Data deletion function part 20 CA list 30 Storage information 100 Storage system 101 Deduplication function part 102 Distributed redundant arrangement function part 103 Data Movement function unit 104 Data deletion function unit 110 Data storage unit 120 Content address list

Claims (8)

A data storage control unit that stores storage target data in a storage device, and generates and stores address data representing a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device;
The data storage control unit stores the already stored storage target data when separately storing in the storage device other storage target data having the same contents as the storage target data already stored in the storage device. The address data representing the location is stored in association with other address data representing the storage location of the other storage target data, and the address data representing the storage location of the storage target data is identified to identify the storage target data Store it in association with the information,
The data deletion processing unit corresponds to the identification information associated with one of the address data among the address data associated with each other, and associates the other address data with the one address data. In addition, when the storage target data stored in the storage position represented by the one address data is deleted from the storage device , and the storage target data is deleted, the storage target data to be deleted The one address associated with the identification information of the different storage target data when the other address data is associated with the one address data representing the storage position of the storage target data different from The data is changed to the other address data and associated with the identification information, and the one address data is Remove the storage target data stored in the storage position represented by the said storage device,
Storage device. The storage device according to claim 1 ,
The data deletion processing unit sets, as an initial value, a flag associated with the address data indicating the storage position of all the storage target data when deleting the storage target data, and the storage target data to be deleted When the other address data is associated with the one address data associated with the identification information of the storage target data different from the storage target data, the storage target data is associated with the identification information of the different storage target data. The one address data is changed to the other address data and associated with the identification information, the flag associated with the changed other address data is set to another value, and the initial value is The storage target data stored in the storage position represented by the address data associated with the flag is stored in the storage device. To remove from the device,
Storage device. The storage device according to claim 2 ,
The data deletion processing unit sets, as an initial value, a flag associated with the address data indicating the storage position of all the storage target data when deleting the storage target data, and the storage target data to be deleted The flag associated with the address data associated with the identification information of the storage target data different from the other is set to another value, and the one of the ones associated with the identification information of the different storage target data When the other address data is associated with the address data, the flag associated with the one address data is not set to another value and remains at the initial value.
Storage device. The storage device according to any one of claims 1 to 3 ,
The data storage control unit is configured to store other storage target data having the same content as the storage target data already stored with respect to the importance set in the storage target data already stored in the storage device. When the importance set in advance is high, the other storage target data is separately stored in the storage device, and the address data indicating the storage location of the already stored storage target data is stored in the other storage Associating and storing other address data indicating the storage position of the target data,
Storage device. The storage apparatus according to any one of claims 1 to 4 ,
The data storage control unit, when moving the storage target data already stored in the storage device to another storage position, stores other storage target data that is a copy of the storage target data already stored. Separately storing in the storage device, and storing the address data indicating the storage position of the storage target data already stored in association with other address data indicating the storage position of the other storage target data,
Storage device. A storage device;
A data storage control unit that stores storage target data in a storage device, and generates and stores address data representing a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device;
The data storage control unit stores the already stored storage target data when separately storing in the storage device other storage target data having the same contents as the storage target data already stored in the storage device. The address data representing the location is stored in association with other address data representing the storage location of the other storage target data, and the address data representing the storage location of the storage target data is identified to identify the storage target data Store it in association with the information,
The data deletion processing unit corresponds to the identification information associated with one of the address data among the address data associated with each other, and associates the other address data with the one address data. In addition, when the storage target data stored in the storage position represented by the one address data is deleted from the storage device , and the storage target data is deleted, the storage target data to be deleted The one address associated with the identification information of the different storage target data when the other address data is associated with the one address data representing the storage position of the storage target data different from The data is changed to the other address data and associated with the identification information, and the one address data is Remove the storage target data stored in the storage position represented by the said storage device,
Storage system. In the information processing device,
A data storage control unit that stores storage target data in a storage device, and generates and stores address data representing a storage position of the storage target data;
A data deletion processing unit that deletes the storage target data stored in the storage device from the storage device, and
The data storage control unit stores the already stored storage target data when separately storing in the storage device other storage target data having the same contents as the storage target data already stored in the storage device. The address data representing the location is stored in association with other address data representing the storage location of the other storage target data, and the address data representing the storage location of the storage target data is identified to identify the storage target data Store it in association with the information,
The data deletion processing unit corresponds to the identification information associated with one of the address data among the address data associated with each other, and associates the other address data with the one address data. In addition, when the storage target data stored in the storage position represented by the one address data is deleted from the storage device , and the storage target data is deleted, the storage target data to be deleted The one address associated with the identification information of the different storage target data when the other address data is associated with the one address data representing the storage position of the storage target data different from The data is changed to the other address data and associated with the identification information, and the one address data is Remove the storage target data stored in the storage position represented by the said storage device,
program. Storing storage target data in a storage device, generating and storing address data representing a storage position of the storage target data, and other storage target data having the same contents as the storage target data already stored in the storage device Is stored separately in the storage device, the address data indicating the storage position of the storage target data already stored is associated with other address data indicating the storage position of the other storage target data, and stored . Storing address data representing a storage position of the storage target data in association with identification information for identifying the storage target data;
When the storage target data stored in the storage device is deleted from the storage device, one of the address data associated with each other is associated with the identification information associated with one of the address data. The one address data is associated with the other address data, and the storage target data stored in the storage position represented by the one address data is deleted from the storage device , and the deletion is performed. When the other address data is associated with the one address data that represents the storage position of the storage target data different from the target storage target data, it is associated with the identification information of the different storage target data. The one address data is changed to the other address data and associated with the identification information Remove the storage target data stored in the storage position where the expressed by one of the address data from said storage device,
Data processing method.

Images