WO2018061161A1 - Appareil de stockage et procédé de commande associé - Google Patents

Appareil de stockage et procédé de commande associé Download PDF

Info

Publication number
WO2018061161A1
WO2018061161A1 PCT/JP2016/078938 JP2016078938W WO2018061161A1 WO 2018061161 A1 WO2018061161 A1 WO 2018061161A1 JP 2016078938 W JP2016078938 W JP 2016078938W WO 2018061161 A1 WO2018061161 A1 WO 2018061161A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
area
storage
compressed
write
Prior art date
Application number
PCT/JP2016/078938
Other languages
English (en)
Japanese (ja)
Inventor
俊哉 関
竹内　久治
義裕 吉井
恭男 渡辺
高岡　伸光
和衛 弘中
智大 川口
井上　浩伸
健一 澤
純 宮下
剛士 小田
純 月岡
友輔 斉藤
Original Assignee
株式会社日立製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立製作所 filed Critical 株式会社日立製作所
Priority to PCT/JP2016/078938 priority Critical patent/WO2018061161A1/fr
Publication of WO2018061161A1 publication Critical patent/WO2018061161A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Definitions

  • the present invention relates to a storage apparatus and a control method therefor, and is suitable for application to a storage apparatus equipped with both a data compression function and a deduplication function, for example.
  • the storage apparatus has a plurality of storage devices for storing data and a storage controller for controlling these storage devices, and aims to provide a large capacity data storage space to the host computer.
  • Storage devices are required to store large amounts of data at low cost.
  • a technique for recording data by lossless compression hereinafter simply referred to as compression
  • compression a technique for recording data by lossless compression
  • Deduplication technology is another technology that reduces the amount of data to be stored.
  • the storage apparatus detects that a plurality of data having the same content exists in the storage apparatus, the deduplication technology leaves only one of them in the storage device in the storage apparatus and stores the remaining data in the storage device. This is a technique to prevent storage.
  • Patent Document 1 discloses a storage device that performs deduplication processing on data that is duplicated among data transferred from a host device and compresses data that has not been deduplicated.
  • the data compression processing unit affects the overhead when decompressing the data. For example, when data is compressed in a large capacity unit, even if only a part of the data is to be read, the data must be decompressed in that capacity unit, and extra overhead is generated. descend. On the other hand, if the data compression processing unit is made too small, the amount of management information increases, and the utilization efficiency of the storage area for storing the management information deteriorates.
  • the compression ratio changes depending on the data content. For this reason, after the storage system compresses the data and stores it in the storage device, when the update data of the data is received from the host device, the size of the update data after compression is smaller than the size of the compressed data stored in the storage device Can also be large. In this case, the compressed update data cannot be overwritten on the area where the compressed data before update is stored, and a different storage method is required.
  • the present invention has been made in view of the above points, and intends to propose a highly useful storage apparatus and control method thereof that can solve the above-mentioned various problems associated with data compression at once. .
  • a virtual volume that is a virtual storage space is provided to a host, and is dynamically stored in response to a data write request to the virtual volume from the host.
  • the storage apparatus to which the area is allocated one or a plurality of storage devices and the storage area of the storage device are dynamically allocated to the virtual volume, and the write target data given from the host together with the write request is stored in the virtual volume.
  • a storage controller that writes to the storage area allocated to the storage device, and the storage controller manages the storage area of the storage device separately into an overwrite area that permits overwriting and an additional writing area that additionally stores data.
  • An overwrite area is allocated, the data to be written is written in the allocated overwrite area, and the data written in the overwrite area is compressed in a second capacity unit smaller than the first capacity and compressed. The data is stored in the additional writing area.
  • a virtual volume that is a virtual storage space is provided to a host, and a storage area is dynamically allocated according to a data write request from the host to the virtual volume.
  • the storage apparatus dynamically allocates one or a plurality of storage devices and a storage area of the storage device to the virtual volume, and writes write target data given from the host together with the write request.
  • a storage controller that writes to the storage area allocated to the virtual volume, and the storage controller includes an overwrite area that allows overwriting, and an additional write area that additionally stores the storage area of the storage device.
  • the storage controller is configured to manage the virtual volume.
  • the capacity unit for allocating the overwrite area to the virtual volume and the capacity unit for compressing the data written to the virtual volume are divided into the second capacity that is a unit for compressing the data. Is made smaller than the first capacity, which is a unit for allocating the overwrite area to the virtual volume, while reducing the amount of extra decompression processing (overhead) when reading compressed data, An increase in management information for managing the area can be prevented. Therefore, it is possible to prevent an increase in the amount of management information while preventing a decrease in access performance of the storage apparatus due to such overhead.
  • the uncompressed data is written in the overwrite area and the compressed data is stored in the additional write area. Can be stored in the storage device without any problem.
  • FIG. 1 is a configuration example of a computer system including a storage device according to an embodiment. It is a figure showing the relationship between a virtual volume and a page. It is a list of management information. It is a structural example of a page management table. It is a structural example of a pool management table. It is a structural example of a virtual volume management table. It is a structural example of a write-once space mapping table. It is an example of a structure of LP conversion table. It is an example of composition of FPMQ and FPTD. It is a list of programs that the storage device has. It is a flowchart of a read process. It is a flowchart of a write process. It is a flowchart of a buffer update program.
  • the processing executed in the storage apparatus may be described using “program” as the subject.
  • the processor (CPU) included in the storage apparatus executes the program to perform the process described in the program. Therefore, the main subject of the process is the processor (CPU), but the explanation becomes redundant. In order to prevent this, the contents of processing may be explained using the program as the subject. Further, part or all of the program may be realized by dedicated hardware.
  • Various programs described below may be provided by a storage medium that can be read by a program distribution server or a computer, and may be installed in each device that executes the program.
  • the computer-readable storage medium is a non-transitory computer-readable medium such as a non-volatile storage medium such as an IC card, an SD card, or a DVD.
  • compression means a process of reducing the data size while maintaining the meaning of the data using a reversible compression algorithm such as the LZW algorithm.
  • data written from the host to the storage apparatus may be compressed.
  • compressed data Data that has been reduced in size as a result of compression processing performed by the storage device
  • uncompressed data data that has not been subjected to compression processing by the storage device
  • decompression The process of returning the compressed data to the original data size using the lossless compression algorithm.
  • “Volume” means a storage space provided by a target device such as a storage device or storage device to an initiator device such as a host computer.
  • the initiator device issues a data write request to a certain position (address) in the storage space, the data is stored in the storage area assigned to that address.
  • the storage apparatus provides a virtual volume formed by so-called Thin Provisioning technology to a host as a volume. In a virtual volume in its initial state (immediately after the virtual volume is defined), no storage area is allocated to each address on the virtual volume.
  • the storage apparatus receives a data write request for an address in the storage space from the initiator device (host), the storage apparatus dynamically determines a storage area to be allocated to the address.
  • Block means an area on the volume.
  • a block is an area having a size equal to the minimum access unit when an initiator device accesses a volume, and has a fixed length.
  • the block size is 512 bytes.
  • LBA Logical Block Address
  • the LBA of the first block of the volume is 0, and subsequent blocks are 1, 2,. . .
  • the serial number is attached.
  • the storage apparatus may use an address other than the LBA as an address for specifying an area on the volume. This will be described in the embodiment.
  • “Duplicate elimination process” is a process of leaving only one data in the storage apparatus and deleting other data from the storage apparatus when there are a plurality of data of the same content in the storage apparatus.
  • the process of determining whether data with the same content exists in the storage device is referred to as a “duplication determination” process.
  • the deduplication processing is processing including duplication determination processing.
  • duplication determination is performed for each data of a predetermined size called a chunk.
  • a chunk In the following embodiment, an example in which the chunk size is 8 KB will be described. However, the chunk size may be other than 8 KB. Data (or chunks) with the same content is called “duplicate data (or duplicate chunks)”.
  • an apparatus that performs duplication determination is characterized by a small size (for example, about 8 bytes) that is calculated based on data to be compared, such as a hash value that is calculated using a hash function.
  • a check code consisting of a quantity is generated, and duplication is determined using the generated check code.
  • a check code generated from data is referred to as “FingerPrint Key”. FingerPrint Key may be abbreviated as FPK.
  • FPK of data A if the value of FPK calculated from data A is H, value H is referred to as FPK of data A. Conversely, data A may be referred to as “data having FPK H”.
  • ⁇ “ Duplicate exclusion range ” means the data search range when determining the presence or absence of duplicate data.
  • the deduplication range is “storage device”
  • the storage device compares the data A with all data in the storage device when determining whether the same data as the data A exists.
  • the scope of deduplication is limited to a part of the storage devices in the storage apparatus (herein referred to as “storage groups”)
  • the storage apparatus The presence or absence of duplicate data is searched from only the data inside.
  • Updating the storage area means rewriting (overwriting) the contents of the data stored in the storage area with new contents.
  • Data that has been stored in a storage area before it is updated is referred to as “pre-update data”.
  • pre-update data data that has been stored in a storage area before it is updated
  • update data data newly written in the storage area
  • Log-structured write or “log-structured” means an operation of sequentially writing data to an unused area of a storage area.
  • a storage area for additional writing is provided.
  • data is written in order from the beginning of the storage area for additional writing.
  • the storage device stores the end address to which data has been written in the immediately preceding write process, and writes data from the address next to this end address when performing the write process.
  • FIG. 1 shows a configuration example of a computer system including a storage apparatus 1 according to this embodiment.
  • the storage device 1 includes a storage controller 10 and a plurality of storage devices 20 connected to the storage controller 10.
  • the storage device 20 is used for the storage device 1 to store write data from a host device such as the host 2.
  • a host device such as the host 2.
  • an HDD (Hard Disk Drive) 22 using a magnetic disk as a storage medium or an SSD (Solid State Drive) 21 adopting a nonvolatile semiconductor memory such as a flash memory as a storage medium is used.
  • the storage device 20 is connected to the storage controller 10 by a transmission line (SAS link) conforming to the SAS (Serial Attached SCSI) standard, a transmission line (PCI link) conforming to the PCI (Peripheral Component Interconnect) standard, or the like.
  • One or more hosts 2 are connected to the storage controller 10.
  • a management host 5 is connected to the storage controller 10.
  • the storage controller 10 and the host 2 are connected via a SAN (Storage Area Network) 3 formed using a fiber channel as an example.
  • the storage controller 10 and the management host 5 are connected via a LAN (Local Area Network) 6 formed using Ethernet (registered trademark) as an example.
  • the storage controller 10 includes at least a processor (also referred to as a CPU) 11, a host interface (also referred to as “host I / F”) 12, a device interface (also referred to as “device I / F”) 13, a cache memory 14, a management I / O. F15 and local memory 16 are provided.
  • the CPU 11, host I / F 12, device I / F 13, cache memory 14, and management I / F 15 are interconnected via an internal switch (also referred to as an internal SW) 17.
  • an internal switch also referred to as an internal SW
  • the device I / F 13 has at least an interface controller and a transfer circuit.
  • the interface controller is a component for converting a protocol (SAS in one example) used in the storage device 20 into a communication protocol (PCI-Express as an example) used in the storage controller 10.
  • the transfer circuit is used when the storage controller 10 transfers data (read, write) to the storage device 20.
  • the host I / F 12 has at least an interface controller and a transfer circuit, like the device I / F 13.
  • the interface controller of the host I / F 12 converts the communication protocol (for example, fiber channel) used in the data transfer path between the host 2 and the storage controller 10 and the communication protocol used in the storage controller 10. Is to do.
  • the CPU 11 performs various controls of the storage device 1.
  • the local memory 16 stores a program executed by the CPU 11 (referred to as a storage control program). Further, the CPU 11 uses a part of the storage area on the local memory 16 as a kind of work area when performing data compression processing.
  • a volatile storage medium such as DRAM or SRAM is used. However, as another embodiment, a nonvolatile memory may be used.
  • the cache memory 14 is used for temporarily storing I / O target data for the storage device 20. Further, it is used for storing various management information of the storage apparatus 1 used by the CPU 11.
  • a volatile storage medium such as DRAM or SRAM is used.
  • the cache memory 14 may be configured using a nonvolatile memory.
  • an auxiliary power source such as a battery may be mounted on the storage device 1 so that the stored contents of the cache memory 14 can be maintained during a power failure.
  • the CPU 11 is configured to be able to access both the local memory 16 and the cache memory 14. However, when the CPU 11 accesses the cache memory 14, it accesses via the internal switch 17. Therefore, the response time when the CPU 11 accesses the cache memory 14 is longer than the response time when the CPU 11 accesses the local memory 16.
  • the management host 5 is a computer for performing management operations of the storage apparatus 1.
  • the management host 5 includes input / output devices (not shown) such as a keyboard and a display, and a user (administrator) can issue a setting instruction to the storage apparatus 1 using the input / output devices.
  • the management host 5 can also display information such as the status of the storage apparatus 1 on an output device such as a display.
  • the storage apparatus 1 can compress data stored in the storage device 20.
  • Data compression is performed by the CPU 11 (compression is performed by executing a program code for performing data compression by the CPU 11).
  • the storage controller 10 may be provided with dedicated hardware for performing data compression, and the storage controller 10 may be configured to perform data compression using this hardware.
  • the one or more storage devices 20 included in the storage apparatus 1 according to the present embodiment provide the storage controller 10 with a storage space of a predetermined size. However, the storage controller 10 does not directly provide the storage space provided by the storage device 20 to the host 2.
  • the storage controller 10 provides the host 2 with one or more virtual storage spaces different from the storage space provided by the storage device 20. This virtual storage space is called a “virtual volume”.
  • a virtual volume (a storage space provided to the host 2) may be referred to as an “overwrite space”.
  • the virtual volume is a volume that is formed using a known Thin Provisioning technology.
  • the storage apparatus 1 receives an access request for an area on the virtual volume from the host 2, the storage apparatus 1 allocates (maps) the storage area of the storage device 20 to the area.
  • the storage device 20 that can be allocated to the virtual volume belongs to a management unit called “pool”.
  • the storage device 1 manages the storage space of the storage device 20 belonging to the pool by dividing it into partial areas of a predetermined size (42 MB as an example). In the present embodiment, this partial area is referred to as “page” or “physical page”.
  • a plurality of pools may be provided in the storage device 1. Each pool is assigned an identification number that is unique within the storage apparatus 1, and this identification number is called a pool number (pool #). A pool with pool #k is denoted as “pool #k”.
  • the storage device 1 allocates the storage area of the storage device 20 to the virtual volume, it allocates for each page. Therefore, the storage apparatus 1 manages the virtual volume by dividing it into an area having the same size as the page.
  • An area of the same size as a page on the virtual volume is called a “virtual page” or “overwrite page”.
  • the size of the virtual page is 42 MB will be described.
  • the size of the virtual page may be other than this.
  • the host 2 When the host 2 accesses the virtual volume, the host 2 issues an access request (read request, write request, etc.) to the storage device 1.
  • the read request and write request include an identifier (Logical Unit Unit Number (LUN) or the like) for specifying the virtual volume to be accessed and information for specifying an area on the virtual volume.
  • the information specifying the area on the virtual volume includes information on the LBA and the length of the area.
  • the storage apparatus 1 receives a write request from the host 2, the storage apparatus 1 specifies a virtual page to be accessed from information (LBA and area length) that specifies an area on the virtual volume included in the write request. .
  • the storage apparatus 1 selects an unused page (a page that has not been allocated to the virtual page) among the pages belonging to the pool, and sets it as the virtual page to be accessed. Assign (map) the selected page. Write data from the host 2 is stored in a page mapped to this access target virtual page.
  • the storage device 1 also stores the correspondence (mapping) between the virtual page and the page assigned to the virtual page in a mapping table (such as a virtual volume management table 300 described later with reference to FIG. 6).
  • a mapping table such as a virtual volume management table 300 described later with reference to FIG. 6.
  • FIG. 2 is a conceptual diagram showing the relationship between a virtual volume and a storage area (page) allocated to the virtual volume.
  • a rectangular object 40 shown in FIG. 2 represents a virtual volume (overwriting space).
  • the cylindrical object 50 represents a pool.
  • the storage device 1 has a storage space different from the virtual volume (overwrite space) 40.
  • This storage space is called “write-once space”.
  • a rectangular object 60 shown in FIG. 2 represents a write-once space.
  • the write-once space 60 is not a storage space that can be accessed by the host 2 but a storage space that can be used only by the storage controller 10.
  • the write-once space 60 is used for the CPU 11 of the storage controller 10 to store the compressed data in the storage device 20.
  • Each virtual page is assigned a unique identification number (a non-negative integer value is used for the identification number) called a virtual page number (sometimes referred to as VP #).
  • a virtual page with a virtual page number n (n is a non-negative integer value) is denoted as “VP # n”.
  • each physical page is given a unique identification number in the storage apparatus 1 called a physical page number (or page #) (this identification number is also a non-negative integer value).
  • page #n a physical page whose physical page number is n (n is a non-negative integer value) is denoted as “page #n”.
  • FIG. 2 an example in which page # 0 is mapped to VP # 0 is shown.
  • VP # 1 and VP # 2 will be described later.
  • the storage apparatus 1 When the storage apparatus 1 receives a write request (and write data) for an area (one or a plurality of blocks) in the virtual page from the host 2, the write data is stored in the block in the physical page mapped to the virtual page. Is done.
  • the storage apparatus 1 manages the correspondence (mapping) between virtual pages and physical pages by storing them in a virtual volume management table 300 described later with reference to FIG.
  • the storage apparatus 1 when the storage apparatus 1 according to the present embodiment receives a data write request from the head of the virtual page to the nth block (except for the case of compressed data storage described later), the data is mapped to the virtual page. Stored in the nth block from the top of the physical page. In order to avoid redundant description, in the following, the storage apparatus 1 stores data in a physical page allocated to a virtual page, and “stores data in a virtual page (or overwrite space)”. It expresses.
  • the write-once space is a storage space formed by so-called Thin Provisioning technology.
  • the storage controller 10 dynamically assigns a page (physical page) to an area on the write-once space, and data is written to the physical page assigned to the write-once space.
  • the storage device 1 manages the write-once space by dividing it into an area having the same size as the physical page. This area is called “additional page”.
  • a unique identification number called an additional page number is also attached to the additional page.
  • the additional page number may be written as an additional page #.
  • the storage device 1 manages the correspondence (mapping) between the additional page and the physical page by storing it in an additional space mapping table 350 (FIG. 7) described later. Further, in order to avoid redundant description, in the following description, the storage apparatus 1 writes data to a physical page assigned to an additional page, and “writes data to the additional page (or additional space). ".
  • the data written in the write-once space is compressed data. Further, an additional write space is provided for each virtual volume. When there are n virtual volumes, there are n additional write spaces.
  • As the physical page allocated to the write-once space an unused physical page is selected from the pool, similarly to the physical page allocated to the virtual page.
  • the physical page allocated to the write-in space is a physical page of pool #k.
  • the storage device 1 performs compression processing on data (uncompressed data) written from the host 2 to the virtual page as necessary.
  • the compressed data generated by the compression process is moved to a location different from the physical page storing the non-compressed data, specifically, to the write-once space. At this time, the presence / absence of duplicate data is also determined.
  • the storage apparatus 1 divides the area in the virtual page into 8 KB partial areas in order from the top of the virtual page, and performs data compression for each partial area.
  • this 8 KB partial area (or data stored in this partial area) is called a “chunk”.
  • a size other than 8 KB may be adopted as the size of the chunk.
  • Data generated by compressing a chunk is called a “compression chunk”.
  • the size of the compressed chunk varies depending on the data contents, but is 512 bytes at the minimum and 8 KB at the maximum. Further, the storage device 1 according to the present embodiment generates a compressed chunk having an integer multiple of 512 bytes.
  • the storage controller 10 When the storage controller 10 (the CPU 11 thereof) generates a compressed chunk, the storage controller 10 adds the compressed chunk to the additional write space.
  • the page assigned to the virtual page When all the chunks of a certain virtual page are compressed and all the compressed chunks are written to the write-once space, the page assigned to the virtual page is discarded. Specifically, the page assigned to the virtual page is changed to an unused state (a state not assigned to the virtual page). As a result, the data written to the overwrite space is moved to the additional write space.
  • a plurality of virtual page chunks may be stored in the physical page allocated to the write-once space. This is because the chunk size is reduced by the compression, and data of a plurality of virtual pages may be accommodated in one physical page.
  • the storage apparatus 1 manages the correspondence (mapping) between the chunks in the overwrite space and the compressed chunks in the write-once space by storing them in an LP conversion table 500 described later with reference to FIG.
  • the storage apparatus 1 again allocates a physical page to the virtual page in the overwrite space, expands the data moved to the write-once space, and expands the data expanded to the physical page allocated to the virtual page. You may make it write back and update (overwrite) the data on this physical page.
  • LA An address used by the storage apparatus 1 to identify a chunk on the overwrite space.
  • LA is an address assigned to each chunk.
  • LA LBA ⁇ 16 Are in a relationship.
  • the LA of the chunk located at the top of the overwrite space is 0, and the LA of the subsequent chunks are 1, 2,. . . It becomes the serial number.
  • the compressed chunk on the write-once space has a variable length and a minimum size of 512 bytes
  • an address is assigned to each 512-byte area (block) on the write-once space. This address is called PA.
  • the PA in the 512-byte area located at the head of the write-once space is 0, and the PA in the subsequent area is 1, 2,. . .
  • the serial number is used.
  • FIG. 3 shows the management information that the storage apparatus 1 has.
  • the storage apparatus 1 has a page management table 200, a virtual volume management table 300, an additional write space mapping table 350, a logical-physical conversion table (LP conversion table) 500, an FPMQ (FingerPrint Match400Queue) 400, and an FPTD (FingerPrint) on the cache memory 14.
  • Table Directory 450 and pool management information 600.
  • FIG. 4 shows an example of a page management table 200 that is information for managing pages belonging to the pool.
  • the page management table 200 is a table provided for each pool.
  • the storage apparatus 1 has n page management tables 200.
  • Each row (record) of the page management table 200 stores physical page information in the pool.
  • Each record has columns of page # (201), DEV # (202), Addr (203), and use (204).
  • Page number (201) stores the page number of the physical page. That is, the record of page # (201) n is a record for managing information of page #n.
  • a page is an area on the storage device 20.
  • DEV # (202) and Addr (203) the identification number of the storage device 20 where the physical page exists and the address in the storage device are stored.
  • the record managing the information of page # 1 stores 0 in DEV # (202) and 0x15000 in Addr (203). Therefore, page # 1 represents that it corresponds to a 42 MB area (an area equal to the size of one physical page) of the storage device 20 with the identification number 0 starting from address 0x15000.
  • Use (204) stores information indicating the usage status of the physical page.
  • “1” is stored in the use (204).
  • “0” is stored in use (204).
  • a page whose use (204) is “1” is called a page in use, and conversely, a page whose use (204) is “0” is called an unused page.
  • the identification number and address of the storage device 20 are registered in the page management table 200 and each page corresponds to a storage area in one storage device 20.
  • other pool (or page) configuration methods may be employed.
  • the storage apparatus 1 forms a single logical storage device (referred to as a logical device) from a plurality of storage devices 20 using RAID (Redundant Arrays of Independent (or Inexpensive) Disks) technology
  • a storage area of a predetermined size for example, 42 MB
  • the DEV # (202) and Addr (203) of the page management table 200 may store the logical device identification number and the address in the logical device.
  • FIG. 5 shows an example of the pool management information 600.
  • the pool management information 600 is a table for storing information about virtual volumes belonging to each pool.
  • the storage apparatus 1 causes each virtual volume to belong to one of the pools.
  • the storage apparatus 1 selects an unused physical page in the pool to which the virtual volume belongs and allocates the physical page to the virtual page.
  • each record of the pool management information 600 information on the pool number (601), virtual volume # (603), and last write position (602) is stored.
  • the pool number (601) stores the pool number.
  • Virtual volume # (603) stores the identification number (referred to as a virtual volume number) of the virtual volume belonging to the pool.
  • a plurality of virtual volume numbers are stored in the virtual volume # (603).
  • the last write position (602) corresponds to each virtual volume that belongs to the corresponding pool (each virtual volume whose virtual volume number is stored in virtual volume # (603) in the same row) and corresponds to that virtual volume.
  • the additional page number of the additional page to which the compressed chunk is written at the end in the attached additional space is stored.
  • the last write position is specified by referring to the final write position (602).
  • FIG. 6 shows an example of the virtual volume management table 300.
  • a virtual volume management table 300 is also stored in the cache memory 14.
  • Each record of the virtual volume management table 300 stores information about virtual pages in the virtual volume. Specifically, information of virtual page # (301), last update time (302), page type (303), page # (304), and pool # (305) is stored in each record.
  • the virtual page number (301) stores the virtual page number of the virtual page.
  • the page # (304) stores the page # of the physical page assigned to the virtual page.
  • Pool # (305) is the pool number of the pool to which the physical page assigned to the virtual page belongs.
  • the storage apparatus 1 stores the time when the write request is received as the last update time (302) of the virtual page.
  • page # (304) and pool # (305) have invalid values (referred to as null, specifically, values such as “ ⁇ 1” that are not used for page # or pool #).
  • null specifically, values such as “ ⁇ 1” that are not used for page # or pool #.
  • each record of the virtual volume management table 300 includes information for specifying a virtual volume (for example, a virtual volume number) in addition to the information such as the virtual page # (301) described above. ) May be included.
  • FIG. 7 shows an example of the write-once space mapping table 350.
  • the write-once space mapping table 350 is management information for managing physical pages mapped to the write-once space. Since the additional write space is provided for each virtual volume, the additional write space mapping table 350 also exists for each virtual volume. In each line's additional page # (351), additional page numbers are stored in ascending order. Each record of the additional space mapping table 350 indicates that the page specified by the page # (352) and the pool # (353) is allocated to the additional page specified by the additional page # (351). Represents. When no page is assigned to the additional writing page, null is stored in page # (352) and pool # (353) of the additional writing page.
  • the relative address of the block in which the compressed chunk is written last among the blocks in the additional write page is stored.
  • the relative address is an address of a block when the address of the first block in the page is set to 0.
  • the storage device 1 refers to the final write position (354) in the page and the final write position (354) of the pool management information 600 to write the compressed chunk. Is identified.
  • the logical-physical conversion table 500 is a table for managing the movement destination of each chunk when the chunk on the overwrite space is compressed and moved to the write-once space.
  • FIG. 8 shows an example of the logical-physical conversion table 500.
  • the logical-physical conversion table 500 may be expressed as an LP conversion table 500.
  • the record of the LP conversion table 500 represents information on the correspondence (mapping) between the chunk on the virtual volume and the recording position of the compressed chunk.
  • Each record is stored in an area of length (504) starting from the address specified by PA (503) after the 8 KB area (chunk) specified by LA (501) of the record is compressed. It represents that.
  • the storage apparatus 1 since the storage apparatus 1 according to the present embodiment performs the deduplication processing, in principle, control is performed so that a plurality of chunks having the same contents are not stored in the write-once space.
  • the outline of the deduplication processing will be described again with reference to FIG.
  • the storage apparatus 1 When the chunk A and chunk B of the VP # 1 of the virtual volume are stored in the write-once space as the compressed chunk a and the compressed chunk b, the storage apparatus 1 additionally writes the VP # 2 chunk. Assume that the process of moving to space is started. If the contents of the chunk D are the same as the chunk B, the compressed chunk of the chunk D is not written to the write-once space. Instead, the storage apparatus 1 uses the same value as the PA (503) of the record storing the chunk B information in the LP conversion table 500, and the PA of the record storing the chunk D information in the LP conversion table 500. (503).
  • FKP is information having a size of 8 bytes.
  • the FKP may have a size other than 8 bytes. A specific calculation method of FKP in the present embodiment will be described later.
  • the storage device 1 records the correspondence between the FPK generated from the chunk and the LA of the chunk in the management information.
  • This management information is called FingerPrint Match Queue (hereinafter referred to as “FPMQ”).
  • FIG. 9 shows an example of the FPMQ 400.
  • the FPMQ 400 is a table having columns of FPK (401) and LA (402), and an FPK generated from an 8 KB area (chunk) starting from LA (402) is stored in the FPK (401). Each row (record) stored in the FPMQ 400 is sorted in the ascending order of the FPK (401).
  • a record of the FPMQ 400 that is, a set of the FPK (401) and the LA (402) may be referred to as “chunk metadata” (or simply “metadata”).
  • FPMQ 400 is provided for each virtual volume. For example, when the storage device 1 moves a chunk of the virtual volume #n (overwrite space) to the write-once space (this chunk is called “chunk A”), by referring to the FPMQ 400 for the virtual volume #n, It is determined whether the same chunk (compressed chunk) as chunk A already exists in the write-once space. When the same FPK as the FPK of the chunk A does not exist in the FPMQ 400 for the virtual volume #n, it can be determined that the same chunk (compressed chunk) as the chunk A does not exist in the additional write space for the pool #n. In the storage apparatus 1 according to this embodiment, the deduplication range is a virtual volume. Therefore, the same chunk (compressed chunk) as chunk A does not exist in virtual volume #n, but if it exists in a virtual volume other than virtual volume #n, chunk A (compressed chunk) is written to virtual volume #n.
  • the storage apparatus 1 has management information that records addresses (addresses on the cache memory 14) in which some records in the FPMQ 400 are stored. .
  • This management information is called FingerPrint Table Directory.
  • FPTD FingerPrint Table Directory
  • FIG. 9 shows an example of FPTD450.
  • the FPTD 450 is also provided for each virtual volume.
  • the FPMQ storage address (452) stores the record having the smallest FPK401 among the records of the FPMQ400, in which the upper 32 bits of the FPK stored in the FPK401 are equal to the first 32 bits (451) of the FPK.
  • An address (address on the cache memory 14) is stored.
  • the FPK 401 having the smallest value among the records having the upper 32 bits of the FPK 401 being “0x00000001” is “0x0000000100000001”.
  • the address storing the record “0x0000000100000001” in the FPK 401 is “ADR2”
  • the FPTD 450 has a record in which the first 32 bits (451) of the FPK is “0x00000001” and the FPMQ storage address 452 is “ADR2”. Is recorded.
  • the deduplication program 1000 After calculating the FPK of the chunk, a process of determining whether the same value as the FPK is stored in the FPK (401) of the FPMQ 400 is performed. Since the search takes time when the number of records stored in the FPMQ 400 is very large, the FPTD 450 is used.
  • the deduplication program 1000 when searching for a record in which “0x0000000100000001” is stored, the deduplication program 1000 refers to the FPTD 450 so that the FPMQ storage address (452) with the FPK leading 32 bits (451) “0x00000001” is “ADR2”. Recognize that there is. Subsequently, the deduplication program 1000 searches the records in the FPMQ 400 for records in which “0x0000000100000001” is stored in order from the record at the address “ADR2” (this is the address on the cache memory 14).
  • the FPTD 450 is information for improving the search speed of information in the FPQ 400, and thus the FPTD 450 is not essential information.
  • the FPMQ 400 stores chunk metadata for each chunk. If there are many chunks that have generated FPK, the size of the FPMQ 400 increases, and the storage area of the cache memory 14 is consumed. In order to avoid excessive consumption of the storage area of the cache memory 14, the storage apparatus 1 according to the present embodiment performs control not to store the chunk metadata in the FPMQ 400 for chunks that satisfy a predetermined condition. Details of this control will be described later.
  • the FPK upper 32 bits (502) and FPMQ registration (505) in FIG. 8 will be described.
  • the upper 32 bits of the FPK of the chunk are stored in the FPK upper 32 bits (502) of the LP conversion table 500.
  • the FPMQ registration (505) is information indicating whether or not the chunk metadata is registered in the FPMQ 400.
  • the storage device 1 registers the metadata of the chunk specified by the LA (501) in the FPMQ 400, “1” is recorded in the FPMQ registration (505), and otherwise “0” in the FPMQ registration (505). Is recorded.
  • the FPK generated from the chunk (and the LA of the chunk) is registered in the FPMQ 400, the FPK and the LA of the chunk may be deleted from the FPMQ 400. Also in this case, the storage apparatus 1 records “0” in the FPMQ registration (505).
  • the local memory 16 is provided with operation information 300 ′ and a buffer 700.
  • the buffer 700 is a memory area used when the CPU 11 performs compression processing or decompression processing.
  • the operation information 300 ′ is information in which a part of the information stored in the virtual volume management table 300 (FIG. 6) is stored. Specifically, the operation information 300 ′ has only the virtual page # (301) and the last update time (302) among the records of the virtual volume management table 300. Therefore, the operation information 300 ′ is not shown in this specification.
  • FIG. 11 shows the flow of read processing executed by the I / O program 100 when the storage apparatus 1 receives a read request as an I / O request from the host 2.
  • FIG. 11 illustrates an example of a case where the data in the area specified by the read request does not exist in the cache memory 14 (case of cache miss).
  • Step 11 Upon receiving a read request from the host 2, the I / O program 100 converts the read destination address (LBA) included in the read request into a virtual page number.
  • LBA read destination address
  • the virtual page number of the read destination area is p will be described.
  • the read destination area specified in the read request does not extend over a plurality of virtual pages will be described.
  • Step 12 The I / O program 100 refers to the virtual volume management table 300 and determines whether the page type (303) of the virtual page number (p) obtained in Step 11 is “1”. If the page type (303) is “1” (step 12: Yes), then step 13 is performed. If not (step 12: No), then step 15 is performed.
  • Step 13 The I / O program 100 refers to the virtual volume management table 300 and identifies the page number (304) of the physical page assigned to the virtual page (VP # p). Further, the I / O program 100 refers to the page management table 200 to identify the DEV # (202) and Addr (203) of the storage device 20 in which the physical page exists.
  • Step 14 The I / O program 100 reads data from the storage device 20 specified in step 13, returns the read data to the host 2, and ends the processing. Note that the data read here may be stored in the cache memory 14. Then, when the read request for the data is received again later, the I / O program 100 can return the data to the host 2 without accessing the storage device 20.
  • Step 15 When step 15 is executed, since the data in the area specified by the read request has been moved to the additional write space, the I / O program 100 stores the read target data (compressed chunk). It is necessary to calculate the address (PA) on the write-once space.
  • PA address
  • the chunk I / O program 100 converts the read destination address (LBA) included in the read request into LA.
  • Step 16 The I / O program 100 refers to the LP conversion table 500, identifies a record in which LA (501) is equal to the LA identified in step 15, and identifies PA (502) and Length (503) of the record. To do. Further, the I / O program 100 calculates the additional page number from the identified PA, and refers to the additional space mapping table 350 to thereby determine the page # of the physical page assigned to the identified additional page. (352) and specify the pool # (353). Further, the I / O program 100 refers to the page management table 200 to identify the DEV # (202) and Addr (203) of the storage device 20 in which the physical page exists.
  • Step 17 The I / O program 100 reads the data (compressed chunk) requested to be read from the storage device 20 specified in Step 16, stores the data in the buffer 700, and decompresses the compressed chunk stored in the buffer 700.
  • the data decompression process to be returned to the host 2 is executed. Details of the data decompression process will be described later. Thereafter, the I / O program 100 ends the process. Note that the data read here may be stored in the cache memory 14.
  • Step 101 Upon receiving a write request from the host 2, the I / O program 100 secures an area on the cache memory 14 in order to receive data to be written by the write request.
  • the I / O program 100 stores the write target data in the secured area. Since this is a process performed also in a known storage device, detailed description thereof is omitted.
  • the I / O program 100 converts the address (LBA) of the write destination area specified in the write request into a virtual page #.
  • LBA address
  • Step 102 The I / O program 100 responds to the host 2 that the write processing has been completed.
  • Step 103 The I / O program 100 updates the last update time (p) of the record whose virtual page # (301) is p among the records of the operation information 300 ′ (FIG. 10) held in the local memory 16 (FIG. 10). 302) stores the current time (the time when step 103 is executed).
  • the I / O program 100 writes the data stored in the cache memory 14 to the storage device 20 at a predetermined opportunity (step 110).
  • This process is called destage.
  • data written to a virtual page (assuming this is VP # p) is destaged, if a physical page is allocated to the virtual page, the data is de-stored in the storage device 29 where the allocated physical page exists. Stage. If no physical page is allocated, the physical page is allocated to the virtual page before destage.
  • the I / O program 100 adds data to the write-once space. Details of this processing will be described later.
  • step 110 need not be performed immediately after step 103, and may be performed at any time. For example, it may be executed when the load on the CPU 11 of the storage device 1 is low, or when the amount of data stored in the cache memory 14 exceeds a predetermined threshold.
  • the buffer update program 150 is a program that is periodically executed.
  • the buffer update program 150 When the execution of the buffer update program 150 is started, the buffer update program 150 reflects the contents of the operation information 300 ′ in the virtual volume management table 300. The flow of processing will be described with reference to FIG.
  • Step 151 The buffer update program 150 selects one record of the operation information 300 '.
  • the value of the virtual page # (101) of this record is p will be described as an example.
  • Step 152 The buffer update program 150 records the last update time of the record of the selected operation information 300 ′ at the last update time (302) of the record whose virtual page # (301) is p in the virtual volume management table 300. .
  • Step 153 The buffer update program 150 deletes the record of the operation information 300 'selected in Step 151 from the local memory 16.
  • Step 154 When the processing up to step 153 is completed for all the records of the operation information 300 '(step 154: Yes), the processing is terminated. When the record of the operation information 300 ′ still remains in the local memory 16 (Step 154: No), the buffer update program 150 repeats the processing from Step 151.
  • the storage apparatus 1 Since the storage apparatus 1 according to this embodiment performs access via the internal switch 17 when the CPU 11 accesses the cache memory 14, the access performance (response time) is the performance when the CPU 11 accesses the local memory 16. It is not good (becomes longer) than. Therefore, the storage apparatus 1 according to the present embodiment temporarily records the information of the last update time in the operation information 300 ′ on the local memory 16, and regularly reflects the contents on the virtual volume management table 300 on the cache memory 14. Take control.
  • the storage apparatus 1 when the storage apparatus 1 has only the local memory 16 and the virtual volume management table 300 is stored in the local memory 16, it is not necessary to perform such control, and I / O
  • the program 100 may directly update the last update time (302) of the virtual volume management table 300 in step 103.
  • the storage apparatus 1 has only the cache memory 14 and all the information is stored in the cache memory 14, it is not necessary to perform the control described above, and the I / O program 100 is executed in step 103.
  • the virtual volume management table 300 on the cache memory 14 may be updated.
  • the deduplication program 1000 performs a process of moving data written in the overwrite space from the host 2 (this data is stored in a physical page once assigned to the overwrite space) to a page assigned to the additional write space. It is a program. As described above, when data (chunk) is moved to a page allocated to the write-once space, the chunk is compressed. Deduplication processing may be performed.
  • Step 1010 The deduplication program 1000 refers to the virtual volume management table 300 and selects a virtual page for moving data to the write-once space.
  • the virtual page selection criteria may be arbitrary. As an example, a virtual page with a page type (303) of “1” is selected such that the difference between the current time and the last update time (302) is equal to or greater than a predetermined threshold and the last update time (302) is the oldest. Good. Since the data (chunk) stored in the write-once space is compressed, the response time at the time of access becomes long. Therefore, it is preferable that data with low access frequency is arranged in the write-once space rather than data with high access frequency.
  • Step 1020 The deduplication program 1000 reads the data of the virtual page selected in Step 1010 from the storage device 20 onto the cache memory 14. If such virtual page data already exists on the cache memory 14, this step 1020 need not be executed.
  • Step 1030 The deduplication program 1000 selects a chunk from the data read onto the cache memory 14 in step 1020.
  • the method for selecting chunks is arbitrary. For example, you may make it select in order from the chunk with small LA.
  • Step 1040 The deduplication program 1000 transfers the chunk selected in step 1030 (or step 1080 described later) to the buffer 700.
  • Step 1050 The deduplication program 1000 calculates the FPK of the chunk transferred to the buffer 700, stores it in the LP conversion table 300, and generates a compressed chunk by compressing the data of the chunk. Details of this processing (hereinafter referred to as FPK calculation / data compression processing) will be described later.
  • the compressed chunk generated here is temporarily stored in the buffer 700.
  • Step 1060 In step 1050, it is determined whether or not data compression is successful. If the data compression is successful (step 1060: Yes), step 1070 is performed next. If not (step 1060: No), step 1080 is performed next.
  • Step 1070 In this step, the deduplication program 1000 stores the compressed chunk in the write-once space 60. In addition, deduplication processing is also performed as necessary. Details of this processing will be described later.
  • Step 1080 The deduplication program 1000 selects the next chunk (chunk that has not yet been processed in steps 1040 to 1070) from the data read onto the cache memory in step 1020.
  • Step 1090 If the next chunk cannot be selected in Step 1080, that is, if the processing of Step 1040 to Step 1070 for all the chunks is completed (Step 1090: Yes), Step 1100 is performed next. If the next chunk can be selected in Step 1080 (Step 1090: No), the deduplication program 1000 performs the processing from Step 1040 again.
  • Step 1100 The deduplication program 1000 discards the data in the overwrite space. Specifically, the deduplication program 1000 updates the contents of the virtual volume management table 300 so that the page allocated to the virtual page selected in step 1010 is not allocated to the virtual page (this virtual The page type (303) of the page is “0”, and “null” is stored in page # (304) and pool # (305). Further, the deduplication program 1000 sets the value of the use (204) of the allocated physical page of the records of the page management table 200 for this physical page to make the allocated physical page unused. Is updated to “0”.
  • step 1070 the details of the processing performed in step 1070 will be described with reference to FIG.
  • Step 6010 The deduplication program 1000 searches whether the same FPK as the FPK obtained in step 1050 is stored in the FPMQ 400. As described above, the deduplication program 1000 searches the FPMQ 400 record in which the same FPK as the FPK obtained in step 1050 is stored by referring to the FPTD 450 and the FPMQ 400.
  • Step 6020 If the same FPK as the FPK obtained in step 1050 is stored in the FPMQ 400 (step 6020: Yes), the deduplication program 1000 next performs step 6030. If the same FPK as the FPK obtained in step 1050 is not stored in the FPQM 400 (step 6020: No), step 6070 is performed next.
  • Step 6030 The deduplication program 1000 refers to the FPMQ 400, the LP conversion table 500, the write space mapping table 350, and the page management table 200, so that a chunk (compression chunk) having the same FPK as the FPK obtained in Step 1050 is obtained. Are read from the storage device 20.
  • the deduplication program 1000 first refers to the FPMQ 400 and the LP conversion table 500 to determine the PA in which the chunk (compression chunk) having the same FPK as the FPK obtained in step 1050 is stored and its length. Identify.
  • this chunk is referred to as “overlapping candidate chunk”, and the PA of the overlapping candidate chunk is referred to as “PA1”.
  • the deduplication program 1000 identifies the page number of the page (physical page) where the duplication candidate chunk is stored by referring to the write-once space mapping table 350, and further refers to the page management table 200.
  • the storage device 20 and the address where the page storing the duplicate candidate chunk exists are specified. This is the same processing as step 16 performed by the I / O program 100.
  • the duplicate elimination program 1000 reads duplicate candidate chunks from the specified storage device 20.
  • Step 6040 The deduplication program 1000 compares the chunk read in step 6030 (compressed chunk) with the contents of the chunk compressed in step 1060, and determines whether or not they are the same. At the time of comparison, both are in a compressed state, but the deduplication program 1000 compares the compressed chunks in byte units. However, as another embodiment, the deduplication program 1000 may temporarily decompress the compressed chunk and compare the decompressed data. As a result of the comparison, if the two match (step 6040: Yes), step 6050 is performed next. When both are different (step 6040: No), step 6070 is performed next.
  • Step 6050 The deduplication program 1000 registers the chunk FPK and the chunk LA in the FPMQ 400. Further, the deduplication program 1000 stores “1” in the FPMQ registration (505) of the record whose LA (501) matches the LA of the processing target chunk among the records of the LP conversion table 500.
  • Step 6060 The duplicate elimination program 1000 stores the PA (503) of the duplicate candidate chunk in the PA (503) of the record whose LA (501) matches the LA of the processing target chunk among the records of the LP conversion table 500. Then, the length of the duplication candidate chunk is stored in Length (504). In this case, the compressed chunk is not stored in the write-once space. Thereafter, the deduplication program 1000 performs the processing after step 1080.
  • Step 6070 The processing content of this step is the same as step 6050.
  • Step 6080 The deduplication program 1000 writes the compressed chunk temporarily stored in the buffer 700 to the write-once space, and deletes the compressed chunk in the buffer 700.
  • the deduplication program 1000 when the deduplication program 1000 writes a compressed chunk in the write-once space, it additionally writes it in the write-once space.
  • the flow of the process of adding the compressed chunk to the additional writing space will be described with reference to FIG.
  • the deduplication program 1000 determines the address (PA) on the write-once space where the compressed chunk is to be added (step 51).
  • the storage apparatus 1 holds information on the final write position of the write space in the pool management information 600 and the write space mapping table 350.
  • the deduplication program 1000 refers to the last write position (602) of the corresponding virtual volume in the pool management information 600, thereby adding the last page to be written in the write space corresponding to the virtual volume. Specify writing page #. Hereinafter, this is referred to as AP1.
  • the deduplication program 1000 refers to the additional write space mapping table 350 to identify the final write position (354) in the page of the record of the additional write page # (351). Hereinafter, this is denoted as RA1.
  • the deduplication program 1000 calculates “AP1 ⁇ page size (42 MB) ⁇ 512 + RA1 + 1” to obtain an address (PA) on the additional write space where the compressed chunk is to be written.
  • PA address
  • the deduplication program 1000 determines whether a physical page is allocated to the additional page including the PA (PA2) determined in step 51 (step 52). For this purpose, the deduplication program 1000 obtains the additional page # of the additional page including PA2 by dividing PA2 by the page size (42 MB) (hereinafter, the additional page # obtained here is “ AP2 "). Further, the deduplication program 1000 refers to the write space mapping table 350, so that a valid value (a value other than null) is stored in the page # (352) of the record of AP2 for the write page # (351). judge. When a valid value is stored in page # (352), a physical page is assigned to the additional page including PA2.
  • a valid value is stored in page # (352)
  • a physical page is assigned to the additional page including PA2.
  • the deduplication program 1000 allocates the physical page to the additional page (step 53). Specifically, the deduplication program 1000 selects one unused physical page from the page management table 200, and each page # and the pool # of the pool to which the page belongs are added to the additional page # (351) as AP2. Are stored in page # (352) and pool # (353). The deduplication program 1000 stores “1” in the use (204) of the physical page selected from the page management table 200. Conversely, if a physical page is assigned to PA2 (step 52: Yes), step 53 is not performed.
  • step 54 the deduplication program 1000 refers to the write space mapping table 350 and the page management table 200 to calculate the address of the storage device 20 to which the compressed chunk is written. Subsequently, the deduplication program 1000 writes the compressed chunk at the address calculated in step 54 (step 55). In step 56, the deduplication program 1000 substitutes the additional write page # of the additional write page in which the compressed chunk has been written this time for the final write position (602) of the pool management information 600, and in the last written write page. Is substituted into the last write position (354) in the page of the additional write space mapping table 350, and the process ends.
  • step 6080 The above is the processing performed in step 6080. Returning to the description of FIG.
  • Step 6090 The deduplication program 1000 stores the compression chunk PA (that is, PA2) in the PA (502) of the record whose LA (501) matches the LA of the processing target chunk among the records of the LP conversion table 500. , Length (503) stores the length (L2) of the compressed chunk. Thereafter, the deduplication program 1000 performs the processing after step 1080.
  • Step 5010 The I / O program 100 selects data to be processed from each data stored in the cache memory 14.
  • There are various methods for selecting data As an example, there is a method of selecting data with the oldest time written from the host 2. In this case, the storage controller 10 needs to store the time written from the host 2 for each data stored in the cache memory 14.
  • a plurality of chunks for example, a plurality of chunks continuous on the virtual volume may be selected.
  • Step 5020 The I / O program 100 specifies the virtual page to which the data selected in Step 5010 is written. Hereinafter, an example in which VP # p is specified in step 5020 will be described. Then, the I / O program 100 refers to the virtual volume management table 300 and determines whether the page type (303) is “0” or “1” for the record whose virtual page # (301) is “p”. If the page type (303) is “1” (step 5020: No), the data write destination is the overwrite space, so step 5110 is executed next. If the page type (303) is “0” (step 5020: Yes), then step 5030 is performed.
  • Step 5030 When this step is executed, it means that the data written in VP # p has been moved to the write-once space. In this case, the I / O program 100 determines whether the data of VP # p that has been moved to the write-once space is returned to the overwrite space again.
  • the I / O program 100 determines to return the data of VP # p that has been moved to the write-once space to the overwrite space again.
  • the I / O program 100 determines to return the data of VP # p that has been moved to the write-once space to the overwrite space again.
  • this determination method is only an example, and it may be determined whether or not the data of VP # p that has been moved to the write-once space is returned to the overwrite space by another determination method. If the determination at step 5030 is negative, then step 5040 is performed, and if the determination at step 5030 is positive, then step 5210 is performed.
  • Step 5040 The I / O program 100 compresses the processing target data. This is the same as the processing performed in steps 1040 to 1060 in FIG. Similar to steps 1040 to 1060, the I / O program 100 also calculates FPK.
  • Step 5050 The I / O program 100 stores the processing target data (compressed chunk) in the write-once space. This process is the same as step 1080 in FIG. That is, the processing from step 6010 to step 6090 in FIG. 15 is performed. After step 5050, the processing target data is deleted from the cache memory 14, and the processing target data and its compressed chunk are deleted from the buffer 700, and the destage processing ends.
  • Step 5210 When it is determined that the data of VP # p that has been moved to the write-once space is returned to the overwrite space, the I / O program 100 stores the data of VP # p that has been moved to the write-once space. All are read from the storage device 20, decompressed data, and temporarily stored in the cache memory 14. For this, the same processing as that described with reference to FIG. 11 may be performed.
  • Step 5220 The I / O program 100 allocates a physical page to VP # p. Specifically, the I / O program 100 selects one unused physical page from the page management table 200, and each of the page # and the pool # of the pool to which the page belongs are stored in the virtual volume management table 300. The virtual page # (301) is stored in the page # (304) and the pool # (305) of the record of p. The I / O program 100 stores “1” in the use (204) of the physical page selected from the page management table 200.
  • Step 5230 The I / O program 100 stores the data read in Step 5210 in the physical page assigned to VP # p. Specifically, the I / O program 100 refers to the page management table 200 as in Step 13 of FIG. 11, and the DEV # (202) of the storage device 20 in which the physical page allocated to VP # p exists. , Addr (203) is specified. The I / O program 100 stores the data read in step 5210 in the specified storage device.
  • Step 5240 The I / O program 100 overwrites the processing target data on the physical page assigned to VP # p, and ends the destage processing.
  • Step 5110 The I / O program 100 allocates a physical page if no physical page is allocated to VP # p. This is the same processing as in step 5220. However, if a physical page has already been allocated to VP # p, it is not necessary to allocate a physical page.
  • Step 5120 Similar to step 5240, the I / O program 100 overwrites the processing target data on the physical page assigned to VP # p, and ends the destage processing.
  • the destage processing described here is an example, and processing different from the processing described above may be performed.
  • the write-once space may be necessarily added without performing the determination in step 5030.
  • control when returning data from the write-once space to the overwrite space, control may be performed to reduce the amount of data to be returned. For example, when a request to update the entire page of VP # p is received from the host 2, data for one page of VP # p is stored in the cache memory 14. In this state, even if data is returned from the write-once space to the overwrite space, if the data stored in the cache memory 14 is destaged, the data written back from the write-once space to the overwrite space is overwritten. Therefore, in such a case, it is more efficient to prevent data from being returned from the write-once space to the overwrite space.
  • DIF Data Integrity Field
  • Step 7010 Each guarantee added to every 512 bytes of data 1100 of the chunk (hereinafter referred to as a target chunk) transferred to the buffer 700 (FIG. 10) of the local memory 16 (FIG. 10) in Step 1040 of FIG. As shown in FIG. 19B, zero padding is executed to fill the LA 1220 of the code 1200 with “0” (replace it with “0”).
  • the reason why the LA 1220 of the target chunk is zero-patented is that the value of the LA 1120 differs depending on the data writing position. Therefore, when the compression processing is performed on the data 1100 with the LA 1120 added to the write target data 1100, This is because even if the data has the same contents, different compression results are obtained, and the same data cannot be deduplicated. Further, when the LA 1220 is compressed without zero padding, the compression effect may be reduced. Therefore, in the present embodiment, when the write target data 1100 is compressed, first, a process of zero-padding the LA 1220 is executed. Thus, the compression effect is expected to be improved by zero-padding LA1220.
  • a predetermined hash function for example, SHA-1
  • Step 7030 As shown in FIG. 19C, the deduplication program 1000 deletes all CRCs 1210 assigned to each 512 bytes of the data of the target chunk stored in the buffer 700 from the data of the target chunk. .
  • Step 7040 The deduplication program 1000 compresses the data of the target chunk stored in the buffer 700 (including the zero-padded LA 1220 represented by reference numeral 1220 ′ in FIG. 19C). Generate compressed chunks. The compressed chunk is temporarily stored in the buffer 700 together with the data before the target chunk is compressed.
  • the reason for compressing the data of the target chunk while adding the zero-padded LA 1220 is to improve the performance.
  • the LA 1220 is deleted from the data of the target chunk and left-justified, it is necessary to create a gap for re-inserting the LA 1220 for every 512 bytes of the decompressed data after decompressing the data.
  • the LA 1220 is not deleted, it is only necessary to rewrite the LA 1220 after decompressing the read data, and the overhead required for memory copy can be reduced.
  • Step 7050 The deduplication program 1000 decompresses the compressed chunk obtained in step 7040.
  • Step 7060 The deduplication program 1000 uses the data obtained by decompressing the compressed chunk in Step 7050 (hereinafter referred to as decompressed data), the data before compression of the target chunk stored in the buffer 700, and Compare This comparison is performed by sequentially comparing the decompressed data and the data before compression of the target chunk in units of 1 bit or 1 byte. By such comparison, it is possible to confirm whether or not the original data can be restored when the data of the target chunk is compressed and decompressed, and it is possible to ensure reliability with respect to data compression and decompression.
  • Step 7070 As a result of the comparison in Step 7060, when the decompressed data matches the data before compression of the target chunk (Step 7070: Yes), Step 7080 is performed next, otherwise (Step 7070: No) Next, step 7100 is performed.
  • Step 7080 The deduplication program 1000 calculates a new CRC for the compression chunk of the target chunk stored in the buffer 700, and the calculated CRC and the logical address of the compression chunk on the buffer 700 (hereinafter referred to as LA ′). To the compressed chunk.
  • the deduplication program 1000 divides the compressed chunk of the target chunk into 512 bytes, and as shown in FIG. 19D, the data of these 512 byte compressed chunks (hereinafter referred to as compressed chunk classified data). ) For each 1110, calculate the CRC 1230 of the compressed chunk classification data 1110. The deduplication program 1000 uses the compressed chunk partition data corresponding to the CRC 1230 for each compressed chunk partition data 1110 calculated in this way and the logical address (LA ′) 1240 of the compressed chunk partition data 1110 on the buffer 700. It is added to 1110. Note that LA ′ 1240 added to the compressed chunk partition data 1110 at this time, as shown in FIG. 19 (E), when the compressed chunk partition data 1110 of the target chunk is subsequently written to the additional write space (see FIG. 19). 16 step 55), the address (LA ′′) 1250 in the write-once additional write space is rewritten.
  • Step 7090 The deduplication program 1000 registers the FPK of the target chunk calculated in step 7020 in the FPK upper 32 bits (502) of the corresponding record in the LP conversion table 500 (FIG. 8), and thereafter ends this processing. .
  • Step 7100 It is determined that the data of the target chunk is handled without being compressed, and thereafter, this processing is terminated.
  • Step 8010 The I / O program 100 compresses the read target data (based on the DEV # (202) and Addr (203) of the storage device 20 specified in step 16 of the read process described above with reference to FIG. (Compressed data) is read from the storage device 20 into the buffer 700.
  • Step 8020 The I / O program 100 determines the integrity of the compressed data read out to the buffer 700 based on the CRC included in every 512 bytes (corresponding to the new CRC 1230 in FIG. 19E).
  • a CRC check to be checked is executed (502). Specifically, as shown in FIG. 21 (A), the I / O program 100, for every 512 bytes of each compressed chunk constituting the read compressed data (that is, for each compressed chunk division data 1110 described above) The CRC is calculated based on the compressed chunk partition data 1110, and the calculated CRC is compared with the CRC 1230 added to the compressed chunk partition data 1110.
  • Step 8030 As a result of the CRC check in Step 8020, the I / O program 100 adds one of the compressed chunk partition data to the CRC calculated based on the compressed chunk partition data 1110 and the compressed chunk partition data 1110. It is determined whether or not a CRC error that does not match the CRC 1230 has occurred. If the CRC calculated based on the compressed chunk partition data 1110 and the CRC 1230 added to the compressed chunk partition data 1110 match for all the compressed chunk partition data 1110 (step 8030: Yes). Next, Step 8100 is performed, and the CRC calculated based on the compressed chunk partition data 1110 does not match the CRC 1230 added to the compressed chunk partition data 1110 for any one of the compressed chunk partition data. If this is the case (step 8030: No), then step 8040 is performed.
  • Step 8040 As shown in FIG. 21B, the I / O program 100 adds a CRC 1230 that is added to every 512 bytes of the compressed data (that is, every compressed chunk division data 1110) from the compressed data read in Step 8010. And LA ′′ are deleted, and the compressed data is decompressed.
  • the I / O program 100 uses the hash function used when calculating the FPK in step 7020 of the FPK calculation / data compression processing from the CRC group for each chunk obtained in this way to obtain a 20-byte hash value. For each chunk, the upper 8 bytes of the calculated hash value are extracted as the FPK of the chunk.
  • Step 8060 For each chunk, the FPK generated in step 8050 is compared with the FPK stored in the FPK upper 32 bits (502) of the LP conversion table for the chunk.
  • Step 8070 For all chunks, if the FPK generated in step 8050 matches the FPK stored in the FPK upper 32 bits (502) of the LP conversion table 500 for that chunk (step 8070: Yes), Next, Step 8080 is performed. If not (Step 8070: No), Step 8120 is performed next.
  • Step 8080 The I / O program 100, as shown in FIG. 21C, provides an error correction code for each 512 bytes of the decompressed data calculated in the decompression process of Step 8040 (hereinafter referred to as CRC (Cyclic Redundancy Code)). 1260 is added to the corresponding 512-byte data (step 8080).
  • CRC Cyclic Redundancy Code
  • the I / O program 100 also stores in the LA 1270 storing the data that the host 2 recognizes LA ′′ (see FIG. 19E) added every 512 bytes of the data. Write back.
  • Step 8090 The I / O program 100 returns the decompressed data 1120 to which the CRC 1260 and LA 1270 are added every 512 bytes in step 8080 to the host 2 and ends the process.
  • Step 8100 The I / O program 100 executes an error response for notifying the host 2 that has transmitted the read request that a read error has occurred, and ends the process.
  • an overwrite space is allocated to a virtual volume in units of 42 MB virtual pages, and data written to the virtual volume is chunked in 8 KB. Since compression is performed in units, decompression processing of the compressed data is performed in units of 8 KB chunks. For example, compared to data compression in units of virtual pages, even unnecessary data at the time of decompression processing of compressed data It is possible to significantly reduce the amount of extra overhead that is extended. Even if the data compression unit is 8 KB, the virtual volume can be managed in units of 42 MB virtual pages. Thus, it is possible to prevent an increase in the amount of management information while preventing a decrease in access performance of the storage apparatus due to such overhead.
  • the storage device 1 since the uncompressed data is written in the overwrite area and the compressed data is stored in the additional write area, the data size after compression of the updated data is larger than that before the update. Even when it becomes larger, it can be stored in the storage device 20 without any problem.
  • the present invention can be applied to a storage apparatus equipped with a data compression function.
  • SYMBOLS 1 ... Storage device, 2 ... Host, 10 ... Storage controller, 11 ... CPU, 14 ... Cache memory, 16 ... Local memory, 20 ... Storage device, 40 ... Overwrite space, 60 ... Additional Write space, 100 ... I / O program, 700 ... Buffer, 500 ... LP conversion table, 1000 ... Deduplication program.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention a pour objectif de fournir un appareil de stockage extrêmement utile ainsi qu'un procédé de commande associé. À cet effet, l'invention concerne un appareil de stockage comprenant un ou plusieurs dispositifs mémoire ainsi qu'une unité de commande de stockage permettant d'attribuer de façon dynamique des zones de mémoire des dispositifs mémoire à des volumes virtuels et d'écrire les données devant être écrites qui sont fournies par un hôte conjointement avec une demande d'écriture dans les zones de mémoire attribuées aux volumes virtuels. L'unité de commande de stockage gère séparément les zones de mémoire des dispositifs mémoire pour une zone d'écrasements permettant des écrasements et une zone d'écritures supplémentaires permettant de stocker des postscripts, attribue la zone d'écrasements en unités d'une première taille prescrite à des zones dans les volumes virtuels, écrit les données devant être écrites dans la zone d'écrasements attribuée, compresse les données écrites dans la zone d'écrasements en unités d'une seconde taille inférieure à la première taille, puis stocke les données compressées dans la zone d'écritures supplémentaires.
PCT/JP2016/078938 2016-09-29 2016-09-29 Appareil de stockage et procédé de commande associé WO2018061161A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/078938 WO2018061161A1 (fr) 2016-09-29 2016-09-29 Appareil de stockage et procédé de commande associé

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/078938 WO2018061161A1 (fr) 2016-09-29 2016-09-29 Appareil de stockage et procédé de commande associé

Publications (1)

Publication Number Publication Date
WO2018061161A1 true WO2018061161A1 (fr) 2018-04-05

Family

ID=61760343

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/078938 WO2018061161A1 (fr) 2016-09-29 2016-09-29 Appareil de stockage et procédé de commande associé

Country Status (1)

Country Link
WO (1) WO2018061161A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022070104A (ja) * 2020-10-26 2022-05-12 株式会社日立製作所 データ圧縮装置及びデータ圧縮方法
JP7475319B2 (ja) 2021-11-16 2024-04-26 株式会社日立製作所 ストレージシステム及びストレージシステムにおけるデータ処理方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006259962A (ja) * 2005-03-16 2006-09-28 Hitachi Ltd ストレージシステム
WO2015087424A1 (fr) * 2013-12-12 2015-06-18 株式会社日立製作所 Dispositif de mémoire et procédé de commande de dispositif de mémoire
WO2015145647A1 (fr) * 2014-03-27 2015-10-01 株式会社日立製作所 Dispositif de stockage, procédé de traitement de données et système de stockage
WO2016088258A1 (fr) * 2014-12-05 2016-06-09 株式会社日立製作所 Système de stockage, programme de sauvegarde, et procédé de gestion de données

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006259962A (ja) * 2005-03-16 2006-09-28 Hitachi Ltd ストレージシステム
WO2015087424A1 (fr) * 2013-12-12 2015-06-18 株式会社日立製作所 Dispositif de mémoire et procédé de commande de dispositif de mémoire
WO2015145647A1 (fr) * 2014-03-27 2015-10-01 株式会社日立製作所 Dispositif de stockage, procédé de traitement de données et système de stockage
WO2016088258A1 (fr) * 2014-12-05 2016-06-09 株式会社日立製作所 Système de stockage, programme de sauvegarde, et procédé de gestion de données

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022070104A (ja) * 2020-10-26 2022-05-12 株式会社日立製作所 データ圧縮装置及びデータ圧縮方法
JP7282066B2 (ja) 2020-10-26 2023-05-26 株式会社日立製作所 データ圧縮装置及びデータ圧縮方法
JP7475319B2 (ja) 2021-11-16 2024-04-26 株式会社日立製作所 ストレージシステム及びストレージシステムにおけるデータ処理方法

Similar Documents

Publication Publication Date Title
JP6553566B2 (ja) メモリシステムおよび制御方法
US20180173632A1 (en) Storage device and method for controlling storage device
JP6685334B2 (ja) ストレージ装置
JP6608468B2 (ja) ストレージ装置及びその制御方法
WO2015162758A1 (fr) Système de stockage
JP6678230B2 (ja) ストレージ装置
JP6677740B2 (ja) ストレージシステム
WO2015162681A1 (fr) Système de stockage et procédé de commande de dispositif de stockage
JP6429963B2 (ja) ストレージ装置及びストレージ装置の制御方法
US20120102260A1 (en) Storage apparatus and data control method
US11928053B2 (en) System garbage collection method and method for garbage collection in solid state disk
JP6298932B2 (ja) ストレージ装置
US20180307440A1 (en) Storage control apparatus and storage control method
US11042316B1 (en) Reordered data deduplication in storage devices
WO2015162755A1 (fr) Dispositif de stockage pour compresser et stocker des données
WO2018061161A1 (fr) Appareil de stockage et procédé de commande associé
US11079956B2 (en) Storage system and storage control method
JP6419662B2 (ja) ストレージシステム及びデータ重複検出方法
WO2020052216A1 (fr) Procédé de récupération d'espace mémoire de système et procédé de récupération d'espace mémoire dans un disque dur à semi-conducteurs
WO2018051446A1 (fr) Système informatique comprenant un système de stockage avec fonction optionnelle de traitement de données, et procédé de commande de stockage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16917713

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16917713

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP