WO2015118680A1 - Dispositif de stockage - Google Patents
Dispositif de stockage Download PDFInfo
- Publication number
- WO2015118680A1 WO2015118680A1 PCT/JP2014/053000 JP2014053000W WO2015118680A1 WO 2015118680 A1 WO2015118680 A1 WO 2015118680A1 JP 2014053000 W JP2014053000 W JP 2014053000W WO 2015118680 A1 WO2015118680 A1 WO 2015118680A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- information
- storage device
- added
- write
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
Definitions
- the present invention relates to a storage device using a semiconductor recording device as a data storage device.
- Patent Document 1 confirms that there is no error using a sum check given to data in order to ensure that the data before compression is correctly input to the compressor in the storage device. Later, it is disclosed that the data is compressed and a new sum check is added to the compressed data.
- the guarantee code is generally composed of an error detection code and some ID (for example, an address at which data is stored).
- the storage device gives a guarantee code to the data when write data is received from the host device. Then, when transferring the write data to the final storage medium, the storage apparatus checks the assigned guarantee code and records only the data that has cleared the check on the final storage medium. With this control, it is possible to detect garbled data generated in the cache or transfer path in the storage apparatus and a control error in the control software of the storage apparatus, and high reliability is guaranteed.
- a storage device using a non-volatile memory as a storage element has recently been used as a cache of a storage device.
- the storage device temporarily records the write data received from the host device in a cache device of a nonvolatile memory that can be stored at high speed. Then, when the load on the storage device is low, the data is taken out from the cache device and recorded in a final storage device composed of an HDD (Hard Disk Drive) or SSD (Solid State Drive).
- HDD Hard Disk Drive
- SSD Solid State Drive
- the storage device may be transferred via a non-volatile memory cache device.
- a non-volatile memory cache device For example, when data is transferred from the SSD to the HDD, a certain amount of data to be copied is temporarily stored in the cache device of the non-volatile memory, and after being stored, the data is collectively written in the HDD. By performing such control, it becomes possible to transfer data to the HDD at a higher speed in the sequential IO processing than in the random IO processing.
- the data stored in the cache is selected according to some rule (for example, the LRU: Last Recently Used algorithm that selects the data that has passed the most time since the last access).
- the process of storing the selected data in the final storage device is called destage.
- FIG. 2 shows a data flow from when the conventional storage apparatus receives the write data 210 until the data is recorded in the storage area in the recording destination volume 230 configured by the final storage device in the storage apparatus. .
- the host interface (host I / F) 124 in the storage apparatus that has received the write data 210 from the host apparatus such as a host computer designates the storage area in the recording volume 230 that is the storage destination for this data.
- a guarantee code 211 including Then, the storage device stores the data 210 with the guarantee code 211 added to the NVM cache memory. After the completion of this operation, the storage apparatus notifies the host apparatus that the write request processing has been completed.
- the storage device performs destage control at an arbitrary timing. Specifically, the data 210 and the guarantee code 211 recorded in the NVM cache memory are acquired and transferred to the disk interface 123. At this time, the storage apparatus instructs the disk interface 123 to check the guarantee code 211 of the data to be transferred, and sets an address for designating the storage area in the recording destination volume 230, which is one of the expected values of the guarantee code 211. Notice.
- the disk interface 123 inspects the guarantee code 211 of the received data 210, and an address that designates a storage area in the recording destination volume 230, which is one of the expected values of the guarantee code 211 notified from the storage device, is given to the data. Make sure that it is.
- the write data 210 and the guarantee code 211 are transferred to the storage area in the recording destination volume 230, thereby allowing the storage device to Write processing that guarantees that no data corruption or incorrect control has been performed is completed.
- the storage apparatus of FIG. 3 has a cache memory having a data compression function.
- This cache memory is referred to herein as an NVM module.
- FIG. 3 shows the data until the storage device acquires the write data 210, compresses it with the NVM module, and records the compressed data in the storage area in the recording volume 230 configured by the final storage device inside the storage device. The flow is shown.
- the storage device assigns a guarantee code 311 based on some rule (for example, the storage address of the NVM module that is the cache device) to the data 210 acquired from the higher-level device, and the cache device 126 configured by the NVM module is given. Control to transfer. After this operation is completed, the storage apparatus 101 notifies the host apparatus that the write request processing has been completed.
- FIG. 3 shows an example in which the guarantee code cannot be assigned at the host interface due to a size change due to compression
- a problem is not a problem that occurs only during data compression.
- the storage apparatus 101 instructs the host interface 124 to give a guarantee code 311 based on some rule, and controls the transfer to the cache apparatus 126 configured by the NVM module.
- the NVM module 126 that has acquired the data 210 to which the guarantee code 311 based on a certain rule has been acquired temporarily stores the data in a nonvolatile memory managed by the NVM module 126.
- the storage device destages data from the NVM module 126 which is a cache device, it is necessary to inspect the guarantee code 312 at the disk interface 123. Therefore, in the configuration of FIG. 3, it is desirable that a new guarantee code 312 for checking with the disk interface 123 is added to the compressed data or the uncompressed data in the NVM module 126 so that the data can be acquired.
- the storage apparatus when the storage apparatus according to the present invention receives write target data from a host apparatus such as a host computer, the storage apparatus provides a first guarantee code to the NVM module operating as a cache apparatus or a final storage apparatus. Assign and record the write target data.
- the storage apparatus issues an instruction to the NVM module to read data in which the second guarantee code different from the first guarantee code is added to the target data.
- the NVM module returns data in which the second guarantee code different from the first guarantee code is added to the target data to the storage device.
- the guarantee code assigned to the data can be changed during the data transfer process in the storage device, and the write data storage location is not determined when the write data is received from the host device. Even if it exists, an appropriate guarantee code can be given later.
- the storage apparatus can manage the data to be controlled with the guarantee code always attached, so that an error check can be performed during the entire data transfer process.
- FIG. 1 is a diagram showing a configuration of a computer system centered on a storage apparatus according to Embodiment 1 of the present invention.
- FIG. 2 is a diagram showing a data flow when a conventional storage device stores write data in a final storage device.
- FIG. 3 is a diagram showing a data flow when a storage device having a cache device having a data compression function stores write data in the final storage device.
- FIG. 4 is a diagram showing an internal configuration of the NVM module.
- FIG. 5 is a diagram showing the configuration of the FM.
- FIG. 6 is a diagram showing the configuration of the physical block.
- FIG. 7 is a diagram showing the concept of associating the LBA0 and LBA1 spaces, which are logical spaces provided by the NVM module of this embodiment to the storage controller, and the PBA space, which is a physical area designating address space.
- FIG. 8 is a diagram showing the contents of the LBA0-PBA conversion table and the LBA1-PBA conversion table.
- FIG. 9 is a diagram showing block management information.
- FIG. 10 is a diagram illustrating a data format handled by the storage apparatus according to the first embodiment of the invention.
- FIG. 11 is a diagram showing the configuration of a sector with a guarantee code.
- FIG. 12 is a diagram showing a write command and response information for the write command.
- FIG. 13 is a diagram showing a read command and response information to the read command.
- FIG. 14 is a diagram showing an LBA1 mapping command and response information for the LBA1 mapping command.
- FIG. 15 is a diagram showing LBA0 mapping command and response information to the LBA0 mapping command.
- FIG. 16 is a diagram for explaining the storage space handled by the storage apparatus according to the first embodiment of the invention.
- FIG. 17 is a flow of write processing executed by the storage apparatus according to Embodiment 1 of the present invention.
- FIG. 18 is a flowchart of destage processing executed by the storage apparatus according to the first embodiment of the invention.
- FIG. 19 is a flow of a read process performed by the NVM module.
- FIG. 20 is a diagram showing the configuration of the storage system related to Example 2 of the present invention.
- FIG. 21 is a diagram for explaining copy processing by the storage apparatus according to the second embodiment of the present invention.
- FM NAND flash memory
- FIG. 16 is a diagram illustrating the storage space handled by the storage apparatus 101 according to the first embodiment.
- the storage apparatus 101 is a compressed volume composed of a final storage device (one or a plurality) such as an HDD 112 or an SSD 111, which is a destage storage destination of compressed data, with respect to a host device 103 such as a host computer. 5500 is concealed, and one or a plurality of virtual decompressed volumes 5000 are recognized from the host device 103 that are recognized as if the compressed data is stored in an uncompressed state (expanded state). The host apparatus 103 sends the read / write request to the storage apparatus 101 by designating the address of the decompression volume 5000.
- the host apparatus 103 When the host apparatus 103 records data in the storage apparatus 101, it designates an address in the decompression volume 5000 and transfers write data to the storage apparatus 101.
- the storage apparatus 101 uses an NVM module 126 having an internal compression function as a cache apparatus.
- the NVM module 126 provides the storage apparatus 101 with two independent logical storage spaces, LBA0 space and LBA1 space, which will be described in detail later.
- the storage apparatus 101 records the write data acquired from the host apparatus in the NVM module 126 by designating the address of the LBA0 space.
- the NVM module 126 compresses and stores the received write data.
- the storage apparatus 101 maps the compressed data of the data recorded by designating the address of the LBA0 space onto the LBA1 space.
- the storage apparatus 101 acquires the compressed data from the NVM module 126 by designating an address in the LBA1 space, and designates the address of the compressed volume 5500 configured by the final storage device such as an HDD or SSD. Stage.
- the compressed volume 5500 is a logical volume configured using storage areas of one or more final storage devices, similar to a volume provided to a host device by a known storage device.
- the storage apparatus 101 acquires compressed data from the compressed volume 5500 in response to a read request received from the host apparatus, and writes to the cache apparatus configured by the NVM module 126 by designating an address on the LBA1 space.
- the storage apparatus 101 maps the decompressed data of the compressed data recorded by designating an address on the LBA1 space to the LBA0 space.
- the storage apparatus 101 acquires the compressed data by specifying the address of LBA0 from the NVM module 126 and transfers it as read data to the host apparatus.
- the storage apparatus 101 additionally writes the compressed data to the compressed volume 5500 when recording the compressed data on the volume.
- the compressed data is recorded sequentially from the head area of the compressed volume in the order of destage.
- the address in the compressed volume that is the recording destination of the compressed data is determined when the storage device controls the destage.
- the storage apparatus 101 When the storage apparatus 101 according to the first embodiment receives write data from the upper apparatus, the first guarantee code including the error detection code calculated from the write data and the storage address of the decompression volume 5000 is received for the write data. Is granted.
- the storage apparatus 101 instructs the NVM module 126 to compress and store the write data including the first guarantee code.
- the storage apparatus 101 notifies the NVM module 126 that stores the data the storage address of the decompressed volume that is the expected value of the first guarantee code, and checks the first guarantee code to the NVM module 126. Let The NVM module 126 checks the first guarantee code for the acquired data.
- the NVM module 126 compresses the data and the first guarantee code together.
- the present invention is not limited to an example in which the first guarantee code and data are compressed together. An area in which only the data is compressed and is controlled in the same way as the compressed data (an area to be transferred by the same control as the transferred data and adjacent (continuous) to the transferred data. It may be recorded in a storage format in which a non-compressed first guarantee code is assigned to the address area that is not addressed).
- the storage apparatus 101 determines an address in the compressed volume 5500 serving as a recording destination, and the address in the compressed volume for the compressed data. To give a second warranty code containing At the same time, the storage apparatus 101 notifies the NVM module 126 of the expected value for the first guarantee code assigned to the data obtained by decompressing the compressed data to be destaged. The NVM module 126 once expands the recorded compressed data, and compares the first guarantee code included in the decompressed data with the expected value notified from the storage apparatus 101. If the first guarantee code matches, the second guarantee code is assigned to the compressed data and transferred to the storage apparatus 101.
- the storage apparatus 101 instructs the disk interface or the final storage device to transfer data when recording the compressed data with the second guarantee code acquired from the NVM module 126 in the compressed volume 5500.
- the storage apparatus 101 notifies the expected value of the second guarantee code including the address in the compressed volume 5500 to the disk interface or the final storage apparatus.
- the disk interface or the final storage device notified of the expected value of the second guarantee code checks the second guarantee code when acquiring the compressed data. Specifically, it is confirmed from the error detection code that there is no error in the data, and it is confirmed that the storage destination address of the volume matches the expected value notified from the storage apparatus.
- the storage apparatus 101 confirms that, in the data transfer in the storage apparatus 101, the control program of the storage apparatus 101 has not mistakenly transferred data different from the data that was originally targeted for access. it can. In addition, it can be guaranteed that no data corruption has occurred in the storage apparatus 101. This guarantee can improve the reliability of the storage apparatus.
- FIG. 1 shows a schematic configuration of a computer system centering on a storage device including a semiconductor recording device (hereinafter referred to as “NVM module”) using an FM relating to the present invention as a recording medium.
- NVM module semiconductor recording device
- FIG. 1 is a semiconductor recording device using FM as a recording medium.
- the storage apparatus 101 includes a plurality of storage controllers 110.
- Each storage controller 110 includes a host interface 124 that connects to the host device 103 and a disk interface 123 that connects to the recording device.
- Examples of the host interface 124 include devices that support protocols such as FC (Fibre Channel), iSCSI (Internet Small Computer System Interface), FCoE (Fibre Channel over Ether), and the disk interface 123 includes FC, SAS (S Examples include devices that support various protocols such as Attached SCSI), SATA (Serial Advanced Technology Attachment), and PCI (Peripheral Component Interconnect) -Express.
- the storage controller 110 includes hardware resources such as a processor 121 and a memory 125.
- the storage controller 110 When the processor 121 executes a control program, the storage controller 110 performs final storage such as the SSD 111 and the HDD 112 in response to a read / write request from the host device 103. Request read / write to the media device.
- Each storage controller 110 has an NVM module 126 to which the present invention is applied. The NVM module 126 can be controlled from the processor 121 via the internal SW 122.
- the storage controller 110 also has a RAID (Redundant Arrays of Inexpensive Disks) parity generation function and a data restoration function using RAID parity, and also has a function of managing a plurality of SSDs 111 and a plurality of HDDs 112 as a RAID group in arbitrary units. Further, it has a function of dividing a RAID group as an LU (Logical Unit) in an arbitrary unit and presenting it as a recording area to the host apparatus 103.
- RAID Redundant Arrays of Inexpensive Disks
- parity is generated according to the specified RAID configuration and written to the recording device.
- receiving a read request from the host apparatus 103 to the LU after reading the data from the recording apparatus, it checks for the presence of data loss, and if data loss is detected, restores the data using RAID parity, Transfer data.
- the storage controller 110 has a function of monitoring and managing the failure, usage status, operation status, etc. of the recording device.
- the storage apparatus 101 is connected to the management apparatus 104 via a network.
- An example of this network is a LAN (Local Area Network). Although this network is omitted for simplification in FIG. 1, it is connected to each storage controller 110 in the storage apparatus 101. This network may be connected by the same network as the SAN 102.
- the management device 104 is a computer having hardware resources such as a processor, a memory, a network interface, and a local input / output device, and software resources such as a management program.
- the management device 104 acquires information from the storage device by a program and displays a management screen.
- the system administrator uses the management screen displayed on the management apparatus to monitor the storage apparatus 101 and control the operation.
- the SSD 111 stores data transferred in response to a write request from the storage controller, retrieves stored data in response to a read request, and transfers the data to the storage controller.
- the disk interface 123 designates the logical storage location for the read / write request by a logical address (hereinafter, LBA: Logical Block Address).
- LBA Logical Block Address
- the plurality of SSDs 111 are divided into a plurality of RAID groups and managed, and are configured such that lost data can be restored when data is lost.
- a plurality of HDDs (Hard Disk Drives) 112 (for example, 120) are provided in the storage apparatus 101 and, like the SSD 111, are connected to a plurality of storage controllers 110 in the same storage apparatus via the disk interface 123.
- the HDD 112 stores data transferred in response to a write request from the storage controller 110, retrieves stored data in response to a read request, and transfers it to the storage controller 110.
- the disk interface 123 designates the logical storage location for the read / write request by a logical address (hereinafter, LBA: Logical Block Address).
- LBA Logical Block Address
- the plurality of HDDs 112 are divided into a plurality of RAID groups and managed, and the lost data can be restored when data is lost.
- the storage controller 110 is connected to the SAN 102 connected to the host device 103 via the host interface 124. Although omitted in FIG. 1 for simplification, a connection path for mutual communication of data and control information between storage controllers is also provided.
- the host device 103 corresponds to, for example, a host computer or a file server that forms the core of the business system.
- the host device 103 includes hardware resources such as a processor, a memory, a network interface, and a local input / output device, and includes software resources such as a device driver, an operating system (OS), and an application program.
- OS operating system
- the host apparatus 103 executes various programs under processor control to perform communication with the storage apparatus 108 and data read / write requests.
- management information such as usage status and operation status of the storage apparatus 101 is acquired by executing various programs under processor control.
- the management unit of the recording apparatus, the recording apparatus control method, the data compression setting, and the like can be designated and changed.
- the NVM module 126 includes an FM controller (FM CTL) 410 and a plurality of (for example, 32) FM 420s.
- FM CTL FM controller
- the FM controller 410 includes a processor 415, a RAM (DRAM) 413, a data compression / decompression unit 418, a parity generation unit 419, a data buffer 416, an I / O interface (I / F) 411, an FM interface (I / F). ) 417, and a switch 414 for mutually transferring data.
- the switch 414 connects the processor 415 in the FM controller 410, the RAM 413, the data compression / decompression unit 418, the parity generation unit 419, the data buffer 416, the I / O interface 411, and the FM interface 417, and addresses the data between the parts. Or route and forward by ID.
- the I / O interface 411 is connected to the internal switch 122 included in the storage controller 110 in the storage apparatus 101, and is connected to each part of the FM controller 410 via the switch 414.
- the I / O interface 411 receives a read / write request and a logical storage location (LBA: Logical Block Address) to be requested from the processor 121 included in the storage controller 110 in the storage apparatus 101, and processes the request. I do. Further, when a write request is made, the write data is received and the write data is recorded in the FM 420. Further, the I / O interface 411 receives an instruction from the processor 121 included in the storage controller 110 and issues an interrupt to the processor 415 in the FM controller internal 410.
- LBA Logical Block Address
- the I / O interface 411 also receives a control command for the NVM module 126 from the processor 121 included in the storage controller 110, and displays the operation status, usage status, current setting value, etc. of the NVM module 126 according to the command.
- the storage controller 110 can be notified.
- the processor 415 is connected to each part of the FM controller 410 via the switch 414 and controls the entire FM controller 410 based on the program and management information recorded in the RAM 413. In addition, the processor 415 monitors the entire FM controller 410 by a periodic information acquisition and interrupt reception function.
- the data buffer 416 stores temporary data during the data transfer process in the FM controller 410.
- the FM interface 417 is connected to the FM 420 by a plurality of buses (for example, 16).
- a plurality (for example, 2) of FM 420 is connected to each bus, and a plurality of FMs 420 connected to the same bus are controlled independently using a CE (Chip Enable) signal that is also connected to the FM 420.
- CE Chip Enable
- the FM interface 417 operates in response to a read / write request instructed by the processor 415. At this time, the FM interface 417 is instructed by the processor 415 as the chip, block, and page numbers as request targets. If it is a read request, the stored data is read from the FM 420 and transferred to the data buffer 416. If it is a write request, the data to be stored is called from the data buffer 416 and transferred to the FM 420.
- the FM interface 417 includes an ECC generation circuit, an ECC data loss detection circuit, and an ECC correction circuit.
- ECC generation circuit When writing data to the FM 420, the data is written with the ECC added. Further, when data is called, the call data from the FM 420 is inspected by the data loss detection circuit using ECC, and when the data loss is detected, the data is corrected by the ECC correction circuit.
- ECC added to the data is added to the data written in the FM 420 and is inspected by the FM interface 417 when it is read from the FM 420. Is different.
- the data compression / decompression unit 418 has a data compression function using a reversible compression algorithm. In addition, there are a plurality of types of data compression algorithms, and a compression level changing function is also provided.
- the data compression / decompression unit 418 reads data from the data buffer 416 according to an instruction from the processor 415, performs a data compression operation that is a data compression operation or an inverse conversion of the data compression by a lossless compression algorithm, and outputs the result again. Write to the data buffer.
- the data compression / decompression unit 418 may be implemented as a logic circuit, or a similar function may be realized by executing a compression / decompression program with a processor.
- the data compression / decompression unit 418 also has a function of verifying a guarantee code attached to data transmitted from the host apparatus 103 and a function of changing the ID of a guarantee code attached to data. This function will be described later.
- the parity generation unit 419 has a function of generating parity that is redundant data required in the RAID technology. Specifically, the parity generation unit 419 includes an XOR operation used in RAID 5 and 6, a Reed-Solomon code or EVENODD used in RAID 6. It has a function to generate diagonal parity calculated by the method.
- the parity generation unit 419 reads data that is a parity generation target from the data buffer 416 in accordance with an instruction from the processor 415, and generates RAID5 or RAID6 parity by the above-described parity generation function.
- the switch 414, I / O interface 411, processor 415, data buffer 416, FM interface 417, data compression / decompression unit 418, and parity generation unit 419 described above are ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate).
- Array may be configured within a single semiconductor element, or may be configured such that a plurality of individual dedicated ICs (Integrated Circuits) are connected to each other.
- a volatile memory such as a DRAM is used for the RAM 413.
- the RAM 413 stores management information of the FM 420 used in the NVM module 126, a transfer list including transfer control information used by each DMA, and the like.
- a part or all of the role of the data buffer 416 for storing data may be included in the RAM 413 and the RAM 413 may be used for data storage.
- the configuration of the NVM module 126 to which the present invention is applied has been described with reference to FIG.
- the NVM module 126 having the flash memory Flash Memory
- the nonvolatile memory to be mounted on the NVM module 126 is not limited to the flash memory.
- a non-volatile memory such as Phase Change RAM or Resistance RAM may be used.
- a configuration may be adopted in which part or all of the FM 420 is a volatile RAM (DRAM or the like).
- the nonvolatile memory area in the FM 420 is composed of a plurality (for example, 4096) of blocks (physical blocks) 502, and stored data is erased in units of physical blocks.
- the FM 420 has an I / O register 501 inside.
- the I / O register 501 is a register having a recording capacity equal to or larger than a physical page size (for example, 8 KB).
- FM 420 operates in accordance with a read / write request instruction from FM interface 417.
- the flow of the write operation is as follows. First, the FM 420 receives a write command, a requested physical block, and a physical page from the FM interface 417. Next, the write data transferred from the FM interface 417 is stored in the I / O register 501. Thereafter, the data stored in the I / O register 501 is written to the designated physical page.
- the flow of read operation is as follows. First, the FM 420 receives a read command, a requested physical block, and a page from the FM interface 417. Next, the data stored in the physical page of the designated physical block is read and stored in the I / O register 501. Thereafter, the data stored in the I / O register 501 is transferred to the FM interface 417.
- the physical block 502 is divided into a plurality of (for example, 128) pages 601, and reading of stored data and writing of data are processed in units of pages.
- the order of writing to the physical page 601 in the block 502 is fixed, and writing is performed in order from the first page. That is, data must be written in the order of Page1, Page2, Page3,.
- overwriting on a written page 601 is prohibited in principle, and when data is overwritten on a written page 601, it is necessary to delete the data in the block 502 to which the page 601 belongs only after the data is erased. Data cannot be written to page 601.
- the NVM module 126 in this embodiment is equipped with a plurality of FM (chips) 420 and manages a storage area composed of a plurality of blocks and a plurality of pages. Then, a logical storage space is provided to the storage controller 110 (the processor 121) to which the storage controller 110 is connected.
- “providing storage space” means that each storage area that the NVM module 126 accesses the storage controller 110 is assigned with an address, and the storage controller 110 to which the NVM module 126 is connected is managed. This means that the processor 121 issues an access request (command) designating the address to enable the reference / update of the data stored in the area specified by the address.
- the physical storage area configured by the FM 420 is managed in a manner uniquely associated with an address space used only within the NVM module 126.
- this physical area designating address space (physical address space) used only within the NVM module 126 will be referred to as a PBA (Physical Block Address) space, and each physical storage area (sector in the PBA space.
- PBA Physical Block Address
- the position (address) of 1 sector is 512 bytes) is described as PBA (Physical Block Address).
- the NVM module 126 of this embodiment manages the association between this PBA and LBA (Logical Block Address) that is the address of each area of the logical storage space provided to the storage apparatus.
- LBA Logical Block Address
- a conventional storage device such as an SSD provides one storage space for a host device (such as a host computer) to which the storage device is connected.
- the NVM module 126 of the present embodiment has two logical storage spaces, and provides two logical storage spaces to the storage controller 110 to which the NVM module 126 is connected. The relationship between the two logical storage spaces LBA and PBA will be described with reference to FIG.
- FIG. 7 is a diagram illustrating a concept of association between the LBA0 space 701 and the LBA1 space 702, which are logical storage spaces provided by the NVM module 126 of the present embodiment to the storage controller 110, and the PBA space 703.
- the NVM module 126 provides two logical storage spaces, an LBA0 space 701 and an LBA1 space 702, to the storage controller 110, which is a host device.
- the addresses assigned to the storage areas on the LBA 0 space 701 are referred to as “LBA 0” or “LBA 0 address”, and the addresses assigned to the storage areas on the LBA 1 space 702 are referred to as “LBA 1”. Or “LBA1 address”.
- the size of the LBA0 space 701 and the size of the LBA1 space 702 are both equal to or smaller than the size of the PBA space. However, even when the size of the LBA0 space 701 is larger than the size of the PBA space, The invention is effective.
- the LBA0 space 701 is a logical storage space for allowing the processor 121 of the storage controller 110 to access the compressed data recorded in the physical storage area configured by the FM 420 as uncompressed data.
- the processor 121 designates an address (LBA0) on the LBA0 space 701 and issues a write request to the NVM module 126
- the NVM module 126 acquires write data from the storage controller 110 and compresses it by the data compression / decompression unit 418. After that, the NVM module 126 records data in the physical storage area on the FM 420 designated by the dynamically selected PBA, and associates LBA0 and PBA.
- the NVM module 126 acquires data (compressed data) from the physical storage area of the FM 420 indicated by the PBA associated with the LBA0. After decompression by the compression / decompression unit 418, the decompressed data is transferred to the storage controller 110 as read data.
- the association between LBA0 and PBA is managed by an LBA0-PBA conversion table described later.
- the LBA1 space 702 is a logical storage space for allowing the storage controller 110 to access the compressed data recorded in the physical storage area configured by the FM 420 as it is (not expanded).
- the processor 121 of the storage controller 110 designates LBA1 and issues a write request to the NVM module 126
- the NVM module 126 acquires data (compressed write data) from the storage controller 110, and the NVM module 126 dynamically
- the data is recorded in the storage area of the FM designated by the selected PBA, and the LBA 1 and the PBA are associated with each other.
- the NVM module 126 acquires data (compressed data) from the physical storage area of the FM 420 indicated by the PBA associated with LBA 1 and reads it to the storage controller 110. Transfer compressed data as data.
- the association between LBA1 and PBA is managed by an LBA1-PBA conversion table described later.
- the area on the PBA space which is the physical storage area in which the compressed data 713 is recorded, may be associated with both the LBA0 space area and the LBA1 space area at the same time.
- the decompressed data of the compressed data 713 is associated with the LBA0 space as the decompressed data 711, and the compressed data 713 is directly associated with the LBA1 space as the compressed data 712.
- the processor 121 specifies LBA0 (assuming that LBA0 is set to 0x000000011000) and writes data to the NVM module 126, the data is compressed by the data compression / decompression unit 418 in the NVM module 126.
- the NVM module 126 is arranged on the dynamically selected PBA space (specifically, any unwritten page among a plurality of pages of the FM 420).
- the data is managed in a state associated with the address 0x000000011000 of the LBA0 space. Thereafter, when the processor 121 issues a request for associating the data associated with 0x000000011000 with the address of the LBA1 space (assuming 0x80000000010) to the NVM module 126, this data is also associated with the LBA1 space.
- the processor 121 when the processor 121 issues a request (command) for reading the data at the LBA1 address 0x80000000010 to the NVM module 126, the processor 121 compresses the data that it has written to the LBA0 address 0x000000011000. Can be read.
- the size of the compressed data generated by the NVM module 126 in the embodiment of the present invention varies depending on the data contents, but it is easy to handle if the size is a multiple of 512 bytes (1 sector), and will be described later.
- the size does not exceed the size of the uncompressed data. That is, when 4 KB data is compressed, the minimum size is 512 bytes and the maximum size is 4 KB.
- the LBA0-PBA conversion table 810 and the LBA1-PBA conversion table 820 will be described with reference to FIG.
- the LBA0-PBA conversion table 810 is stored in the RAM 413 in the NVM module 126, and includes information on the NVM module LBA0 (811), the NVM module PBA (812), and the PBA length (813).
- the processor 415 of the NVM module 126 receives the LBA 0 specified at the time of the read request from the host device, and then uses the LBA 0 to obtain the PBA indicating the location where the actual data is stored.
- the NVM module 126 records update data (write data) in a physical storage area different from the PBA in which the pre-update data is recorded, and converts the PBA and PBA length in which the update data is recorded into an LBA0-PBA conversion. Record in the corresponding part of the table and update the LBA0-PBA conversion table. By operating in this manner, the NVM module 126 enables (pseudo) overwriting of data in the area on the LBA0 space.
- the NVM module LBA0 (811) is a logical area of the LBA0 space provided by the NVM module 126 arranged in units of 4 KB in order (each address (LBA0) in the LBA0 space is attached to each sector (512 bytes). Have been).
- LBA0 each address
- PBA NVM module PBA
- the association between the NVM module LBA0 (811) and the NVM module PBA (812) is managed in units of 4 KB (8 sectors).
- the association between the NVM module LBA0 (811) and the NVM module PBA (812) may be managed in an arbitrary unit other than the 4 KB unit.
- the NVM module PBA (812) is a field for storing the head address of the PBA associated with the NVM module LBA0 (811).
- the physical storage area of the PBA space is divided and managed for every 512 bytes (one sector).
- a value (PBA) of “XXX” is associated as PBA (Physical Block Address) associated with the NVM module LBA0 (811) “0x000_0000_0000”. This value is an address that uniquely indicates a storage area among a plurality of FMs 420 mounted on the NVM module 126.
- the actual storage size of the 4 KB data specified in the NVM module LBA0 (811) is recorded.
- the storage size is recorded by the number of sectors.
- PBA 4 KB data starting from LBA 0 [0x000_0000_0000] is compressed and stored in the 1 KB area from PBA “XXX” to “XXX + 1”. Represents.
- the NVM module 126 in this embodiment compresses uncompressed data instructed by the processor 121 of the storage controller 110 in units of 4 KB.
- the processor 121 receives a write request for 8 KB data (uncompressed data) starting from the address (0x000_0000_0000) in the LBA0 space
- 4 KB data in the address range 0x000_0000_0000 to 0x000_0000_0007 (in the LBA0 space) is used as a unit.
- Compressed data is generated by compression, and then compressed data is generated by compressing 4 KB data in the address range 0x000_0000_0008 to 0x000_0000_000F as a unit, and each compressed data is written in the physical storage area of the FM 420.
- the present invention is not limited to a mode in which data is compressed in units of 4 KB, and the present invention is effective even in a configuration in which data is compressed in other units.
- the LBA1-PBA conversion table 820 is stored in the DRAM 413 in the NVM module 126, and includes two pieces of information of the NVM module LBA1 (821) and the NVM module PBA (822).
- the processor 415 of the NVM module 126 receives the LBA1 specified at the time of the read request from the host device, and then converts the received LBA1 into a PBA indicating the location where the data is stored using the LBA1-PBA conversion table 820. To do.
- the NVM module LBA1 (821) is a logical area of the LBA1 space provided by the NVM module 126 arranged in order for each sector (a numerical value 1 in the NVM module LBA1 (821) means one sector (512 bytes). To do). This is because the NVM module 126 in this embodiment is described on the premise that the association between the NVM module LBA1 (821) and the NVM module PBA (822) is managed in units of 512B, but this NVM module LBA1 (821). ) And the NVM module PBA (822) are not limited to the mode managed in 512B units, and may be managed in any unit.
- the LBA1 space is a space that directly maps the PBA space that is the physical storage space where the compressed data is stored, and is preferably equal to the PBA division management size. Divide and manage with.
- the NVM module PBA (822) is a field for storing the head address of the PBA associated with LBA1.
- the PBA value “ZZZ” is associated with the NVM module LBA1 “0x800_0000_0002”.
- This PBA value is an address that uniquely indicates a storage area on a certain FM 420 mounted on the NVM module 126. Accordingly, when “0x800_0000_0002” is received as the read request destination start address (LBA1), “ZZZ” is acquired as the physical read destination start address in the NVM module 126.
- LBA1 the read request destination start address
- a value indicating “unallocated” is stored in the NVM module PBA (822).
- the above is the contents of the LBA0-PBA logical-physical conversion table 810 and the LBA1-PBA logical-physical conversion table 820 used by the NVM module 126.
- the block management information 900 is stored in the DRAM 413 in the NVM module 126 and includes items of an NVM module PBA 901, an NVM chip number 902, a block number 903, and an invalid PBA amount 904.
- the NVM module PBA 901 is a field for storing a PBA value that uniquely identifies each area in all the FMs 420 managed by the NVM module 126.
- the NVM module PBA 901 is divided and managed in units of blocks.
- FIG. 9 shows an example in which the head address is stored as the NVM module PBA value. For example, in the row (entry) where the value of the NVM module PBA 901 is “0x000_0000_0000”, information about the PBA range from “0x000_0000_0000” to “0x000_0000_0FFF” is stored.
- the NVM chip number 902 is a field for storing a number for uniquely specifying the FM Chip 420 mounted on the NVM module 126.
- the block number 903 is a field for storing the block number in the FM Chip 420 specified by the stored value of the NVM Chip number 902.
- the invalid PBA amount 904 is a field for storing the invalid PBA amount of the block specified by the stored value of the block number 903 in the FM Chip specified by the stored value of the NVM Chip number 902.
- the invalid PBA amount is associated with the LBA0 space and / or LBA1 space specified by the NVM module LBA0 (811) and the NVM module LBA1 (821) in the LBA0-PBA conversion table 810 and the LBA1-PBA conversion table 820. This is the amount of the area (on the PBA space) that was later released from the association.
- the PBA associated with the NVM module LBA0 or LBA1 by the LBA0-PBA conversion table 810 or the LBA1-PBA conversion table 820 is referred to as an effective PBA in this specification.
- the invalid PBA area is inevitably generated when a pseudo-overwrite is attempted in a non-volatile memory where data cannot be overwritten.
- the NVM module 126 records the update data in an unwritten PBA (different from the PBA in which the pre-update data is written) at the time of data update, and the NVM module PBA 812 of the LBA0-PBA conversion table 810. And the PBA length 813 field are rewritten to the start address and PBA length of the PBA area in which the update data is recorded. At this time, the association by the LBA0-PBA conversion table 810 is released for the PBA area in which the pre-update data is recorded.
- the NVM module 126 also checks the LBA1-PBA conversion table 820 and sets an area that is not associated in the LBA1-PBA conversion table as an invalid PBA area.
- the NVM module 126 counts the amount of invalid PBA for each block, which is the minimum erase unit of FM, and preferentially selects a block with a large amount of invalid PBA as a garbage collection target area.
- the block number 0 of the NVM chip number 0 managed by the NVM module 126 has an invalid PBA area of 160 KB.
- garbage collection when the total amount of invalid PBA areas managed by the NVM module 126 exceeds a predetermined garbage collection start threshold (depletion of unwritten pages), blocks including invalid PBA areas are erased and unwritten. Create a PBA area. This operation is called garbage collection.
- garbage collection When an effective PBA area is included in an erasure target block at the time of garbage collection, it is necessary to copy the effective PBA area to another block before erasing the block. Since this data copy involves a write operation to the FM, the destruction of the FM progresses, and resources such as the processor of the NVM module 126 and the bus bandwidth are consumed as the copy operation, which causes a decrease in performance. For this reason, it is desirable that the number of valid PBA areas be as small as possible.
- the NVM module 126 refers to the block management information 900 at the time of garbage collection, and deletes the effective PBA by sequentially deleting the blocks having a larger storage value of the invalid PBA amount 904 (including many invalid PBA areas). Operates to reduce the amount of space copy.
- the amount of area released from the association with the NVM modules LBA0 (811) and LBA1 (821) is managed by the PBA amount (number of sectors). Is not limited to this management unit. For example, instead of the PBA amount, there may be a mode in which the number of pages that are the minimum writing unit is managed.
- the above is the content of the block management information 900 used by the NVM module to which the present invention is applied.
- FIG. 10 shows the data format generated by each component of the storage device as 1000, 1010, 1020, 1030, and 1040 in order when the host device writes 4 KB data.
- the storage device of the first embodiment compresses and manages data in units of 4 KB, but the present invention is not limited to the storage device in which the management unit of compressed data in the storage device is 4 KB.
- the data compression / decompression unit 418 of the NVM module 126 that performs data compression according to the first embodiment is a unit that compresses and decompresses data in units of 4 KB.
- the present invention is a unit for compressing and decompressing data. Is not limited to 4 KB.
- the first data format 1000 indicates 4 KB data transferred from the host device 103 to the storage device 101.
- the minimum access unit when the host apparatus 103 accesses the volume of the storage apparatus 101 is 512 B (bytes), and the storage apparatus 101 can be configured to specify data in 512 B units.
- the 512B unit data that can be designated as an address by the host device 103 is referred to as a sector, and 4 KB data is handled as eight sectors.
- the case where the sector is 512B will be described.
- the present invention is not limited to this sector unit. For example, one sector may be 4 KB.
- the storage apparatus 101 When the storage apparatus 101 receives a 4 KB request in the data format 1000, it causes the host interface 124 to generate a guarantee code.
- an 8B guarantee code is assigned to each 512B sector, which is a data designation unit. This configuration is described in detail in FIG. FIG. 11 shows an example in which an 8B guarantee code is assigned to the 512B sector.
- the sector to which the guarantee code is assigned is referred to as a sector with a guarantee code.
- the sector 1100 with a guarantee code is composed of 512B data 1101 and 8B guarantee code, and the 8B guarantee code is composed of 2B CRC 1102 and 6B ID 1103.
- a guarantee code composed only of CRC and ID will be described, but the present invention is not limited to this example. The present invention is applied if it is necessary to change the ID 1103 of the guarantee code during the control of the apparatus.
- CRC 1102 is a 2B CRC (Cyclic Redundancy Check) code generated using 512B data 1101. If the bit value constituting the data 1101 changes for some reason and an error bit is generated, a change in the bit value can be detected by performing an inspection using this CRC.
- CRC Cyclic Redundancy Check
- the processor 121 of the storage apparatus 101 instructs the hardware (for example, the host interface 124) to transfer, the received hardware detects a bit error using the data 1101 and CRC 1102 of the sector with a guarantee code to be transferred. . If no error bit is detected, data transfer is performed. On the other hand, if an error bit is detected, the data transfer is stopped and the processor 121 is notified of the error.
- Example 1 although the example which set CRC to 2B was shown, this invention is not limited to the size of this CRC. The amount of CRC may be increased according to the required detection capability. Further, although the CRC of the first embodiment is shown as an example generated only from the data portion, the present invention is not limited to this CRC generation method. For example, a CRC may be created from data and ID. In this case, it is possible to detect an error bit generated in the ID. On the other hand, it is necessary to recalculate the CRC when changing the ID.
- ID 1103 is an ID that can identify a sector with a guarantee code.
- the logical address (LBA) of the decompression volume that is a virtual recording destination of uncompressed data is used as the ID.
- LBA logical address
- a configuration in which a sector can be uniquely identified from an ID is described.
- the reliability is not significantly impaired. I do not care.
- the probability that the same ID is assigned is very low, the same ID may be assigned to different sectors.
- the data size of the ID 1103 is 6B is shown, but the present invention is not limited to this data size.
- the data size of ID 1103 may be made larger than 6B in order to reduce the probability of transferring erroneous data.
- the ID used in the storage apparatus guarantees the correctness of the operation of the storage apparatus 101.
- the processor 121 notifies the hardware that performs the transfer of the expected ID value.
- the hardware instructed to transfer acquires the ID 1103 in the sector 1100 with a guarantee code to be transferred, and confirms that it matches the expected value.
- the hardware instructed to transfer notifies the processor 121 of an error.
- a compressed sector with a guarantee code which will be described later, stores compressed data in the data 1101, and is given a guarantee code including a CRC created from the compressed data and an ID assigned to the compressed data. Since the size of each field of the compressed sector with a guarantee code is the same as that of the sector 1100 with a guarantee code, detailed description thereof is omitted.
- FIG. 10 shows an example in which compressed data of 1210B is generated as a result of compressing a sector with a guarantee code of 4160B, but compressed data of other sizes may be generated depending on the data contents.
- compression Data indicates “Compression Data” in the figure.
- FIG. 10 shows an example in which compressed data of 1210B is generated as a result of compressing a sector with a guarantee code of 4160B, but compressed data of other sizes may be generated depending on the data contents.
- FIG. 10 shows an example in which data 1101 and a guarantee code are compressed together in a sector with a guarantee code will be described.
- the present invention is not limited to this example.
- only the data 1101 is compressed in the sector with the guarantee code, and the guarantee code is not compressed.
- compressed data 1020 having a configuration in which uncompressed guarantee codes are collected and compressed at the beginning or end of data may be created.
- the NVM module 126 manages the compressed data on a sector basis as well as the uncompressed data. Therefore, the 326B data is padded after the compressed data of 1210B to obtain data of the size of 1536B (that is, the size of 3 sectors). Then, 1536B data is divided into three compressed sectors ("CompData" in the figure) (see data format 1030), and data 1040 is generated by padding an 8B area for a guarantee code at the end of each compressed sector. (The black box portion in the figure means a guarantee code padding area of the compressed sector. Also, when the data 1040 is generated, no guarantee code is stored in the padding area). This data is recorded in the FM 420.
- the NVM module 126 Since the minimum recording unit of FM is a page, for example, a size such as 8 KB or 16 KB, the NVM module 126 stores a certain amount of data in the data buffer 416 in the NVM module 126, and the data exceeding the minimum recording unit. A plurality of write data is collectively recorded in the FM 420 at the timing when the error is accumulated.
- the processor 121 of the storage apparatus 101 reads out data in which an 8B area for a guarantee code is added to each compressed sector by issuing a read command to be described later to the NVM module 126.
- the NVM module 126 assigns a compression sector guarantee code to each compression sector. More specifically, when the storage controller 110 (the processor 121) issues a read command to the NVM module 126, the ID of the guarantee code to be given to the compression sector is included in the read command.
- the NVM module 126 when transferring the compressed sector to the storage controller 110, stores the 2B CRC generated from the compressed sector in the padded guarantee code area and the 6B specified by the storage controller 110. Are generated, and a compressed sector 1050 with a guarantee code is generated. Thereafter, the NVM module 126 transfers the compressed sector 1050 with a guarantee code as read data to the controller 110.
- NVM Module Control Command 1 Write Command Next, commands supported by the NVM module 126 to which the present invention is applied will be described.
- the NVM module 126 analyzes the content of the received command, performs predetermined processing, and sends one response (response information) after the processing is completed. Reply to the storage controller.
- This process is realized by the processor 415 in the NVM module 126 executing a command processing program stored in the RAM 413.
- the command includes information necessary for the NVM module 126 to perform a predetermined process. For example, in the case of a write command that instructs the NVM module 126 to write data, the command includes a write command and information (such as the write data write position and data length) required for the write. Contains.
- FIG. 12 is a diagram showing a write command and response information to the write command supported by the NVM module 126 according to the first embodiment of the present invention.
- the write command 1210 of the NVM module in this embodiment includes, as command information, an operation code (Opcode) 1211, a command ID 1212, an LBA 0/1 start address 1213, an LBA 0/1 length 1214, a compression necessity flag 1215, a write data address 1216, It consists of an expected value 1217 of the guarantee code ID and a seed 1218 of the guarantee code ID.
- Opcode operation code
- Operation code 1211 and command ID 1212 are fields that exist in common with each command supported by the NVM module 126.
- the operation code 1211 is a field in which information for notifying the NVM module 126 of the command type is stored. As an example, a value of 0x01 is stored in the field for a write command, and a value of 0x02 is stored for a read command.
- the NVM module 126 that has acquired the command recognizes that the notified command is a write command by referring to this field.
- the command ID 1212 is a field for storing a unique ID of the command.
- the command issuer storage controller 110
- the specified ID is given.
- the storage controller 110 stores an ID that can uniquely identify the command in the command ID 1212.
- the completion of the command is recognized by acquiring the ID included in the response information.
- the LBA 0/1 start address 1213 is a field for designating the head address of the write destination logical space (LBA 0 space or LBA 1 space).
- LBA 0 space in the embodiment of the present invention is a space in the range of addresses 0x000_0000_0000 to 0x07F_FFFF_FFFF
- the LBA1 space is defined as a space in the range after the address 0x800_0000_0000.
- an address in the range from 0x000_0000_0000 to 0x07F_FFFF_FFFF is stored in 1 start address 1213, it is recognized that an address in the LBA0 space has been designated, and if an address in the range from 0x800_0000_0000 to 0x8FF_FFFF_FFFF is designated, the address in the LBA1 space is designated Can be recognized.
- the address according to the address space of the LBA0 space or the LBA1 space is specified (that is, whether data is compressed and written or whether data is written as it is without being compressed)
- a method other than the method described above can be adopted as a method for recognizing. For example, there may be a method of identifying the LBA0 space and the LBA1 space according to the contents of the operation code 1211.
- the LBA 0/1 length 1214 is a field for designating the range (length) of the data recording destination LBA 0 or LBA 1 starting from the LBA 0/1 start address 1213.
- the NVM module 126 associates the PBA space area for storing write data with the LBA0 space upper area or the LBA1 space upper area within the range indicated by the LBA0 or LBA1 start address 1213 and the LBA0 / 1 length 1214 described above. I do.
- the compression necessity flag 1215 is a field for designating whether or not to compress the write target data indicated by this command.
- the storage controller 110 creates a write command, if the size reduction effect due to data compression cannot be expected for the write target data (for example, when it is already recognized as data compressed by image compression or the like), “0” is set in this field. Is stored, the NVM module 126 is notified that compression is not necessary. Conversely, if a value other than 0 is stored in this field, the NVM module 126 performs compression. In the first embodiment, when writing to the LBA1 space, “0” is stored in this field in order to explicitly notify that the write target data has already been compressed and no further compression processing is necessary. As another embodiment, when data is written to the LBA1 space, the NVM module 126 may determine that compression of transfer data is not always necessary. In that case, the field of the compression necessity flag 1215 is not necessary.
- the write data address 1216 is a field for storing the start address of the current storage destination (for example, the DRAM 125 of the storage controller 110) of the write target data indicated by this command.
- the NVM module 126 acquires write data by reading data existing in the range of the length specified by the LBA 0/1 length (1214) from the position specified by this field. When the write target data is discretely stored in a plurality of areas, a plurality of addresses are stored in this field. As another embodiment, the NVM module 126 refers to the pointer information so that this field stores pointer information (address of an area in which the list is stored) for storing a plurality of addresses. The write data address may be acquired.
- the expected value 1217 of the guarantee code ID is a field for storing the expected value of the guarantee code assigned to the write target data.
- 6B data is used as the ID of the guarantee code. For this reason, this field also specifies an ID expectation value of 6B.
- the NVM module 126 that has received the command checks whether the ID of the guarantee code added to each 512B of the data to be written matches the expected value 1217 of the guarantee code ID.
- a specific method of the inspection is as follows. When the write data size is 4 KB (8 sectors), the NVM module 126 performs a comparison operation for the IDs of the sectors with eight guarantee codes. First, expected values of IDs of eight guarantee code-added sectors are generated from expected values 1217 of guarantee code IDs. The expected value of the ID of the sector with the first guarantee code in the write data is the expected value 1217 of the guarantee code ID, and the expected value of the ID of the sector with the guarantee code thereafter is the expected value 1217 of the guarantee code ID one by one. The added value. As described above, in the case of a 4 KB write request, each of the eight sectors with a guarantee code is compared with an expected value obtained by adding a value of 0 to 7 to the expected value 1217 of the guarantee code ID.
- the write data coming from the host apparatus 103 is recorded in the NVM module 126 that is a cache.
- the host interface 124 of the storage apparatus 101 assigns 2B CRC and 6B ID as a guarantee code for each 512B data acquired from the host apparatus 103.
- the ID given at the time of the write operation is the lower 6 bytes of the logical address (LBA) of the decompression volume 5000 to which the write data is written. A value is added as an ID.
- LBA logical address
- the present invention is not limited to an aspect in which the write destination address of the decompression volume 5000 is used as the guarantee code ID.
- the NVM module 126 when LBA0 is designated as the LBA0 / 1 start address 1213, the contents of the ID part of the guarantee code of each sector of the decompressed data are set to the expectation of the guarantee code ID.
- the LBA1 is specified as the LBA0 / 1 start address 1213, the contents of the ID part of the guarantee code of each sector of the compressed data are expected of the guarantee code ID. It operates to check based on the value stored in the value 1217 field. Therefore, when the storage apparatus 101 reads data (compressed data) from the final storage device constituting the compressed volume and records it in the area on the LBA1 space of the NVM module 126, the ID of the guarantee code of each sector of the compressed data Perform part inspection.
- the new guarantee code ID type 1218 is a field used when changing the guarantee code at the time of writing or when attaching a new guarantee code to the compressed data.
- the new guarantee code ID seed 1218 stores an ID value newly added to the data of the first sector of the write target data in the write request. In the first embodiment, 6B data is used as the guarantee code ID. For this reason, this field also specifies the seed of 6B.
- a value indicating invalidity for example, a value not used as a guarantee code ID such as 0xFFFFFFFF) is stored in the field of the new guarantee code ID seed 1218.
- the new guarantee code seed 1218 is not used in the storage apparatus 101 shown in the first embodiment during normal write processing, that is, when write data from the host apparatus 103 is stored in the NVM module 126. This is because, as described above, when the write data from the host device 103 is stored in the NVM module 126, the address on the compressed volume in which the data is stored is not determined.
- the storage apparatus 101 writes data to the NVM module 126, if the newly assigned guarantee code is known, the new guarantee code seed 1218 field may be used.
- the processor 121 of the storage apparatus 101 When changing the guarantee code at the time of data writing to the NVM module 126, the processor 121 of the storage apparatus 101 stores information that becomes the seed of the new guarantee code in the new guarantee code seed 1218 field of the write command. Upon receiving the instruction, the NVM module 126 generates a new guarantee code from the new guarantee code seed 1218 included in the write command and stores the write data in the FM 420, and a sector with guarantee code (or compression with guarantee code). (Sector) guarantee code is changed and stored in FM.
- the NVM module 126 when the write request size specified by the LBA 0/1 length 1214 is 4 KB, the NVM module 126 generates eight new guarantee codes from the new guarantee code ID seed 1218. Specifically, the NVM module 126 sets the ID of the sector with the guarantee code at the head of the 4 KB (8 sectors) data to be transferred as the value specified by the new guarantee code ID seed 1218 and sets the next guarantee code.
- the ID value of the attached sector is a value obtained by incrementing the seed of the new guarantee code by one.
- the ID part included in the sector guarantee code with a guarantee code (in each sector of the decompressed data)
- the ID generated based on the value stored in the seed 1218 field of the new guarantee code ID is stored in the ID part of the guarantee code included), and LBA1 is designated as the LBA0 / 1 start address 1213
- the ID part in the guarantee code of the compressed sector with the guarantee code (the ID part of the guarantee code added to each sector of the compressed data) is generated based on the value stored in the seed 1218 field of the new guarantee code ID. To store stored IDs.
- the write response information 1220 includes a command ID 1221, a status 1222, and a compressed data length 1223.
- a command ID 1221 a command ID 1221
- a status 1222 a status 1222
- a compressed data length 1223 a compressed data length
- the command ID 1221 and the status 1222 are common information included in the response of each command supported by the NVM module 126.
- the command ID 1221 is a field for storing a number that can uniquely identify a completed command.
- the status 1222 is a field for notifying the command request source (the processor 121 of the storage apparatus 101) of the completion or error of the command.
- the command request source the processor 121 of the storage apparatus 101
- a number for identifying the cause of the error is stored.
- the compressed data length 1223 is a field for recording the data length when the written data is reduced by data compression.
- the processor 121 that has issued the command can acquire the compressed data size of the written data by acquiring information in this field included in the write response information 1220.
- this field is invalid because compressed data is recorded.
- NVM Module Control Command 2 Read Command Next, a read command supported by the NVM module to which the present invention is applied will be described. This read command instructs the replacement of the guarantee code, which is a characteristic operation of the present invention.
- FIG. 13 is a diagram showing an NVM module read command and response information to the read command in the present embodiment.
- the read command 1310 of the NVM module in this embodiment includes, as command information, an operation code 1311, a command ID 1312, an LBA0 / 1 start address 1313, an LBA0 / 1 length 1314, an expansion necessity flag 1315, a read data address 1316, and a guarantee code ID. Expected value 1317 and guarantee code ID seed 1318.
- an example of a command based on the above information will be described, but there may be additional information above. Since the command ID 1312 has the same contents as the previous write command, description thereof is omitted.
- the operation code 1311 is a field for notifying the command type to the NVM module 126, and the NVM module 126 that has acquired the command recognizes that the command notified by this field is a compressed data size acquisition command.
- the LBA 0/1 start address 1313 is a field for designating the start address of the logical space (LBA 0 space or LBA 1 space) of the read destination. Similarly to the write command, if LBA0 is specified as the LBA0 / 1 start address 1313, the decompressed data is returned to the request source, and if LBA1 is specified, the unexpanded data is returned to the request source. Return to
- the LBA 0/1 length 1314 is a field for designating the range of the recording destination LBA 0 or LBA 1 starting from the LBA 0/1 start address 1313.
- the NVM module 126 identifies the PBA associated with the LBA0 or LBA1 area in the range indicated by the LBA0 or LBA1 start address 1313 and the LBA0 / 1 length 1314 described above, and the identified PBA. Data is acquired from the associated physical area, and read processing is performed by transferring the data to the storage apparatus.
- the decompression necessity flag 1315 is a field for designating the necessity of decompression of the read target data indicated by this command.
- “0” is stored in this field to notify the NVM module 126 that decompression is unnecessary.
- the decompression necessity flag 1315 is used to explicitly tell that decompression is unnecessary.
- the NVM module 126 may determine that it is not necessary to decompress the acquired data when reading the LBA1 space. In this case, the decompression necessity flag 1315 may not be provided.
- the head address (for example, an address in the DRAM 125) of the output destination area of the read target data is designated.
- the read data data having a length designated by the LBA 0/1 length 1314 is continuously stored from the area of the address designated by the read data address 1316.
- the write data address of the write command there is also an aspect in which a plurality of combinations of the read data address 1316 and the data length can be specified as read command parameters, and data can be output to discrete areas. It can be.
- the guarantee code ID expected value 1317 is a field in which the storage apparatus stores the expected value of the guarantee code of the read target data. In the first embodiment, 6B data is used as the guarantee code ID. For this reason, this field also specifies an ID expectation value of 6B.
- the NVM module 126 notified of the command checks whether the guarantee code ID included in each read target data 512B matches the expected value 1317 of the guarantee code ID.
- the NVM module 126 compares the IDs of the eight sectors with guarantee codes.
- the expected values of the IDs of the eight sectors with guarantee codes are generated from the expected value 1317 of the guarantee code ID.
- the expected value of the ID of the sector with the first guarantee code of the transfer is the expected value 1317 of the guarantee code ID
- the expected value of the ID of the next sector with the guarantee code is a value obtained by incrementing the expected value 1317 of the guarantee code ID by one. It becomes.
- each of the eight sectors with a guarantee code is compared with an expected value obtained by adding the expected value 1317 of the guarantee code ID.
- the expanded sector with a guarantee code is inspected.
- a read command is sent to the host apparatus 103 when the storage apparatus 101 destages the data recorded in the cache NVM module 126 to a volume (compressed volume) or the data recorded in the NVM module 126. Used when transferring.
- the storage controller 110 receives the changed guarantee code from the NVM module 126. Get the appended data.
- the NVM module 126 gives a new guarantee code to the compressed data.
- the NVM module 126 When the NVM module 126 assigns the new guarantee code, for the purpose of guaranteeing the consistency of the data, the NVM module 126 decompresses the compressed data once to generate a sector with the guarantee code, and then checks the sector with the guarantee code. Specifically, a CRC is generated from the decompressed data, and compared with the CRC of the guarantee code included in the decompressed sector with the guarantee code, and it is confirmed whether there is a difference between the two. Further, it is confirmed that the guarantee code ID included in the decompressed sector with the guarantee code matches the expected value 1317 of the guarantee code ID. If the CRC or ID does not match, an error is notified to the processor 121 of the storage apparatus 101. If they match, a new guarantee code for compressed data is assigned, and the data (compressed data) with the new guarantee code is transferred to the storage controller 110.
- the new guarantee code ID type 1318 is a field for designating an ID for attaching a new guarantee code to data at the time of reading.
- the new guarantee code ID type 1318 stores an ID value newly assigned to the sector with guarantee code or the sector with guarantee code compression at the head of the read request.
- 6B data is used as the guarantee code ID. For this reason, this field also specifies the seed of 6B.
- each of the expanded data is stored in the ID part of the guarantee code of the sector, and compression is performed when LBA1 is designated as the LBA0 / 1 start address 1313.
- An ID generated based on the value stored in the seed 1318 field of the new guarantee code ID is stored in the ID part of the guarantee code of each sector of the data.
- the processor 121 of the storage apparatus 101 When the processor 121 of the storage apparatus 101 wants to change the guarantee code when reading data from the NVM module 126, it stores the new guarantee code ID seed in the new guarantee code ID seed 1318 field of the read command. Upon receiving the instruction, the NVM module 126 generates a new guarantee code from the new guarantee code seed 1318 at the time of data read, and changes the guarantee code of the sector with the guarantee code read from the FM or the compressed sector with the guarantee code to store the data. Transfer to device.
- the NVM module when the write request size specified by the LBA 0/1 length 1104 is 4 KB, the NVM module generates eight new guarantee codes from the new guarantee code ID seed 1318. Specifically, the NVM module uses the ID of the sector with the first guarantee code for transfer as the value specified by the seed 1318 of the new guarantee code ID, and the expected ID of the sector with the next guarantee code is the new guarantee code. The value specified by the code ID seed 1318 is incremented by one. In this way, a 4 KB write request is generated by adding a value from 0 to 7 to the value specified by the new guarantee code ID seed 1318 for each of eight sectors with guarantee codes. Give an ID.
- FIG. 13 shows an example in which one expected value 1317 of the guarantee code ID is included in one read command 1310, the present invention is not limited to this number.
- a plurality of sectors with guarantee codes or compressed sectors with guarantee codes constituting the read target data designated by one read command are not necessarily constituted by consecutive guarantee codes ID.
- the ID value of the sector with a guarantee code constituting the data obtained by decompressing the compressed data may not be continuous.
- data A ′ obtained by compressing data recorded in the area A of the decompression volume and data B ′ obtained by compressing data recorded in the area B of the decompression volume are recorded in the NVM module. It is conceivable that the LBA1 space is continuously mapped. In this case, the storage apparatus can acquire A ′ and B ′ with a single read command.
- the read command 1310 may include a plurality of guarantee code expected values.
- the read command 1310 includes a data length to which the A ′ guarantee code expected value is applied and a B ′ guarantee code expectation. A field for storing information on the data length to which the value is applied is added.
- the read command 1310 may include a pointer indicating the start address of the data area in which a plurality of expected values of the guarantee code ID are described. In this case, a field describing the size of the data area describing a plurality of expected values of the guarantee code ID is also added to the read command 1310.
- the read response 1320 includes a command ID 1321 and a status 1322 as in the previous write response. This content is the same as the write response described above. In the present embodiment, an example of response information based on the above information will be described, but there may be additional information above.
- FIG. 14 is a diagram schematically showing an LBA1 mapping command 1410 supported by the NVM module 126 in this embodiment and response information for the LBA1 mapping command.
- the NVM module 126 compresses and writes the data written by designating the LBA 0 area to the FM 420.
- the storage controller 110 maps LBA1 different from LBA0 using the LBA1 mapping command 1410 in order to write the compressed data recorded in the FM 420 to the final storage medium in a compressed state.
- the LBA1 mapping command 1410 of the NVM module 126 in the present embodiment is composed of an operation code 1411, a command ID 1412, an LBA0 start address 1413, an LBA0 length 1414, and an LBA1 start address 1415 as command information.
- an example of a command based on the above information will be described, but there may be additional information above.
- the LBA 0 start address 1413 is a field for designating a head address for designating the LBA 0 area of the target data for mapping the compressed data to the LBA 1.
- the LBA0 length 1414 is a field for designating a range of LBA0 starting from the LBA0 start address 1413 to be mapped to LBA1. As with the compressed data size acquisition command, the LBA 0 start address 1413 and the LBA 0 length 1414 are limited to multiples of 8 sectors (4 KB).
- the LBA1 start address 1415 is a field for designating the start address of LBA1 to be mapped.
- the processor 121 of the storage controller 110 manages the data size of the data to be mapped (data size when compressed) and the information on the LBA1 space area that can be mapped (the LBA1 space area where other data is not mapped). Based on this information, the head address of the area on the LBA1 space where the mapping target data can be mapped is stored in the LBA1 start address 1415 field, and the command is issued to the NVM module 126.
- LBA1 start addresses 1415 a specification that specifies a plurality of LBA1 start addresses 1415 is adopted, that is, a configuration in which an LBA0 space area specified by an LBA0 start address 1413 and an LBA0 length 1414 is mapped to a discrete LBA1 space area. It is also possible to take
- the NVM module 126 maps the compressed data associated with the LBA0 space in the range indicated by the LBA0 start address 1413 and the LBA0 length 1414 from the LBA1 start address 1415 over an area corresponding to the compressed data size. More specifically, the PBA (NVM module PBA812) associated with the LBA0 space in the range indicated by the LBA0 start address 1413 and the LBA0 length 1414 is acquired by referring to the LBA0-PBA conversion table. Then, referring to the LBA1-PBA conversion table, from the LBA1 start address 1415, the PBA acquired in the PBA 822 in the LBA1 range (entry specified by the NVM module LBA1 (821)) is the same size as the total size of the acquired PBA. Enter the address.
- the LBA1 mapping response 1420 includes a command ID 1421 and a status 1422.
- a command ID 1421 includes a command ID 1421 and a status 1422.
- response information based on the above information will be described, but there may be additional information above.
- FIG. 15 is a diagram showing an LBA0 mapping command supported by the NVM module 126 in this embodiment and response information to the LBA0 mapping command.
- the LBA0 mapping command 1510 of the NVM module 126 in the present embodiment is configured by an operation code 1511, a command ID 1512, an LBA1 start address 1513, an LBA1 length 1514, and an LBA0 start address 1515 as command information.
- an example of a command based on the above information will be described, but there may be additional information above.
- the LBA1 start address 1513 is a field for designating the start address of the range of the LBA1 space of the compressed data to be mapped.
- the LBA1 length 1514 is a field for designating the range of the LBA1 space starting from the LBA1 start address 1513 to be mapped to LBA0.
- the LBA 0 start address 1515 is a field for designating the start address of LBA 0 to be mapped. Since the storage controller 110 manages the usage state of the LBA0 / 1 space based on the management information of the data cached in the NVM module 126 managed by the storage controller 110, the LBA0 area that can be mapped based on the management status And the start address is written in the LBA 0 start address 1515.
- the address that can be specified as the LBA0 start address 1515 is limited to a multiple of 8 sectors (4 KB).
- the compressed data associated with the LBA1 space in the range indicated by the LBA1 start address 1513 and the LBA1 length 1514 is expanded from the LBA0 start address 1515. Mapping is performed over the area for the data size. More specifically, the PBA associated with the LBA in the range indicated by the LBA1 start address 1513 and the LBA1 length 1514 is acquired by referring to the LBA1-PBA conversion table. Then, referring to the LBA0-PBA conversion table, the address of the acquired PBA is entered from the LBA0 start address 1515 into the PBA822 in the LBA0 range that is the same size as the decompressed size of the compressed data.
- the LBA 0 mapping response 1520 includes only information common to other command response information (command ID 1521, status 1522), and thus description thereof is omitted. A configuration in which additional information other than the common information is included in the LBA 0 mapping response 1520 may be used.
- Storage device operation Write operation (up to cache storage) Subsequently, a write process performed by the storage apparatus 101 according to the first embodiment of the present invention will be described with reference to FIG.
- the storage apparatus 101 obtains the write request designating the expansion volume area and the write target data (write data) from the higher level apparatus 103 and stores the write data in the cache (NVM module 126). A write completion response is transferred to the host device 103.
- the first step S1501 of the write operation is a step in which the storage apparatus 101 receives a write request from the host apparatus.
- the processor 121 of the storage apparatus 101 is notified from the host interface 124 that there is a request from the host apparatus 103, and recognizes that it is a write request from the information notified from the host interface 124, and the data size of the write request Get the address of the write request.
- step S1502 the processor 121 of the storage apparatus 101 secures a cache area for storing write data.
- the processor 121 stores information on the usage status of the storage space of the NVM module 126, specifically, an area storing data on the LBA0 space / LBA1 space of the NVM module 126, and decompression corresponding to the data stored in the area.
- Information about the address of the volume or the compressed volume is managed in the DRAM 125, and this information is hereinafter referred to as cache information.
- the processor 121 refers to the cache information, and selects an area on the LBA0 space of the NVM module 126 that is not larger than the write data size specified by the write request acquired in S1501 from the area that has not yet stored data. Secure.
- This securing process is the same as the disk cache securing process performed by a well-known storage device, so detailed description will be omitted, but the write data storage area (LBA0 space area) is used by other processes. This is a process of locking so as not to be performed and updating the contents of the cache information.
- step S1503 the processor 121 of the storage apparatus 101 instructs the host interface 124 to acquire write data.
- the processor 121 instructs the host interface 124 to transfer data, and notifies the write destination address (LBA) in the decompressed volume designated by the write command from the higher level device 103 as the seed of the guarantee code ID.
- the host interface 124 that has been instructed to transfer data acquires write data from the host device 103 and divides the data by 512B (size of one sector in the first embodiment). Then, a 2B CRC is generated from the 512B data. Further, a 6B ID for each divided 512B data is generated from the type of guarantee code ID instructed by the processor 121.
- the host interface 124 assigns an 8B ID, which is a combination of the 2B CRC and the 6B ID, to each 512B data, generates a sector with a 520B guarantee code, and transfers it to the DRAM 125 of the storage apparatus 101.
- the first embodiment shows an example in which the data of the host interface 124 is transferred to the NVM module 126 via the DRAM 125 of the storage apparatus 101, but the present invention is not limited to this transfer path. For example, it may be transferred directly from the host interface 124 to the NVM module 126.
- step S1504 the processor 121 of the storage apparatus 101 instructs the NVM module 126 to write data. More specifically, the processor 121 instructs the NVM module 126 to write using the write command 1210. At this time, the write command 1210 is given the lower 6B of the decompressed volume address designated by the upper device 103 as the expected value 1217 of the guarantee code ID. In the field of the new guarantee code ID seed 1218, a value notifying that it is invalid is entered.
- the storage apparatus 101 performs a process for failure (for example, notifying the host apparatus 103 of an error and canceling the write process).
- a process for failure for example, notifying the host apparatus 103 of an error and canceling the write process.
- the NVM module 126 determines that the write data is intended for the storage apparatus 101 and sends the write data to the data compression / decompression unit 418. Compress and record.
- step S1506 following step S1505 the processor 121 of the storage apparatus 101 acquires the compressed size of the data recorded by writing from the NVM module 126. More specifically, the compressed size is acquired by referring to the compressed data length 1223 of the write response information 1220 transferred from the NVM module 126.
- the processor 121 that has acquired the size after compression uses the LBA0 address of the NVM module 126 that recorded the write data in the cache information and the address of the decompression volume in order to transfer the compressed data to the compressed volume in a later destage operation. And the compressed data length are managed in association with each other.
- FIG. 18 is a flowchart showing the destage operation of the first embodiment.
- S1601 which is the first step of the destage operation, is a step of determining data to be destaged.
- the storage apparatus 101 manages the last access time from the higher level apparatus 103 for each data stored in the NVM module 126 that is a cache.
- an LRU (Least Recently Used) algorithm is used to determine data to be destaged. For this reason, the processor 121 selects the data stored in the NVM module 126 as data to be destaged in order from the data with the oldest access time.
- step S1602 following S1601 the post-compression data size of the destage target data determined in S1601 is acquired.
- the processor 121 acquires the compressed data length of the destage target data recorded in the LBA0 space of the NVM module 126 from the cache information in which the compressed data length is recorded in S1506 in the write operation.
- Step S1603 following S1602 is a step of securing a compressed volume area that is a destage destination of the write data.
- the storage apparatus 101 stores (destages) write data
- the storage apparatus 101 performs additional writing from the head address of the compressed volume. Therefore, whenever the destaging process is performed on the compressed volume, the storage apparatus 101 stores the next address after the end address on the compressed volume in which the destage target data is written as the next write start position.
- the area of the data length after compression of the destage target data acquired in S1602 is secured from the next write start position on the compression volume.
- the position (address) on the compressed volume in which write data (destage target data) is stored is determined for the first time at this point. If the compressed volume is a so-called logical volume composed of a plurality of final storage devices, the address on the compressed volume determined here and the address on the final storage device where the write data is actually stored are Although not necessarily the same, since the relationship between the address on the compressed volume and the address on the final storage device is fixedly determined, when the address on the compressed volume is determined, the address on the final storage device is also uniquely determined. Therefore, the determination of the position (address) on the compressed volume where the write data is stored is equivalent to the determination of the address on the final storage device.
- the processor 121 manages, in the DRAM 125, information on the correspondence between the (virtual) storage destination address on the decompression volume (the decompression volume address specified by the host apparatus 103 in the write request) and the storage destination address on the compression volume. (Hereinafter, this information is referred to as inter-volume mapping information).
- inter-volume mapping information the processor 121 determines the correspondence between the (virtual) storage destination address on the decompressed volume and the storage destination address on the compressed volume in which the destage target data is stored. The process of recording is also performed.
- This inter-volume mapping information is used to specify the storage address of the read target data on the compressed volume (final storage device) when the storage apparatus 101 receives a data read request from the host apparatus 103.
- step S1604 following S1603, the compressed data of the destage target data recorded in the area on the LBA0 space of the NVM module 126 is mapped to the LBA1 space of the NVM module 126.
- the processor 121 of the storage apparatus 101 manages the information on the usage status of the storage space of the NVM module 126 as cache information, the processor 121 can be secured by referring to this cache information (yet An unused (unused) LBA1 space area that is not associated with compressed data is acquired. Then, by issuing an LBA1 mapping command 1410 to the NVM module 126, the compressed data is associated with the acquired area on the LBA1 space.
- the storage device when the storage device is destaged, in order to acquire (read) the compressed data from the NVM module 126, it is mapped to the LBA1 space independent of the LBA0 space, and the data on the LBA1 space (compressed data)
- the present invention is not limited to this method.
- the same LBA0 space address as that at the time of writing may be specified, and the compressed data may be transferred to the storage device without being decompressed.
- S1604 is not necessary.
- Step S1605 following S1604 is a step in which the storage controller 110 acquires compressed data from the area on the LBA1 space mapped in S1604. Since this step is a characteristic operation of the first embodiment, the process flow will be described in detail below.
- the processor 121 issues a read command 1310 to the NVM module 126 in order to acquire compressed data.
- the read command includes an expected value 1317 of the guarantee code ID to be inspected and a new guarantee code ID to be newly assigned to the compressed sector constituting the read data. Contains seed 1318.
- the cache information is referred to and the data on the decompressed volume (virtual The storage destination address (the decompressed volume address specified by the upper apparatus 103 in the write request) is acquired, and the lower 6B of this address is set as the expected value of the guarantee code ID to be checked.
- the lower 6B of the start address of the area in the compressed volume secured in step S1603 is used as the ID of the guarantee code ID assigned to the compressed sector constituting the read data.
- the processor 415 of the NVM module 126 acquires the read command 1310 issued by the processor 121 of the storage controller 110.
- the processor 415 obtains data associated with the LBA1 area designated by the LBA1 start address 1313 and the LBA1 length 1314 of the read command 1310 from the FM 420 and stores the data in the data buffer 416.
- the acquisition target data may exist not in the FM 420 but in the data buffer 416 (this is because the storage controller 110 in the NVM module 126 in the data write processing received from the host apparatus 103 described with reference to FIG. 17).
- the NVM module 126 temporarily stores the data in the data buffer 416 after issuing an instruction to write data by issuing a write command, but acquires the data from the processor 121 before writing the data to the FM 420. In this case, the process of acquiring data from the FM 420 is not performed.
- step S1703 From step S1703 to at least step S1707 following step S1702, the data compression / decompression unit 418 performs as a series of processes without the processor 415 of the NVM module 126 being intervened.
- the processor 415 instructs the data compression / decompression unit 418 to continuously execute steps S1703 to S1708.
- the data compression / decompression unit 418 that has received the instruction generates a sector with a guarantee code by decompressing the acquired data in step S1703.
- Step S1704 following step S1703 is a step of checking the guarantee code for the decompressed data generated in S1703.
- the data compression / decompression unit 418 confirms that no error has occurred in the sector with the guarantee code using the CRC of the sector with the guarantee code that is the decompressed data, and the compressed data can be decompressed correctly. Confirm. Further, it is confirmed whether the guarantee code ID matches the expected value 1317 of the guarantee code ID given to the command, and the compressed data requested to be read from the processor 121 of the storage apparatus 101 is the data assumed by the storage apparatus 101. Confirm that it is compressed data.
- step S1705 the process branches according to the comparison result in step S1704. If the guarantee code ID of the sector with the guarantee code constituting the decompressed data matches the expected value 1317 of the guarantee code ID given to the read command, the process proceeds to S1706. On the other hand, if they do not match, the error is transferred to the processor 121 of the storage apparatus 101 because the storage control is incorrect.
- Step S1706 which transitions from step S1705, is a step of generating guarantee code information for the compressed data for which it is confirmed that the destage target data is the data intended by the storage.
- the data compression / decompression unit 418 generates a compressed sector by dividing the compressed data every 512 bytes, and then generates a 2B CRC from the data of each compressed sector. Further, the ID of each compressed sector is generated from the seed 1318 of the new guarantee code ID given to the read command.
- the data compression / decompression unit 418 adds the CRC and ID for the compression sector generated in S1706 to the compression sector, and generates a compression sector with a guarantee code.
- step S1708 the processor 415 transfers the compressed sector with the guarantee code generated in S1707 to the storage controller 110.
- step S1709 the processor 415 notifies the processor 121 of the storage controller 110 that the read command has been completed.
- the storage apparatus 101 has stored the destage target data stored in the compressed state as the data that it requested access to, and the data content has an error. It can be confirmed that the data after compression of the destage target data can be expanded. A desired guarantee code can be assigned to the compressed sector with the guarantee code recorded in the compressed volume.
- step S1606 the processor 121 of the storage apparatus 101 records the compressed data constituted by the compressed sector with the guarantee code acquired in S1605 in the compressed volume 5500. More specifically, the processor 121 controls the disk interface 123 to transfer the compressed data to one or more final storage devices (SSD 111 or HDD 112) constituting the compressed volume. At this time, the processor 121 notifies the expected value of the guarantee code ID of the transfer data to the disk interface 123. This expected value is the lower 6B of the address (LBA) on the compressed volume.
- LBA lower 6B of the address
- the disk interface 123 Upon receiving the instruction, the disk interface 123 checks the CRC of the guarantee code of the compressed sector with the guarantee code constituting the transfer data and confirms that no error bit has occurred, and then notifies the ID of the guarantee code and the processor 121. After confirming that the ID of the guarantee code matches the expected value, the data is transferred to the SSD 111 or HDD 112 as the final storage device. If a CRC error or ID mismatch is detected in this check, the disk interface 123 stops data transfer to the final storage device and notifies the processor 121 of the error.
- the read operation according to the first embodiment is started when the processor 121 receives a read request from the higher-level device 103.
- the processor 121 acquires the read target address of the decompression volume included in the read request, and converts it into the address of the compression volume associated with the read target address of the decompression volume. This conversion is performed by referring to the inter-volume mapping information described in the description of the destage processing.
- the address of the compressed volume is converted into an address on the final storage device that constitutes the compressed volume.
- the processor 121 reads the compressed sector with the guarantee code from the address on the final storage device.
- the processor 121 notifies the disk interface 123 of the expected value of the guarantee code ID of the compressed sector with the guarantee code to be read.
- This expected value is the lower 6B of the address of the compressed volume 5500 in which the compressed sector with the guarantee code is stored as described above.
- the disk interface 123 Upon receiving the instruction, the disk interface 123 reads the compressed sector with the guarantee code from the final storage device, and checks the guarantee code. If a CRC error or ID mismatch is detected in this guarantee code check, the disk interface 123 notifies the processor 121 of the error.
- the compressed data with the guarantee code that has passed the guarantee code check is transferred to the storage controller 110 and temporarily stored in the DRAM 125.
- the processor 121 records the compressed sector with the guarantee code acquired from the disk interface 123 in the area on the LBA1 space of the NVM module 126.
- the processor 121 writes data to the NVM module 126 using the write command 1210.
- the lower 6B of the address in the compressed volume 5500 in which the compressed sector with the guarantee code is stored is entered.
- the NVM module 126 acquires a compressed sector with a guarantee code from the storage controller 110, and checks the guarantee code. If a CRC error or ID mismatch is detected in this guarantee code check, the NVM module 126 notifies the processor 121 of the error.
- the processor 121 issues an LBA0 mapping command 1510 to the NVM module 126, thereby mapping the compressed data recorded in the LBA1 space of the NVM module 126 to the LBA0 space so that the decompressed data can be taken out.
- the processor 121 reads the data obtained by decompressing the compressed data using the LBA0 space.
- the processor 121 issues a read command 1310 to the NVM module 126.
- This read command 1310 includes the lower 6B of the address in the decompression volume 5000, which is the storage destination of the virtual read target data, as the expected value of the guarantee code.
- the processor 415 in the NVM module 126 controls the data compression / decompression unit 418 to record one or a plurality of guarantee codes recorded by designating LBA1 at the time of writing.
- the compressed data composed of attached compressed sectors is decompressed to generate read target data composed of one or more sectors with guarantee codes.
- the CRC is inspected using the guarantee code, and it is confirmed that the ID matches the value generated based on the expected value 1217 of the guarantee code ID given to the read command. Thereafter, the data is transferred to the storage controller 110. In this check, if a CRC error or a mismatch with the expected ID value is detected, the NVM module 126 notifies the processor 121 of the error.
- the storage controller 110 that has acquired the sector with the guarantee code constituting the read target data from the NVM module 126 instructs the host interface 124 to transfer the sector with the guarantee code to the host device 103.
- the processor 121 notifies the host interface 124 of the lower 6B of the address in the decompression volume 5000 that is the storage destination of the virtual read target data as the expected value of the guarantee code.
- the host interface 124 confirms that no error bit has occurred for each sector with a guarantee code using the CRC, and uses the expected value of the ID to read data requested by the host device 103. Check that there is. When an error is detected in this inspection, the error is notified to the processor 121.
- the host interface 124 deletes the guarantee code from each sector with the guarantee code, and generates one or more sectors. Then, data composed of one or more sectors is transferred as read request data from the host device 103 to the host device 103, and the read process is terminated.
- the above is the configuration and processing contents of the storage apparatus and NVM module according to the first embodiment.
- the write data is compressed and the data size after compression is determined.
- the storage location of the write data on the final storage device cannot be determined. For this reason, it is difficult to add a guarantee code including information such as a data storage destination address to data in order to improve reliability.
- a storage device when a storage device receives a write request and write data specifying a volume write destination address from a host device, it is generated based on the volume write destination address in the host interface of the storage device.
- a CRC code generated from the first ID and the write data is added to the write data as a guarantee code, and transferred to the NVM module which is a cache device.
- an error can be detected when an error occurs in data during the data transfer process from the host interface to the cache device.
- the data (compressed data) is destaged to the final storage device, that is, when the storage location (write destination address) of the data on the final storage device is determined, the NVM module serving as the cache device is stored on the final storage device.
- the second ID generated based on the write destination address is added to the data as a guarantee code, and the storage device transmits the data with the second ID added to the final storage device.
- the storage device receives the write data from the host device and sends it to the final storage device.
- error checking is possible in the entire process until the write data is stored. Similarly, it is possible to perform error checking in the entire process from when the storage device reads data requested from the host device from the final storage device and transmits the data to the host device via the host interface.
- guarantee codes there are two types of guarantee codes: a guarantee code added to a compressed sector and a guarantee code added to uncompressed data (expanded data).
- the NVM module 126 adds or checks a guarantee code, with the exception of some exceptions, any guarantee code is added depending on whether the address specified in the LBA0 / 1 start address field of the read command or write command is LBA0 or LBA1.
- the present invention is not limited to this method. In the command, use a method such as including information that explicitly indicates whether the guarantee code added to the compressed sector or the guarantee code added to the decompressed data is to be added / inspected. Also good.
- the ID of the new guarantee code is added to the compressed sector with the guarantee code in the process of reading the compressed data from the NVM module 126 and destaging to the final storage device.
- a new guarantee code ID may be added when data is written to 126.
- migration source volume data stored in a certain compression volume 5500
- migration destination volume another compression volume
- the data stored at the address a of the migration source volume may be stored at the address b of the migration destination volume (a and b are different values).
- guarantee code ID part of the compressed sector with the guarantee code.
- the processor 121 of the storage apparatus 101 reads the compressed data (compressed sector with guarantee code) from the area of the final storage device corresponding to the LBA at address a of the migration source volume, and temporarily stores it in the DRAM 125. This process is the same as the read process of the storage apparatus described in the first embodiment.
- the processor 121 records the compressed sector with the guarantee code stored in the DRAM 125 in the area on the LBA1 space of the NVM module 126 by issuing a write command to the NVM module 126.
- This process is also similar to the read process of the storage apparatus described in the first embodiment.
- the new guarantee code ID seed 1218 field of the write command contains the LBA (data storage address) of the migration destination volume. This is different from the read processing of the storage apparatus described in the first embodiment in that the lower 6B of (address b) is stored and a write command is issued.
- the NVM module 126 Upon receiving the instruction, the NVM module 126 acquires the compressed sector with the guarantee code from the storage controller 110, checks the guarantee code, and if no error is found, the NVM module 126 is included in the guarantee code of the compressed sector with the guarantee code. The ID is changed to the value specified by the new guarantee code ID seed 1218 and then stored in the FM 420 or the data buffer 416.
- the processor 121 reads the compressed sector with the guarantee code by issuing a read command designating the LBA1 address storing the compressed sector with the guarantee code stored in the NVM module 126 in the previous process to the NVM module 126.
- the read compressed sector with a guarantee code is written in the area of the final storage device corresponding to the LBA at address b of the migration destination volume.
- the guarantee code (ID portion) of the compression sector already stores the lower 6B value of the address b of the migration destination volume. Here, there is no need to replace the warranty code.
- the processor 121 stores an invalid value in the field of the seed 1318 of the new guarantee code ID of the read command issued here.
- data for which the guarantee code has not been replaced is read from the NVM module 126, and the processor 121 stores the data in the final storage medium via the disk interface.
- the data inspection based on the guarantee code of the sector with the guarantee code is performed, and the process of writing the read compressed sector with the guarantee code to the final storage device In the disk interface, the data inspection based on the guarantee code is performed in the same manner as the destage processing described in the first embodiment.
- the guarantee code can be replaced when data is moved from the migration source volume to the migration destination volume, and error checking can be performed appropriately during the entire data transfer process from the migration source volume to the migration destination volume. It becomes possible.
- the guarantee code When data is moved from the migration source volume to the migration destination volume, it is not always necessary to replace the guarantee code when storing the compressed sector with the guarantee code in the NVM module 126.
- the guarantee code When data is stored in the NVM module 126 from the final storage device, the guarantee code is not reassigned.
- the compressed sector with the guarantee code is read from the NVM module 126 and written to the final storage medium constituting the migration destination volume, the guarantee is performed. The code may be replaced.
- the NVM module plays the same role as the SSD 111 in the storage device shown in FIG.
- the DRAM 125 is used as a cache of the storage device.
- FIG. 20 shows the configuration of the storage apparatus 1801 in the second embodiment. Since many components of the storage apparatus 1801 in the second embodiment are common to the storage apparatus 101 in the first embodiment, the differences will be mainly described below, and the components common to the first embodiment will not be described. Omitted.
- a plurality of NVM modules 1826 are connected by the internal switch 122 in the storage controller 110.
- the NVM module 1826 is a module having the same function as that of the NVM module 126 described in the first embodiment, and is a final storage device that receives a command directly (without passing through a disk interface) from the processor 121.
- the storage apparatus 1801 configures one or a plurality of volumes from one or more NVM modules 1826, and provides the configured volumes to the host apparatus 103.
- FIG. 21 shows that, based on a copy instruction from the upper level apparatus 103, the storage apparatus 1801 stores data stored in a volume A6000 composed of one or a plurality of NVM modules 1826 to which the present invention is applied. Alternatively, an operation of copying to a volume B 6500 constituted by a plurality of NVM modules 1826 is shown.
- the copy instruction from the host device 103 is to notify the storage device 1801 of the address where the copy source data is stored and the address of the copy destination of the data, for example, as in the SCSI Extended Copy command.
- the storage apparatus 1801 of the second embodiment Similar to the storage apparatus of the first embodiment, the storage apparatus 1801 of the second embodiment generates a sector with a guarantee code by adding a guarantee code to each sector of data at the host interface 124, and this sector with the guarantee code is converted into an NVM.
- the data is stored in the area above the LBA0 space of the module 1826. Therefore, the data is stored in the NVM module 1826 in a compressed state.
- the address (LBA) in the volume A6000 in which the sector with the guarantee code is recorded is recorded in the ID portion of the guarantee code of the sector with the guarantee code. Note that the storage apparatus 1801 maintains the reliability by inspecting the guarantee code by the NVM module 1826 at the time of writing.
- the storage apparatus 1801 copies the guarantee code ID in the volume B when copying (writing) the data in the volume A 6000 with the address in the volume A 6000 as the guarantee code ID to the volume B 6500. Change to the address of and read.
- the processor 121 of the storage apparatus 1801 that has received a data copy instruction from the volume A to the volume B from the higher-level apparatus 103 identifies the area of the volume A and the area of the volume B for which the copy instruction has been issued.
- the read command is issued to one or a plurality of NVM modules 1826 storing the data of the volume A area.
- the read command 1310 shown in FIG. 13 is used for this read command.
- the read command 1310 stores an LBA 0 start address 1313 and an LBA 0 length 1314 for notifying the data storage position (position on the LBA 0 space) in each NVM module 1826 constituting the volume A area.
- An address in the volume A is stored as the expected code value 1317, and an address in the volume B to be replaced is stored as the new guarantee code seed 1318.
- Each of the one or more NVM modules 1826 constituting the volume A area receives a read command from the processor 121.
- the processing performed by each NVM module 1826 that has received the read command performs processing similar to the processing described in FIG. 17 of the first embodiment, and therefore will be described with reference to FIG.
- the processing from S1701 to S1704 is the same as that described in the first embodiment. That is, the data associated with the LBA 0 start address 1313 and the LBA 0 length 1314 is acquired from the FM 420 or the data buffer 416 and decompressed, and the CRC and ID of the guarantee code are set for one or more sectors with a guarantee code constituting the data. Check the error used.
- the storage controller 110 acquires a sector with a guarantee code, which has been changed with respect to the guarantee code ID, from the NVM module 1826 and transfers it to one or a plurality of NVM modules constituting the volume B. At this time, the storage apparatus 1801 transfers the write command 1210 shown in FIG. 12 to one or a plurality of NVM modules 1826 constituting the volume B.
- Each of the one or more NVM modules 1826 constituting the volume B area receives the write command 1210 from the storage apparatus 1801.
- This write command 1210 includes an LBA 0 start address 1313, an LBA 0 length 1314 indicating a new storage location of data in each NVM module 1826 constituting the volume B area, and an expected value 1317 of the guarantee code.
- the expected value 1317 of the guarantee code is an address in the copy destination volume B, and is the same as that notified by the processor 121 as the seed of the new guarantee code when reading from the NVM module 1826 constituting the volume A. .
- the NVM module 1826 that has received the write command 1210 receives a sector with a guarantee code to which a new guarantee code is assigned as write data from the storage controller 110, and checks the guarantee code. Specifically, it is confirmed that there is no bit error by using the CRC of the guarantee code, and it is confirmed that the guarantee code ID matches the expected value 1317 of the guarantee code assigned to the write command.
- the above is the data copy operation performed by the storage apparatus of the second embodiment.
- the storage apparatus can be controlled in a state where a guarantee code is always added to the data of the data copy. For this reason, when erroneous data is copied due to a deficiency in the control program, an error notification can be acquired.
- the guarantee code is added and inspected in each final storage device (NVM module) connected to the storage controller 110, the burden of the guarantee code addition and the inspection processing conventionally performed in the storage controller is reduced. Can be offloaded. In general, since a large number of final storage devices are connected to the storage controller, the burden of adding a guarantee code and processing for inspection is large. In the storage apparatus according to the second embodiment of the present invention, since a plurality of NVM modules add and check a guarantee code only for data written to / read from itself, the processing load of guarantee code addition and check Can be dispersed.
- the NVM module 1826 is described as being the same as the NVM module 126 described in the first embodiment. However, the NVM module 1826 is not necessarily the same as the NVM module 126 according to the first embodiment. Absent.
- the NVM module 126 includes a data compression / decompression unit 418 and has a function of compressing and storing write data.
- the NVM module 1826 in the second embodiment is required to compress and store data. Therefore, a configuration without the data compression / decompression unit 418 may be used.
- the write data from the host device 103 is stored in the NVM module 1826 in an uncompressed state, and the data copied from the volume A to the device constituting the volume B is also stored in the uncompressed state.
- the operation is the same as in the second embodiment described above.
- another element in the NVM module such as the I / O interface 411, may be provided with means for checking and adding the guarantee code.
- the parity generation unit 419 is not essential.
- the function of the NVM module providing two logical storage spaces, LBA0 space and LBA1 space, to the host device is not essential.
- the NVM module like a normal SSD or the like, it is a storage device that provides only a single storage space, and is an NVM module having a configuration in which a means for checking and adding a guarantee code is added.
- the objects and effects of the present invention can be achieved.
- the storage controller 110 does not have the disk interface 123 for connecting the final storage apparatus, and the SSD 111 and the HDD 112 are not connected to the storage controller 110.
- the present invention is not limited to the storage apparatus having such a configuration. The present invention is effective even when a disk interface is connected to the internal switch 122 and an SSD or HDD is connected to the disk interface.
- the data stored in the volume A composed of one or a plurality of NVM modules 1826 is copied to the volume B composed of the one or a plurality of NVM modules 1826.
- volume B is configured as the volume B ′ configured from the SSD 111 or the HDD 112 and the data stored in the volume A configured from one or more NVM modules 1826 is copied to the volume B ′.
- the present invention is effective.
- the NVM module to which the present invention is applied has a function of changing the guarantee code given to the data and sending it to the request source, or storing it in the NVM module.
- the storage apparatus can manage data to be controlled with a guarantee code always attached. For this reason, the storage apparatus can perform data transfer while confirming correctness of control, and the reliability of the apparatus can be improved.
- Storage device 102 SAN 103: Host device 104: Management device 110: Storage controller 111: SSD 112: HDD 121: Processor 122: Internal SW 123: Disk interface 124: Host interface 125: DRAM 126: NVM module 410: FM controller 411: I / O interface 413: RAM 414: Switch 416: Data buffer 417: FM interface 418: Data compression / decompression unit 419: Parity generation unit
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Le dispositif de stockage d'après la présente invention comporte un module NVM qui fait office de dispositif à mémoire cache ou de dispositif de stockage final. Lorsqu'il reçoit des données d'écriture provenant d'un dispositif hôte, le dispositif de stockage ajoute un premier code de vérification aux données d'écriture et stocke les données obtenues dans le module NVM. Lorsque les données d'écriture stockées sont lues ultérieurement, le dispositif de stockage délivre une instruction ordonnant au module NVM de lire les données obtenues en ajoutant aux données d'écriture un second code de vérification différent du premier. Lorsqu'il reçoit cette instruction, le module NVM ajoute aux données d'écriture un second code de vérification différent du premier et renvoie les données obtenues au dispositif de stockage.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2014/053000 WO2015118680A1 (fr) | 2014-02-10 | 2014-02-10 | Dispositif de stockage |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2014/053000 WO2015118680A1 (fr) | 2014-02-10 | 2014-02-10 | Dispositif de stockage |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015118680A1 true WO2015118680A1 (fr) | 2015-08-13 |
Family
ID=53777506
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/053000 WO2015118680A1 (fr) | 2014-02-10 | 2014-02-10 | Dispositif de stockage |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2015118680A1 (fr) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019212175A (ja) * | 2018-06-07 | 2019-12-12 | 株式会社日立製作所 | 記憶制御システム及び記憶制御方法 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2006072435A (ja) * | 2004-08-31 | 2006-03-16 | Hitachi Ltd | ストレージシステムおよびデータ記録方法 |
JP2008009635A (ja) * | 2006-06-28 | 2008-01-17 | Hitachi Ltd | ストレージシステム及びそのデータ保護方法 |
-
2014
- 2014-02-10 WO PCT/JP2014/053000 patent/WO2015118680A1/fr active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2006072435A (ja) * | 2004-08-31 | 2006-03-16 | Hitachi Ltd | ストレージシステムおよびデータ記録方法 |
JP2008009635A (ja) * | 2006-06-28 | 2008-01-17 | Hitachi Ltd | ストレージシステム及びそのデータ保護方法 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019212175A (ja) * | 2018-06-07 | 2019-12-12 | 株式会社日立製作所 | 記憶制御システム及び記憶制御方法 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9811277B2 (en) | Information processing system | |
US9304685B2 (en) | Storage array system and non-transitory recording medium storing control program | |
JP5937697B2 (ja) | ストレージシステム | |
US20180173632A1 (en) | Storage device and method for controlling storage device | |
US8850114B2 (en) | Storage array controller for flash-based storage devices | |
US7831764B2 (en) | Storage system having plural flash memory drives and method for controlling data storage | |
US8862808B2 (en) | Control apparatus and control method | |
KR101735866B1 (ko) | 카피-백 동작을 수행하기 위한 방법 및 제어기 | |
JP5722500B2 (ja) | リモートコピーシステム、及びリモートコピー制御方法 | |
US9465561B2 (en) | Storage system and storage control method | |
JP6062060B2 (ja) | ストレージ装置、ストレージシステム、及びストレージ装置制御方法 | |
JP6429963B2 (ja) | ストレージ装置及びストレージ装置の制御方法 | |
US20050144382A1 (en) | Method, system, and program for managing data organization | |
US20060156059A1 (en) | Method and apparatus for reconstructing data in object-based storage arrays | |
US10019315B2 (en) | Control device for a storage apparatus, system, and method of controlling a storage apparatus | |
JP6666540B2 (ja) | ストレージ制御装置、及びプログラム | |
WO2015075837A1 (fr) | Mémoire et procédé de commande associé | |
US7716519B2 (en) | Method and system for repairing partially damaged blocks | |
US11487428B2 (en) | Storage control apparatus and storage control method | |
US10691550B2 (en) | Storage control apparatus and storage control method | |
JP6451769B2 (ja) | ストレージ制御装置、及びストレージ制御プログラム | |
WO2015118680A1 (fr) | Dispositif de stockage | |
JP6163588B2 (ja) | ストレージシステム | |
US11221790B2 (en) | Storage system | |
US20190205044A1 (en) | Device for restoring lost data due to failure of storage drive |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14881826 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14881826 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: JP |