US20180260148A1 - Large scale implementation of a plurality of open channel solid state drives - Google Patents
Large scale implementation of a plurality of open channel solid state drives Download PDFInfo
- Publication number
- US20180260148A1 US20180260148A1 US15/456,308 US201715456308A US2018260148A1 US 20180260148 A1 US20180260148 A1 US 20180260148A1 US 201715456308 A US201715456308 A US 201715456308A US 2018260148 A1 US2018260148 A1 US 2018260148A1
- Authority
- US
- United States
- Prior art keywords
- network
- open channel
- pages
- metadata server
- memory structure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0688—Non-volatile semiconductor memory arrays
Definitions
- This invention generally relates to large scale implementation of a plurality of open channel (OC) solid state drives (SSDs) over a network.
- OC open channel
- SSDs solid state drives
- Modern data centers are comprised of magnetic hard disk drives (HDD) and/or SSDs.
- HDD hard disk drives
- SSDs operate using different technology than HDDs, SSDs provide the same interface and functionality as HDDs for compatibility with existing data center operating systems.
- a typical SSD is comprised of non-volatile NAND media (comprised of memory blocks and pages capable of retaining data when power is disconnected), a controller and an interface (e.g., PCIe, SAS, SATA, or any other interface).
- NAND SSDs are equipped with a flash translation layer (FTL) in the SSD controller that translates logical block addresses from a host device (e.g., addresses used in read, write commands from a computer) to low-level flash operations for the associated physical block and page addresses of the NAND SSD.
- FTL flash translation layer
- the embedded FTL also performs numerous other functions, including: error code correction (i.e., use of redundant data, or parity data to recover a message with errors); garbage collection (i.e., identifying/invalidating stale data in blocks of memory for deletion to free space for future writes); scheduling (i.e., controlling the order in which read and write commands from the host device are executed); over-provisioning (i.e., SSD memory reserved for maintaining write speed); and wear-leveling (i.e., spreading writes across the blocks of the SSD as evenly as possible to ensure that all blocks of the device wear at a roughly similar rate).
- error code correction i.e., use of redundant data, or parity data to recover a message with errors
- garbage collection i.e., identifying/invalidating stale data in blocks of memory for deletion to free space for future writes
- scheduling i.e., controlling the order in which read and write commands from the host device are executed
- over-provisioning i.e., SSD memory reserved for maintaining write speed
- a traditional host device HDD interface is not capable of taking full advantage of the performance of a NAND SSD.
- the host device is not capable of issuing low-level commands that govern how data is programmed and erased in the SSD and the SSD is not aware of when the host device will issue read and write commands. Accordingly, while both the FTL and the host device employ best effort approaches for performing their independent functions, the result is still inefficient utilization of NAND SSD resources, including unpredictable data storage and increased wear-leveling.
- the Linux 4.4 kernel is an example of an operating system kernel that supports open-channel SSDs which follow the NVM Express specification, by providing an abstraction layer called LightNVM.
- open-channel SSDs require significant overhead as the host device operating system must manage the functionality of each open-channel SSD.
- a modern data center i.e., a system
- This storage includes a physical layer of disks (e.g., open-channel SSDs) and software abstraction that exposes the disks to applications that control physical storage within such disks. Separating the software abstraction layer from the physical layer for each open-channel SSD has the disadvantage of consuming an increasingly larger proportion of system resources as additional open-channel SSDs are deployed within the system.
- a method of implementing a plurality of open channel SSDs over a network includes receiving a request over a network to create an abstracted memory structure having a first capacity, identifying a plurality of open channel SSDs accessible over the network that have memory blocks with free pages, determining that the identified memory blocks with free pages have a cumulative capacity greater than or equal to the first capacity, allocating free pages from the plurality of open channel SSDs to create the abstracted memory structure, maintaining physical addresses corresponding to the allocated free pages of the plurality of open channel SSDs, and assigning the abstracted memory structure to a host device connected to the network.
- the request to create the abstracted memory structure further comprises identifying a reliability. In one embodiment, allocating memory from the plurality of open channel SSDs further comprises identifying memory blocks having the smallest erase count.
- the method further comprises assigning the abstracted memory structure to a second host device connected to the network. In another embodiment, the method further comprises assigning a part of the abstracted memory structure to a second host device connected to the network.
- the method further comprises, detecting a total capacity of the network. In one embodiment, the method further comprises reducing the total capacity of the network by reserving a second capacity. In another embodiment, the method further comprises detecting a total capacity assigned to all preexisting abstracted memory structures of the network. In another embodiment, the method further comprises detecting a total capacity of written data within all preexisting abstracted memory structures of the network.
- the method further comprises maintaining the abstracted memory structure within a non-volatile memory. In one embodiment, the method further comprises, maintaining an access control list identifying that the abstracted memory structure is assigned to the host device.
- a method of implementing a plurality of open channel SSDs over a network includes receiving a command and identification information from a host device over a network, using the identification information to identify an abstracted memory structure comprised of a plurality of open channel SSDs connected to the network, determining that the host device has access to the abstracted memory structure, identifying a physical address within the abstracted memory structure corresponding to the command, wherein the physical address comprises a memory block and a memory page of one of the plurality of open channel SSDs accessible via the network, sending the physical address to the host device, and detecting that the host device issued the command over the network using the physical address.
- the method further comprises detecting that the command is a read command. In another embodiment, the method further comprises detecting that the command is a write command. In another embodiment, the method further comprises detecting a capacity associated with the write command.
- the method further comprises detecting a trigger that initiates a bad block management process.
- the detecting a trigger comprises detecting a failure to execute the command issued by the host device.
- the method further comprises identifying bad blocks associated with a failure to execute the command issued by the host device.
- the method further comprises identifying a reliability mechanism associated with the bad blocks and repairing the bad blocks using the detected reliability mechanism. In one embodiment, the method further comprises updating a bad block list to identify the bad blocks.
- FIG. 1 is a block diagram of one embodiment implementing an open channel SSD.
- FIG. 2 is a block diagram of one embodiment implementing a plurality of open channel SSDs over a network.
- FIG. 3 is a block diagram of one embodiment implementing a plurality of open channel SSDs over a network.
- FIG. 4 is a block diagram of one embodiment of implementing abstracted memory structures for a plurality of open channel SSDs over a network.
- FIG. 5 is a block diagram of one embodiment of maintaining physical mapping information for a plurality of open channel SSDs over a network.
- FIG. 6 is a flow chart of steps for one embodiment of allocating memory from a plurality of open channel SSDs over a network.
- FIG. 7 is a flow chart of steps for one embodiment of garbage collection from a plurality of open channel SSDs over a network.
- FIG. 8 is a flow chart of steps for one embodiment of managing bad blocks in a plurality of open channel SSDs over a network.
- FIG. 9 is a flow chart of steps for one embodiment of implementing a plurality of open channel SSDs over a network.
- FIG. 1 is a block diagram illustrating one embodiment of a system 100 for implementing an open channel SSD 101 .
- Open channel SSD 101 includes a NAND memory 103 and a controller 105 .
- Controller 105 does not include an FTL, and instead, has the limited functionality of performing error correction and interfacing with a host device 109 (e.g., a computer) through an adaptor 107 over a connection 111 .
- Adaptor 107 can be any type of standard interface, including, but not limited to, Peripheral Component Interconnect Express (PCIe), Serial Attached SCSI (SAS), Serial ATA (SATA), or any other interface known in the art.
- Open channel SSD 101 has a standard form factor (e.g., 2.5 inch NVME, SAS or SATA, M.2, U.2 or any other standard interface) and can be inserted into any standard computer platform (e.g., a server).
- PCIe Peripheral Component Interconnect Express
- SAS Serial Attached SCSI
- Controller 105 of open channel SSD 101 exposes the physical geometry of the NAND memory 103 to the host 109 .
- the geometry of NAND memory 103 includes, for example, the number of blocks, the number of pages in a block and the size of each page, among other parameters.
- the host 109 uses a driver 113 that encapsulates a NAND API to communicate with controller 105 to retrieve the geometry of the NAND memory 103 and execute program, read and erase commands on specific blocks and pages of the NAND memory 103 .
- FIG. 2 is a block diagram illustrating one embodiment of implementing a plurality of open channel SSDs over a network 200 .
- an open channel device 201 includes a plurality of open channel SSDs 203 , a CPU 205 , and a network interface card 207 .
- the plurality of open channel SSDs 203 are connected to the open channel device 201 using a standard interface including, but not limited to, Peripheral Component Interconnect Express (PCIe), Serial Attached SCSI (SAS), Serial ATA (SATA), or any other interface known in the art.
- PCIe Peripheral Component Interconnect Express
- SAS Serial Attached SCSI
- SATA Serial ATA
- the CPU 205 functions as the controller for the plurality of open channel SSDs 203 and exposes the geometry of each of the plurality of open channel SSDs 203 to the remote host 213 through a driver 209 that encapsulates an open channel API.
- CPU 205 may also perform other functions unrelated to the plurality of open channel SSDs 203 .
- the driver 209 communicates with CPU 205 to identify a device ID corresponding to each of the plurality of open channel SSDs 203 .
- the network interface card 207 enables Ethernet connectivity between open channel device 201 and remote host 213 .
- Remote host 213 accesses open channel device 201 over network connection 211 by implementing a transport protocol such as iSCSI, NVMe over Fabric (NVMeoF) or any other transport protocol known in the art. Remote host 213 can further access any one of the open channel SSDs 203 through driver 209 by using the corresponding device ID. Remote host 213 can use driver 209 to communicate with controller 205 and execute program, read and erase commands on specific blocks and pages of each of the plurality of open channel SSDs 203 .
- a transport protocol such as iSCSI, NVMe over Fabric (NVMeoF) or any other transport protocol known in the art.
- Driver 209 can use driver 209 to communicate with controller 205 and execute program, read and erase commands on specific blocks and pages of each of the plurality of open channel SSDs 203 .
- remote host 213 accesses an open channel device 221 over a network connection 231 .
- the open channel device 221 includes an open channel SSD 223 (similar in function to each of the plurality of open channel SSDs 203 ), a controller 225 that exposes the geometry of open channel SSD 223 , and a network interface card 227 (similar in function to network interface card 207 ). As illustrated in FIG. 2 , any number of open channel devices 221 may be connected to the network such that remote host 213 can access each of the open channel devices 221 over network connection 231 .
- Network connections 211 and 213 may comprise any type of wired or wireless network connections known in the art.
- FIG. 3 is a block diagram illustrating one embodiment of implementing a plurality of open channel SSDs over a network 300 .
- An open channel device 301 includes a plurality of open channel SSDs 303 , a CPU 305 , and a network interface card 307 (all of which have similar functionality to the corresponding components 201 , 203 , 205 , and 207 of FIG. 2 ).
- An open channel device, 321 includes an open channel SSD 323 , a controller 325 that exposes the geometry of open channel SSD 323 , and a network interface card 327 (all of which have similar functionality to the corresponding components 221 , 223 , 225 , 227 and 229 of FIG. 2 ).
- Open channel SSD 323 (identified in FIG. 3 as OCSSD j) and each of the plurality of open channel SSDs 303 (identified in FIG. 3 as OCSSD i) may be identified by any naming convention (e.g., numeral, character and the like).
- One or more remote hosts 313 are capable of accessing open channel device 301 and/or open channel device 321 over a network switch 329 .
- Network switch 329 is comprised of a plurality of nodes (identified in FIG. 3 as nodes A-H). Nodes within network 300 may be identified by any convention (e.g., numeral, character and the like).
- each of the plurality of open channel SSDs accessible via a particular node of network switch 329 has a unique identifier.
- every open channel SSD in network 300 has a unique identifier.
- one or more management stations 333 are capable of accessing open channel device 301 and/or open channel device 331 over network switch 329 .
- Remote host 313 communicates with devices over network 300 through node A of network switch 329 over network connection 317 .
- a driver 309 encapsulates an open channel API that allows remote host(s) 313 to communicate with open channel device 301 and/or open channel device 321 .
- Management station 333 communicates with devices over network 300 through node D of network switch 329 over network connection 319 .
- a driver 329 encapsulates an open channel API that allows management station(s) 333 to communicate with open channel device 301 and/or open channel device 321 .
- Metadata server 351 includes a mapping of nodes and devices within network 300 by maintaining a list of identifiers (e.g., Internet Protocol (IP) address, or any other unique identifier). Any device within network 300 can send a request to metadata server 351 to identify the network location of any other device in network 300 .
- IP Internet Protocol
- Any management station 333 may send a request to identify the network location of one or more remote host(s) 313 , open channel device 301 , open channel device 321 , or any open channel SSD 303 or 323 .
- any remote host 313 may send a request to identify the network location of open channel device 301 , open channel device 303 , or any open channel SSD 303 or 323 .
- any number of nodes may be added to network 300 by adding additional network switches 329 .
- any number of devices may be added to the network 300 by adding devices to available nodes. When nodes and/devices are added or removed from network 300 , the mapping of nodes and devices within metadata server 351 is updated to reflect such changes.
- management station 333 sends a request to metadata server 351 to create one or more abstracted memory structures using the plurality of open channel SSDs 303 that comprise open channel device(s) 301 and/or the open channel SSDs 323 that comprise open channel devices 321 .
- the request to create an abstracted memory structure may include various parameters including capacity (e.g., 1 GigaByte) and reliability (e.g., RAID1 mirrored storage), among other parameters.
- Management station 333 may request that metadata server 351 use any type of reliability scheme for creating an abstracted memory structures, including, for example, eager simple (i.e., a single array mapping virtual address to physical page location), RAID1 mirror (i.e., two or more arrays mapping virtual address to physical page location, such that a corrupted array can be recovered using one or more of the mirrored arrays), RAIDS (i.e., an array mapping virtual address to physical page location and parity location, such that corrupted data can be recovered using the parity information), or any other reliability scheme known in the art.
- eager simple i.e., a single array mapping virtual address to physical page location
- RAID1 mirror i.e., two or more arrays mapping virtual address to physical page location, such that a corrupted array can be recovered using one or more of the mirrored arrays
- RAIDS i.e., an array mapping virtual address to physical page location and parity location, such that corrupted data can be recovered using the parity information
- Management station 333 may request that the abstracted memory structure is comprised of part or all of the open channel SSD memory capacity comprising any of the following: one or more nodes of network switch 329 ; one or more devices connected to network switch 329 ; one or more blocks within a particular open channel SSD 303 or 323 ; one or more pages within a block of a particular open channel SSD 303 or 323 ; and or any combination thereof.
- Metadata server 351 creates the abstracted memory structure(s) in response to the request from management station 333 and maintains a mapping of the nodes, devices, blocks and/or pages comprising such subsystem(s) (i.e., the physical NAND mapping).
- management station 333 requests that metadata server 351 assign one or more abstracted memory structures to one or more host devices 313 connected to network 300 over network switch 329 .
- one or more abstracted memory structures may be assigned to a single host device 313 .
- one or more abstracted memory structures may be shared by a plurality of host devices 313 .
- a part of an abstracted memory structure may be assigned to one or more host devices 313 .
- Metadata server 351 maintains a list of all abstracted memory structures and controls whether one or more host devices 313 have access to one or more of such subsystems.
- management station 333 requests that the metadata server 351 create an abstracted memory structure for such host device.
- the metadata server 351 does not create the requested abstracted memory structure for the new host device 313 unless the sum of the requested capacity for the new abstracted memory structure and the total capacity of abstracted memory structures assigned to preexisting host devices does not exceed the total page capacity of network 300 .
- a portion of the total page capacity of network 300 is reserved as an over-provision for maintaining speed and performance. Accordingly, in this embodiment, the total page capacity of network 300 is reduced by the portion of the total page capacity reserved as an over-provision.
- metadata server 333 may assign an aggregate capacity greater than the total capacity of network 300 if previously assigned capacity is not fully written with data (i.e., assigned capacity is not fully used). In this embodiment, if data utilization exceeds the total capacity, the metadata server 333 sends an alert to one or more of the management stations 333 .
- any node comprising network 300 can send a request to metadata server 351 to create an abstracted memory structure.
- the mapping of abstracted memory systems is maintained with a centralized global repository (not shown) in communication with metadata server 351 .
- the centralized global repository is comprised of a non-volatile memory (e.g., PROM, EPROM, EEPROM, FRAM, NVRAM, flash, or any other type of non-volatile memory known in the art).
- a plurality of metadata servers 351 and/or centralized global repositories are used for redundancy and to maintain high speed performance.
- FIG. 4 is a block diagram illustrating one embodiment of implementing abstracted memory structures 400 for a plurality of open channel SSDs over a network.
- management station 333 sends a request to metadata server 351 to create one or more abstracted memory structures 400 using the plurality of open channel SSDs 303 that comprise open channel device(s) 301 and/or the open channel SSDs 323 that comprise open channel devices 321 .
- the abstracted memory structures 400 may comprise any form as illustrated by FIG. 4 .
- the abstracted memory structure 400 comprises one or more subsystems 403 .
- Each subsystem 403 has a unique identifier that is maintained by the metadata server 351 in a subsystem list 401 .
- Each subsystem 403 may further be comprised of one or more namespaces 407 .
- Each subsystem 403 includes subsystem information 405 that describes parameters of the subsystem.
- Subsystem information 405 includes subsystem capacity (e.g., 1 Gigabyte), an access control list that identifies the devices (e.g., host devices 300 ) that have access to a particular subsystem over network 300 , and/or a list of namespace information that comprises each subsystem 403 , among other parameters.
- Each namespace 407 includes header information 409 a that describes parameters of the namespace and virtual pages 409 b , 409 c and 409 d that comprise namespace 407 .
- a namespace 407 may be comprised of any number of virtual pages.
- Header information 409 a includes namespace capacity (e.g., 500 Megabytes), and mapping between the virtual pages comprising the namespace and the location of the physical pages (i.e., the location of the physical memory pages across one or more open channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters.
- virtual pages 409 b , 409 c and 409 d identify the location of the corresponding physical pages by referencing the node ID, device ID, block and page where the physical page resides (e.g., network Node E, device OCSSD 1, Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond to virtual pages 409 b , 409 c and 409 d .
- the abstracted memory structure 400 comprised of subsystems 403 and namespaces 407 meets the NVMe specification or any other specification known in the art.
- the abstracted memory structure 400 comprises one or more volumes 413 .
- Each volume 413 has a unique identifier that is maintained by the metadata server 351 in a volume list 411 .
- Each volume 413 may further be comprised of one or more logical unit numbers (LUNs) 417 .
- Each volume 413 includes volume information 415 that describes parameters of the volume.
- Volume information 415 may include volume capacity (e.g., 1 Gigabyte), an access control list that identifies the devices (e.g., host devices 300 ) that have access to a particular volume over network 300 , and/or a list of LUN information that comprises each volume 413 , among other parameters.
- Each LUN 417 may include header information 419 a that describes parameters of the LUN and virtual pages 419 b , 419 c and 419 d that comprise LUN 417 .
- An LUN 417 may be comprised of any number of virtual pages.
- Header information 419 a includes LUN capacity (e.g., 500 Megabytes), and mapping between the virtual pages comprising the LUN and the location of the physical pages (i.e., the location of the physical memory pages across one or more open channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters.
- virtual pages 419 b , 419 c and 419 d identify the location of the corresponding physical pages by referencing the node ID, device ID, block and page where the physical page resides (e.g., network Node E, device OCSSD 1, Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond to virtual pages 419 b , 419 c and 419 d .
- the abstracted memory structure 400 comprised of volumes 413 and LUNs 417 meets the SCSI specification or any other specification known in the art.
- the abstracted memory structure 400 comprises one or more partitions 423 .
- Each partition 423 has a unique identifier that is maintained by the metadata server 351 in a partition list 421 .
- Each partition 423 may further be comprised of one or more files 427 .
- Each partition 423 includes partition information 425 that describes parameters of the partition.
- Partition information 425 includes partition capacity (e.g., 1 Gigabyte), an access control list that identifies the devices (e.g., host devices 300 ) that have access to a particular partition over network 300 , and/or a list of file information that comprises each partition 423 , among other parameters.
- Each file 427 includes header information 429 a that describes parameters of the file and virtual pages 429 b , 429 c and 429 d that comprise file 427 .
- a file 427 may be comprised of any number of virtual pages.
- Header information 429 a includes file capacity (e.g., 500 Megabytes), and mapping between the virtual pages comprising the file and the location of the physical pages (i.e., the location of the physical memory pages across one or more open channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters.
- virtual pages 429 b , 429 c and 429 d identify the location of the corresponding physical pages by referencing the node ID, device ID, block and page where the physical page resides (e.g., network Node E, device OCSSD 1, Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond to virtual pages 429 b , 429 c and 429 d .
- the abstracted memory structure 400 comprised of partitions 423 and files 427 is a file system that meets any known file system specification, including, for example, NTFS, FAT32, FAT, HPFS, or any other specification known in the art.
- the abstracted memory structure 400 comprises one or more objects 433 .
- Each object 433 has a unique identifier that is maintained by the metadata server 351 in an object list 431 .
- Each object 433 is further comprised of virtual pages 439 b , 439 c and 439 d .
- Each object 433 includes object information 435 that describes parameters of the object.
- Object information 435 includes object capacity (e.g., 100 Megabytes), object metadata identifying the type of object (e.g., EXIF for a JPG image), an access control list that identifies the devices (e.g., host devices 300 ) that have access to a particular object over network 300 , and/or mapping between the virtual pages comprising the object and the location of the physical pages (i.e., the location of the physical memory pages across one or more open channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters.
- An object 437 may be comprised of any number of virtual pages.
- virtual pages 439 b , 439 c and 439 d identify the location of the corresponding physical pages by referencing the node ID, device ID, block and page where the physical page resides (e.g., network Node E, device OCSSD 1, Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond to virtual pages 439 b , 439 c and 439 d.
- FIG. 5 is a block diagram illustrating one embodiment of maintaining physical mapping information 500 for a plurality of open channel SSDs over a network.
- metadata server 351 maintains a list of all nodes that comprise network 300 , and the list of open channel devices 301 and 321 that comprise each node.
- network 300 is comprised of one or more nodes, including node 501 .
- Metadata server 351 maintains a list of nodes including node information 503 corresponding to node 501 .
- Node information 503 identifies node parameters, including a node identifier (e.g., IP address or equivalent), number of devices comprising the node, identifiers for each device comprising the node, and vendor information for each device, among other parameters.
- node 501 is comprised of one or more devices, including device 505 .
- Metadata server 351 maintains a list of devices comprising node 501 , including device information 507 corresponding to device 505 .
- Device information 507 identifies device parameters, including device geometry (i.e., the number of blocks that comprise the device, the number of pages per block, and the size of each page), among other parameters.
- device 505 is comprised of blocks 509 , 511 and 513 .
- metadata server 351 maintains block information for each of blocks 509 , 511 and 513 , including an erase counter (i.e., the number of times a block has been erased), block usage (i.e., whether the block is free, partially written or fully written), status (i.e., whether the block is functional/okay or non-functional/bad), and a pointer referencing the block, among other information.
- an erase counter i.e., the number of times a block has been erased
- block usage i.e., whether the block is free, partially written or fully written
- status i.e., whether the block is functional/okay or non-functional/bad
- a pointer referencing the block among other information.
- block information for block 509 indicates the block is “free,” has been erased 500 times, has a status of “ok,” and can be identified by “PTR 1.”
- Block information for block 511 indicates the block is “partial[ly]” used, has been erased 600 times, has a status of
- Metadata server 351 further maintains a bad block list 529 and a free block list 531 .
- bad block list 529 and free block list 531 are maintained as part of device information 507 of device 505 .
- bad block list 529 and free block list 531 contain bad block and free block information for each of the plurality of open channel SSDs over the network.
- metadata server 351 updates bad block list 529 with the pointer corresponding to the bad block. For example, the pointer for bad block 511 (PTR 2) is identified on bad block list 529 .
- metadata server 351 updates free block list 531 with the pointer corresponding to the block and identifies the block erase count. For example, the pointer for free block 509 (PTR1) is identified on free block list 531 . In a preferred embodiment, the blocks identified on free block list 531 are sorted by erase count.
- device 505 is further comprised of block 515 .
- Block 515 is comprised of pages 517 , 519 and 521 .
- Metadata server 351 maintains page information for each of pages 517 , 519 and 521 , including status of the page data and a pointer referencing the page, among other information. For example, page 517 has a status of “valid” (i.e., includes valid data), page 519 has a status of “dirty” (i.e., includes old/invalid data, and page 521 has a status of “free” (i.e., has no data).
- Blocks having pages containing “dirty” data i.e., dirty data
- a page with dirty data is, for example, updated or new data that is not stored anywhere else, or data that would otherwise be permanently lost because it is not stored in non-volatile memory.
- device 505 may be comprised of any number of blocks, and each block may be comprised of any number of pages.
- Node 501 may be comprised of any number of devices.
- network 300 may be comprised of any number of nodes.
- FIG. 5 illustrates the use of pointers to identify a particular block or page in network 300 , any identifier can be used instead of or in addition to a pointer.
- metadata server 351 updates the node list.
- metadata server 351 updates the device list.
- FIG. 6 is a flow chart of steps 600 illustrating one embodiment of allocating memory from a plurality of open channel SSDs over a network.
- management station 333 sends a request to metadata server 351 to create one or more abstracted memory structures 400
- metadata server 351 allocates physical pages and blocks that comprise one or more memory devices attached to one or more nodes within network 300 .
- metadata server 351 maintains lists identifying the nodes within network 300 , all devices connected to a particular node, and blocks and pages of each device.
- metadata server 351 allocates memory for the abstracted data structure by allocated physical blocks and pages from one or more devices connected to one or more nodes within network 300 .
- metadata server 351 allocates physical blocks and/or pages corresponding to the abstracted memory structure for the write or program operation.
- metadata server 351 may allocate additional capacity to an abstracted memory structure 400 when the abstracted memory structure 400 is fully written with data such that no further data may be written.
- Metadata server 351 receives a request to allocate memory.
- the request to allocate memory is either sent by management station 333 or by a host device 313 regarding one or more existing or one or more new abstracted memory structures 400 .
- Metadata server 351 calculates the number of memory pages that must be allocated to satisfy the memory allocation request.
- metadata server 351 allocates memory capacity using a simple eager algorithm.
- the method steps 600 moves from step 601 to step 603 .
- metadata server 351 identifies blocks within the applicable abstracted memory structure 400 that are partially written (e.g., partially written block 511 ).
- metadata server 351 determines whether the identified pages from partially written blocks are greater than or equal to the number of pages required for allocation (step 601 ). If the identified pages are sufficient to satisfy the allocation request, the method steps proceed to step 613 and metadata server 351 allocates the identified pages.
- step 611 metadata server 351 identifies free blocks with the smallest erase count on free block list 531 (e.g., free block 509 identified by “PTR1”).
- metadata server 351 uses only free blocks to identify a sufficient number of pages to satisfy the allocation request.
- metadata server 351 uses pages from partially written blocks identified in step 603 in combination with pages from blocks on free block list 531 to satisfy the allocation request. The method steps 600 then proceed back to step 609 to determine whether the pages identified in step 611 are sufficient to satisfy the allocation request.
- metadata server 351 allocates memory pages for an object from the same blocks where the object is stored.
- the method steps 600 moves from step 601 to step 605 .
- metadata server 351 identifies free pages on partially written blocks where the object is stored (e.g., partially written block 511 ).
- metadata server 351 determines whether the identified pages from partially written blocks are greater than or equal to the number of pages required for allocation (step 601 ). If the identified pages are sufficient to satisfy the allocation request, the method steps proceed to step 613 and metadata server 351 allocates the identified pages.
- step 611 metadata server 351 identifies free blocks with the smallest erase count on free block list 531 (e.g., free block 509 identified by “PTR1”).
- metadata server 351 uses only free blocks to identify a sufficient number of pages to satisfy the allocation request.
- metadata server 351 uses pages from partially written blocks identified in step 605 in combination with pages from blocks on free block list 531 to satisfy the allocation request. The method steps 600 then proceed back to step 609 to determine whether the pages identified in step 611 are sufficient to satisfy the allocation request.
- metadata server 351 allocates memory pages in accordance with a particular reliability policy (e.g., from different devices on the same node or different devices on different nodes of network 300 ). For example, if the specified reliability policy is RAID 5 arranged in groups of N units, allocated pages should come from at least N different devices for device-level reliability or N different nodes for node-level reliability (e.g., N may be defined as 3 units or any other number of units).
- the method steps 600 moves from step 601 to step 607 .
- metadata server 351 identifies free pages on partially written blocks of different devices or different nodes, as specified by the reliability policy (e.g., partially written block 511 ).
- metadata server 351 determines whether the identified pages from partially written blocks are greater than or equal to the number of pages required for allocation (step 601 ). If the identified pages are sufficient to satisfy the allocation request, the method steps proceed to step 613 and metadata server 351 allocates the identified pages. If the identified pages are not sufficient to satisfy the allocation request, the method steps proceed to step 611 and metadata server 351 identifies free blocks with the smallest erase count on free block list 531 (e.g., free block 509 identified by “PTR1”) of different devices or different nodes, as specified by the reliability policy. In one embodiment, at step 611 , metadata server 351 uses only free blocks to identify a sufficient number of pages to satisfy the allocation request.
- metadata server 351 uses pages from partially written blocks identified in step 607 in combination with pages from blocks on free block list 531 to satisfy the allocation request.
- the method steps 600 then proceed back to step 609 to determine whether the pages identified in step 611 are sufficient to satisfy the allocation request.
- the metadata server 351 allocates such pages in response to the request. If the pages identified in step 611 are not sufficient to satisfy the allocation request, the metadata server 351 responds with a message indicating that the abstracted memory structure 400 has insufficient capacity for the requested allocation. In one embodiment, metadata server 351 may resolve such insufficient capacity for the requested allocation by allocating unused capacity within network 300 or unused capacity from another abstracted memory subsystem within network 300 .
- block information e.g., blocks 509 , 511 and 513 identifying block information
- page information e.g., block 515 with page 517 , 519 and 521 identifying page information
- the reliability mechanism e.g., mirror, RAID and the like.
- the abstracted memory structure information e.g., 405 , 415 , 425 or 435
- header information e.g., 409 a , 419 a or 429 a
- the reliability mechanism e.g., mirror, RAID, and the like.
- FIG. 7 is a flow chart of steps for one embodiment of garbage collection from a plurality of open channel SSDs over a network.
- Metadata server 351 maintains page information for each of pages 517 , 519 and 521 , including status of the page data and a pointer referencing the page, among other information.
- the metadata server 351 executes a sequence of garbage collection steps to process pages containing “dirty” data (e.g., page 519 ).
- metadata server 351 detects a trigger within network 300 that causes the metadata server to initiate the garbage collection process within one or more abstracted memory structures 400 .
- the management station 333 may set a threshold number of free blocks at which the metadata server 351 initiates the garbage collection process for the respective abstracted memory structure 400 .
- the threshold may be set by any computer within network 300 , including remote host(s) 313 or metadata server(s) 351 .
- the threshold for garbage collection for one or more abstracted memory structures 400 may be periodic.
- metadata server 351 identifies blocks having pages with dirty data within the respective abstracted memory structure(s) 400 using page information within each block (e.g., block 515 ) of each device (e.g., device 505 ) attached to each node (e.g., node 501 ) of network 300 that comprise the respective abstracted memory structure(s) 400 .
- valid pages are pages with data that is not invalid or stale and pages with dirty data are a subset of the valid pages.
- metadata server 351 reads the valid pages (e.g., pages 517 and 521 ) from each block (e.g., block 515 ) that contains pages with dirty data (e.g., page 519 ) and determines the number of valid pages read. In one embodiment, the metadata server 351 also identifies whether the valid pages read are associated with a particular namespace 407 within a subsystem 403 , a particular LUN 417 within a volume 413 , a particular file 427 within a partition 423 , or a particular object 433 .
- metadata server 351 allocates new pages equivalent to the number of valid pages read. As discussed with respect to FIG. 6 , allocation of new pages may be performed in various different ways.
- metadata server 351 writes the valid pages read (step 705 ) to the allocated pages (step 707 ).
- the metadata server 351 updates the physical block and page information for the valid pages written to the allocated pages (step 709 ). In one embodiment, metadata server 351 updates the physical block and page information to the allocated pages (step 707 ) for the valid pages (step 705 ) that correspond to a particular namespace 407 , LUN 417 , file 427 or object 433 .
- the metadata server 351 sends a command erasing the blocks with pages having dirty data (e.g., block 515 ).
- FIG. 8 is a flow chart 800 of steps for one embodiment of managing bad blocks in a plurality of open channel SSDs over a network.
- Metadata server 351 maintains block information for each of blocks 509 , 511 and 513 , including status of the block and a pointer referencing the block, among other information.
- the metadata server 351 executes a process for managing bad blocks (e.g., block 511 ).
- metadata server 351 detects a trigger within network 300 that causes the metadata server to initiate the bad block management process within one or more abstracted memory structures 400 .
- the management station 333 may set a threshold number of bad blocks at which the metadata server 351 initiates the bad block management process for the respective abstracted memory structure 400 .
- the threshold may be set by any computer within network 300 , including remote host(s) 313 or metadata server(s) 351 .
- the threshold for initiating the bad block management process for one or more abstracted memory structures 400 may be periodic.
- the threshold for initiating the bad block management process may be related to a command failure, such as a failure to execute a read or write command initiated by one of the computers (e.g., remote host(s) 313 , management station(s) 333 , and/or metadata server(s) 351 ) within network 300 .
- a command failure such as a failure to execute a read or write command initiated by one of the computers (e.g., remote host(s) 313 , management station(s) 333 , and/or metadata server(s) 351 ) within network 300 .
- metadata server 351 identifies bad blocks within the respective abstracted memory structure(s) 400 using block information for each block (e.g., blocks 509 , 511 and 513 ) of each device (e.g., device 505 ) attached to each node (e.g., node 501 ) of network 300 that comprise the respective abstracted memory structure(s) 400 .
- metadata server 351 determines whether pages on the bad blocks are protected by a reliability mechanism (e.g., mirror, RAID and the like).
- metadata server 351 identifies the reliability mechanism from block information (e.g., blocks 509 , 511 and 513 identifying block information) and/or page information (e.g., block 515 with page 517 , 519 and 521 identifying page information). In another embodiment, metadata server 351 identifies the reliability mechanism from the abstracted memory structure information (e.g., 405 , 415 , 425 or 435 ) and/or header information (e.g., 409 a , 419 a or 429 a ). If the pages on the bad block are protected by a reliability mechanism, per flow chart 800 , the process moves from step 805 to step 807 .
- block information e.g., blocks 509 , 511 and 513 identifying block information
- page information e.g., block 515 with page 517 , 519 and 521 identifying page information.
- metadata server 351 identifies the reliability mechanism from the abstracted memory structure information (e.g., 405 , 415
- metadata server 351 allocates new pages to repair the pages on the bad blocks (e.g., as discussed with respect to FIG. 6 ).
- metadata server 351 repairs the bad pages using the appropriate reliability mechanism. For example, if the bad pages are protected by mirror reliability, metadata server 351 identifies the physical location of the mirrored pages and reads the data from the mirrored pages. If the bad pages are protected by RAID reliability, metadata server 351 uses the parity pages to repair the bad pages.
- metadata server 351 writes the repaired pages (step 809 ) to the newly allocated pages (step 807 ).
- metadata server 351 updates the physical block and page information for the repaired pages written to the allocated pages.
- metadata server 351 marks the status of the bad blocks as “bad” (e.g., block 511 ) and updates the bad block list 529 by adding a reference (e.g., pointer “PTR2”), referencing the bad block.
- a reference e.g., pointer “PTR2”
- any reference may be used within bad block list 529 to identify bad blocks (e.g., block 511 ).
- step 805 if metadata server 351 determines that the pages are not protected by a reliability mechanism, per the flow chart 800 , the process moves from step 805 to step 815 .
- metadata server 351 sets the status of the bad blocks as “bad” (e.g., block 511 ) and updates the bad block list 529 by adding a reference (e.g., pointer “PTR2”), referencing the bad block.
- a reference e.g., pointer “PTR2”
- FIG. 9 depicts a flow chart 900 showing steps for one embodiment of implementing a plurality of open channel SSDs over a network.
- a remote host 313 issues a command to metadata server 351 .
- multiple remote hosts 313 may simultaneously issue one or more commands to metadata server 351 .
- the process moves from step 901 to step 903 .
- the read command includes identification information for the abstracted memory structure (e.g., identifying subsystem 403 and namespace 407 , volume 413 and LUN 417 , partition 423 and file 427 , or object 423 ) and the length of the write command.
- metadata server 351 allocates physical pages in response to the write command (e.g., as discussed with respect to FIG. 6 ).
- metadata server 351 returns mapping information corresponding to the write command to remote host 313 .
- the mapping information identifies the location of the physical block and pages corresponding to the write command.
- the mapping information identifies the node (e.g., node E), the device (e.g., open channel device 301 ), the open channel SSD (e.g., OCSSD 303 ), and the block and page numbers corresponding to the write command.
- remote host 313 uses the physical block and page mapping information to issue one or more write commands to write the data to each page of the corresponding OCSSD(s) (e.g., OCSSDs 303 and/or 323 ) identified in the mapping information.
- remote host 313 uses the physical block and page mapping information for each of the mirror locations to issue one or more write commands to write the data to each page of the corresponding OCSSD(s) (e.g., OCSSDs 303 and/or 323 ) identified in the mapping information for each mirror location.
- OCSSD(s) e.g., OCSSDs 303 and/or 323
- remote host 313 uses the physical block and page mapping information to issue one or more write commands to write the data and parity data to each page of the corresponding OCSSD(s) identified in the mapping information.
- the metadata server 351 updates mapping information identifying the physical blocks and pages where the data is written.
- Metadata server 351 also updates block information (e.g., sets block usage as “partial” or “full”) and page information (e.g., sets page status as “valid”). If the data written by remote host 313 was previously written at another location (i.e., old data), metadata server 351 updates the status of the page information containing the old data as “dirty” (e.g., page 519 ) for the next garbage collection process (as discussed with respect to FIG. 7 ).
- block information e.g., sets block usage as “partial” or “full”
- page information e.g., sets page status as “valid”. If the data written by remote host 313 was previously written at another location (i.e., old data), metadata server 351 updates the status of the page information containing the old data as “dirty” (e.g., page 519 ) for the next garbage collection process (as discussed with respect to FIG. 7 ).
- the process moves from step 901 to step 905 .
- the read command includes identification information for the abstracted memory structure (e.g., identifying subsystem 403 and namespace 407 , volume 413 and LUN 417 , partition 423 and file 427 , or object 423 ) and the virtual pages corresponding to the command.
- metadata server 351 returns mapping information corresponding to the read command to remote host 313 .
- the mapping information identifies the location of the physical block and pages corresponding to the read command.
- the mapping information identifies the node (e.g., node E), the device (e.g., open channel device 301 ), the open channel SSD (e.g., OCSSD 303 ), and the block and page numbers corresponding to the read request.
- the metadata server 351 sends mapping information for all mirror locations comprising the requested data.
- host device 313 determines for each page the mirror location from which to read the data.
- the metadata server 351 sends mapping information including data pages and parity pages.
- the remote host 313 may use the parity pages to repair the data if there is a read error.
- remote host 313 issues one or more read commands to read the data from each page of the corresponding OCSSD(s) identified in the mapping information.
- the method steps 900 for a read command terminate at step 907 .
- remote host 313 may issue a get command read an object or a put command to write an object such as a JPG image (e.g., object 433 ).
- the method steps for getting or putting an object are similar to the method steps for reading or writing data as discussed above.
Abstract
Description
- This invention generally relates to large scale implementation of a plurality of open channel (OC) solid state drives (SSDs) over a network.
- Modern data centers are comprised of magnetic hard disk drives (HDD) and/or SSDs. Although SSDs operate using different technology than HDDs, SSDs provide the same interface and functionality as HDDs for compatibility with existing data center operating systems. A typical SSD is comprised of non-volatile NAND media (comprised of memory blocks and pages capable of retaining data when power is disconnected), a controller and an interface (e.g., PCIe, SAS, SATA, or any other interface).
- NAND SSDs are equipped with a flash translation layer (FTL) in the SSD controller that translates logical block addresses from a host device (e.g., addresses used in read, write commands from a computer) to low-level flash operations for the associated physical block and page addresses of the NAND SSD. The embedded FTL also performs numerous other functions, including: error code correction (i.e., use of redundant data, or parity data to recover a message with errors); garbage collection (i.e., identifying/invalidating stale data in blocks of memory for deletion to free space for future writes); scheduling (i.e., controlling the order in which read and write commands from the host device are executed); over-provisioning (i.e., SSD memory reserved for maintaining write speed); and wear-leveling (i.e., spreading writes across the blocks of the SSD as evenly as possible to ensure that all blocks of the device wear at a roughly similar rate). Because the FTL of NAND SSDs can accommodate the logical interface of host devices, NAND SSDs easily integrate with standard hard disk drive (HDD) compatible interfaces and offload overhead that would otherwise be needed for the host device to perform the functions of the FTL.
- Although traditional NAND SSDs are easy to integrate and have the advantage of the FTL offloading host device overhead, a traditional host device HDD interface is not capable of taking full advantage of the performance of a NAND SSD. For example, the host device is not capable of issuing low-level commands that govern how data is programmed and erased in the SSD and the SSD is not aware of when the host device will issue read and write commands. Accordingly, while both the FTL and the host device employ best effort approaches for performing their independent functions, the result is still inefficient utilization of NAND SSD resources, including unpredictable data storage and increased wear-leveling.
- One technique for increasing efficiency between a host-device with a conventional HDD-interface and the FTL of a NAND SSD is through an application program interface (API) that shares valuable information between such devices (i.e., through Storage Intelligence). However, this has not proven to be an achievable approach as the complexity of coordinating FTL functionality with host device functionality can further reduce efficiency of the NAND SSD.
- Another technique involves using a NAND SSD that does not have a firmware FTL, which requires the host device operating system to manage the physical solid-state storage. The Linux 4.4 kernel is an example of an operating system kernel that supports open-channel SSDs which follow the NVM Express specification, by providing an abstraction layer called LightNVM. However, open-channel SSDs require significant overhead as the host device operating system must manage the functionality of each open-channel SSD.
- For example, a modern data center (i.e., a system) stores and process petabytes of data on a monthly basis. This storage includes a physical layer of disks (e.g., open-channel SSDs) and software abstraction that exposes the disks to applications that control physical storage within such disks. Separating the software abstraction layer from the physical layer for each open-channel SSD has the disadvantage of consuming an increasingly larger proportion of system resources as additional open-channel SSDs are deployed within the system.
- Accordingly, there is an unmet demand for a system that can easily scale to accommodate any number of open-channel SSDs and can efficiently manage a plurality of open-channel SSDs in groups or pools rather than at the individual device level, thereby reducing system overhead, increasing the efficiency of garbage collection and reducing wear on individual open-channel SSDs.
- In one embodiment, a method of implementing a plurality of open channel SSDs over a network includes receiving a request over a network to create an abstracted memory structure having a first capacity, identifying a plurality of open channel SSDs accessible over the network that have memory blocks with free pages, determining that the identified memory blocks with free pages have a cumulative capacity greater than or equal to the first capacity, allocating free pages from the plurality of open channel SSDs to create the abstracted memory structure, maintaining physical addresses corresponding to the allocated free pages of the plurality of open channel SSDs, and assigning the abstracted memory structure to a host device connected to the network.
- In one embodiment, the request to create the abstracted memory structure further comprises identifying a reliability. In one embodiment, allocating memory from the plurality of open channel SSDs further comprises identifying memory blocks having the smallest erase count.
- In one embodiment, the method further comprises assigning the abstracted memory structure to a second host device connected to the network. In another embodiment, the method further comprises assigning a part of the abstracted memory structure to a second host device connected to the network.
- In one embodiment, the method further comprises, detecting a total capacity of the network. In one embodiment, the method further comprises reducing the total capacity of the network by reserving a second capacity. In another embodiment, the method further comprises detecting a total capacity assigned to all preexisting abstracted memory structures of the network. In another embodiment, the method further comprises detecting a total capacity of written data within all preexisting abstracted memory structures of the network.
- In one embodiment, the method further comprises maintaining the abstracted memory structure within a non-volatile memory. In one embodiment, the method further comprises, maintaining an access control list identifying that the abstracted memory structure is assigned to the host device.
- In one embodiment, a method of implementing a plurality of open channel SSDs over a network includes receiving a command and identification information from a host device over a network, using the identification information to identify an abstracted memory structure comprised of a plurality of open channel SSDs connected to the network, determining that the host device has access to the abstracted memory structure, identifying a physical address within the abstracted memory structure corresponding to the command, wherein the physical address comprises a memory block and a memory page of one of the plurality of open channel SSDs accessible via the network, sending the physical address to the host device, and detecting that the host device issued the command over the network using the physical address.
- In one embodiment, the method further comprises detecting that the command is a read command. In another embodiment, the method further comprises detecting that the command is a write command. In another embodiment, the method further comprises detecting a capacity associated with the write command.
- In one embodiment, the method further comprises detecting a trigger that initiates a bad block management process. In one embodiment, the detecting a trigger comprises detecting a failure to execute the command issued by the host device. In one embodiment, the method further comprises identifying bad blocks associated with a failure to execute the command issued by the host device.
- In one embodiment, the method further comprises identifying a reliability mechanism associated with the bad blocks and repairing the bad blocks using the detected reliability mechanism. In one embodiment, the method further comprises updating a bad block list to identify the bad blocks.
-
FIG. 1 is a block diagram of one embodiment implementing an open channel SSD. -
FIG. 2 is a block diagram of one embodiment implementing a plurality of open channel SSDs over a network. -
FIG. 3 is a block diagram of one embodiment implementing a plurality of open channel SSDs over a network. -
FIG. 4 is a block diagram of one embodiment of implementing abstracted memory structures for a plurality of open channel SSDs over a network. -
FIG. 5 is a block diagram of one embodiment of maintaining physical mapping information for a plurality of open channel SSDs over a network. -
FIG. 6 is a flow chart of steps for one embodiment of allocating memory from a plurality of open channel SSDs over a network. -
FIG. 7 is a flow chart of steps for one embodiment of garbage collection from a plurality of open channel SSDs over a network. -
FIG. 8 is a flow chart of steps for one embodiment of managing bad blocks in a plurality of open channel SSDs over a network. -
FIG. 9 is a flow chart of steps for one embodiment of implementing a plurality of open channel SSDs over a network. -
FIG. 1 is a block diagram illustrating one embodiment of asystem 100 for implementing an open channel SSD 101. Open channel SSD 101 includes aNAND memory 103 and acontroller 105.Controller 105 does not include an FTL, and instead, has the limited functionality of performing error correction and interfacing with a host device 109 (e.g., a computer) through anadaptor 107 over aconnection 111.Adaptor 107 can be any type of standard interface, including, but not limited to, Peripheral Component Interconnect Express (PCIe), Serial Attached SCSI (SAS), Serial ATA (SATA), or any other interface known in the art. Open channel SSD 101 has a standard form factor (e.g., 2.5 inch NVME, SAS or SATA, M.2, U.2 or any other standard interface) and can be inserted into any standard computer platform (e.g., a server). -
Controller 105 of open channel SSD 101 exposes the physical geometry of theNAND memory 103 to thehost 109. The geometry ofNAND memory 103 includes, for example, the number of blocks, the number of pages in a block and the size of each page, among other parameters. Thehost 109 uses adriver 113 that encapsulates a NAND API to communicate withcontroller 105 to retrieve the geometry of theNAND memory 103 and execute program, read and erase commands on specific blocks and pages of theNAND memory 103. -
FIG. 2 is a block diagram illustrating one embodiment of implementing a plurality of open channel SSDs over anetwork 200. In one embodiment, anopen channel device 201 includes a plurality ofopen channel SSDs 203, aCPU 205, and anetwork interface card 207. The plurality ofopen channel SSDs 203 are connected to theopen channel device 201 using a standard interface including, but not limited to, Peripheral Component Interconnect Express (PCIe), Serial Attached SCSI (SAS), Serial ATA (SATA), or any other interface known in the art. TheCPU 205 functions as the controller for the plurality ofopen channel SSDs 203 and exposes the geometry of each of the plurality ofopen channel SSDs 203 to theremote host 213 through adriver 209 that encapsulates an open channel API.CPU 205 may also perform other functions unrelated to the plurality ofopen channel SSDs 203. Thedriver 209 communicates withCPU 205 to identify a device ID corresponding to each of the plurality ofopen channel SSDs 203. Thenetwork interface card 207 enables Ethernet connectivity betweenopen channel device 201 andremote host 213. -
Remote host 213 accessesopen channel device 201 overnetwork connection 211 by implementing a transport protocol such as iSCSI, NVMe over Fabric (NVMeoF) or any other transport protocol known in the art.Remote host 213 can further access any one of theopen channel SSDs 203 throughdriver 209 by using the corresponding device ID.Remote host 213 can usedriver 209 to communicate withcontroller 205 and execute program, read and erase commands on specific blocks and pages of each of the plurality ofopen channel SSDs 203. - In one embodiment,
remote host 213 accesses anopen channel device 221 over anetwork connection 231. Theopen channel device 221 includes an open channel SSD 223 (similar in function to each of the plurality of open channel SSDs 203), acontroller 225 that exposes the geometry ofopen channel SSD 223, and a network interface card 227 (similar in function to network interface card 207). As illustrated inFIG. 2 , any number ofopen channel devices 221 may be connected to the network such thatremote host 213 can access each of theopen channel devices 221 overnetwork connection 231. - In another embodiment, there may be any number of
open channel devices 201 and/or any number ofopen channel devices 221 comprising the plurality of open channel SSDs implemented overnetwork 200.Network connections -
FIG. 3 is a block diagram illustrating one embodiment of implementing a plurality of open channel SSDs over anetwork 300. Anopen channel device 301 includes a plurality ofopen channel SSDs 303, aCPU 305, and a network interface card 307 (all of which have similar functionality to the correspondingcomponents FIG. 2 ). An open channel device, 321 includes anopen channel SSD 323, acontroller 325 that exposes the geometry ofopen channel SSD 323, and a network interface card 327 (all of which have similar functionality to the correspondingcomponents FIG. 2 ). In one embodiment, there may be any number ofopen channel devices 301 and/or any number ofopen channel devices 321 comprising the plurality of open channel SSDs implemented overnetwork 300. Open channel SSD 323 (identified inFIG. 3 as OCSSD j) and each of the plurality of open channel SSDs 303 (identified inFIG. 3 as OCSSD i) may be identified by any naming convention (e.g., numeral, character and the like). - One or more
remote hosts 313 are capable of accessingopen channel device 301 and/oropen channel device 321 over anetwork switch 329.Network switch 329 is comprised of a plurality of nodes (identified inFIG. 3 as nodes A-H). Nodes withinnetwork 300 may be identified by any convention (e.g., numeral, character and the like). In a preferred embodiment, each of the plurality of open channel SSDs accessible via a particular node ofnetwork switch 329 has a unique identifier. In another embodiment, every open channel SSD innetwork 300 has a unique identifier. Similar toremote host 313, one ormore management stations 333 are capable of accessingopen channel device 301 and/oropen channel device 331 overnetwork switch 329. -
Remote host 313 communicates with devices overnetwork 300 through node A ofnetwork switch 329 overnetwork connection 317. Adriver 309 encapsulates an open channel API that allows remote host(s) 313 to communicate withopen channel device 301 and/oropen channel device 321.Management station 333 communicates with devices overnetwork 300 through node D ofnetwork switch 329 overnetwork connection 319. Adriver 329 encapsulates an open channel API that allows management station(s) 333 to communicate withopen channel device 301 and/oropen channel device 321. - One or
more metadata servers 351 are connected to node G ofnetwork switch 329 overnetwork connection 315.Metadata server 351 includes a mapping of nodes and devices withinnetwork 300 by maintaining a list of identifiers (e.g., Internet Protocol (IP) address, or any other unique identifier). Any device withinnetwork 300 can send a request tometadata server 351 to identify the network location of any other device innetwork 300. For example, anymanagement station 333 may send a request to identify the network location of one or more remote host(s) 313,open channel device 301,open channel device 321, or anyopen channel SSD remote host 313 may send a request to identify the network location ofopen channel device 301,open channel device 303, or anyopen channel SSD network 300 by adding additional network switches 329. Further, any number of devices may be added to thenetwork 300 by adding devices to available nodes. When nodes and/devices are added or removed fromnetwork 300, the mapping of nodes and devices withinmetadata server 351 is updated to reflect such changes. - In one embodiment,
management station 333 sends a request tometadata server 351 to create one or more abstracted memory structures using the plurality ofopen channel SSDs 303 that comprise open channel device(s) 301 and/or theopen channel SSDs 323 that compriseopen channel devices 321. The request to create an abstracted memory structure may include various parameters including capacity (e.g., 1 GigaByte) and reliability (e.g., RAID1 mirrored storage), among other parameters.Management station 333 may request thatmetadata server 351 use any type of reliability scheme for creating an abstracted memory structures, including, for example, eager simple (i.e., a single array mapping virtual address to physical page location), RAID1 mirror (i.e., two or more arrays mapping virtual address to physical page location, such that a corrupted array can be recovered using one or more of the mirrored arrays), RAIDS (i.e., an array mapping virtual address to physical page location and parity location, such that corrupted data can be recovered using the parity information), or any other reliability scheme known in the art. -
Management station 333 may request that the abstracted memory structure is comprised of part or all of the open channel SSD memory capacity comprising any of the following: one or more nodes ofnetwork switch 329; one or more devices connected to networkswitch 329; one or more blocks within a particularopen channel SSD open channel SSD Metadata server 351 creates the abstracted memory structure(s) in response to the request frommanagement station 333 and maintains a mapping of the nodes, devices, blocks and/or pages comprising such subsystem(s) (i.e., the physical NAND mapping). - In one embodiment,
management station 333 requests that metadataserver 351 assign one or more abstracted memory structures to one ormore host devices 313 connected to network 300 overnetwork switch 329. In one embodiment, one or more abstracted memory structures may be assigned to asingle host device 313. In another embodiment, one or more abstracted memory structures may be shared by a plurality ofhost devices 313. In another embodiment, a part of an abstracted memory structure may be assigned to one ormore host devices 313.Metadata server 351 maintains a list of all abstracted memory structures and controls whether one ormore host devices 313 have access to one or more of such subsystems. - In one embodiment, when a
new host device 313 is added tonetwork 300,management station 333 requests that themetadata server 351 create an abstracted memory structure for such host device. In one embodiment, themetadata server 351 does not create the requested abstracted memory structure for thenew host device 313 unless the sum of the requested capacity for the new abstracted memory structure and the total capacity of abstracted memory structures assigned to preexisting host devices does not exceed the total page capacity ofnetwork 300. In one embodiment, a portion of the total page capacity ofnetwork 300 is reserved as an over-provision for maintaining speed and performance. Accordingly, in this embodiment, the total page capacity ofnetwork 300 is reduced by the portion of the total page capacity reserved as an over-provision. In another embodiment,metadata server 333 may assign an aggregate capacity greater than the total capacity ofnetwork 300 if previously assigned capacity is not fully written with data (i.e., assigned capacity is not fully used). In this embodiment, if data utilization exceeds the total capacity, themetadata server 333 sends an alert to one or more of themanagement stations 333. - In one embodiment, any
node comprising network 300 can send a request tometadata server 351 to create an abstracted memory structure. In another embodiment the mapping of abstracted memory systems is maintained with a centralized global repository (not shown) in communication withmetadata server 351. In one embodiment, the centralized global repository is comprised of a non-volatile memory (e.g., PROM, EPROM, EEPROM, FRAM, NVRAM, flash, or any other type of non-volatile memory known in the art). In another embodiment, a plurality ofmetadata servers 351 and/or centralized global repositories are used for redundancy and to maintain high speed performance. -
FIG. 4 is a block diagram illustrating one embodiment of implementingabstracted memory structures 400 for a plurality of open channel SSDs over a network. As discussed with respect toFIG. 3 ,management station 333 sends a request tometadata server 351 to create one or moreabstracted memory structures 400 using the plurality ofopen channel SSDs 303 that comprise open channel device(s) 301 and/or theopen channel SSDs 323 that compriseopen channel devices 321. Theabstracted memory structures 400 may comprise any form as illustrated byFIG. 4 . - In one embodiment, the
abstracted memory structure 400 comprises one ormore subsystems 403. Eachsubsystem 403 has a unique identifier that is maintained by themetadata server 351 in asubsystem list 401. Eachsubsystem 403 may further be comprised of one ormore namespaces 407. Eachsubsystem 403 includessubsystem information 405 that describes parameters of the subsystem.Subsystem information 405 includes subsystem capacity (e.g., 1 Gigabyte), an access control list that identifies the devices (e.g., host devices 300) that have access to a particular subsystem overnetwork 300, and/or a list of namespace information that comprises eachsubsystem 403, among other parameters. Eachnamespace 407 includesheader information 409 a that describes parameters of the namespace andvirtual pages namespace 407. Anamespace 407 may be comprised of any number of virtual pages.Header information 409 a includes namespace capacity (e.g., 500 Megabytes), and mapping between the virtual pages comprising the namespace and the location of the physical pages (i.e., the location of the physical memory pages across one or moreopen channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters. In one embodiment,virtual pages device OCSSD 1,Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond tovirtual pages abstracted memory structure 400 comprised ofsubsystems 403 andnamespaces 407 meets the NVMe specification or any other specification known in the art. - In one embodiment, the
abstracted memory structure 400 comprises one ormore volumes 413. Eachvolume 413 has a unique identifier that is maintained by themetadata server 351 in avolume list 411. Eachvolume 413 may further be comprised of one or more logical unit numbers (LUNs) 417. Eachvolume 413 includesvolume information 415 that describes parameters of the volume.Volume information 415 may include volume capacity (e.g., 1 Gigabyte), an access control list that identifies the devices (e.g., host devices 300) that have access to a particular volume overnetwork 300, and/or a list of LUN information that comprises eachvolume 413, among other parameters. EachLUN 417 may includeheader information 419 a that describes parameters of the LUN andvirtual pages LUN 417. AnLUN 417 may be comprised of any number of virtual pages.Header information 419 a includes LUN capacity (e.g., 500 Megabytes), and mapping between the virtual pages comprising the LUN and the location of the physical pages (i.e., the location of the physical memory pages across one or moreopen channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters. In one embodiment,virtual pages device OCSSD 1,Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond tovirtual pages abstracted memory structure 400 comprised ofvolumes 413 andLUNs 417 meets the SCSI specification or any other specification known in the art. - In one embodiment, the
abstracted memory structure 400 comprises one ormore partitions 423. Eachpartition 423 has a unique identifier that is maintained by themetadata server 351 in apartition list 421. Eachpartition 423 may further be comprised of one ormore files 427. Eachpartition 423 includespartition information 425 that describes parameters of the partition.Partition information 425 includes partition capacity (e.g., 1 Gigabyte), an access control list that identifies the devices (e.g., host devices 300) that have access to a particular partition overnetwork 300, and/or a list of file information that comprises eachpartition 423, among other parameters. Eachfile 427 includesheader information 429 a that describes parameters of the file andvirtual pages file 427. Afile 427 may be comprised of any number of virtual pages.Header information 429 a includes file capacity (e.g., 500 Megabytes), and mapping between the virtual pages comprising the file and the location of the physical pages (i.e., the location of the physical memory pages across one or moreopen channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters. In one embodiment,virtual pages device OCSSD 1,Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond tovirtual pages abstracted memory structure 400 comprised ofpartitions 423 and files 427 is a file system that meets any known file system specification, including, for example, NTFS, FAT32, FAT, HPFS, or any other specification known in the art. - In one embodiment, the
abstracted memory structure 400 comprises one ormore objects 433. Eachobject 433 has a unique identifier that is maintained by themetadata server 351 in anobject list 431. Eachobject 433 is further comprised ofvirtual pages object 433 includesobject information 435 that describes parameters of the object.Object information 435 includes object capacity (e.g., 100 Megabytes), object metadata identifying the type of object (e.g., EXIF for a JPG image), an access control list that identifies the devices (e.g., host devices 300) that have access to a particular object overnetwork 300, and/or mapping between the virtual pages comprising the object and the location of the physical pages (i.e., the location of the physical memory pages across one or moreopen channel SSDs 323 and/or 321 on the same or different devices of a single node or different nodes of the network), among other parameters. An object 437 may be comprised of any number of virtual pages. In one embodiment,virtual pages device OCSSD 1,Block 1, Page 1). Any naming convention or identifier may be used to identify the location of the physical pages that correspond tovirtual pages -
FIG. 5 is a block diagram illustrating one embodiment of maintainingphysical mapping information 500 for a plurality of open channel SSDs over a network. As discussed with respect toFIG. 3 ,metadata server 351 maintains a list of all nodes that comprisenetwork 300, and the list ofopen channel devices network 300 is comprised of one or more nodes, includingnode 501.Metadata server 351 maintains a list of nodes includingnode information 503 corresponding tonode 501.Node information 503 identifies node parameters, including a node identifier (e.g., IP address or equivalent), number of devices comprising the node, identifiers for each device comprising the node, and vendor information for each device, among other parameters. In one embodiment,node 501 is comprised of one or more devices, includingdevice 505.Metadata server 351 maintains a list ofdevices comprising node 501, includingdevice information 507 corresponding todevice 505.Device information 507 identifies device parameters, including device geometry (i.e., the number of blocks that comprise the device, the number of pages per block, and the size of each page), among other parameters. In one embodiment,device 505 is comprised ofblocks embodiment metadata server 351 maintains block information for each ofblocks block 509 indicates the block is “free,” has been erased 500 times, has a status of “ok,” and can be identified by “PTR 1.” Block information forblock 511, indicates the block is “partial[ly]” used, has been erased 600 times, has a status of “bad” and can be identified by “PTR 2.” -
Metadata server 351 further maintains abad block list 529 and afree block list 531. In one embodiment,bad block list 529 andfree block list 531 are maintained as part ofdevice information 507 ofdevice 505. In another embodiment,bad block list 529 andfree block list 531 contain bad block and free block information for each of the plurality of open channel SSDs over the network. In one embodiment, when block information for a particular block identifies the block as “bad,”metadata server 351 updatesbad block list 529 with the pointer corresponding to the bad block. For example, the pointer for bad block 511 (PTR 2) is identified onbad block list 529. In one embodiment, when block information for a particular block identifies the block as “free,”metadata server 351 updatesfree block list 531 with the pointer corresponding to the block and identifies the block erase count. For example, the pointer for free block 509 (PTR1) is identified onfree block list 531. In a preferred embodiment, the blocks identified onfree block list 531 are sorted by erase count. - In one embodiment,
device 505 is further comprised ofblock 515.Block 515 is comprised ofpages Metadata server 351 maintains page information for each ofpages page 517 has a status of “valid” (i.e., includes valid data),page 519 has a status of “dirty” (i.e., includes old/invalid data, andpage 521 has a status of “free” (i.e., has no data). Blocks having pages containing “dirty” data (i.e., dirty data) are processed during a garbage collection process executed by themetadata server 351. In one embodiment, a page with dirty data is, for example, updated or new data that is not stored anywhere else, or data that would otherwise be permanently lost because it is not stored in non-volatile memory. - In one
embodiment device 505 may be comprised of any number of blocks, and each block may be comprised of any number of pages. In another embodiment,Node 501 may be comprised of any number of devices. In another embodiment,network 300 may be comprised of any number of nodes. AlthoughFIG. 5 illustrates the use of pointers to identify a particular block or page innetwork 300, any identifier can be used instead of or in addition to a pointer. In one embodiment when a node is added or removed from thenetwork 300,metadata server 351 updates the node list. In another embodiment, when a device is added or removed from a node,metadata server 351 updates the device list. -
FIG. 6 is a flow chart ofsteps 600 illustrating one embodiment of allocating memory from a plurality of open channel SSDs over a network. As discussed with respect toFIG. 3 , whenmanagement station 333 sends a request tometadata server 351 to create one or moreabstracted memory structures 400,metadata server 351 allocates physical pages and blocks that comprise one or more memory devices attached to one or more nodes withinnetwork 300. As discussed with respect toFIG. 5 ,metadata server 351 maintains lists identifying the nodes withinnetwork 300, all devices connected to a particular node, and blocks and pages of each device. In one embodiment, when amanagement station 333 requests that metadataserver 351 create anabstracted memory structure 400,metadata server 351 allocates memory for the abstracted data structure by allocated physical blocks and pages from one or more devices connected to one or more nodes withinnetwork 300. In another embodiment, when ahost device 313 issues a command to write or program data to a particularabstracted memory structure 400,metadata server 351 allocates physical blocks and/or pages corresponding to the abstracted memory structure for the write or program operation. In another embodiment,metadata server 351 may allocate additional capacity to anabstracted memory structure 400 when theabstracted memory structure 400 is fully written with data such that no further data may be written. - At
step 601,metadata server 351 receives a request to allocate memory. The request to allocate memory is either sent bymanagement station 333 or by ahost device 313 regarding one or more existing or one or more newabstracted memory structures 400.Metadata server 351 calculates the number of memory pages that must be allocated to satisfy the memory allocation request. - In one embodiment,
metadata server 351 allocates memory capacity using a simple eager algorithm. In this embodiment, the method steps 600 moves fromstep 601 to step 603. Atstep 603,metadata server 351 identifies blocks within the applicableabstracted memory structure 400 that are partially written (e.g., partially written block 511). Atstep 609,metadata server 351 determines whether the identified pages from partially written blocks are greater than or equal to the number of pages required for allocation (step 601). If the identified pages are sufficient to satisfy the allocation request, the method steps proceed to step 613 andmetadata server 351 allocates the identified pages. If the identified pages are not sufficient to satisfy the allocation request, the method steps proceed to step 611 andmetadata server 351 identifies free blocks with the smallest erase count on free block list 531 (e.g.,free block 509 identified by “PTR1”). In one embodiment, atstep 611,metadata server 351 uses only free blocks to identify a sufficient number of pages to satisfy the allocation request. In another embodiment,metadata server 351 uses pages from partially written blocks identified instep 603 in combination with pages from blocks onfree block list 531 to satisfy the allocation request. The method steps 600 then proceed back to step 609 to determine whether the pages identified instep 611 are sufficient to satisfy the allocation request. - In one embodiment,
metadata server 351 allocates memory pages for an object from the same blocks where the object is stored. In this embodiment, the method steps 600 moves fromstep 601 to step 605. Atstep 605,metadata server 351 identifies free pages on partially written blocks where the object is stored (e.g., partially written block 511). Atstep 609,metadata server 351 determines whether the identified pages from partially written blocks are greater than or equal to the number of pages required for allocation (step 601). If the identified pages are sufficient to satisfy the allocation request, the method steps proceed to step 613 andmetadata server 351 allocates the identified pages. If the identified pages are not sufficient to satisfy the allocation request, the method steps proceed to step 611 andmetadata server 351 identifies free blocks with the smallest erase count on free block list 531 (e.g.,free block 509 identified by “PTR1”). In one embodiment, atstep 611,metadata server 351 uses only free blocks to identify a sufficient number of pages to satisfy the allocation request. In another embodiment,metadata server 351 uses pages from partially written blocks identified instep 605 in combination with pages from blocks onfree block list 531 to satisfy the allocation request. The method steps 600 then proceed back to step 609 to determine whether the pages identified instep 611 are sufficient to satisfy the allocation request. - In one
embodiment metadata server 351 allocates memory pages in accordance with a particular reliability policy (e.g., from different devices on the same node or different devices on different nodes of network 300). For example, if the specified reliability policy is RAID 5 arranged in groups of N units, allocated pages should come from at least N different devices for device-level reliability or N different nodes for node-level reliability (e.g., N may be defined as 3 units or any other number of units). In this embodiment, the method steps 600 moves fromstep 601 to step 607. Atstep 607,metadata server 351 identifies free pages on partially written blocks of different devices or different nodes, as specified by the reliability policy (e.g., partially written block 511). Atstep 609,metadata server 351 determines whether the identified pages from partially written blocks are greater than or equal to the number of pages required for allocation (step 601). If the identified pages are sufficient to satisfy the allocation request, the method steps proceed to step 613 andmetadata server 351 allocates the identified pages. If the identified pages are not sufficient to satisfy the allocation request, the method steps proceed to step 611 andmetadata server 351 identifies free blocks with the smallest erase count on free block list 531 (e.g.,free block 509 identified by “PTR1”) of different devices or different nodes, as specified by the reliability policy. In one embodiment, atstep 611,metadata server 351 uses only free blocks to identify a sufficient number of pages to satisfy the allocation request. In another embodiment,metadata server 351 uses pages from partially written blocks identified instep 607 in combination with pages from blocks onfree block list 531 to satisfy the allocation request. The method steps 600 then proceed back to step 609 to determine whether the pages identified instep 611 are sufficient to satisfy the allocation request. - In each of the preceding embodiments, if the pages identified in
step 611 are sufficient to satisfy the allocation request, atstep 613, themetadata server 351 allocates such pages in response to the request. If the pages identified instep 611 are not sufficient to satisfy the allocation request, themetadata server 351 responds with a message indicating that theabstracted memory structure 400 has insufficient capacity for the requested allocation. In one embodiment,metadata server 351 may resolve such insufficient capacity for the requested allocation by allocating unused capacity withinnetwork 300 or unused capacity from another abstracted memory subsystem withinnetwork 300. - In one embodiment, block information (e.g., blocks 509, 511 and 513 identifying block information) and/or page information (e.g., block 515 with
page -
FIG. 7 is a flow chart of steps for one embodiment of garbage collection from a plurality of open channel SSDs over a network. As discussed with respect toFIG. 5 ,Metadata server 351 maintains page information for each ofpages metadata server 351 executes a sequence of garbage collection steps to process pages containing “dirty” data (e.g., page 519). Atstep 701,metadata server 351 detects a trigger withinnetwork 300 that causes the metadata server to initiate the garbage collection process within one or moreabstracted memory structures 400. In one embodiment, for eachabstracted memory structure 400, themanagement station 333 may set a threshold number of free blocks at which themetadata server 351 initiates the garbage collection process for the respectiveabstracted memory structure 400. In another embodiment, the threshold may be set by any computer withinnetwork 300, including remote host(s) 313 or metadata server(s) 351. In another embodiment, the threshold for garbage collection for one or moreabstracted memory structures 400 may be periodic. - At
step 703,metadata server 351 identifies blocks having pages with dirty data within the respective abstracted memory structure(s) 400 using page information within each block (e.g., block 515) of each device (e.g., device 505) attached to each node (e.g., node 501) ofnetwork 300 that comprise the respective abstracted memory structure(s) 400. In one embodiment, valid pages are pages with data that is not invalid or stale and pages with dirty data are a subset of the valid pages. In order to ensure that the pages with dirty data are written to an allocated page before such pages are erased instep 713, atstep 705,metadata server 351 reads the valid pages (e.g.,pages 517 and 521) from each block (e.g., block 515) that contains pages with dirty data (e.g., page 519) and determines the number of valid pages read. In one embodiment, themetadata server 351 also identifies whether the valid pages read are associated with aparticular namespace 407 within asubsystem 403, aparticular LUN 417 within avolume 413, aparticular file 427 within apartition 423, or aparticular object 433. - At
step 707,metadata server 351 allocates new pages equivalent to the number of valid pages read. As discussed with respect toFIG. 6 , allocation of new pages may be performed in various different ways. Atstep 709,metadata server 351 writes the valid pages read (step 705) to the allocated pages (step 707). Atstep 711, themetadata server 351 updates the physical block and page information for the valid pages written to the allocated pages (step 709). In one embodiment,metadata server 351 updates the physical block and page information to the allocated pages (step 707) for the valid pages (step 705) that correspond to aparticular namespace 407,LUN 417, file 427 orobject 433. Atstep 713, themetadata server 351 sends a command erasing the blocks with pages having dirty data (e.g., block 515). -
FIG. 8 is aflow chart 800 of steps for one embodiment of managing bad blocks in a plurality of open channel SSDs over a network. As discussed with respect toFIG. 5 ,Metadata server 351 maintains block information for each ofblocks metadata server 351 executes a process for managing bad blocks (e.g., block 511). Atstep 801,metadata server 351 detects a trigger withinnetwork 300 that causes the metadata server to initiate the bad block management process within one or moreabstracted memory structures 400. In one embodiment, for eachabstracted memory structure 400, themanagement station 333 may set a threshold number of bad blocks at which themetadata server 351 initiates the bad block management process for the respectiveabstracted memory structure 400. In another embodiment, the threshold may be set by any computer withinnetwork 300, including remote host(s) 313 or metadata server(s) 351. In another embodiment, the threshold for initiating the bad block management process for one or moreabstracted memory structures 400 may be periodic. In a preferred embodiment, the threshold for initiating the bad block management process may be related to a command failure, such as a failure to execute a read or write command initiated by one of the computers (e.g., remote host(s) 313, management station(s) 333, and/or metadata server(s) 351) withinnetwork 300. - At
step 803,metadata server 351 identifies bad blocks within the respective abstracted memory structure(s) 400 using block information for each block (e.g., blocks 509, 511 and 513) of each device (e.g., device 505) attached to each node (e.g., node 501) ofnetwork 300 that comprise the respective abstracted memory structure(s) 400. Atstep 805,metadata server 351 determines whether pages on the bad blocks are protected by a reliability mechanism (e.g., mirror, RAID and the like). In one embodiment,metadata server 351 identifies the reliability mechanism from block information (e.g., blocks 509, 511 and 513 identifying block information) and/or page information (e.g., block 515 withpage metadata server 351 identifies the reliability mechanism from the abstracted memory structure information (e.g., 405, 415, 425 or 435) and/or header information (e.g., 409 a, 419 a or 429 a). If the pages on the bad block are protected by a reliability mechanism, perflow chart 800, the process moves fromstep 805 to step 807. Atstep 807,metadata server 351 allocates new pages to repair the pages on the bad blocks (e.g., as discussed with respect toFIG. 6 ). Atstep 809,metadata server 351 repairs the bad pages using the appropriate reliability mechanism. For example, if the bad pages are protected by mirror reliability,metadata server 351 identifies the physical location of the mirrored pages and reads the data from the mirrored pages. If the bad pages are protected by RAID reliability,metadata server 351 uses the parity pages to repair the bad pages. Atstep 811,metadata server 351 writes the repaired pages (step 809) to the newly allocated pages (step 807). Atstep 813,metadata server 351 updates the physical block and page information for the repaired pages written to the allocated pages. Atstep 815,metadata server 351 marks the status of the bad blocks as “bad” (e.g., block 511) and updates thebad block list 529 by adding a reference (e.g., pointer “PTR2”), referencing the bad block. In one embodiment, any reference may be used withinbad block list 529 to identify bad blocks (e.g., block 511). - At
step 805, ifmetadata server 351 determines that the pages are not protected by a reliability mechanism, per theflow chart 800, the process moves fromstep 805 to step 815. As discussed above, atstep 815,metadata server 351 sets the status of the bad blocks as “bad” (e.g., block 511) and updates thebad block list 529 by adding a reference (e.g., pointer “PTR2”), referencing the bad block. -
FIG. 9 depicts aflow chart 900 showing steps for one embodiment of implementing a plurality of open channel SSDs over a network. For the sake of clarity, theflow chart 900 will be described in connection with the embodiments ofFIG. 3 ,FIG. 4 andFIG. 6 . Atstep 901, aremote host 313 issues a command tometadata server 351. In one embodiment, multipleremote hosts 313 may simultaneously issue one or more commands tometadata server 351. - If the command issued by
remote host 313 is a write command, the process moves fromstep 901 to step 903. In one embodiment, the read command includes identification information for the abstracted memory structure (e.g., identifyingsubsystem 403 andnamespace 407,volume 413 andLUN 417,partition 423 and file 427, or object 423) and the length of the write command. Atstep 903,metadata server 351 allocates physical pages in response to the write command (e.g., as discussed with respect toFIG. 6 ). Atstep 905,metadata server 351 returns mapping information corresponding to the write command toremote host 313. In one embodiment, the mapping information identifies the location of the physical block and pages corresponding to the write command. In another embodiment, the mapping information identifies the node (e.g., node E), the device (e.g., open channel device 301), the open channel SSD (e.g., OCSSD 303), and the block and page numbers corresponding to the write command. Atstep 907,remote host 313 uses the physical block and page mapping information to issue one or more write commands to write the data to each page of the corresponding OCSSD(s) (e.g.,OCSSDs 303 and/or 323) identified in the mapping information. In one embodiment, if the allocated pages are protected by mirror reliability,remote host 313 uses the physical block and page mapping information for each of the mirror locations to issue one or more write commands to write the data to each page of the corresponding OCSSD(s) (e.g.,OCSSDs 303 and/or 323) identified in the mapping information for each mirror location. In another embodiment, if the allocated pages are protected by RAID reliability,remote host 313 uses the physical block and page mapping information to issue one or more write commands to write the data and parity data to each page of the corresponding OCSSD(s) identified in the mapping information. Atstep 909, themetadata server 351 updates mapping information identifying the physical blocks and pages where the data is written.Metadata server 351 also updates block information (e.g., sets block usage as “partial” or “full”) and page information (e.g., sets page status as “valid”). If the data written byremote host 313 was previously written at another location (i.e., old data),metadata server 351 updates the status of the page information containing the old data as “dirty” (e.g., page 519) for the next garbage collection process (as discussed with respect toFIG. 7 ). - If the command issued by
remote host 313 is a read command, the process moves fromstep 901 to step 905. In one embodiment, the read command includes identification information for the abstracted memory structure (e.g., identifyingsubsystem 403 andnamespace 407,volume 413 andLUN 417,partition 423 and file 427, or object 423) and the virtual pages corresponding to the command. Atstep 905,metadata server 351 returns mapping information corresponding to the read command toremote host 313. In one embodiment, the mapping information identifies the location of the physical block and pages corresponding to the read command. In another embodiment, the mapping information identifies the node (e.g., node E), the device (e.g., open channel device 301), the open channel SSD (e.g., OCSSD 303), and the block and page numbers corresponding to the read request. In one embodiment, if the pages corresponding to the read command are protected by mirror reliability,metadata server 351 sends mapping information for all mirror locations comprising the requested data. In this embodiment,host device 313 determines for each page the mirror location from which to read the data. In another embodiment, if the page corresponding to the read command are protected by RAID reliability, themetadata server 351 sends mapping information including data pages and parity pages. In this embodiment, theremote host 313 may use the parity pages to repair the data if there is a read error. Atstep 907,remote host 313 issues one or more read commands to read the data from each page of the corresponding OCSSD(s) identified in the mapping information. The method steps 900 for a read command terminate atstep 907. - In another embodiment
remote host 313 may issue a get command read an object or a put command to write an object such as a JPG image (e.g., object 433). The method steps for getting or putting an object are similar to the method steps for reading or writing data as discussed above. - Other objects, advantages and embodiments of the various aspects of the present invention will be apparent to those who are skilled in the field of the invention and are within the scope of the description and the accompanying Figures. For example, but without limitation, structural or functional elements might be rearranged, or method steps reordered, consistent with the present invention. Similarly, principles according to the present invention could be applied to other examples, which, even if not specifically described here in detail, would nevertheless be within the scope of the present invention.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/456,308 US10073640B1 (en) | 2017-03-10 | 2017-03-10 | Large scale implementation of a plurality of open channel solid state drives |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/456,308 US10073640B1 (en) | 2017-03-10 | 2017-03-10 | Large scale implementation of a plurality of open channel solid state drives |
Publications (2)
Publication Number | Publication Date |
---|---|
US10073640B1 US10073640B1 (en) | 2018-09-11 |
US20180260148A1 true US20180260148A1 (en) | 2018-09-13 |
Family
ID=63406418
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/456,308 Active US10073640B1 (en) | 2017-03-10 | 2017-03-10 | Large scale implementation of a plurality of open channel solid state drives |
Country Status (1)
Country | Link |
---|---|
US (1) | US10073640B1 (en) |
Cited By (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10423361B2 (en) * | 2017-12-21 | 2019-09-24 | Microsoft Technology Licensing, Llc | Virtualized OCSSDs spanning physical OCSSD channels |
CN111913892A (en) * | 2019-05-09 | 2020-11-10 | 北京忆芯科技有限公司 | Providing open channel storage devices using CMBs |
WO2021002896A1 (en) * | 2019-07-01 | 2021-01-07 | Western Digital Technologies, Inc. | Open channel vector command execution |
US10893050B2 (en) | 2016-08-24 | 2021-01-12 | Intel Corporation | Computer product, method, and system to dynamically provide discovery services for host nodes of target systems and storage resources in a network |
US10901624B1 (en) | 2019-07-01 | 2021-01-26 | Western Digital Technologies, Inc. | Dummy host command generation for supporting higher maximum data transfer sizes (MDTS) |
US10970231B2 (en) * | 2016-09-28 | 2021-04-06 | Intel Corporation | Management of virtual target storage resources by use of an access control list and input/output queues |
US20220027265A1 (en) * | 2020-07-27 | 2022-01-27 | Alibaba Group Holding Limited | Method and system for facilitating fast crash recovery in a storage device |
US11263132B2 (en) | 2020-06-11 | 2022-03-01 | Alibaba Group Holding Limited | Method and system for facilitating log-structure data organization |
US11281575B2 (en) | 2020-05-11 | 2022-03-22 | Alibaba Group Holding Limited | Method and system for facilitating data placement and control of physical addresses with multi-queue I/O blocks |
US11327929B2 (en) | 2018-09-17 | 2022-05-10 | Alibaba Group Holding Limited | Method and system for reduced data movement compression using in-storage computing and a customized file system |
US11354200B2 (en) | 2020-06-17 | 2022-06-07 | Alibaba Group Holding Limited | Method and system for facilitating data recovery and version rollback in a storage device |
US11372774B2 (en) | 2020-08-24 | 2022-06-28 | Alibaba Group Holding Limited | Method and system for a solid state drive with on-chip memory integration |
US11379447B2 (en) | 2020-02-06 | 2022-07-05 | Alibaba Group Holding Limited | Method and system for enhancing IOPS of a hard disk drive system based on storing metadata in host volatile memory and data in non-volatile memory using a shared controller |
US11379127B2 (en) | 2019-07-18 | 2022-07-05 | Alibaba Group Holding Limited | Method and system for enhancing a distributed storage system by decoupling computation and network tasks |
US11379155B2 (en) | 2018-05-24 | 2022-07-05 | Alibaba Group Holding Limited | System and method for flash storage management using multiple open page stripes |
US11385833B2 (en) | 2020-04-20 | 2022-07-12 | Alibaba Group Holding Limited | Method and system for facilitating a light-weight garbage collection with a reduced utilization of resources |
US11385795B2 (en) | 2015-12-21 | 2022-07-12 | Intel Corporation | Method and apparatus to enable individual non volatile memory express (NVMe) input/output (IO) queues on differing network addresses of an NVMe controller |
US11416365B2 (en) | 2020-12-30 | 2022-08-16 | Alibaba Group Holding Limited | Method and system for open NAND block detection and correction in an open-channel SSD |
US11422931B2 (en) | 2020-06-17 | 2022-08-23 | Alibaba Group Holding Limited | Method and system for facilitating a physically isolated storage unit for multi-tenancy virtualization |
US11449455B2 (en) | 2020-01-15 | 2022-09-20 | Alibaba Group Holding Limited | Method and system for facilitating a high-capacity object storage system with configuration agility and mixed deployment flexibility |
US11449386B2 (en) | 2020-03-20 | 2022-09-20 | Alibaba Group Holding Limited | Method and system for optimizing persistent memory on data retention, endurance, and performance for host memory |
US11461262B2 (en) | 2020-05-13 | 2022-10-04 | Alibaba Group Holding Limited | Method and system for facilitating a converged computation and storage node in a distributed storage system |
US11461173B1 (en) | 2021-04-21 | 2022-10-04 | Alibaba Singapore Holding Private Limited | Method and system for facilitating efficient data compression based on error correction code and reorganization of data placement |
US11476874B1 (en) | 2021-05-14 | 2022-10-18 | Alibaba Singapore Holding Private Limited | Method and system for facilitating a storage server with hybrid memory for journaling and data storage |
US11487465B2 (en) | 2020-12-11 | 2022-11-01 | Alibaba Group Holding Limited | Method and system for a local storage engine collaborating with a solid state drive controller |
US11494115B2 (en) | 2020-05-13 | 2022-11-08 | Alibaba Group Holding Limited | System method for facilitating memory media as file storage device based on real-time hashing by performing integrity check with a cyclical redundancy check (CRC) |
US11507499B2 (en) | 2020-05-19 | 2022-11-22 | Alibaba Group Holding Limited | System and method for facilitating mitigation of read/write amplification in data compression |
US11556277B2 (en) | 2020-05-19 | 2023-01-17 | Alibaba Group Holding Limited | System and method for facilitating improved performance in ordering key-value storage with input/output stack simplification |
US11617282B2 (en) | 2019-10-01 | 2023-03-28 | Alibaba Group Holding Limited | System and method for reshaping power budget of cabinet to facilitate improved deployment density of servers |
US11726699B2 (en) | 2021-03-30 | 2023-08-15 | Alibaba Singapore Holding Private Limited | Method and system for facilitating multi-stream sequential read performance improvement with reduced read amplification |
US11734115B2 (en) | 2020-12-28 | 2023-08-22 | Alibaba Group Holding Limited | Method and system for facilitating write latency reduction in a queue depth of one scenario |
US11768709B2 (en) | 2019-01-02 | 2023-09-26 | Alibaba Group Holding Limited | System and method for offloading computation to storage nodes in distributed system |
US11816043B2 (en) | 2018-06-25 | 2023-11-14 | Alibaba Group Holding Limited | System and method for managing resources of a storage device and quantifying the cost of I/O requests |
US11934684B2 (en) | 2021-12-14 | 2024-03-19 | Western Digital Technologies, Inc. | Maximum data transfer size per tenant and command type |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3866016A4 (en) * | 2018-11-20 | 2021-11-10 | Huawei Technologies Co., Ltd. | Method and apparatus for deleting index entry in memory |
KR20200113047A (en) | 2019-03-20 | 2020-10-06 | 삼성전자주식회사 | Operation method of open-channel storage device |
KR20210039871A (en) | 2019-10-02 | 2021-04-12 | 삼성전자주식회사 | Storage system managing meta data, Host system controlling storage system and Operating method of storage system |
TWI805231B (en) * | 2022-02-18 | 2023-06-11 | 慧榮科技股份有限公司 | Data storage device and control method for non-volatile memory |
TWI802279B (en) * | 2022-02-18 | 2023-05-11 | 慧榮科技股份有限公司 | Data storage device and control method for non-volatile memory |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05265840A (en) | 1991-12-31 | 1993-10-15 | Internatl Business Mach Corp <Ibm> | Computer system and data processing method |
US6092168A (en) | 1996-10-25 | 2000-07-18 | Hewlett-Packard Co. | Data storage system and method for deallocating space by writing and detecting a predefined data pattern |
US7765378B1 (en) | 2001-06-01 | 2010-07-27 | Sanbolic, Inc. | Utilization of memory storage |
CN100461156C (en) | 2001-11-09 | 2009-02-11 | 无锡永中科技有限公司 | Integrated data processing system |
US7243201B2 (en) | 2004-07-22 | 2007-07-10 | International Business Machines Corporation | Application-based commit for local storage subsystems and remote storage subsystems |
US8255456B2 (en) | 2005-12-30 | 2012-08-28 | Citrix Systems, Inc. | System and method for performing flash caching of dynamically generated objects in a data communication network |
JP5224498B2 (en) | 2007-02-28 | 2013-07-03 | 学校法人早稲田大学 | MEMORY MANAGEMENT METHOD, INFORMATION PROCESSING DEVICE, PROGRAM CREATION METHOD, AND PROGRAM |
US9466025B2 (en) | 2009-02-27 | 2016-10-11 | Mckesson Financial Holdings | Method, apparatus and computer program product for loading content items |
US9251295B2 (en) | 2011-08-31 | 2016-02-02 | International Business Machines Corporation | Data filtering using filter icons |
US9128820B1 (en) | 2012-06-18 | 2015-09-08 | Western Digital Technologies, Inc. | File management among different zones of storage media |
US20140047159A1 (en) * | 2012-08-10 | 2014-02-13 | Sandisk Technologies Inc. | Enterprise server with flash storage modules |
US9405621B2 (en) | 2012-12-28 | 2016-08-02 | Super Talent Technology, Corp. | Green eMMC device (GeD) controller with DRAM data persistence, data-type splitting, meta-page grouping, and diversion of temp files for enhanced flash endurance |
CA2906635C (en) | 2013-03-15 | 2019-08-06 | BeulahWorks, LLC | Knowledge capture and discovery system |
CN104166651B (en) | 2013-05-16 | 2017-10-13 | 阿里巴巴集团控股有限公司 | Method and apparatus based on the data search integrated to homogeneous data object |
US9542118B1 (en) * | 2014-09-09 | 2017-01-10 | Radian Memory Systems, Inc. | Expositive flash memory control |
US10437486B2 (en) * | 2016-05-24 | 2019-10-08 | Samsung Electronics Co., Ltd. | Method and apparatus for tenant-aware storage sharing platform |
-
2017
- 2017-03-10 US US15/456,308 patent/US10073640B1/en active Active
Cited By (40)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11385795B2 (en) | 2015-12-21 | 2022-07-12 | Intel Corporation | Method and apparatus to enable individual non volatile memory express (NVMe) input/output (IO) queues on differing network addresses of an NVMe controller |
US10893050B2 (en) | 2016-08-24 | 2021-01-12 | Intel Corporation | Computer product, method, and system to dynamically provide discovery services for host nodes of target systems and storage resources in a network |
US10970231B2 (en) * | 2016-09-28 | 2021-04-06 | Intel Corporation | Management of virtual target storage resources by use of an access control list and input/output queues |
US11630783B2 (en) | 2016-09-28 | 2023-04-18 | Intel Corporation | Management of accesses to target storage resources |
US10423361B2 (en) * | 2017-12-21 | 2019-09-24 | Microsoft Technology Licensing, Llc | Virtualized OCSSDs spanning physical OCSSD channels |
US11093177B2 (en) * | 2017-12-21 | 2021-08-17 | Microsoft Technology Licensing, Llc | Virtualized OCSSDs spanning physical OCSSD channels |
US11379155B2 (en) | 2018-05-24 | 2022-07-05 | Alibaba Group Holding Limited | System and method for flash storage management using multiple open page stripes |
US11816043B2 (en) | 2018-06-25 | 2023-11-14 | Alibaba Group Holding Limited | System and method for managing resources of a storage device and quantifying the cost of I/O requests |
US11327929B2 (en) | 2018-09-17 | 2022-05-10 | Alibaba Group Holding Limited | Method and system for reduced data movement compression using in-storage computing and a customized file system |
US11768709B2 (en) | 2019-01-02 | 2023-09-26 | Alibaba Group Holding Limited | System and method for offloading computation to storage nodes in distributed system |
CN111913892A (en) * | 2019-05-09 | 2020-11-10 | 北京忆芯科技有限公司 | Providing open channel storage devices using CMBs |
KR102645983B1 (en) * | 2019-07-01 | 2024-03-08 | 웨스턴 디지털 테크놀로지스, 인코포레이티드 | Open channel vector command execution |
US10901624B1 (en) | 2019-07-01 | 2021-01-26 | Western Digital Technologies, Inc. | Dummy host command generation for supporting higher maximum data transfer sizes (MDTS) |
US10901733B1 (en) | 2019-07-01 | 2021-01-26 | Western Digital Technologies, Inc. | Open channel vector command execution |
KR20210084632A (en) * | 2019-07-01 | 2021-07-07 | 웨스턴 디지털 테크놀로지스, 인코포레이티드 | Open Channel Vector Command Execution |
WO2021002896A1 (en) * | 2019-07-01 | 2021-01-07 | Western Digital Technologies, Inc. | Open channel vector command execution |
US11379127B2 (en) | 2019-07-18 | 2022-07-05 | Alibaba Group Holding Limited | Method and system for enhancing a distributed storage system by decoupling computation and network tasks |
US11617282B2 (en) | 2019-10-01 | 2023-03-28 | Alibaba Group Holding Limited | System and method for reshaping power budget of cabinet to facilitate improved deployment density of servers |
US11449455B2 (en) | 2020-01-15 | 2022-09-20 | Alibaba Group Holding Limited | Method and system for facilitating a high-capacity object storage system with configuration agility and mixed deployment flexibility |
US11379447B2 (en) | 2020-02-06 | 2022-07-05 | Alibaba Group Holding Limited | Method and system for enhancing IOPS of a hard disk drive system based on storing metadata in host volatile memory and data in non-volatile memory using a shared controller |
US11449386B2 (en) | 2020-03-20 | 2022-09-20 | Alibaba Group Holding Limited | Method and system for optimizing persistent memory on data retention, endurance, and performance for host memory |
US11385833B2 (en) | 2020-04-20 | 2022-07-12 | Alibaba Group Holding Limited | Method and system for facilitating a light-weight garbage collection with a reduced utilization of resources |
US11281575B2 (en) | 2020-05-11 | 2022-03-22 | Alibaba Group Holding Limited | Method and system for facilitating data placement and control of physical addresses with multi-queue I/O blocks |
US11494115B2 (en) | 2020-05-13 | 2022-11-08 | Alibaba Group Holding Limited | System method for facilitating memory media as file storage device based on real-time hashing by performing integrity check with a cyclical redundancy check (CRC) |
US11461262B2 (en) | 2020-05-13 | 2022-10-04 | Alibaba Group Holding Limited | Method and system for facilitating a converged computation and storage node in a distributed storage system |
US11507499B2 (en) | 2020-05-19 | 2022-11-22 | Alibaba Group Holding Limited | System and method for facilitating mitigation of read/write amplification in data compression |
US11556277B2 (en) | 2020-05-19 | 2023-01-17 | Alibaba Group Holding Limited | System and method for facilitating improved performance in ordering key-value storage with input/output stack simplification |
US11263132B2 (en) | 2020-06-11 | 2022-03-01 | Alibaba Group Holding Limited | Method and system for facilitating log-structure data organization |
US11354200B2 (en) | 2020-06-17 | 2022-06-07 | Alibaba Group Holding Limited | Method and system for facilitating data recovery and version rollback in a storage device |
US11422931B2 (en) | 2020-06-17 | 2022-08-23 | Alibaba Group Holding Limited | Method and system for facilitating a physically isolated storage unit for multi-tenancy virtualization |
US20220027265A1 (en) * | 2020-07-27 | 2022-01-27 | Alibaba Group Holding Limited | Method and system for facilitating fast crash recovery in a storage device |
US11354233B2 (en) * | 2020-07-27 | 2022-06-07 | Alibaba Group Holding Limited | Method and system for facilitating fast crash recovery in a storage device |
US11372774B2 (en) | 2020-08-24 | 2022-06-28 | Alibaba Group Holding Limited | Method and system for a solid state drive with on-chip memory integration |
US11487465B2 (en) | 2020-12-11 | 2022-11-01 | Alibaba Group Holding Limited | Method and system for a local storage engine collaborating with a solid state drive controller |
US11734115B2 (en) | 2020-12-28 | 2023-08-22 | Alibaba Group Holding Limited | Method and system for facilitating write latency reduction in a queue depth of one scenario |
US11416365B2 (en) | 2020-12-30 | 2022-08-16 | Alibaba Group Holding Limited | Method and system for open NAND block detection and correction in an open-channel SSD |
US11726699B2 (en) | 2021-03-30 | 2023-08-15 | Alibaba Singapore Holding Private Limited | Method and system for facilitating multi-stream sequential read performance improvement with reduced read amplification |
US11461173B1 (en) | 2021-04-21 | 2022-10-04 | Alibaba Singapore Holding Private Limited | Method and system for facilitating efficient data compression based on error correction code and reorganization of data placement |
US11476874B1 (en) | 2021-05-14 | 2022-10-18 | Alibaba Singapore Holding Private Limited | Method and system for facilitating a storage server with hybrid memory for journaling and data storage |
US11934684B2 (en) | 2021-12-14 | 2024-03-19 | Western Digital Technologies, Inc. | Maximum data transfer size per tenant and command type |
Also Published As
Publication number | Publication date |
---|---|
US10073640B1 (en) | 2018-09-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10073640B1 (en) | Large scale implementation of a plurality of open channel solid state drives | |
US10542089B2 (en) | Large scale implementation of a plurality of open channel solid state drives | |
US20180173632A1 (en) | Storage device and method for controlling storage device | |
US9606914B2 (en) | Apparatus, system, and method for allocating storage | |
US8832371B2 (en) | Storage system with multiple flash memory packages and data control method therefor | |
US9684591B2 (en) | Storage system and storage apparatus | |
US9015425B2 (en) | Apparatus, systems, and methods for nameless writes | |
WO2015162758A1 (en) | Storage system | |
JP5816303B2 (en) | Storage system including flash memory and storage control method | |
US20150254186A1 (en) | Information processing device having a plurality of types of memory caches with different characteristics | |
US9116622B2 (en) | Storage system having nonvolatile semiconductor storage device with nonvolatile semiconductor memory | |
JP2020035300A (en) | Information processing apparatus and control method | |
US11928053B2 (en) | System garbage collection method and method for garbage collection in solid state disk | |
US20110246701A1 (en) | Storage apparatus and its data control method | |
US20190243758A1 (en) | Storage control device and storage control method | |
US9170750B2 (en) | Storage apparatus and data copy control method | |
JP6817340B2 (en) | calculator | |
JP6605762B2 (en) | Device for restoring data lost due to storage drive failure | |
JP2022116209A (en) | virtual storage system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: TOSHIBA MEMORY CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KABUSHIKI KAISHA TOSHIBA;REEL/FRAME:043397/0380 Effective date: 20170706 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: K.K. PANGEA, JAPAN Free format text: MERGER;ASSIGNOR:TOSHIBA MEMORY CORPORATION;REEL/FRAME:055659/0471 Effective date: 20180801 Owner name: TOSHIBA MEMORY CORPORATION, JAPAN Free format text: CHANGE OF NAME AND ADDRESS;ASSIGNOR:K.K. PANGEA;REEL/FRAME:055669/0401 Effective date: 20180801 Owner name: KIOXIA CORPORATION, JAPAN Free format text: CHANGE OF NAME AND ADDRESS;ASSIGNOR:TOSHIBA MEMORY CORPORATION;REEL/FRAME:055669/0001 Effective date: 20191001 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |