WO2020177017A1 - Method and apparatus for managing non-volatile memory and flash as computational object interfaced solid-state drive - Google Patents
Method and apparatus for managing non-volatile memory and flash as computational object interfaced solid-state drive Download PDFInfo
- Publication number
- WO2020177017A1 WO2020177017A1 PCT/CN2019/076703 CN2019076703W WO2020177017A1 WO 2020177017 A1 WO2020177017 A1 WO 2020177017A1 CN 2019076703 W CN2019076703 W CN 2019076703W WO 2020177017 A1 WO2020177017 A1 WO 2020177017A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- storage
- storage device
- data
- computation
- host computer
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
Definitions
- Apparatuses and methods consistent with the present disclosure relate generally to storage devices, and more particularly, to a storage device including computational object interface, a method for managing and operating the same, and a system including the same.
- big data For example, as processing power of a computers has increased, big data applications that perform computations using large amounts of data have become feasible and, as a result, mainstream. Due to the nature of their large amount, big data are usually stored in an object-based storage system. For example, the data are physically stored in a database or a storage device, and an object is used as a logic container for grouped data that a user wishes to store or retrieve.
- a host computer may extract a desired data from among the big data for a computation, and the extracted data may be transmitted from the storage system to the host computer.
- a storage device may comprise: an interface configured to receive an object-oriented command from a host; and a central processing unit including one or more cores configured to perform computation on data stored in the storage device based on the object-oriented command.
- the storage device may be implemented as a solid state drive (SSD) .
- the storage device may further comprise a first storage configured to store an object attribute and a second storage configured to store an object data.
- the first storage may comprise a plurality of non-volatile memories and the second storage may comprise a plurality of NAND flash memories.
- the storage device may further comprise an object computing manager configured to manage the computation in the storage device.
- the storage device may further comprise an object storage manager configured to manage the storage of the object attribute and the object data in the first storage and second storage, respectively.
- the object storage manager may comprise an object attribute translation table that maps an object ID to an abject attribute address in the first storage, and an object data address translation table that maps a combination of the object ID and an offset to an object data address in the second storage.
- the storage device may further comprise a volatile memory, wherein the volatile memory may be one of dynamic random access memory (DRAM) , a static random access memory (SRAM) , and a tightly-coupled memory (TCM) .
- the computation may comprise at least one of a compression, a compaction, a cyclic redundancy check (CRC) , or a scrubbing.
- a method for operating a storage device may comprise: receiving, by the storage device, an object-oriented command from a host; performing, by the storage device, a computation on data stored in the storage device; and transmitting, by the storage device, a computation result to the host.
- the method may further comprise: storing an object attribute in a first storage of the storage device; and storing an object data in a second storage of the storage device.
- the objected-oriented command may be a low-level application programming interface (API) command generated by the host.
- the first storage may comprise a plurality of non-volatile memories and the second storage may comprise a plurality of NAND flash memories.
- the computation in the storage device may comprise at least one of a compression, a compaction, a cyclic redundancy check (CRC) , or a scrubbing.
- a data processing system comprising a host computer and a storage device.
- the host computer may comprise: a processor; a memory; and an object storage controller configured to generate an object-oriented command.
- the storage device may comprise: a volatile memory; an interface configured to receive the object-oriented command from the host computer; and a central processing unit including at least one core configured to perform a computation on data stored in the storage device.
- the storage device may further comprise a first storage configured to store an object attribute and a second storage configured to store an object.
- the object storage controller of the host computer may further comprise: an application programming interface (API) library configured to invoke a high-level API; and an object storage driver configured to convert the high-level API into a low-level API to generate the object-oriented command.
- the storage device may be further configured to transmit a computation result to the host computer.
- FIG. 1 is a block diagram illustrating a data processing system comprising a host computer and an object interfaced storage device, consistent with some embodiments of the present disclosure.
- FIG. 2 is a schematic diagram illustrating an object computing management scheme, consistent with some embodiments of the present disclosure.
- FIG. 3 is a schematic diagram illustrating an object storage management scheme, consistent with some embodiments of the present disclosure.
- FIG. 4 is a flow chart illustrating operations performed in a data processing system, consistent with some embodiments of the present disclosure.
- FIG. 5 is a schematic diagram illustrating operations performed in a storage device, consistent with some embodiments of the present disclosure.
- Big data are physically stored in a database or a storage device, and an object is used as a logic container for grouped data that a user wishes to store or retrieve.
- An object may contain two parts, i.e., an object data and an object attribute.
- the object data and the object attribute are stored together in a single storage.
- such a storage method may suffer from its disadvantage because an object attribute usually has a smaller size than an object data.
- a host computer may extract a desired data from among the big data for a computation, and the extracted data may be transmitted from the storage system to the host computer.
- extraction of the desired data from among big data is time consuming.
- a persistent data transmission from the storage system to the host computer as well as the computation in the host computer consumes a large amount of power in the host computer, causing inefficient energy distribution among different components of the computer.
- the data transmission consumes bandwidth of the host computer and may cause data traffic.
- the disclosed embodiments provide a storage device that can support an object interface and execute an object related operations in the storage device.
- the object related operations may include performing computation of the data stored in the storage device.
- the objected related operations may further include object storage management in the storage device.
- Each object is provided with a unique ID for identification.
- the object data is stored on a NAND flash and the object attribute is stored on a nonvolatile memory.
- the term “or” encompasses all possible combinations, except where infeasible. For example, if it is stated that a database may include A or B, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.
- a data processing system 100 may comprise a host computer 110 and a data storage device 130.
- Host computer 110 may comprise a processor 112, a memory 114, a user interface 116, and a storage device controller 118.
- Storage device controller 118 may include an object storage API library 120 and an object storage driver 122.
- Processor 104 can comprise multiple processors. In some embodiments, storage device controller 118 may be implemented in processor 104.
- Storage device 130 may comprise a central computing unit (CPU) 140 comprising a plurality of cores, a dynamic random access memory (DRAM) 142, a plurality of non-volatile memories 144, a plurality of flash memories 146, an object interface 132, and an object manager 134.
- Object manager 134 may include an object storage manager 136 and an object computing manager 138.
- storage device 130 may be implemented as a solid state drive (SSD) .
- DRAM 142 may function as a main storage of storage device 130 and may be replaced by any other types of volatile memories such as a static random access memory (SRAM) , and a tightly-coupled memory (TCM) .
- SRAM static random access memory
- TCM tightly-coupled memory
- non-volatile memory 144 may be, but not limited to, a read-only memory (ROM) , a flash memory, a ferroelectric RAM, a SSD, a hard disk drive, a magnetic tape, or an optical disc.
- flash memory 146 may be a NAND flash memory or a NOR flash memory.
- Host computer 110 may be in communication with data storage device 120 via wired or wireless communications.
- Host computer 110 may include, but is not limited to, a personal computer, a workstation, a network computer, a server computer, an application processor, or any device having one or more processors.
- Memory 114 may store instructions related to host computer 110 or storage device 130, and processor 112 is configured (via its circuitry) to execute the instructions.
- User interface 116 may include a display and an input device.
- the display may be configured to display applications running in host computer 110 or the progress of data processing in data processing system 110, in some embodiments.
- the display may be any type of a computer output surface and projecting mechanism that shows text and graphic images, including (but not limited to) cathode ray tube (CRT) , liquid crystal display (LCD) , light- emitting diode (LED) , gas plasma, a touch screen, or other image projection technologies, for displaying information to a computer user.
- the input device may be any type of a computer hardware equipment used to provide data and control signals from an operator to host computer 110.
- the input device may include, but is not limited to, a keyboard, a mouse, a scanner, a digital camera, a joystick, a trackball, cursor direction keys, a touchscreen monitor, or audio/video commanders, etc., for communicating direction information and command selections to processor or for controlling cursor movement on display.
- Host computer 110 may need to extract data stored in storage device 130 and perform computation on the data. Instead of extracting and transmitting the required data to host computer 110 for a computation, data processing system performs in-storage computation based on a command generated by host computer 110.
- host computer 110 may provide storage device controller 118 including an object storage application programming interface (API) library 120.
- object storage API library 120 invokes an object-oriented high-level API.
- Host computer 110 may further provide object storage driver 122, which may convert the object-oriented high-level API into object-oriented low-level API, e.g., a binary code or a binary file, to generate an object-oriented command.
- Storage device controller 118 may transmit the object-oriented command to storage device 130.
- object interface 132 may be configured to receive the object-oriented command transmitted from host computer 110. Object interface 132 may further store the object-oriented command in a storage of storage device 130 or provide the command to CPU 140 for processing an object related operation. In some embodiments, the object-oriented command may be a command for performing in-storage computation in a core of CPU 140 based on the command.
- the in-storage computing operations may include, but is not limited to, a compression operation, a compaction operation, a cyclic redundancy check (CRC) , and a scrubbing operation.
- Object computing manager 138 may manage the in-storage computing operation as shown in FIG. 2.
- object computing manager 138 may assign a unique object ID to each computation task, classify a type of computation, and assign a specific core among the plurality of cores of CPU 140.
- object computing manager 138 may assign a specific core of CPU 140 to a computation task based on the type of computation. For example, as shown in FIG. 2, both the computations having object ID 1 and 2 are compression type computations, and object computing manager 138 assigns core #1 of CPU 140 for this computation.
- the type of computation is cyclic redundancy check (CRC) and object computing manager 138 assigns core #2 of CPU 140 for this CRC.
- object computing manager 138 may assign a specific core of CPU 140 for a computation task based on a complexity level of computation or an estimated amount of time for the computation.
- the object computing manager 138 transmits the computation results to host computer 110 through object interface 132.
- the need for host computer 110 to extract data from the storage device and perform computation is eliminated, leading to a faster data processing and reduced power consumption in the host computer. Also, by eliminating the data transmission between the host computer and the storage device, data traffic is reduced, and the bandwidth of the host computer is reserved for other purposes, leading to improved efficiency and speed of the host computer.
- object manager 134 may further include object storage manager 136 that is configured to manage data storage in storage device 130.
- An object may contain two parts, i.e., an object data and an object attribute. Instead of storing the object data and the object attribute in a single storage, object storage manager 136 may allocate a space in flash memory 146 to store the object data and allocate a space in non-volatile memory 144 to store the object attribute, as shown in FIG. 3.
- Object storage manager 136 may include object translation layer including an object attribute address translation table and an object data address translation table. As shown in the object attribute address translation table, an object storage manager (not shown) may assign a unique ID to each object, and each ID is mapped to an object attribute address on a non-volatile memory 144.
- the object storage management module may also allocate the space on flash memory 146, which stores the object’s data. As shown in the object data address translation table, the object storage manager may assign one or more offset values to each object ID, depending on the size of the object data. Object storage manager may further map each object ID having a specified offset value to an object data address (e.g., physical page number) of flash memory 146. For example, object ID #1 has two corresponding offsets, i.e., offset 0 and offset 4096, and offset 0 is mapped to physical page number 1 of flash memory 146 and offset 4096 is mapped to physical page number 2 of flash 146. The object storage manager may manage and update the object attribute address translation table and the object data address translation table.
- object data address translation table e.g., physical page number
- mapping scheme allows a separate storage of the object attribute and object data, i.e., the two types of data that are significantly vary in size.
- the nonvolatile memory and the flash memory By separately storing the object attribute and the object data in different memories (the nonvolatile memory and the flash memory) , the space of the memories is more efficiently utilized, and the data retrieval became easier and faster.
- object storage manager 136 and object computing manager 138 may be functional blocks of the instructions (or software) stored in a storage of storage device 130 (e.g., DRAM 142) , and CPU 140 executes the instructions (or software) to perform the functions of object storage manager 136 and object computing manager 138 as described above.
- storage device controller 118 including object storage API library 120 and object storage driver 122 may be functional blocks of the instructions (or software) stored in memory 114, and processor 112 executes the instructions (or software) to perform the functions of object storage API library 120 and object storage driver 122 as described above.
- FIG. 4 a schematic diagram illustrating operations performed in a data processing system, consistent with some embodiments of the present disclosure.
- an application is initiated in a host computer, such as host computer 110 of FIG. 1.
- the application may invoke an object-oriented high-level API in object storage API library of the host computer, such as object storage API library 120 of storage device controller 118 of FIG. 1.
- an object storage driver (e.g., object storage driver 122 of storage device controller 118) of the host computer may interpret the high-level API and convert it into a corresponding object-oriented low-level command.
- the object storage drive can send the command to the storage device, such as storage device 130 of FIG. 1.
- the storage device may execute the command in the storage device and send the execution results back to the host application.
- FIG. 5 a schematic diagram illustrating operations performed in a storage device, such as storage device 130 of FIG. 1, consistent with some embodiments of the present disclosure.
- the storage device may receive an object-oriented command from a host, such as host computer 110 of FIG. 1.
- the command is a command for performing a computation in the storage device.
- an execution of the command in the storage device may invoke scheduling of the computation task that is associated with specific objects and running the task on a specified CPU core.
- the storage device executes the command and performs computations on data stored in the storage device.
- the in-storage computation may include, but is not limited to, a compression operation, a compaction operation, a cyclic redundancy check (CRC) , and a scrubbing operation.
- the storage device transmits computation results to the host. In this way, an object-oriented computation in the storage device is realized, and the storage device performs the computation specified by the host computer, thereby leading to a faster data processing, reduced power consumption, and reduced data traffic in the host computer.
- the command may provide address space management in the storage device.
- the storage device may store the object attribute in a first storage of the storage device. For example, when the command is executed in the storage device, the storage device may allocate a space in a nonvolatile memory to store an object attribute.
- the storage device may store the object data in a second storage of the storage device. For example, when the command is executed in the storage device, the storage device may allocate a space in a flash memory store the object data. In this way, an object-oriented storage management in the storage device is realized and the object attribute and the object data are separately stored, leading to an efficient utilization of storages, and faster retrieval of the stored data.
- Example embodiments are described above with reference to flowchart illustrations or block diagrams of methods, apparatus (systems) and computer program products. It will be understood that each block of the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by computer program product or instructions on a computer program product. These computer program instructions may be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine (e.g., host computer 110 or storage device 130) , such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart or block diagram block or blocks.
- a machine e.g., host computer 110 or storage device 130
- These computer program instructions may also be stored in a computer readable medium that can direct a hardware processor core of a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium form an article of manufacture including instructions which implement the function/act specified in the flowchart or block diagram block or blocks.
- the computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart or block diagram block or blocks.
- the computer readable medium may be a non-transitory computer readable storage medium.
- a computer readable storage medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
- a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
- Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, IR, etc., or any suitable combination of the foregoing.
- Computer program code for carrying out operations for example embodiments may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
- the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
- the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) .
- LAN local area network
- WAN wide area network
- Internet Service Provider for example, AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
- each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function (s) .
- the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
- each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
- exemplary is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word is intended to present concepts in a concrete fashion.
- figure numbers or figure reference labels in the claims is intended to identify one or more possible embodiments of the claimed subject matter to facilitate the interpretation of the claims. Such use is not to be construed as necessarily limiting the scope of those claims to the embodiments shown in the corresponding figures.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Techniques For Improving Reliability Of Storages (AREA)
Abstract
A storage device and a method for operating the same is provided. The storage device may comprise: an interface configured to receive a command from a host; and a central processing unit including one or more cores configured to perform computation on data stored in the storage device and transmit a computation result to the host. The storage device may further comprise a first storage configured to store an object attribute and a second storage configured to store an object data.
Description
Apparatuses and methods consistent with the present disclosure relate generally to storage devices, and more particularly, to a storage device including computational object interface, a method for managing and operating the same, and a system including the same.
Technological advances have prompted applications that rely on large volumes of data ( “big data” ) . For example, as processing power of a computers has increased, big data applications that perform computations using large amounts of data have become feasible and, as a result, mainstream. Due to the nature of their large amount, big data are usually stored in an object-based storage system. For example, the data are physically stored in a database or a storage device, and an object is used as a logic container for grouped data that a user wishes to store or retrieve. A host computer may extract a desired data from among the big data for a computation, and the extracted data may be transmitted from the storage system to the host computer.
However, while calculations on large data can produce valuable applications, extraction of the desired data from among big data is time consuming. Also, a persistent data transmission from the storage system to the host computer as well as the computation in the host computer consumes a large amount of power in the host computer, causing inefficient energy distribution among different components of the computer. Further, the data transmission consumes bandwidth of the host computer and may cause data traffic.
SUMMARY
According to some embodiments of the present disclosure, there is provided a storage device. The storage device may comprise: an interface configured to receive an object-oriented command from a host; and a central processing unit including one or more cores configured to perform computation on data stored in the storage device based on the object-oriented command. The storage device may be implemented as a solid state drive (SSD) .
The storage device may further comprise a first storage configured to store an object attribute and a second storage configured to store an object data. The first storage may comprise a plurality of non-volatile memories and the second storage may comprise a plurality of NAND flash memories.
The storage device may further comprise an object computing manager configured to manage the computation in the storage device. The storage device may further comprise an object storage manager configured to manage the storage of the object attribute and the object data in the first storage and second storage, respectively. The object storage manager may comprise an object attribute translation table that maps an object ID to an abject attribute address in the first storage, and an object data address translation table that maps a combination of the object ID and an offset to an object data address in the second storage.
The storage device may further comprise a volatile memory, wherein the volatile memory may be one of dynamic random access memory (DRAM) , a static random access memory (SRAM) , and a tightly-coupled memory (TCM) . In the storage device, the computation may comprise at least one of a compression, a compaction, a cyclic redundancy check (CRC) , or a scrubbing.
According to some embodiments of the present disclosure, there is provided a method for operating a storage device. The method may comprise: receiving, by the storage device, an object-oriented command from a host; performing, by the storage device, a computation on data stored in the storage device; and transmitting, by the storage device, a computation result to the host.
The method may further comprise: storing an object attribute in a first storage of the storage device; and storing an object data in a second storage of the storage device.
In the method, the objected-oriented command may be a low-level application programming interface (API) command generated by the host. In the method, the first storage may comprise a plurality of non-volatile memories and the second storage may comprise a plurality of NAND flash memories. In the method, the computation in the storage device may comprise at least one of a compression, a compaction, a cyclic redundancy check (CRC) , or a scrubbing.
According to some embodiments of the present disclosure, there is provided a data processing system comprising a host computer and a storage device. The host computer may comprise: a processor; a memory; and an object storage controller configured to generate an object-oriented command. The storage device may comprise: a volatile memory; an interface configured to receive the object-oriented command from the host computer; and a central processing unit including at least one core configured to perform a computation on data stored in the storage device.
In the system, the storage device may further comprise a first storage configured to store an object attribute and a second storage configured to store an object.
In the system, the object storage controller of the host computer may further comprise: an application programming interface (API) library configured to invoke a high-level API; and an object storage driver configured to convert the high-level API into a low-level API to generate the object-oriented command. In the system, the storage device may be further configured to transmit a computation result to the host computer.
BRIEF DESCRIPTION OF FIGURES
FIG. 1 is a block diagram illustrating a data processing system comprising a host computer and an object interfaced storage device, consistent with some embodiments of the present disclosure.
FIG. 2 is a schematic diagram illustrating an object computing management scheme, consistent with some embodiments of the present disclosure.
FIG. 3 is a schematic diagram illustrating an object storage management scheme, consistent with some embodiments of the present disclosure.
FIG. 4 is a flow chart illustrating operations performed in a data processing system, consistent with some embodiments of the present disclosure.
FIG. 5 is a schematic diagram illustrating operations performed in a storage device, consistent with some embodiments of the present disclosure.
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise represented. The implementations set forth in the following description of exemplary embodiments do not represent all implementations consistent with the invention. Instead, they are merely examples of apparatuses and methods consistent with aspects related to the invention as recited in the appended claims. For example, although some embodiments are described in the context of utilizing non-volatile memory and flash memory for storage of objects, the disclosure is not so limited. Other types of storage device, current available or future developed, can be similarly applied.
Big data are physically stored in a database or a storage device, and an object is used as a logic container for grouped data that a user wishes to store or retrieve. An object may contain two parts, i.e., an object data and an object attribute. In conventional methods, the object data and the object attribute are stored together in a single storage. However, such a storage method may suffer from its disadvantage because an object attribute usually has a smaller size than an object data.
Also, conventionally, a host computer may extract a desired data from among the big data for a computation, and the extracted data may be transmitted from the storage system to the host computer. However, extraction of the desired data from among big data is time consuming. Also, a persistent data transmission from the storage system to the host computer as well as the computation in the host computer consumes a large amount of power in the host computer, causing inefficient energy distribution among different components of the computer. Further, the data transmission consumes bandwidth of the host computer and may cause data traffic.
The disclosed embodiments provide a storage device that can support an object interface and execute an object related operations in the storage device. The object related operations may include performing computation of the data stored in the storage device. By performing the computation in the data storage device, the need for the host computer to extract data from the storage device and compute the data is eliminated, leading to faster data processing. Moreover, by eliminating the data transmission and computation in the host computer, the power consumption and the data traffic in the host computer are reduced.
The objected related operations may further include object storage management in the storage device. Each object is provided with a unique ID for identification. The object data is stored on a NAND flash and the object attribute is stored on a nonvolatile memory. By separately storing the object data and the object attribute that have different sizes, the efficiency of storage usage is improved and data retrieval became easier and faster.
As used herein, unless specifically stated otherwise, the term “or” encompasses all possible combinations, except where infeasible. For example, if it is stated that a database may include A or B, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.
Reference is now made to FIG. 1, a block diagram illustrating a data processing system comprising a host computer and an object interfaced storage device, consistent with some embodiments of the present disclosure. As shown in FIG. 1, a data processing system 100 may comprise a host computer 110 and a data storage device 130. Host computer 110 may comprise a processor 112, a memory 114, a user interface 116, and a storage device controller 118. Storage device controller 118 may include an object storage API library 120 and an object storage driver 122. Processor 104 can comprise multiple processors. In some embodiments, storage device controller 118 may be implemented in processor 104.
User interface 116 may include a display and an input device. The display may be configured to display applications running in host computer 110 or the progress of data processing in data processing system 110, in some embodiments. The display may be any type of a computer output surface and projecting mechanism that shows text and graphic images, including (but not limited to) cathode ray tube (CRT) , liquid crystal display (LCD) , light- emitting diode (LED) , gas plasma, a touch screen, or other image projection technologies, for displaying information to a computer user. The input device may be any type of a computer hardware equipment used to provide data and control signals from an operator to host computer 110. The input device may include, but is not limited to, a keyboard, a mouse, a scanner, a digital camera, a joystick, a trackball, cursor direction keys, a touchscreen monitor, or audio/video commanders, etc., for communicating direction information and command selections to processor or for controlling cursor movement on display.
In storage device 130, object interface 132 may be configured to receive the object-oriented command transmitted from host computer 110. Object interface 132 may further store the object-oriented command in a storage of storage device 130 or provide the command to CPU 140 for processing an object related operation. In some embodiments, the object-oriented command may be a command for performing in-storage computation in a core of CPU 140 based on the command. The in-storage computing operations may include, but is not limited to, a compression operation, a compaction operation, a cyclic redundancy check (CRC) , and a scrubbing operation. Object computing manager 138 may manage the in-storage computing operation as shown in FIG. 2.
References are now made to FIG. 2, a schematic diagram illustrating an object computing management scheme, consistent with some embodiments of the present disclosure. As shown in FIG. 2, object computing manager 138 may assign a unique object ID to each computation task, classify a type of computation, and assign a specific core among the plurality of cores of CPU 140. In some embodiments, object computing manager 138 may assign a specific core of CPU 140 to a computation task based on the type of computation. For example, as shown in FIG. 2, both the computations having object ID 1 and 2 are compression type computations, and object computing manager 138 assigns core # 1 of CPU 140 for this computation. On the other hand, for a computation task having object ID 100, the type of computation is cyclic redundancy check (CRC) and object computing manager 138 assigns core # 2 of CPU 140 for this CRC. In some embodiments, object computing manager 138 may assign a specific core of CPU 140 for a computation task based on a complexity level of computation or an estimated amount of time for the computation. When the computation task is done, the object computing manager 138 transmits the computation results to host computer 110 through object interface 132.
By performing the computation in the storage device, the need for host computer 110 to extract data from the storage device and perform computation is eliminated, leading to a faster data processing and reduced power consumption in the host computer. Also, by eliminating the data transmission between the host computer and the storage device, data traffic is reduced, and the bandwidth of the host computer is reserved for other purposes, leading to improved efficiency and speed of the host computer.
Referring back to FIG. 1, object manager 134 may further include object storage manager 136 that is configured to manage data storage in storage device 130. An object may contain two parts, i.e., an object data and an object attribute. Instead of storing the object data and the object attribute in a single storage, object storage manager 136 may allocate a space in flash memory 146 to store the object data and allocate a space in non-volatile memory 144 to store the object attribute, as shown in FIG. 3.
References are now made to FIG. 3, a schematic diagram illustrating an object storage management scheme, consistent with some embodiments of the present disclosure. Object storage manager 136 may include object translation layer including an object attribute address translation table and an object data address translation table. As shown in the object attribute address translation table, an object storage manager (not shown) may assign a unique ID to each object, and each ID is mapped to an object attribute address on a non-volatile memory 144.
The object storage management module may also allocate the space on flash memory 146, which stores the object’s data. As shown in the object data address translation table, the object storage manager may assign one or more offset values to each object ID, depending on the size of the object data. Object storage manager may further map each object ID having a specified offset value to an object data address (e.g., physical page number) of flash memory 146. For example, object ID # 1 has two corresponding offsets, i.e., offset 0 and offset 4096, and offset 0 is mapped to physical page number 1 of flash memory 146 and offset 4096 is mapped to physical page number 2 of flash 146. The object storage manager may manage and update the object attribute address translation table and the object data address translation table. Such a mapping scheme allows a separate storage of the object attribute and object data, i.e., the two types of data that are significantly vary in size. By separately storing the object attribute and the object data in different memories (the nonvolatile memory and the flash memory) , the space of the memories is more efficiently utilized, and the data retrieval became easier and faster.
Referring back to FIG. 1, in some embodiments, object storage manager 136 and object computing manager 138 may be functional blocks of the instructions (or software) stored in a storage of storage device 130 (e.g., DRAM 142) , and CPU 140 executes the instructions (or software) to perform the functions of object storage manager 136 and object computing manager 138 as described above. In some embodiments, storage device controller 118 including object storage API library 120 and object storage driver 122 may be functional blocks of the instructions (or software) stored in memory 114, and processor 112 executes the instructions (or software) to perform the functions of object storage API library 120 and object storage driver 122 as described above.
Reference is now made to FIG. 4, a schematic diagram illustrating operations performed in a data processing system, consistent with some embodiments of the present disclosure. As shown in FIG. 4, at step 410, an application is initiated in a host computer, such as host computer 110 of FIG. 1. At step 420, the application may invoke an object-oriented high-level API in object storage API library of the host computer, such as object storage API library 120 of storage device controller 118 of FIG. 1.
At step 430, an object storage driver (e.g., object storage driver 122 of storage device controller 118) of the host computer may interpret the high-level API and convert it into a corresponding object-oriented low-level command. After the conversion, at step 440, the object storage drive can send the command to the storage device, such as storage device 130 of FIG. 1. After receiving the command, at step 450, the storage device may execute the command in the storage device and send the execution results back to the host application.
Reference is now made to FIG. 5, a schematic diagram illustrating operations performed in a storage device, such as storage device 130 of FIG. 1, consistent with some embodiments of the present disclosure. As shown in FIG. 5, at step 510, the storage device may receive an object-oriented command from a host, such as host computer 110 of FIG. 1. In some embodiments, the command is a command for performing a computation in the storage device. For example, an execution of the command in the storage device may invoke scheduling of the computation task that is associated with specific objects and running the task on a specified CPU core. At step 520, the storage device executes the command and performs computations on data stored in the storage device. The in-storage computation may include, but is not limited to, a compression operation, a compaction operation, a cyclic redundancy check (CRC) , and a scrubbing operation. At step 530, the storage device transmits computation results to the host. In this way, an object-oriented computation in the storage device is realized, and the storage device performs the computation specified by the host computer, thereby leading to a faster data processing, reduced power consumption, and reduced data traffic in the host computer.
In some embodiments, the command may provide address space management in the storage device. At step 540, the storage device may store the object attribute in a first storage of the storage device. For example, when the command is executed in the storage device, the storage device may allocate a space in a nonvolatile memory to store an object attribute. At step 550, the storage device may store the object data in a second storage of the storage device. For example, when the command is executed in the storage device, the storage device may allocate a space in a flash memory store the object data. In this way, an object-oriented storage management in the storage device is realized and the object attribute and the object data are separately stored, leading to an efficient utilization of storages, and faster retrieval of the stored data.
Example embodiments are described above with reference to flowchart illustrations or block diagrams of methods, apparatus (systems) and computer program products. It will be understood that each block of the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by computer program product or instructions on a computer program product. These computer program instructions may be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine (e.g., host computer 110 or storage device 130) , such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a hardware processor core of a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium form an article of manufacture including instructions which implement the function/act specified in the flowchart or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart or block diagram block or blocks.
Any combination of one or more computer readable medium (s) may be utilized. The computer readable medium may be a non-transitory computer readable storage medium. A computer readable storage medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (anon-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM, EEPROM or Flash memory) , an optical fiber, a cloud storage, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, IR, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for example embodiments may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) .
The flowchart and block diagrams in the figures illustrate examples of the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function (s) . It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It is understood that the described embodiments are not mutually exclusive, and elements, components, materials, or steps described in connection with one example embodiment may be combined with, or eliminated from, other embodiments in suitable ways to accomplish desired design objectives.
Reference herein to “some embodiments” or “some exemplary embodiments” mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment. The appearance of the phrases “one embodiment” “some embodiments” or “some exemplary embodiments” in various places in the specification do not all necessarily refer to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments.
It should be understood that the steps of the example methods set forth herein are not necessarily required to be performed in the order described, and the order of the steps of such methods should be understood to be merely example. Likewise, additional steps may be included in such methods, and certain steps may be omitted or combined, in methods consistent with various embodiments.
As used in this application, the word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word is intended to present concepts in a concrete fashion.
Additionally, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form.
The use of figure numbers or figure reference labels in the claims is intended to identify one or more possible embodiments of the claimed subject matter to facilitate the interpretation of the claims. Such use is not to be construed as necessarily limiting the scope of those claims to the embodiments shown in the corresponding figures.
Although the elements in the following method claims, if any, are recited in a particular sequence with corresponding labeling, unless the claim recitations otherwise imply a particular sequence for implementing some or all of those elements, those elements are not necessarily intended to be limited to being implemented in that particular sequence.
It will be further understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of described embodiments may be made by those skilled in the art without departing from the scope as expressed in the following claims.
Claims (20)
- A storage device, comprising:an interface configured to receive an object-oriented command from a host; anda central processing unit including one or more cores configured to perform a computation based on the object-oriented command.
- The storage device of claim 1, further comprising an object computing manager configured to manage the computation in the storage device.
- The storage device of any one of claims 1 and 2, further comprising:a first storage configured to store an object attribute; anda second storage configured to store an object data.
- The storage device of claim 3, further comprising an object storage manager configured to manage the storage of the object attribute and the object data in the first storage and second storage, respectively.
- The storage device of claim 4, wherein the object storage manager comprises an object attribute translation table that maps an object identifier to an object attribute address in the first storage, and an object data address translation table that maps a combination of the object identifier and an offset to an object data address in the second storage.
- The storage device of claim 3, wherein the first storage comprises a plurality of non-volatile memories.
- The storage device of claim 3, wherein the second storage comprises a plurality of NAND flash memories.
- The storage device of any one of claims 1-7, wherein the storage device comprises a solid state drive (SSD) .
- The storage device of any one of claims 1-8, further comprising a volatile memory, wherein the volatile memory is one of dynamic random access memory (DRAM) , a static random access memory (SRAM) , or a tightly-coupled memory (TCM) .
- The storage device of any one of claims 1-9, wherein the computation comprises at least one of a compression, a compaction, a cyclic redundancy check, or a scrubbing.
- A method for operating a storage device, comprising:receiving, by the storage device, an object-oriented command from a host;performing, by the storage device, a computation on data stored in the storage device; andtransmitting, by the storage device, a computation result to the host.
- The method of claim 11, wherein the objected-oriented command is a low-level application programming interface (API) command generated by the host.
- The method of any one of claims 11 and 12, further comprising:storing an object attribute in a first storage of the storage device; andstoring an object data in a second storage of the storage device.
- The method of claim 13, wherein the first storage comprises a plurality of non-volatile memories.
- The method of claim 13, wherein the second storage comprises a plurality of NAND flash memories.
- The method of any one of claims 11-15, wherein the computation in the storage device comprises at least one of a compression, a compaction, a cyclic redundancy check (CRC) , or a scrubbing.
- A system for data processing, comprising:a host computer that comprises:a processor,a memory, andan object storage controller configured to generate an object-oriented command;anda storage device that comprises:an interface configured to receive the object-oriented command from the host computer, anda central processing unit including at least one core configured to perform a computation on data stored in the storage device.
- The system of claim 17, wherein the storage device further comprises:a first storage configured to store an object attribute; anda second storage configured to store an object data.
- The system of claim 17, wherein the object storage controller of the host computer further comprises:an application programming interface (API) library configured to invoke a high-level API; andan object storage driver configured to convert the high-level API into a low-level API to generate the object-oriented command.
- The system of claim 17, wherein the storage device is further configured to transmit a computation result to the host computer.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201980089504.6A CN113330410A (en) | 2019-03-01 | 2019-03-01 | Method and apparatus for managing non-volatile memory and flash memory as compute object interface solid state drives |
PCT/CN2019/076703 WO2020177017A1 (en) | 2019-03-01 | 2019-03-01 | Method and apparatus for managing non-volatile memory and flash as computational object interfaced solid-state drive |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2019/076703 WO2020177017A1 (en) | 2019-03-01 | 2019-03-01 | Method and apparatus for managing non-volatile memory and flash as computational object interfaced solid-state drive |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020177017A1 true WO2020177017A1 (en) | 2020-09-10 |
Family
ID=72337341
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/076703 WO2020177017A1 (en) | 2019-03-01 | 2019-03-01 | Method and apparatus for managing non-volatile memory and flash as computational object interfaced solid-state drive |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN113330410A (en) |
WO (1) | WO2020177017A1 (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1602490A (en) * | 2001-12-11 | 2005-03-30 | 汤姆森许可贸易公司 | Storage device |
CN101636712A (en) * | 2006-12-06 | 2010-01-27 | 弗森多系统公司(dba弗森-艾奥) | Apparatus, system, and method for servicing object requests within a storage controller |
-
2019
- 2019-03-01 WO PCT/CN2019/076703 patent/WO2020177017A1/en active Application Filing
- 2019-03-01 CN CN201980089504.6A patent/CN113330410A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1602490A (en) * | 2001-12-11 | 2005-03-30 | 汤姆森许可贸易公司 | Storage device |
CN101636712A (en) * | 2006-12-06 | 2010-01-27 | 弗森多系统公司(dba弗森-艾奥) | Apparatus, system, and method for servicing object requests within a storage controller |
Also Published As
Publication number | Publication date |
---|---|
CN113330410A (en) | 2021-08-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN104008064B (en) | The method and system compressed for multi-level store | |
US20150089382A1 (en) | Application context migration framework and protocol | |
US10446174B2 (en) | File system for shingled magnetic recording (SMR) | |
US20140156925A1 (en) | Selection of allocation policy and format for virtual machine disk images | |
US10120920B2 (en) | Increasing storage space for processes impacting data storage systems | |
US9910872B2 (en) | Grid based data mobility | |
US9705986B2 (en) | Elastic scalability of a content transformation cluster | |
US9389997B2 (en) | Heap management using dynamic memory allocation | |
US11341044B2 (en) | Reclaiming storage resources | |
US9471366B2 (en) | Virtual machine disk image backup using block allocation area | |
US11099761B2 (en) | Method, device and computer program product for expanding storage space | |
US20190079982A1 (en) | Content transformations using a transformation node cluster | |
US10579520B2 (en) | Multi-ring shared, traversable, and dynamic advanced database | |
CN112988060A (en) | Key value storage device and method for operating key value storage device | |
US20240143377A1 (en) | Overlay container storage driver for microservice workloads | |
US20200073868A1 (en) | Space-efficient methodology for representing label information in large graph data for fast distributed graph query | |
CN113791990A (en) | Open channel storage device management using FTL on virtual machines | |
US10346424B2 (en) | Object processing | |
WO2020177017A1 (en) | Method and apparatus for managing non-volatile memory and flash as computational object interfaced solid-state drive | |
CN111104347A (en) | Heap memory block searching method, device, equipment and storage medium | |
US11983573B2 (en) | Mapping telemetry data to states for efficient resource allocation | |
US8510530B1 (en) | Memory management for programs operating asynchronously | |
US10509578B2 (en) | Logical address space for storage resource pools | |
US8966133B2 (en) | Determining a mapping mode for a DMA data transfer | |
US11169996B2 (en) | Query optimizer injection for dynamically provisioned database containers |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19918452 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19918452 Country of ref document: EP Kind code of ref document: A1 |