CN113330410A - Method and apparatus for managing non-volatile memory and flash memory as compute object interface solid state drives - Google Patents

Abstract

A memory device and a method of operating the same are provided. The storage device may include: an interface configured to receive a command from a host; a central processing unit comprising one or more cores configured to perform computations on data stored in the storage device and to communicate results of the computations to a host. The storage device may further include a first memory configured to store object attributes and a second memory configured to store object data.

Description

Method and apparatus for managing non-volatile memory and flash memory as compute object interface solid state drives

Technical Field

Apparatuses and methods consistent with the present disclosure relate generally to storage apparatuses, and more particularly, to a storage apparatus including a compute object interface, a method for managing and operating the storage apparatus, and a system including the storage apparatus.

Background

Technological advances have made applications dependent on large amounts of data ("big data"). For example, as computer processing power increases, large data applications that perform computations using large amounts of data become feasible and therefore mainstream. Due to the large amount of data, large data is often stored in object-based storage systems. For example, data is physically stored in a database or storage device and objects serve as logical containers for packet data that a user wishes to store or retrieve. The host may extract the required data from the big data for calculation, and may transfer the extracted data from the storage system to the host.

However, while computation of big data may yield valuable applications, extracting the required data from big data is time consuming. Furthermore, the continuous transmission of data from the storage system to the host and the computation in the host consume a large amount of power in the host, which results in inefficient energy distribution among the different components of the computer. In addition, data transmission consumes bandwidth of the host and may result in data traffic.

Disclosure of Invention

According to some embodiments of the present disclosure, a storage device is provided. The storage device may include: an interface configured to receive an object-oriented command from a host; a central processing unit comprising one or more cores configured to perform computations on data stored in a storage device based on the object-oriented commands. The storage device may be implemented as a Solid State Drive (SSD).

The storage device may further include a first memory configured to store the object attributes and a second memory configured to store the object data. The first memory may include a plurality of non-volatile memories and the second memory may include a plurality of NAND flash memories.

The storage device may also include an object computation manager configured to manage computations in the storage device. The storage device may further include an object store manager configured to manage storage of object attributes and object data in the first and second memories, respectively. The object store manager may include an object property translation table that maps object IDs to object property addresses in the first memory, and an object data address translation table that maps a combination of an object ID and an offset to an object data address in the second store.

The memory device may further include a volatile memory, wherein the volatile memory may be one of a Dynamic Random Access Memory (DRAM), a Static Random Access Memory (SRAM), and a Tightly Coupled Memory (TCM). In the storage device, the computing may include at least one of compression, Cyclic Redundancy Check (CRC), or cleaning.

According to some embodiments of the present disclosure, a method for operating a storage device is provided. The method can comprise the following steps: the storage device receiving an object-oriented command from a host; the storage device calculates data stored in the storage device; the storage device transmits the calculation result to the host.

The method may further comprise: storing the object attributes in a first memory of a storage device; and storing the object data in a second storage of the storage device.

In the method, the object-oriented commands may be low-level Application Programming Interface (API) commands generated by the host. In the method, the first memory may include a plurality of non-volatile memories and the second memory may include a plurality of NAND flash memories. In the method, the computing in the storage device may include at least one of compression, Cyclic Redundancy Check (CRC), or cleaning.

According to some embodiments of the present disclosure, a data processing system is provided that includes a host and a storage device. The host may include: a processor; a memory; and an object storage controller configured to generate object-oriented commands; the storage device may include: a volatile memory; an interface configured to receive an object-oriented command from a host; a central processing unit comprising at least one core configured to perform computations on data stored in a storage device.

In the system, the storage device may further include a first memory configured to store the object attribute and a second memory configured to store the object.

In the system, the object storage controller of the host may further include: an Application Programming Interface (API) library for calling a high-level API; an object storage driver configured to convert the high-level API to the low-level API to generate object-oriented commands. In this system, the storage device may also be used to communicate the results of the calculations to the host.

Drawings

FIG. 1 is a block diagram illustrating a data processing system including a host and an object interface storage device consistent with some embodiments of the present disclosure.

FIG. 2 is a schematic diagram illustrating an object computation management scheme consistent with some embodiments of the present disclosure.

FIG. 3 is a schematic diagram illustrating an object storage management scheme consistent with some embodiments of the present disclosure.

FIG. 4 is a flow chart illustrating operations performed in a data processing system consistent with certain embodiments of the present disclosure.

Fig. 5 is a schematic diagram illustrating operations performed in a storage device consistent with some embodiments of the present disclosure.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, in which like numerals refer to the same or similar elements throughout the different views unless otherwise specified. The embodiments set forth in the exemplary embodiments described below do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with aspects related to the invention as set forth in the claims below. For example, although some embodiments utilizing non-volatile memory and flash memory to store objects are described above and below, the present disclosure is not so limited. Other types of storage devices currently available or developed in the future may be similarly applied.

Large data is physically stored in a database or storage device and objects are used as logical containers for packet data that a user wishes to store or retrieve. An object may contain two parts, object data and object attributes. In the conventional method, object data and object attributes are stored together in a single memory. However, this storage approach may have its drawbacks because the size of the object attributes is typically smaller than the size of the object data.

Further, conventionally, the host may extract desired data from the big data for calculation, and the extracted data may be transferred from the storage system to the host. However, extracting the required data from the big data is time consuming. Furthermore, the continuous transmission of data from the storage system to the host and the computation in the host may consume a large amount of power in the host, resulting in inefficient energy distribution among the different components of the computer. In addition, data transmission consumes bandwidth of the host and may result in data traffic.

The disclosed embodiments provide a storage device that can support object interfaces and perform operations related to objects in the storage device. The object-related operations may include performing computations on data stored in the storage device. By performing computations in the data storage device, the need for a host to extract data from the storage device and compute the data is eliminated, making data processing faster. Furthermore, by eliminating data transmission and computation in the host, power consumption and data traffic in the host is reduced.

The object-related operations may also include object storage management in the storage device. Each object has a unique ID for identification. The object data is stored in the NAND flash memory and the object attributes are stored in the non-volatile memory. By storing object data and object attributes with different sizes separately, the storage use efficiency is improved, and data retrieval becomes easier and faster.

As used herein, unless specifically stated otherwise, the term "or" encompasses all possible combinations, unless otherwise not feasible. For example, if it is stated that a database may contain a or B, then the database may contain a or B or both unless explicitly stated otherwise or otherwise not feasible. As a second example, if it is stated that a database may contain A, B or C, then, unless explicitly stated otherwise or otherwise not feasible, the database may contain A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

Referring now to FIG. 1, shown is a block diagram of a data processing system including a host and an object interface storage device consistent with certain embodiments of the present disclosure. As shown in FIG. 1, data processing system 100 may include a host 110 and a data storage device 130. The host 110 may include a processor 112, a memory 114, a user interface 116, and a storage device controller 118. The storage device controller 118 may include an object storage API library 120 and an object storage drive 122. The processor 104 may include a plurality of processors. In some embodiments, the storage device controller 118 may be implemented in the processor 104.

The storage 130 may include a central computing unit (CPU)140 that includes a plurality of cores, a Dynamic Random Access Memory (DRAM)142, a plurality of non-volatile memories 144, a plurality of flash memories 146, an object interface 132, and an object manager 134. The object manager 134 may include an object storage manager 136 and an object computation manager 138. In some embodiments, storage device 130 may be implemented as a Solid State Drive (SSD). In some embodiments, DRAM 142 may be used as the main memory for storage device 130 and may be replaced by any other type of volatile memory, such as Static Random Access Memory (SRAM) and Tightly Coupled Memory (TCM). In some embodiments, the non-volatile memory 144 may be, but is not limited to, a Read Only Memory (ROM), flash memory, ferroelectric RAM, SSD, hard disk drive, magnetic tape, or optical disk. In some embodiments, the flash memory 146 may be a NAND flash memory or a NOR flash memory. Host 110 may communicate with data storage device 120 via wired or wireless communication.

Host 110 may include, but is not limited to, a personal computer, a workstation, a network computer, a server computer, an application processor, or any device having one or more processors. The memory 114 may store instructions associated with the host 110 or the storage device 130, and the processor 112 is configured (by its circuitry) to execute the instructions.

The user interface 116 may include a display and an input device. In some embodiments, the display may be configured to display an application running in the host 110 or the progress of data processing in the data processing system 110. The display may be any type of computer output interface for displaying information to a computer user and a projection mechanism that displays text and graphical images, including, but not limited to, Cathode Ray Tubes (CRTs), Liquid Crystal Displays (LCDs), Light Emitting Diodes (LEDs), gas plasmas, touch screens, or other image projection technology. The input devices may be any type of computer hardware device for providing data and control signals from an operator to the host 110. Input devices may include, but are not limited to, a keyboard, mouse, scanner, digital camera, joystick, trackball, cursor direction keys, touch screen monitor or audio/video commander to communicate direction information and command selections to a processor or to control cursor movement on a display.

The host 110 may need to extract data stored in the storage device 130 and perform calculations on the data. Rather than extracting and transmitting the required data to host 110 for computation, the data processing system performs in-storage computations based on commands generated by host 110. For example, the host 110 may provide a storage device controller 118 that includes an object storage Application Programming Interface (API) library 120. The object store API library 120 calls an object-oriented high-level API when a particular application in the host is launched. The host 110 may also provide an object storage driver 122 that may convert the object-oriented high-level API into an object-oriented low-level API, such as binary code or a binary file, to generate object-oriented commands. The storage device controller 118 may transmit object-oriented commands to the storage device 130.

In the storage device 130, the object interface 132 may be configured to receive object-oriented commands sent from the host 110. The object interface 132 may further store object-oriented commands in memory of the storage device 130 or provide commands to the CPU 140 for processing object-related operations. In some embodiments, the object-oriented command may be a command for performing an in-memory computation in the core of CPU 140 based on the command. The in-memory computation operations may include, but are not limited to, compression operations, Cyclic Redundancy Checks (CRCs), and scrubbing operations. The object computation manager 138 may manage in-storage computation operations as shown in FIG. 2.

Referring now to FIG. 2, a schematic diagram of an object computation management scheme is shown, consistent with some embodiments of the present disclosure. As shown in FIG. 2, the object computation manager 138 may assign a unique object ID to each computation task, classify the type of computation, and assign a particular core among the cores of the CPU 140. In some embodiments, the object compute manager 138 may assign a particular core of the CPU 140 to a compute task based on the compute type. For example, as shown in FIG. 2, the computations with object ID 1 and ID 2 are compression type computations for which the object computation manager 138 assigns core #1 of the CPU 140. On the other hand, for a computation task with object ID 100, the computation type is Cyclic Redundancy Check (CRC) object computation manager 138 assigns core #2 of CPU 140 for this CRC. In some embodiments, the object computation manager 138 may allocate a particular core of the CPU 140 for a computation task based on a complexity level of the computation or an estimated amount of time of the computation. When the computing task is completed, the object compute manager 138 communicates the results of the computation to the host 110 through the object interface 132.

By performing the computations in the storage device, the need for the host 110 to extract data from the storage device and perform the computations is eliminated, resulting in faster data processing and reduced power consumption in the host. And, by eliminating data transmission between the host and the storage device, data traffic is reduced and bandwidth of the host is reserved for other uses, thereby improving efficiency and speed of the host.

Referring back to FIG. 1, the object manager 134 may also include an object storage manager 136 configured to manage data storage in the storage device 130. An object may contain two parts, object data and object properties. Rather than storing the object data and object properties in a single memory, the object storage manager 136 may allocate space in the flash memory 146 to store the object data and allocate space in the non-volatile memory 144 to store the object properties, as shown in FIG. 3.

Reference is now made to FIG. 3, which is a schematic diagram illustrating an object storage management scheme, consistent with certain embodiments of the present disclosure. The object store manager 136 may include an object translation layer that includes an object property address translation table and an object data address translation table. As shown by the object attribute address translation table, an object store manager (not shown) may assign a unique ID to each object, and each ID is mapped to an object attribute address on non-volatile memory 144.

The object storage management module may also allocate space on the flash memory 146 that stores data for the objects. As shown by the object data address translation table, the object storage manager may assign one or more offset values to each object ID depending on the size of the object data. The object storage manager may also map each object ID with a specified offset value to an object data address (e.g., a physical page number) of flash memory 146. For example, object ID #1 has two corresponding offsets, offset 0 and offset 4096, with offset 0 being mapped to physical page number 1 of flash memory 146 and offset 4096 being mapped to physical page number 2 of flash memory 146. This mapping scheme allows for the separate storage of object attributes and object data, i.e., two types of data that differ greatly in size. By storing object attributes and object data in different memories (non-volatile memory and flash memory), respectively, memory space can be more efficiently utilized, making data retrieval easier and faster.

Referring back to FIG. 1, in some embodiments, the object store manager 136 and the object compute manager 138 may be functional blocks of instructions (or software) stored in a memory of the storage device 130 (e.g., DRAM 142) that the CPU 140 executes to perform the functions of the object store manager 136 and the object compute manager 138 as described above. In some embodiments, the storage device controller 118, including the object storage API library 120 and the object storage driver 122, may be functional blocks of instructions (or software) stored in the memory 114 that the processor 112 executes to perform the functions of the object storage API library 120 and the object storage driver 122 as described above.

Reference is now made to FIG. 4, which is a schematic diagram illustrating operations in a data processing system, consistent with some embodiments of the present disclosure. As shown in FIG. 4, at step 410, an application is launched in a host (e.g., host 110 of FIG. 1). At step 420, the application may call an object-oriented high-level API in the host's object storage API library, such as object storage API library 120 of storage device controller 118 of FIG. 1.

At step 430, the host's object storage driver (e.g., object storage driver 122 of storage device controller 118) may interpret the high-level API and translate it into corresponding object-oriented low-level commands. After the conversion, the subject storage drive may send a command to a storage device, such as storage device 130 of FIG. 1, at step 440. After receiving the command, the storage device may execute the command in the storage device and send the execution back to the host application at step 450.

Reference is made to fig. 5, which is a schematic diagram illustrating operations performed in a memory device (e.g., memory device 130 of fig. 1) consistent with some embodiments of the present disclosure. As shown in FIG. 5, at step 510, the storage device may receive an object-oriented command from a host (e.g., host 110 of FIG. 1). In some embodiments, the command is a command to perform a calculation in a storage device. For example, execution of a command in a storage device may invoke scheduling of a computational task associated with a particular object and run the task on a specified CPU core. In step 520, the storage device executes the command and performs calculations on the data stored in the storage device. In-memory computations may include, but are not limited to, compress operations, Cyclic Redundancy Checks (CRCs), and cleanup operations. At step 530, the storage device sends the computed result to the host. Thus, object-oriented computation is realized in the storage device, and host-specified computation is performed by the storage device, so that the data processing speed in the host is higher, the power consumption is reduced, and the data flow is reduced.

In some embodiments, the command may provide address space management in the storage device. At step 540, the storage device may store the object properties in a first memory of the storage device. For example, when a command is executed in a storage device, the storage device may allocate space in non-volatile memory to store object properties. In step 550, the storage device may store the object data in a second memory of the storage device. For example, when a command is executed in a storage device, the storage device may allocate space in flash memory to store object data. Thus, object-oriented storage management in the storage device is realized, object attributes and object data are stored separately, efficient utilization of storage is realized, and rapid retrieval of the stored data is realized.

Example embodiments are described above with reference to flowchart illustrations or block diagrams of methods, apparatus (systems) and computer program products. It will be understood that each block of the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by instructions on a computer program product or computer program product. These computer program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine (e.g., host 110 or storage device 130), such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a hardware processor core, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart or block diagram block or blocks.

Any combination of one or more computer-readable media may be utilized. The computer readable medium may be a non-transitory computer readable storage medium such as, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or flash memory), an optical fiber, cloud storage, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, IR, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for example embodiments may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate examples of the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts of special purpose hardware and computer instructions, or combinations of special purpose hardware and computer instructions.

It should be understood that the described embodiments are not mutually exclusive and that elements, components, materials or steps described in connection with one exemplary embodiment may be combined with or removed from other embodiments in suitable ways to achieve the desired design goals.

Reference herein to "some embodiments" or "some example embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment. The appearances of the phrase "one embodiment," "some embodiments," or "some exemplary embodiments" in various places in the specification are not necessarily all referring to the same embodiments, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments.

It should be understood that the steps of the example methods set forth herein do not necessarily need to be performed in the order described, and the order of the steps of these methods should be understood to be merely examples. Likewise, in methods consistent with various embodiments, additional steps may be included in such methods, and certain steps may be omitted or combined.

As used in this application, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word is intended to present concepts in a concrete fashion.

In addition, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from context to be directed to a singular form.

The use of figure numbers or figure reference labels in the claims is intended to identify one or more possible embodiments of the claimed subject matter to facilitate the interpretation of the claims. Such use should not be construed as necessarily limiting the scope of those claims to the embodiments shown in the respective figures.

Although the elements of the following method claims, if any, are recited in a particular order with corresponding labeling, unless the claim recitations otherwise imply a particular order for implementing some or all of those elements, those elements are not intended to be limited to being practiced in that particular order.

It will also be understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of the embodiments described may be made by those skilled in the art without departing from the scope of the invention as expressed in the subjoined claims.

Claims (20)

1. A storage device, comprising:

an interface configured to receive an object-oriented command from a host; and

a central processing unit comprising one or more cores, the central processing unit configured to perform computations based on the object-oriented commands.

2. The storage device of claim 1, further comprising:

an object computation manager configured to manage computations in the storage device.

3. The storage device of any of claims 1 and 2, further comprising:

a first memory configured to store object properties; and

a second memory configured to store object data.

4. The storage device of claim 3, further comprising:

an object store manager configured to manage storage of object attributes and object data in the first and second memories, respectively.

5. The storage device of claim 4, wherein the object store manager comprises:

an object property translation table mapping object identifiers to object property addresses in said first memory, an

An object data address translation table that maps the combination of the object identifier and the offset to an object data address of the second memory.

6. The storage device of claim 3, wherein the first memory comprises a plurality of non-volatile memories.

7. The storage device of claim 3, wherein the second memory comprises a plurality of NAND flash memories.

8. The storage device of any of claims 1 to 7, wherein the storage device comprises a Solid State Drive (SSD).

9. The storage device of any of claims 1-8, further comprising a volatile memory, wherein the volatile memory is one of a Dynamic Random Access Memory (DRAM), a Static Random Access Memory (SRAM), or a Tightly Coupled Memory (TCM).

10. The storage device of any of claims 1-9, wherein the computing comprises at least one of compression, cyclic redundancy check, or scrubbing.

11. A method of operating a storage device, comprising:

receiving, by a storage device, an object-oriented command from a host;

computing, by a storage device, data stored in the storage device; and

the calculation result is transmitted to the host by the storage device.

12. The method of claim 11, wherein the object-oriented command is a low-level Application Programming Interface (API) command generated by a host.

13. The method of any of claims 11 and 12, further comprising:

storing the object attributes in a first memory of a storage device; and

the object data is stored in a second memory of the storage device.

14. The method of claim 13, wherein the first memory comprises a plurality of non-volatile memories.

15. The method of claim 13, wherein the second memory comprises a plurality of NAND flash memories.

16. The method of any of claims 11 to 15, wherein the computing in the storage device comprises at least one of compression, Cyclic Redundancy Check (CRC), or cleaning.

17. A data processing system comprising:

a host, comprising:

a processor for processing the received data, wherein the processor is used for processing the received data,

a memory, and

an object storage controller configured to generate object-oriented commands;

and

a storage device, comprising:

an interface configured to receive the object-oriented command from the host, an

A central processing unit comprising at least one core, the central processing unit configured to perform computations on data stored in the storage device.

18. The system of claim 17, wherein the storage device further comprises:

a first memory configured to store object properties; and

a second memory configured to store object data.

19. The system of claim 17, wherein the object storage controller of the host further comprises:

an Application Programming Interface (API) library configured to call a high-level API; and

an object storage driver configured to convert the high-level API to a low-level API to generate the object-oriented commands.

20. The system of claim 17, wherein the storage device is further configured to transmit the calculation results to the host.

Images