CN114296638A

CN114296638A - Storage and calculation integrated solid state disk controller, solid state disk, data storage system and method

Info

Publication number: CN114296638A
Application number: CN202111506094.6A
Authority: CN
Inventors: 方浩俊; 黄运新; 俞伟
Original assignee: Shenzhen Dapu Microelectronics Co Ltd
Current assignee: Shenzhen Dapu Microelectronics Co Ltd
Priority date: 2021-12-10
Filing date: 2021-12-10
Publication date: 2022-04-08
Anticipated expiration: 2041-12-10
Also published as: CN114296638B

Abstract

The embodiment of the application relates to the field of solid state disk application, and discloses a storage and calculation integrated solid state disk controller, a solid state disk, a data storage system and a data processing method among the storage and calculation systems, wherein the storage and calculation integrated solid state disk controller comprises a storage system used for storing data; a computing system for performing data calculations; and the storage and calculation path module is connected with the storage system and the calculation system and is used for realizing data transmission processing between the storage system and the calculation system. By designing the storage and calculation path module, data transmission between the storage system and the computing system is realized, and the design of the hardware path between the storage system and the computing system in the solid state disk controller can be optimized, so that the storage system and the computing system can realize internal transmission standardization, the software development amount is reduced, and the software development efficiency of the solid state disk is improved.

Description

Storage and calculation integrated solid state disk controller, solid state disk, data storage system and method

Technical Field

The application relates to the field of solid state disk application, in particular to a storage and computation integrated solid state disk controller, a solid state disk, a data storage system and a data processing method between storage and computation systems.

Background

Solid State Drives (SSD), which are hard disks made of Solid State electronic memory chip arrays, include a control unit and a memory unit (FLASH memory chip or DRAM memory chip). At present, a considerable part of solid state disk systems are Dynamic Random Access Memories (DRAMs), so that SSDs have a large data cache space for caching data. Flash memory (NAND Flash) is the main storage medium for solid state disks.

The solid state disk has become a main device for data storage, and under the background of big data application, mass data transmission occupies various buses and network bandwidths, and meanwhile, the computing power of a CPU contained in the solid state disk is stronger and stronger. In order to reduce the transmission of large data volume, various requirements for integration of storage and calculation are derived, such as Edge calculation (Edge calculation) and Machine Learning (Machine Learning), in storage, that is, calculation processing is performed in a data side solid state disk, and only results and partial data are transmitted, so that the transmission of mass data is reduced, and the loads of a bus and a network are reduced.

At present, a dual-system architecture is introduced into a storage-computing integrated solid state disk: one system is used for data storage processing, called a storage system, the other system is used for data calculation processing, called a computing system, and the two systems communicate through an Inter-Process Communication (IPC) mechanism to complete the interaction of data and information. The IPC communication mechanism is originally used for inter-core communication, is mainly based on a message mechanism, and when the IPC communication mechanism is used for a storage and computation integrated system, a user-defined protocol stack is needed, namely, currently, no matter a storage system or a computation system, a plurality of software development is needed, for example, the IPC communication mechanism is used as a specific device under the computation system to carry out drive development, and a data path is continuously developed under the storage system. Under the condition of existence of a custom protocol stack, application and popularization under a general system are limited to a certain extent.

In the process of implementing the present application, the applicant finds that the current technical solution has at least the following technical problems: the communication mechanism between the internal storage systems of the storage-computer integrated solid state disk controller is not friendly to software development, which results in insufficient software development efficiency of the solid state disk.

Disclosure of Invention

The embodiment of the application aims to provide a storage and computation integrated solid-state hard disk controller, a solid-state hard disk, a data storage system and a data processing method among storage systems, and solves the technical problem that the communication mechanism among the internal storage systems of the existing storage and computation integrated solid-state hard disk controller is not friendly to software development, so that the software development efficiency of the solid-state hard disk is insufficient, the software development amount is reduced, and the software development efficiency of the solid-state hard disk is improved.

In order to solve the above technical problem, an embodiment of the present application provides the following technical solutions:

in a first aspect, an embodiment of the present application provides a storage and computation integrated solid-state hard disk controller, configured to store and compute integrated solid-state storage, including:

the storage system is used for storing data;

a computing system for performing data calculations; and the storage and calculation path module is connected with the storage system and the calculation system and is used for realizing data transmission between the storage system and the calculation system.

In some embodiments, a computation path module, comprises:

the host module processing unit is used for simulating the functions of the host end;

the equipment module processing unit is used for simulating the functions of an equipment end;

the path processing unit is connected with the host module processing unit and the equipment module processing unit and is used for processing the data requirements of the host module processing unit and the equipment module processing unit;

and the data path interface is connected with the path processing unit and is used for processing the interface requirement of the internal data bus.

In some embodiments, the computation path module further comprises:

the first system interface is connected with the host module processing unit and is used for carrying out interactive operation with the host module processing unit;

and the second system interface is connected with the equipment module processing unit and used for carrying out interactive operation with the equipment module processing unit.

In some embodiments, the storage-integrated solid state hard disk controller further comprises:

a first host interface for interfacing with a first host;

and the second host interface is used for butting the second host.

In some embodiments of the present invention, the,

a storage system comprising a first cluster of processors for processing data storage operations;

a computing system includes a second cluster of processors to process data computation operations.

In some embodiments, the storage system further comprises:

the front-end module is used for processing a communication protocol with the first host and distributing data storage operation sent by the first host;

the data processing module is connected with the front-end module and is used for processing a data path;

the mapping table management module is connected with the data processing module and is used for managing the mapping table and managing the granularity of data written into the flash memory;

and the back-end module is connected with the mapping table management module and the flash memory medium and is used for reading and writing flash memory data and managing flash memory commands.

In some embodiments of the present invention, the,

the computing system and the storage system share one dynamic random access memory and a dynamic random access memory controller, and the dynamic random access memory is connected with the dynamic random access memory controller;

the first processor cluster is connected with the storage and computation path module, the second processor cluster is connected with the storage and computation path module, and the front-end module is connected with the first processor cluster and the storage and computation path module.

Processing unit in some embodiments,

the first processor cluster comprises an application processor and/or a real-time processor;

the second cluster of processors includes an application processor and/or a real-time processor.

In a second aspect, an embodiment of the present application provides a solid state disk, including:

a storage-and-computation-integrated solid state hard disk controller as in the first aspect;

and the at least one flash memory medium is connected with the solid state hard disk controller.

In a third aspect, an embodiment of the present application provides a data storage system, including:

the solid state disk of the second aspect;

host system, communication connection solid state hard drives, host system includes:

a first host for interfacing with a storage system;

and the second host is used for butting the computing system.

In a fourth aspect, an embodiment of the present application provides a data processing method between storage systems, which is applied to the storage system in the third aspect, and the method includes:

receiving an operation task command sent by a first host or a second host, wherein the operation task command comprises information of at least one operation task;

and converting at least one operation task into IO operation, and forwarding the IO operation to the storage system or directly receiving the IO operation by the storage system through the storage access module so that the storage system processes the IO operation.

In a fifth aspect, the present application further provides a non-volatile computer-readable storage medium storing computer-executable instructions for enabling a solid state disk to perform the data processing method between the storage systems according to the fourth aspect.

The beneficial effects of the embodiment of the application are that: different from the situation of the prior art, the storage and calculation integrated solid-state hard disk controller provided by the embodiment of the application is applied to a solid-state hard disk, and comprises a storage system for storing data; a computing system for performing data calculations; and the storage and calculation path module is connected with the storage system and the calculation system and is used for realizing data transmission between the storage system and the calculation system. By designing the storage and calculation path module, data transmission between the storage system and the calculation system is realized, and the internal hardware path design of the solid state disk controller can be optimized, so that the storage system and the calculation system can realize internal transmission standardization, the software development amount is reduced, and the software development efficiency of the solid state disk is improved.

Drawings

One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.

Fig. 1 is a schematic structural diagram of a solid state disk hardware frame according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a solid state disk provided in an embodiment of the present application;

fig. 3 is a schematic structural diagram of a solid state disk hardware frame according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a solid-state hard disk controller according to an embodiment of the present application;

fig. 5 is a hardware system diagram of a solid-state hard disk controller according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a host and a storage device according to an embodiment of the present disclosure;

FIG. 7 is a diagram of a standard PCIe hierarchy provided by an embodiment of the present application and a PCIe hierarchy implemented within a controller of the present application;

fig. 8 is a schematic diagram of an internal system of a computing-integrated solid-state hard disk controller according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a storage and computation integrated solid-state hard disk controller according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of another storage and computation integrated solid-state hard disk controller provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of the structure of the deposit path module of FIG. 9;

FIG. 12 is a schematic diagram illustrating a connection between a storage access module and a computing system and a storage system according to an embodiment of the present disclosure;

FIG. 13 is an interaction diagram of a storage access module, a computing system and a storage system according to an embodiment of the present disclosure;

FIG. 14 is a schematic diagram of a hardware system framework of a solid state disk with integral storage according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of a first processor cluster according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of a second processor cluster according to an embodiment of the present application;

fig. 17 is a schematic diagram of an IO operation between a solid state disk and a host according to an embodiment of the present application;

FIG. 18 is a schematic structural diagram of a data storage system according to an embodiment of the present application;

fig. 19 is a flowchart illustrating a data processing method between computing systems according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In addition, the technical features mentioned in the embodiments of the present application described below may be combined with each other as long as they do not conflict with each other.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a solid state disk hardware frame in the prior art;

as shown in fig. 1, a Solid State Drive (SSD) generally includes a Solid State drive Controller (SSD), i.e., a main Controller (SSD Controller), a Connector (Connector), a flash memory array, a cache unit, and other peripheral units.

The solid state hard disk controller is used as a control operation unit for managing an SSD internal system; flash memory arrays (NAND Flash), as memory cells for storing data, including user data and system data, typically present multiple channels (channels, abbreviated CH), one Channel being independently connected to a set of NAND Flash, e.g. CH0/CH1 … … CHx. The characteristic of the Flash memory (NAND Flash) is that before writing, erasing must be carried out, and the erasing times of each Flash memory are limited; the cache unit is used for caching the mapping table, and is generally a Dynamic Random Access Memory (DRAM). The Connector (Connector) is used to connect a host, for example: a PC or a server, and other peripheral units may include serial ports, sensors, registers, power chips, and the like.

Referring to fig. 2 again, fig. 2 is a schematic structural diagram of a solid state disk provided in the embodiment of the present application;

as shown in fig. 2, the solid state disk includes a flash memory medium and a solid state disk controller connected to the flash memory medium. The solid state disk is in communication connection with the host computer in a wired or wireless mode and is used for realizing data interaction.

The Flash memory medium, which is a storage medium of a solid state disk, is also called a Flash memory, a Flash memory or a Flash granule, belongs to one of storage devices, is a nonvolatile memory, can store data for a long time under the condition of no current supply, and has storage characteristics equivalent to that of a hard disk, so that the Flash memory medium can become the basis of the storage medium of various portable digital devices.

The FLASH memory medium can be Nand FLASH, the Nand FLASH uses a single transistor as a storage unit of a binary signal, the structure of the Nand FLASH is very similar to that of a common semiconductor transistor, the difference is that a floating gate and a control gate are added to the single transistor of the Nand FLASH, the floating gate is used for storing electrons, the surface of the floating gate is coated by a layer of silicon oxide insulator and is coupled with the control gate through a capacitor, when negative electrons are injected into the floating gate under the action of the control gate, the storage state of the single crystal of the Nand FLASH is changed from '1' to '0', when the negative electrons are removed from the floating gate, the storage state is changed from '0' to '1', and the insulator coated on the surface of the floating gate is used for trapping the negative electrons in the floating gate to realize data storage. That is, the Nand FLASH memory cell is a floating gate transistor, and data is stored in the form of electric charge using the floating gate transistor. The amount of charge stored is related to the magnitude of the voltage applied to the floating gate transistor.

A Nand FLASH comprises at least one Chip, each Chip is composed of a plurality of Block physical blocks, and each Block physical Block comprises a plurality of Page pages. The Block physical Block is the minimum unit of Nand FLASH for executing the erasing operation, the Page is the minimum unit of Nand FLASH for executing the reading and writing operation, and the capacity of one Nand FLASH is equal to the number of the Block physical blocks and the number of the Page pages contained in one Block physical Block. Specifically, the flash memory medium 200 may be classified into SLC, MLC, TLC and QLC according to different levels of the voltages of the memory cells.

The solid state hard disk controller comprises a data converter, a processor, a buffer, a flash memory controller and an interface.

And the data converter is respectively connected with the processor and the flash memory controller and is used for converting the binary data into hexadecimal data and converting the hexadecimal data into the binary data. Specifically, when the flash memory controller writes data to the flash memory medium, the binary data to be written is converted into hexadecimal data by the data converter, and then written to the flash memory medium. When the flash memory controller reads data from the flash memory medium, hexadecimal data stored in the flash memory medium is converted into binary data by the data converter, and then the converted data is read from the binary data page register. The data converter may include, among other things, a binary data register and a hexadecimal data register. The binary data register may be used to store data converted from hexadecimal to binary, and the hexadecimal data register may be used to store data converted from binary to hexadecimal.

The processor is connected with the data converter, the buffer, the flash memory controller and the interface respectively, wherein the processor, the data converter, the buffer, the flash memory controller and the interface can be connected through a bus or other modes, and the processor is used for operating the nonvolatile software program, the instructions and the modules stored in the buffer, so that any method embodiment of the application is realized.

The buffer is mainly used for buffering read/write commands sent by the host and read data or write data acquired from the flash memory medium according to the read/write commands sent by the host. The cache, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The buffer may include a storage program area that may store an operating system, an application program required for at least one function. In addition, the buffer may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the cache optionally includes memory that is remotely located from the processor. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The buffer may be a Static Random Access Memory (SRAM), a Coupled Memory (TCM), or a Double data rate Synchronous Dynamic Random Access Memory (DDR SRAM).

The flash memory controller is connected with the flash memory medium, the data converter, the processor and the buffer and is used for accessing the flash memory medium at the rear end and managing various parameters and data I/O of the flash memory medium; or, an interface and a protocol for providing access, implementing a corresponding SAS/SATA target protocol end or NVMe protocol end, acquiring an I/O instruction sent by a host, decoding and generating an internal private data result to wait for execution; or, the core processing module is used for taking charge of the FTL (Flash translation layer).

The interface is connected with the host, the data converter, the processor and the buffer and used for receiving data sent by the host or receiving data sent by the processor and realizing data transmission between the host and the processor, and the interface can be an SATA-2 interface, an SATA-3 interface, an SAS interface, an MSATA interface, a PCI-E interface, an NGFF interface, a CFast interface, an SFF-8639 interface and an M.2NVME/SATA protocol.

Referring to fig. 3, fig. 3 is a schematic structural diagram of a solid state disk hardware frame according to an embodiment of the present disclosure;

compared with the solid state disk hardware framework in fig. 1, the solid state disk hardware framework has two interfaces, namely Connector0 and Connector1, which are respectively used for data storage and data calculation, and the problem that a calculation application channel and a data storage channel compete with each other for bus bandwidth is solved through separation of host interfaces.

Referring to fig. 4 again, fig. 4 is a schematic structural diagram of a solid state hard disk controller according to an embodiment of the present disclosure;

as shown in fig. 4, the solid state disk controller includes two interfaces, namely a first interface and a second interface, for data storage and data calculation, respectively.

At present, due to the demand of computing application, a dual system architecture running in a solid state disk is introduced: one system for data storage processing, referred to as a storage system, and another system for data computing processing or application processing, referred to as a computing system. The two systems communicate with each other through an IPC (Inter-Process Communication) mechanism to complete the interaction of data and information. The IPC is realized by software FIFO, hardware FIFO, shared memory, hardware IPC logic circuit, etc.

Referring to fig. 5 again, fig. 5 is a schematic diagram of a hardware system of a solid state hard disk controller according to an embodiment of the present disclosure;

as shown in fig. 5, the solid state hard disk controller has a hardware IPC design inside, that is, includes an IPC system, and data processing between the first processor cluster and the second processor cluster is implemented by the IPC system, where the IPC system has an IPC software system in software design. The data transmission path is realized by a hardware system and a software system, for example: and realizing a data transmission path required by data calculation processing or application processing through the application IO path.

In terms of hardware, information of two hardware systems, namely a first processor cluster and a second processor cluster, is interacted through an IPC (inter-processor control) hardware interface, and data interaction is realized through DDR (double data rate) sharing on a large amount of data.

In terms of software, the two software systems both have a hierarchical structure, so the IO operations transmitted between the two software systems are transmitted through the IPC system.

The data path exists as a Specific device for the software of the computing system, and a bottom driver (running in an SDL, Specific Drive Layer) is developed for dispatching or reading IO operations, that is, information of the IO operations is written or read into an interface of a FIFO/shared memory/hardware IPC logic circuit; an intermediate program (running in MDL, Middle Layer) is developed to meet os (linux) docking requirements.

However, because the IPC system needs a custom protocol stack in software design, both the storage system and the computing system need many software developments, such as development of specific device drivers under the computing system and development of data paths under the storage system, which results in insufficient software development efficiency of the solid state disk.

In view of this, the present application provides a storage and computation integrated solid state disk controller, a solid state disk, a storage system, and a data processing method between storage systems, so as to improve software development efficiency of the solid state disk.

The technical scheme of the application is specifically explained in the following by combining the drawings in the specification.

Referring to fig. 6, fig. 6 is a schematic diagram of a host and a storage device according to an embodiment of the present disclosure;

as shown in fig. 6, the host and the storage device communicate via a standard PCIe bus (via the PCIe protocol), where the PCIe bus is a PCB transmission line in the system, including slots, patch cords, and the like. In the traditional solid state disk integrating storage and computation, two systems exist in the same solid state disk controller, a standard channel between the host and the equipment between the two systems is not designed, and the standardized data channel is realized in the solid state disk controller, so that the internal transmission standardization of the storage system and the computing system can be realized.

Referring to fig. 7, fig. 7 is a schematic diagram of a standard PCIe hierarchy and a PCIe hierarchy implemented in a controller of the present application according to an embodiment of the present application;

as shown in fig. 7, the standard PCIe hierarchy includes two PCIe devices, a first PCIe Device (PCIe Device a) and a second PCIe Device (PCIe Device B), each PCIe Device including a core Layer (DeviceCore), a PCIe core hardware/software interface (PCIe core hardware/software interface), a transaction Layer (transactionaliyer), a Data Link Layer (Data Link Layer), and a Physical Layer (Physical Layer). It is understood that in the PCIe architecture, a Data message is first generated in a Core Layer (Device Core) of a Device, then passes through a Transaction Layer (Transaction Layer), a Data Link Layer (Data Link Layer) and a Physical Layer (Physical Layer) of the Device, and is finally transmitted. And the data at the receiving end also needs to pass through the physical layer, the data link layer and the transaction layer and finally reach the core layer.

It will be appreciated that the main problem is the high cost if the complete host and device modules are implemented in a solid state hard disk controller, internal transmission is clearly not suitable for transmitting such a mode, and there is no need for the analog circuit part (generally referred to as the PHY part) of the two-terminal interface, and even the control circuit does not need full functionality. Also taking the PCIe hierarchy as an example, as shown in fig. 7, the left side is a standard hierarchy, which is seen as an application layer for software, and is not concerned with specific other hierarchies, so that the present application implements interaction between two systems (a and B) inside the solid-state controller, and implements data transmission required by the application layer by replacing other hierarchies with a data transmission layer.

In short, with the implementation described above, two systems within the solid state hard disk controller are considered as one standard host system and one standard storage device system, thereby simplifying software design.

Referring to fig. 8, fig. 8 is a schematic diagram of an internal system of a computing integrated solid state hard disk controller according to an embodiment of the present application;

as shown in fig. 8, the computing system and the storage system communicate with each other through a storage path, wherein the computing system serves as a standard host system, the storage system serves as a standard storage system, and the computing system performs operations such as reading and writing operations on the storage system through a path having a standard protocol.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a storage and computation integrated solid state hard disk controller according to an embodiment of the present application;

as shown in fig. 9, the integrated-storage solid-state hard disk controller 100 includes: the storage system 10, the computing system 20 and the storage path module 30, wherein the storage system 10 and the computing system 20 are connected through the storage path module 30.

Referring to fig. 10 again, fig. 10 is a schematic structural diagram of another storage and computation integrated solid state hard disk controller according to an embodiment of the present application;

as shown in fig. 10, the integrated-storage solid-state hard disk controller 100 includes: a first host interface 41, a second host interface 42, a computation path module 30, a cache control module 21, a processing module 11, a data path processing module 22, and a flash control module 23.

The first host interface 41 is configured to interface with a first host, for example: the interface can be SATA-2 interface, SATA-3 interface, SAS interface, MSATA interface, PCI-E interface, NGFF interface, CFast interface, SFF-8639 interface and M.2NVME/SATA protocol.

The second host interface 42 is configured to interface with a second host, for example: the interface can be SATA-2 interface, SATA-3 interface, SAS interface, MSATA interface, PCI-E interface, NGFF interface, CFast interface, SFF-8639 interface and M.2NVME/SATA protocol.

The storage path module 30 is used for implementing a connection between the storage system 10 and the computing system 20 to implement data transmission between the storage system 10 and the computing system 20.

The cache control module 21 includes a cache, and is mainly used for caching the read/write command sent by the host and the read data or write data acquired from the flash memory medium according to the read/write command sent by the host. The cache, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The buffer may include a storage program area that may store an operating system, an application program required for at least one function. In addition, the buffer may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the cache optionally includes memory that is remotely located from the processor. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The buffer may be a Static Random Access Memory (SRAM), a Coupled Memory (TCM), or a Double data rate Synchronous Dynamic Random Access Memory (DDR SRAM).

The processing module 11 includes a processor, which is connected to the data converter, the buffer, the flash memory controller, and the interface, where the processor, the data converter, the buffer, the flash memory controller, and the interface may be connected through a bus or other means, and the processor is configured to run the nonvolatile software program, the instructions, and the modules stored in the buffer, so as to implement any method embodiment of the present application. Or, an interface and a protocol for providing access, implementing a corresponding SAS/SATA target protocol end or NVMe protocol end, acquiring an I/O instruction sent by a host, decoding and generating an internal private data result to wait for execution; or, the core processing module is used for taking charge of the FTL (Flash translation layer).

The datapath processing module 22 includes datapath hardware logic modules, such as: and the Data Path Processor is used for accelerating the processing of the stored Data.

The flash memory control module 23 includes a flash memory controller, connected to the flash memory medium, the data converter, the processor, and the buffer, and configured to access the flash memory medium at the back end and manage various parameters and data I/O of the flash memory medium.

Referring to fig. 11 again, fig. 11 is a schematic structural diagram of the computation path module in fig. 9;

as shown in fig. 11, the computation path module 30 includes: the host module processing unit 31, the device module processing unit 32, the path processing unit 33, the data path interface 34, the second system interface 35, and the first system interface 36, wherein the host module processing unit 31 is connected to the path processing unit 33, the path processing unit 33 is connected to the device module processing unit 32, the data path interface 34 is connected to the path processing unit 33, the second system interface 35 is connected to the host module processing unit 31, and the first system interface 36 is connected to the device module processing unit 32.

It will be appreciated that within the solid state hard disk controller, there is effectively no Host-side (Host) hardware module for the computing system that manages a standard storage device, such as a PCIe Host device-PCIe RC (Root Complex); for storage systems, there is also no Device side (Device) hardware module for the internal computing system host, such as PCIe EP devices, so the storage path module needs to implement the emulation functions of both devices, as well as the processing functions of the data transmission path.

Specifically, the host module processing unit 31 is configured to simulate functions of a host, such as: host-side communication functions such as: PCIe RC functions, i.e., Root Complex (RC), i.e., functions of a Root node (Root) in the PCIe system, wherein the PCIe RC functions refer to functions that communicate with other parts of the computer system, such as: the CPU accesses the memory through the Root Complex (Root Complex), or accesses the PCIe device in the PCIe system through the Root Complex (Root Complex). The functions of the host side belong to a part of a standard protocol and are realized according to the standard protocol definition. In the embodiment of the present application, the host module processing unit 31 includes a simulator.

It is understood that peripheral component interconnect express (PCIe) is a high-speed short-distance communication interface, and is widely used in computers, test instruments, and other devices. The main components of the PCIe system include a Root node (Root), a Switch node (Switch), and an end node (Endpoint).

Wherein, Root is responsible for managing all buses and nodes in the PCIe system, and is a bridge for communication between a Central Processing Unit (CPU) and an Endpoint in the PCIe system; the Switch is used as a data forwarding node and connected with the Switch and the Endpoint; an Endpoint is an end device, such as a Peripheral (Peripheral). In the PCIe system, the Endpoint and the Endpoint cannot directly communicate with each other, and the Endpoint must pass through Root.

Specifically, the device module processing unit 32 is configured to simulate a function of a device side, for example: PCIe EP function (PCIe Endpoint), i.e., a function of an end node (Endpoint) of the PCIe system. The functions of the equipment end belong to one part of a standard protocol and are realized according to the standard protocol definition. In the present embodiment, the device module processing unit 32 includes a simulator.

It should be noted that the host module processing unit 31 and the device module processing unit 32 only need to simulate part of the functions, because data is transmitted through the internal data bus without the need of simulating circuits/interfaces. For example: the host module processing unit 31 and the Device module processing unit 32 are configured to simulate a data transmission function, as shown in fig. 7, the interaction between the first PCIe Device (PCIe Device a) and the second PCIe Device (PCIe Device B) is implemented in the solid state disk controller, and the data transmission layer replaces other layers to implement data transmission required by the application layer, which is equivalent to the function of the host module processing unit and the Device module processing unit for implementing the data transmission layer.

Specifically, the path processing unit 33 is connected to the host module processing unit 31 and the device module processing unit 32, and is configured to process data requirements of the host module processing unit 31 and the device module processing unit 32, for example: the method is used for processing the requirements of data forwarding, format change and the like.

Specifically, the data path interface 34 is connected to the path processing unit 33, and is configured to handle the interface requirement of the internal data bus.

Specifically, the first system interface 36 is coupled to the equipment module processing unit 32 for interoperation with the equipment module processing unit 32 and for communication between the storage system and the storage access module.

Specifically, the second system interface 35 is connected to the host module processing unit 31, and is used for performing an interactive operation with the host module processing unit 31 and for communicating the computing system and the storage path module.

Referring to fig. 12, fig. 12 is a schematic diagram illustrating a connection between a processing unit and a storage path module, a computing system, and a storage system according to an embodiment of the present disclosure;

as shown in fig. 12, the storage system includes a first processor cluster, a DDR path, and an NVMe path, and the computing system includes a second processor cluster and a DDR path, where the first system interfaces with the first processor cluster of the storage system, the first processor cluster connects with the DDR path, the data path interfaces with the DDR path and the NVMe path, the second system interfaces with the second processor cluster of the storage system, and the second processor cluster connects with the DDR path. The DDR Access is used for being connected with a DDR Memory, namely a Double Data Rate Synchronous Random Access Memory (DDR); the NVMe access is used to interface the NVMe controller.

Referring to fig. 13, fig. 13 is an interaction diagram of a data path module, a computing system and a storage system according to an embodiment of the present disclosure;

as shown in fig. 13, the second processor cluster of the processing unit computing system sends an operation command to the first processor cluster of the storage system through the second system interface of the storage path module, through the host module processing unit, the path processing unit, and the data path interface, and by the NVMe path of the storage system.

The first processor cluster of the storage system sends a response to the second processor cluster of the computing system through the first system interface of the storage path module, the device module processing unit, the path processing unit and the host module processing unit, and the second system interface.

And the data block acquired by the back-end module reaches the second processor cluster through the NVMe path and the DDR path, and is calculated and processed by the second processor cluster.

Referring to fig. 14 again, fig. 14 is a schematic diagram of a hardware system framework of a solid state disk with integral storage according to an embodiment of the present application;

as shown in fig. 14, where Host1 is a first Host, Host2 is a second Host, Host1IF is a first Host interface, Host2 IF is a second Host interface, CPU Cluster1 is a first processor Cluster, CPU Cluster2 is a second processor Cluster, iHost2Device Channel is a storage path Module, FEModule is a front-end Module, BE Module is a back-end Module, nand flash is a flash memory, DDR is a DDR memory, and DDRController/PHY is a DDR controller for controlling and handling interface requirements of DRAM, where the DDR controller includes a DDR interface.

The first processor Cluster (CPU Cluster 1) comprises a data storage processing system, the second processor Cluster (CPU Cluster2) comprises a data computing application system, both the data storage processing system and the data computing application system have a layered structure, and IO operations transmitted between the data storage processing system and the data computing application system are transmitted through an iHost2Device Channel. Specifically, the transmission is completed by a data processing unit under a standard protocol, and the data processing unit comprises data, an operation command and the like. It is understood that, for the corresponding driver, the dispatching or reading of the IO operation is to write or read information of the IO operation to or from the first system interface or the second system interface of the computation path module.

Specifically, a first Host (Host1) interacts with a Flash memory medium (NAND Flash Array) through a first Host interface (Host1 IF), a front-end Module (FE Module), and a back-end Module (BE Module);

the second Host (Host2) interacts with the Flash memory medium (NAND Flash Array) via a second Host interface (Host2 IF), a second processor Cluster (CPU Cluster2), a DDR memory (DDR), a DDR Controller (DDR Controller/PHY), a storage Channel Module (iHost2Device Channel), a front end Module (FE Module), and a back end Module (BE Module).

Referring to fig. 15 again, fig. 15 is a schematic structural diagram of a first processor cluster according to an embodiment of the present disclosure;

as shown in fig. 15, the first processor cluster 11 includes a data storage processing system 110 for performing data storage operations, and the data storage processing system 110 includes:

a Front-End module 111, i.e. (Front End, FE), connected to the data storage interface for processing the communication protocol with the host system and distributing the data storage operation sent by the host system;

a Data processing module 112, i.e., (Data Process, DP), connected to the front-end module 111, for processing Data paths, specifically, for taking charge of command level Data processing, such as caching Data;

a mapping table management module 113 (i.e. Flash Translation Layer, FTL) connected to the data processing module 112 for managing the mapping table and the granularity of data written in the Flash memory;

the Back End module 114, i.e., (Back End, BE), is connected to the mapping table management module 113 and the flash medium, and is used for reading and writing data of the flash and managing commands of the flash.

In an embodiment of the application, the first cluster of processors comprises an application processor and/or a real-time processor.

Referring to fig. 16 again, fig. 16 is a schematic structural diagram of a second processor cluster according to an embodiment of the present application;

as shown in fig. 16, the second processor cluster 21 includes a data computing application system 210, wherein the data computing application system 210 is configured to perform data computing operations, and wherein the data computing application system 210 includes: an operating system module 211, a data computing application module 212, an NVMe device module 213, and a PCIe device module 214.

Specifically, the operating system module 211 is configured to perform a bottom layer operation of the operating system, where the operating system module 211 is an operating system layer (OS Kernel), that is, an operating system Kernel, and is configured to perform the bottom layer operation of the operating system, including bottom layer operations such as command parsing and code compiling;

specifically, the data calculation application module 212 is connected to the operating system module 211 and configured to perform application processing related to a data calculation task sent by the host. Due to the fact that the application layer is developed to have very high universality based on an Operating System (OS), related hardware characteristics can be separated, and the application comprises List Intersection of a search engine, retrieval of MySQL and the like, wherein the data computing application module comprises an application layer (APL).

Specifically, the NVMe device module 213 is connected to the operating system module 211, the data calculation application module 212, and the PCIe device module 214, and configured to convert the data calculation task into an IO operation and send the corresponding IO operation to the NVMe memory. In the embodiment of the present application, the NVMe device module 213 includes an NVMe device layer (NVMeDeviceLayer).

Specifically, the PCIe device module 214 is connected to the NVMe device module 213, the operating system module 211, and the front-end module of the first processor cluster, and is configured to send an IO operation to the front-end module of the first processor cluster. In an embodiment of the present application, the PCIe Device module 214 includes a PCIe Device Layer (PCIe Device Layer).

In the embodiment of the present application, the data computing application system 210 further includes an interface module 215, where the interface module 215 is connected to the data computing application module 212 and the PCIe device module 214, and the interface module 215 is configured to send an IO operation, or the interface module 215 is communicatively connected to a host interface and configured to receive a host command sent by a host.

In an embodiment of the application, the second cluster of processors 21 comprises application processors and/or real-time processors.

In the embodiment of the application, the data storage processing system and the data computing application system independently run in parallel. By running the data storage processing system on the first processor cluster for specially processing storage transactions and running the data computing application system on the second processor cluster for specially processing computing application transactions, the processing efficiency of the first processor cluster and the second processor cluster can be improved.

Referring to fig. 17, fig. 17 is a schematic diagram illustrating an IO operation between a solid state disk and a host according to an embodiment of the present application;

as shown in fig. 17, the second processor Cluster (CPUCluster 2) is used as an internal host (InternalHost) in the function of the solid state disk controller, and the first processor Cluster (CPU Cluster 1) is used as an internal device (InternalDevice) in the function of the solid state disk controller, and realizes a standard storage device path via an internal virtual host-to-device connection (VirtualHost2device connect).

It can be seen that the computing system can complete information or data interaction with the storage system by using a standard storage Device path, typically, a PCIe Device Layer (PCIe Device Layer) path or an NVMe Device Layer (NVMe Device Layer) path, without using a specific Device path, so that the computing system can perform operations such as reading and writing operations on the storage system through the standard path. For example: the front-end module (FE) is connected with a PCIe Device Layer, an NVMe Device Layer and an Application Layer (APL) of the computing system through a back-end module (BE) of the storage system, a mapping table management module (FTL) and a data path module (DP), so that interaction between the computing system and the storage system is realized.

Referring to fig. 18, fig. 18 is a schematic structural diagram of a data storage system according to an embodiment of the present application;

as shown in fig. 18, the data storage system 400 includes: the solid state disk 200 is in communication connection with the host system 300, wherein the solid state disk 200 includes a storage and computation integrated solid state disk controller 100 and a flash memory medium 220, and the storage and computation integrated solid state disk controller 100 is connected with the flash memory medium 220.

Specifically, the storage-integrated solid state hard disk controller 100 includes: the storage system 10, the computing system 20, the storage access module 30, the first host interface 41 and the second host interface 42, wherein the first host interface 41 is connected to the storage system 10, the second host interface 42 is connected to the computing system 20, and the storage access module 30 is respectively connected to the storage system 10 and the computing system 20, so as to implement interaction of data and information between the storage system 10 and the computing system 20.

In the embodiment of the present application, the storage system 10 is connected to the flash storage medium 220 to perform IO operations of the flash storage medium.

Specifically, the host system 300 includes a first host 310 and a second host 320, the first host 310 is communicatively connected to the storage system 10 through a first host interface 41, the first host 310 sends data storage operations to the storage system 10 through the first host interface 41 to enable the storage system 10 to process the data storage operations, the second host 320 is communicatively connected to the computing system 20 through a second host interface 42, and the second host 320 sends data calculation operations to the computing system 20 through the second host interface 42 to enable the computing system 20 to process the data calculation operations.

It is understood that the first host and the second host may be the same host or different hosts, for example: the first host is a local host, and the second host is a cloud server. First, second and third are not necessarily sequential, but are merely descriptive terms.

In the embodiment of the application, the storage and calculation integrated solid state hard disk controller is applied to a storage and calculation integrated solid state hard disk, and comprises a storage system for storing data; a computing system for performing data calculations; and the storage and calculation path module is connected with the storage system and the calculation system and is used for realizing data transmission between the storage system and the calculation system. Through designing the storage and calculation path module, data transmission between the storage system and the computing system is realized, the internal hardware path design of the solid state disk controller can be optimized, so that internal IO transmission standardization of the storage system and the computing system can be realized, the software development amount is reduced, and the software development efficiency of the solid state disk is improved.

Referring to fig. 19, fig. 19 is a schematic flowchart illustrating a data processing method between computing systems according to an embodiment of the present disclosure;

the data processing method between the storage systems is applied to the data storage system mentioned in the above embodiment, the data storage system includes a solid state disk and a host system, and specifically, the data processing method between the storage systems is applied to the storage-integrated solid state disk controller in the solid state disk of the above embodiment, where the storage-integrated solid state disk controller includes a storage system, a computing system, and a storage access module.

As shown in fig. 19, the data processing method includes:

step S10: receiving an operation task command sent by a first host or a second host, wherein the operation task command comprises information of at least one operation task;

specifically, the storage and computation integrated solid state hard disk controller receives an operation task command sent by a host system, specifically, receives the operation task command sent by a first host through a first host interface, or receives the operation task command sent by a second host through a second host interface, where the operation task command includes information of at least one operation task.

Step S20: and converting at least one operation task into IO operation, and forwarding the IO operation to the storage system or directly receiving the IO operation by the storage system through the storage access module so that the storage system processes the IO operation.

If the first host interface receives an operation task command sent by the first host, the storage system converts at least one operation task corresponding to the operation task command into an IO operation, the IO operation is sent to the storage and computation path module, and the storage and computation path module forwards the IO operation to the computing system, so that the computing system processes the IO operation.

Specifically, when receiving an operation task command sent by a first host, a first processor cluster of the storage system converts an operation task corresponding to the operation task command into an IO operation, and the IO operation is sent to a second processor cluster of the computing system through a first system interface of the storage path module, the device module processing unit, the path processing unit, the host module processing unit, and then to a second system interface by the second system interface, so that the second processor cluster of the computing system processes the IO operation to obtain an IO operation result, for example: and calculating the data.

If the second host interface receives an operation task command sent by the second host, the computing system converts at least one operation task corresponding to the operation task command into an IO operation, sends the IO operation to the storage and computation path module, and forwards the IO operation to the storage system by the storage and computation path module so that the storage system processes the IO operation.

Specifically, when receiving an operation task command sent by a second host, a second processor cluster of the computing system converts an operation task corresponding to the operation task command into an IO operation, and sends the IO operation to an NVMe path of the storage system through a second system interface of the storage path module, a host module processing unit, a path processing unit, and a data path interface, and sends the IO operation to a first processor cluster of the storage system through the NVMe path, so that the first processor cluster of the storage system processes the IO operation to obtain an IO operation result, for example: and acquiring the queried data.

In an embodiment of the present application, a data processing method between computing systems is provided, which is applied to a data storage system, and includes: receiving an operation task command sent by a first host or a second host, wherein the operation task command comprises information of at least one operation task; and converting at least one operation task into IO operation, and forwarding the IO operation to the storage system or directly receiving the IO operation by the storage system through the storage access module so that the storage system processes the IO operation. By receiving the operation task command sent by the first host or the second host, the storage access module forwards the IO operation corresponding to the operation task command to the storage system or directly receives the IO operation by the storage system, so that the storage system processes the IO operation.

Embodiments of the present application also provide a non-volatile computer storage medium, which stores computer-executable instructions, which are executed by one or more processors, and can enable the one or more processors to execute the data processing method between the computing systems in any of the above method embodiments, for example, execute the above-described steps shown in fig. 19.

The above-described embodiments of the apparatus or device are merely illustrative, wherein the unit modules described as separate parts may or may not be physically separate, and the parts displayed as module units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or partially contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method of the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; within the context of the present application, features from the above embodiments or from different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the present application as described above, which are not provided in detail for the sake of brevity; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A storage-integrated solid state hard disk controller for storage-integrated solid state storage, comprising:

the storage system is used for storing data;

a computing system for performing data calculations;

and the storage and calculation path module is connected with the storage system and the calculation system and is used for realizing data transmission between the storage system and the calculation system.

2. The storage-computing integrated solid state hard disk controller of claim 1, wherein the storage path module comprises:

a path processing unit, connected to the host module processing unit and the device module processing unit, for processing data requirements of the host module processing unit and the device module processing unit;

3. The computing integrated solid state disk controller of claim 2, wherein the computing path module further comprises:

4. The computing-integrated solid state hard disk controller of claim 1, further comprising:

a first host interface for interfacing with a first host;

and the second host interface is used for butting the second host.

5. The storage-computing integrated solid state hard disk controller of claim 1, wherein the storage system comprises a first cluster of processors configured to process data storage operations;

the computing system includes a second cluster of processors to process data computation operations.

6. The memory-computing integrated solid state hard disk controller of claim 5, wherein the storage system further comprises:

and the rear-end module is connected with the mapping table management module and the flash memory medium and is used for reading and writing flash memory data and managing flash memory commands.

7. The computing-integrated solid state disk controller of claim 6,

the first processor cluster is connected with the storage and computation access module, the second processor cluster is connected with the storage and computation access module, and the front-end module is connected with the first processor cluster and the storage and computation access module.

8. A solid state disk, comprising:

the computing-integrated solid state disk controller of any of claims 1-7;

and the at least one flash memory medium is connected with the storage and calculation integrated solid state hard disk controller.

9. A data storage system, comprising:

the solid state disk of claim 8;

the host system is in communication connection with the solid state disk, and comprises:

a first host for interfacing with the storage system;

a second host for interfacing the computing system.

10. A data processing method between storage systems, applied to the data storage system according to claim 9, the method comprising:

and converting the at least one operation task into IO operation, and forwarding the IO operation to the storage system or directly receiving the IO operation by the storage system through the storage access module so that the storage system processes the IO operation.