CN113448517B - Solid state disk big data writing processing method, device, equipment and medium - Google Patents
Solid state disk big data writing processing method, device, equipment and medium Download PDFInfo
- Publication number
- CN113448517B CN113448517B CN202110627597.2A CN202110627597A CN113448517B CN 113448517 B CN113448517 B CN 113448517B CN 202110627597 A CN202110627597 A CN 202110627597A CN 113448517 B CN113448517 B CN 113448517B
- Authority
- CN
- China
- Prior art keywords
- data
- module
- writing
- request
- block data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000007787 solid Substances 0.000 title claims abstract description 37
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000004044 response Effects 0.000 claims abstract description 20
- 238000012216 screening Methods 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 11
- 230000000977 initiatory effect Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 abstract description 10
- 230000008569 process Effects 0.000 description 10
- 239000000872 buffer Substances 0.000 description 7
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Abstract
The invention discloses a solid state disk big block writing data processing method, device, equipment and medium. The method comprises the following steps: screening a large-block data writing request from writing requests sent by a host; splitting large block data to be written corresponding to the large block data writing request into a plurality of small block data by using a DM module and mounting the small block data to a linked list; the DM module sends a write request to the LKM module, wherein the content of the write request comprises an LBA (logical block addressing) first address of large block data to be written and the total number of small block data; the LKM module receives the write request and performs cache allocation to generate a plurality of write request results, and then the write request results are returned to the DM module; the DM module searches the context content of each small block of data from the mounted linked list by taking the small block of data as a unit according to the received request results and initiates a data moving request to the DMA module; and releasing the DM module resource and the LKM module resource in response to the DMA module completing the data movement. The scheme of the invention reduces the communication times among the modules, optimizes the efficiency of the large block writing, reduces the writing delay and improves the writing bandwidth.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a solid state disk big block writing data processing method, device, equipment and medium.
Background
Compared with a traditional SSD (Solid State Drive), the NVME protocol-based solution of ZNS (short for connected Namespace) has the characteristics of write-play size and low cost. Generally, the ZNS SSD only supports sequential writing, so before the ZNS SSD is configured and used, it needs to obtain configuration information of all the zones inside the SSD, and then can accurately determine the writing pointers and states of the zones. For example, ZNS 128KB sequential writing generally needs to go through the processes of unpacking, applying for write buffer (buff), DMA (direct memory access unit) data transfer, write completion, and the like, and each Process needs IPC (Inter Process Communication, which refers to a message body for Communication between internal modules of the SSD) message Communication.
To facilitate understanding of some data washing processes, a brief description of several modules inside the SSD is provided here: the DM module is a short name of dispatch Manager and mainly processes a request issued by a host; the LKM module is a loadable kernel module used by the Linux kernel for expanding the functions of the Linux kernel, and is mainly responsible for writing host data into a Flash memory (NAND Flash).
Currently, the process for ZNS 128KB sequential writes is as follows: when the ZNS SSD firmware receives a 128KM write request, firstly, a DM module is required to be unpacked into 32 4KB, each 4KB sends a write request to an LKM module, the LKM module returns a write-back cache (buff) address, the DM module sends a data moving request to the DMA module, the DMA module completes moving, then the DM module sends the write completion of the 4KB to the LKM module, and finally the LKM module releases the buff, and the operation is repeatedly executed for 32 times; therefore, it can be seen that in the conventional 128KB sequential write process, multiple write applications need to be performed, and repeated communication is needed among modules, so that the write efficiency is low, and a large bandwidth is occupied, and therefore, improvement is urgently needed in the conventional large-block data write processing.
Disclosure of Invention
In view of the foregoing, there is a need to provide a method, an apparatus, a device and a medium for processing write chunk data of a solid state disk.
According to a first aspect of the present invention, a solid state disk write chunk data processing method is provided, where the method includes:
screening a large-block data writing request from writing requests sent by a host;
splitting large block data to be written corresponding to the large block data writing request into a plurality of small block data by using a DM module and mounting the small block data to a linked list;
sending a write request to an LKM module by a DM module, wherein the content of the write request comprises an LBA (logical block addressing) first address of large block data to be written and the total number of small block data;
receiving the write request by the LKM module, performing cache allocation to generate a plurality of write request results, and returning the write request results to the DM module, wherein each request result comprises a write cache address and the number of continuous small block data from the write cache address, and the sum of the number of continuous small block data in all the request results is equal to the total number of small block data;
the DM module searches the context content of each small block of data from the mounted linked list by taking the small block of data as a unit according to the received request results and initiates a data moving request to the DMA module;
and releasing the DM module resource and the LKM module resource in response to the DMA module completing the data movement.
In some embodiments, the step of screening large block data write requests from write requests sent by the host comprises:
the method comprises the steps that a writing request issued by a host is received by ZNS solid state disk firmware, and the data size corresponding to the writing request is obtained;
and responding to the data size corresponding to the write request which is more than or equal to the preset value, and taking the write request issued by the host as a large-block data write request.
In some embodiments, the step of splitting, by the DM module, the to-be-written large block data corresponding to the large block data write request into a plurality of small block data and mounting the small block data to the linked list includes:
splitting the large block data into a plurality of small blocks of data by taking 4KB as a unit by utilizing a DM module;
apply for the context of each 4KB small block of data and mount to the linked list.
In some embodiments, in response to the DMA module completing the data movement, the step of releasing the DM module resource and the LKM module resource includes:
responding to DMA to move a small block of data from the host to the write cache, and sending a move completion message back to the DM module;
the DM module sends a write completion message to the LKM module;
responding to the LKM receiving the writing completion message, releasing resources of the LKM corresponding to a certain small data block which completes data movement;
and the DM module releases the resources corresponding to the small data blocks which have completed the data movement.
In some embodiments, the step of the DM module retrieving context content of each small block of data from the mounted linked list in units of small blocks of data according to the result of receiving several requests and initiating a data moving request to the DMA module includes:
in response to that the DM receives that the LKM returns a certain request result, retrieving the context of the LBA first address from the mounted linked list according to the LBA first address and the continuous 4KB stroke number starting from the LBA first address, and deleting the corresponding element from the linked list;
a move request is sent by the DM to the DMA module.
In some embodiments, the method further comprises:
and in response to the fact that the context of all the small block data of the DM module is completely released, returning the completion of writing the large block data to be written to the host.
In some embodiments, the size of the large block of data to be written is 128 KB.
According to a second aspect of the present invention, there is provided a solid state disk write chunk data processing apparatus, including:
the request screening unit is used for screening the large block data writing requests from the writing requests sent by the host;
the splitting unit is used for splitting the large block data to be written corresponding to the large block data writing request into a plurality of small block data by using the DM module and mounting the small block data to the linked list;
a write request sending unit, configured to send a write request to an LKM module by a DM module, where the content of the write request includes an LBA header address of a large block of data to be written and a total number of small block of data;
the writing cache allocation unit is used for receiving the writing request by the LKM module, performing cache allocation to generate a plurality of writing request results and returning the writing request results to the DM module, wherein each request result comprises a writing cache address and the number of continuous small block data from the writing cache address, and the sum of the number of continuous small block data of all the request results is equal to the total number of small block data;
a moving request unit, which is used for the DM module to retrieve the context content of each small block data from the mounted linked list by taking the small block data as a unit according to the received request results and to initiate a data moving request to the DMA module;
and the resource release unit is used for releasing the DM module resource and the LKM module resource after the DMA module finishes data transfer.
According to a third aspect of the present invention, there is also provided a computer apparatus comprising:
at least one processor; and
the storage stores a computer program which can run on the processor, and the processor executes the solid state disk block writing data processing method when executing the program.
According to the fourth aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed by a processor, executes the foregoing solid state disk write chunk data processing method.
The processing method for writing big data on the solid state disk comprises the steps of splitting a big data block to be written corresponding to a big data writing request into a plurality of small data blocks through a DM module, mounting the small data blocks to a linked list, sending a writing request to an LKM module by the DM module, performing cache allocation by the LKM module to generate a plurality of writing request results, returning the writing request results to the DM module, retrieving the context content of each small data block from the mounted linked list by taking the small data blocks as a unit according to the received request results by the DM module, initiating a data moving request to a DMA module, releasing the DM module resources and the LKM module resources when the DMA module completes the data moving, reducing the writing requests into one writing request compared with the traditional writing scheme, reducing the quantity of IPC messages of results returned by the LKM module, obviously reducing the communication times among the modules, and optimizing the efficiency of writing the big data block, effectively reduces the writing delay and obviously improves the writing bandwidth.
In addition, the invention also provides a solid state disk big block data writing processing device, a computer device and a computer readable storage medium, which can also achieve the technical effects and are not repeated herein.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a processing method for writing large block data in a solid state disk according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a solid state disk write chunk data processing apparatus according to an embodiment of the present invention;
fig. 3 is an internal structural view of a computer device according to another embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In one embodiment, referring to FIG. 1, the present invention provides a method for manufacturing a semiconductor device
S100, screening a big data writing request from the writing requests sent by the host;
s200, splitting the large block data to be written corresponding to the large block data writing request into a plurality of small block data by using a DM module and mounting the small block data to a linked list;
s300, sending a write request to an LKM module by a DM module, wherein the content of the write request comprises an LBA (logical block addressing) first address of large block data to be written and the total number of small block data;
s400, an LKM module receives the writing request, performs cache allocation to generate a plurality of writing request results, and returns the writing request results to a DM module, wherein each request result comprises a writing cache address and the number of continuous small block data from the writing cache address, and the sum of the number of continuous small block data in all the request results is equal to the total number of small block data;
it should be noted that, because the LKM module allocates continuous buffers to the large block of write data as much as possible when allocating the buffers, that is, if there is a space capable of continuously supplying the large block of data in the buffers, the LKM module replies one request result, and if there is not enough buffer space, taking the data obtained by splitting the 128KB large block of data into 32 4KB small blocks as an example, the LKM module may reply two request results according to the size of the actual space, where one request result can provide buffers for 16 small blocks of data, and the other group provides buffers for the remaining 16 smaller blocks of data, the number of times of replying the request result by the LKM module is not fixed, but the number of times of IPC message communication between the LKM module and the DM module is greatly reduced compared with the case of replying one request result for each 4KB small block of data.
S500, the DM module searches the context content of each small block of data from the mounted linked list by taking the small block of data as a unit according to the received plurality of request results and initiates a data moving request to the DMA module;
s600, responding to the DMA module to finish data movement, releasing the DM module resource and the LKM module resource.
The processing method for writing the big block data of the solid state disk comprises the steps of splitting a large block data to be written corresponding to a large block data writing request into a plurality of small block data through a DM module, mounting the small block data to a linked list, sending a writing request to an LKM module by the DM module, performing cache allocation by the LKM module to generate a plurality of writing request results, returning the writing request results to the DM module, retrieving the context content of each small block data from the mounted linked list by the small block data according to the received request results by the DM module, initiating a data moving request to an DMA module, releasing DM module resources and LKM module resources when the DMA module finishes data moving, reducing the writing requests into one writing request compared with the traditional writing scheme, reducing the number of IPC messages of results returned by the LKM module, obviously reducing the number of communication among modules, optimizing the efficiency of writing the large block data, effectively reduces the writing delay and obviously improves the writing bandwidth.
In a further embodiment, the aforementioned step S100 comprises the following sub-steps:
s110, receiving a write request issued by a host by using ZNS solid state disk firmware, and acquiring the data size corresponding to the write request;
and S120, in response to the fact that the data size corresponding to the write request is larger than or equal to a preset value, taking the write request issued by the host as a large-block data write request.
In another embodiment, the step S200 specifically includes the following steps:
s210, splitting the large block data into a plurality of small block data by using a DM module and taking 4KB as a unit;
s220, applying for the context of each 4KB small block of data and mounting the context to a linked list.
In another embodiment, the foregoing step S600 specifically includes the following sub-steps:
s610, responding to the DMA to move a certain small block data from the host to the write cache, and sending a move completion message back to the DM module;
s620, the DM module sends a write completion message to the LKM module;
s630, responding to that the LKM receives the writing completion message, releasing resources of the LKM corresponding to a certain small data block which completes data movement;
and S640, the DM module releases the resources corresponding to the small data blocks after the data movement is finished.
In yet another embodiment, the aforementioned step S500 comprises the following sub-steps:
s510, in response to that the DM receives that the LKM returns a certain request result, retrieving the context of the LBA first address from the mounted linked list according to the LBA first address and the continuous 4KB stroke number starting from the LBA first address, and deleting the corresponding element from the linked list;
s520, the DM sends a moving request to the DMA module.
In a further embodiment, on the basis of the preceding embodiment, the method of the present invention further comprises:
s700, in response to the fact that the context of all the small block data of the DM module is completely released, the writing completion of the large block data to be written is returned to the host.
In yet another embodiment, the size of the large block of data to be written is 128 KB. Specifically, when the host sends a 128KB large block write request, the DM module splits the 128KB data into 32 4KB small blocks of data after receiving the request, which is different from the conventional write method in that the DM module sends the total split number of strokes to the LKM module in some communication messages instead of sending a request for each 4KB small block of data, thereby greatly reducing the number of times of communication between the DM module and the LKM module.
In another embodiment, to facilitate understanding of the technical solution of the present invention, the following takes an example of processing 128KB large block write data in a ZNS SSD firmware, because in the processing process of a large block write such as 128k, in order to improve the write bandwidth, it is necessary to reduce IPC interaction across modules as much as possible, and a specific method for processing large block data written in a solid state disk includes the following steps:
step one, ZNS SSD firmware receives a 128KB write request issued by a HOST (HOST);
step two, a DM module in firmware splits a 128KB request into 32 4KB small block data for processing, applies for the context of each 4KB small block data, and mounts the context to a linked list;
step three, the DM module sends an IPC write request to the LKM module, wherein payload in the request is an LBA initial address (startLBA) and the number of data strokes of the split 4KB small block, namely 32 strokes;
step four, after receiving the IPC write request of the DM module, the LKM module allocates a write cache according to the startLBA and the stroke number of 4KB, and takes the address of the write cache and the stroke number of 4KB continuous from the address as payload returned to the IPC message of the DM module;
step five, after receiving the IPC message returned by the LKM module, the DM module retrieves the context of the LBA from the context linked list according to the startLBA and the continuous 4KB small block data pen number from the address, deletes the element from the linked list, and then initiates a DMA IPC request; the whole process is completed until the context linked list is changed to be empty, which means that 32 pens of 4KB small data blocks are processed;
step six, the DMA module moves data from a HOST (HOST) to a write cache, and then returns IPC information to the DM module to complete;
step seven, the DM module sends the written IPC information to the LKM module, and each 4KB small block of data is sent once;
step eight, the DM module releases the context of the moved 4KB small block data;
step nine, when the DM module 32 completes releasing all 4KB of context, returns the 128KB write completion to the HOST (HOST).
The method of the invention reduces the 32-pen 4KB writing request into 1 pen, also reduces the number of the IPC messages returned by the LKM writing buff, optimizes the writing efficiency of large blocks such as 128KB and the like, reduces the writing delay and obviously improves the writing bandwidth.
In another embodiment, referring to fig. 2, the present invention further provides a solid state disk write block data processing apparatus 80, including:
a request screening unit 81, configured to screen a chunk data write request from write requests sent by a host;
a splitting unit 82, configured to split, by using a DM module, the large block data to be written corresponding to the large block data writing request into multiple small block data, and mount the multiple small block data to a linked list;
a write request sending unit 83, configured to send a write request to the LKM module by the DM module, where the content of the write request includes an LBA header address of the large block data to be written and a total number of small block data;
a write cache allocation unit 84, configured to receive the write request by the LKM module, perform cache allocation to generate a plurality of write request results, and return the write request results to the DM module, where each request result includes a write cache address and a number of consecutive small block data from the write cache address, and a sum of the number of consecutive small block data of all request results is equal to a total number of small block data;
a move request unit 85, configured to retrieve, by the DM module, context content of each small block of data from the mounted linked list in units of small blocks of data according to the received multiple request results, and initiate a data move request to the DMA module;
and a resource releasing unit 86, configured to release the DM module resource and the LKM module resource after the DMA module completes data transfer.
It should be noted that, for specific limitations of the solid state disk large block data writing processing apparatus, reference may be made to the above limitations of the solid state disk large block data writing processing method, and details are not described herein again. All or part of each unit in the solid state disk writing block data processing device can be realized by software, hardware and a combination thereof. The units can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
According to another aspect of the present invention, a computer device is provided, and the computer device may be a server, and its internal structure is shown in fig. 3. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. When executed by a processor, the computer program implements the solid state disk big block data writing processing method described above, and specifically, the method includes the following steps:
screening a large-block data writing request from writing requests sent by a host;
splitting large block data to be written corresponding to the large block data writing request into a plurality of small block data by using a DM module and mounting the small block data to a linked list;
sending a write request to an LKM module by a DM module, wherein the content of the write request comprises an LBA (logical block addressing) first address of large block data to be written and the total number of small block data;
receiving the write request by the LKM module, performing cache allocation to generate a plurality of write request results, and returning the write request results to the DM module, wherein each request result comprises a write cache address and the number of continuous small block data from the write cache address, and the sum of the number of continuous small block data in all the request results is equal to the total number of small block data;
the DM module searches the context content of each small block of data from the mounted linked list by taking the small block of data as a unit according to the received request results and initiates a data moving request to the DMA module;
and releasing the DM module resource and the LKM module resource in response to the DMA module completing the data movement.
In some embodiments, the step of screening large block data write requests from write requests sent by the host comprises:
the method comprises the steps that a writing request issued by a host is received by ZNS solid state disk firmware, and the data size corresponding to the writing request is obtained;
and responding to the data size corresponding to the write request which is more than or equal to the preset value, and taking the write request issued by the host as a large-block data write request.
In some embodiments, the step of splitting, by the DM module, the to-be-written large block data corresponding to the large block data write request into a plurality of small block data and mounting the small block data to the linked list includes:
splitting the large block data into a plurality of small blocks of data by taking 4KB as a unit by utilizing a DM module;
apply for the context of each 4KB small block of data and mount to the linked list.
In some embodiments, in response to the DMA module completing the data movement, the step of releasing the DM module resource and the LKM module resource includes:
responding to DMA to move a small block of data from the host to the write cache, and sending a move completion message back to the DM module;
the DM module sends a write completion message to the LKM module;
responding to the LKM receiving the writing completion message, releasing resources of the LKM corresponding to a certain small data block which completes data movement;
and the DM module releases the resources corresponding to the small data blocks which have completed the data movement.
In some embodiments, the step of the DM module retrieving context content of each small block of data from the mounted linked list in units of small blocks of data according to the result of receiving several requests and initiating a data moving request to the DMA module includes:
in response to that the DM receives that the LKM returns a certain request result, retrieving the context of the LBA first address from the mounted linked list according to the LBA first address and the continuous 4KB stroke number starting from the LBA first address, and deleting the corresponding element from the linked list;
a move request is sent by the DM to the DMA module.
In some embodiments, the method further comprises:
and in response to the fact that the context of all the small block data of the DM module is completely released, returning the completion of writing the large block data to be written to the host.
In some embodiments, the size of the large block of data to be written is 128 KB. According to still another aspect of the present invention, there is provided a computer-readable storage medium, on which a computer program is stored, the computer program, when executed by a processor, implementing the above-mentioned solid state disk write chunk data processing method, specifically, comprising performing the following steps:
screening a large-block data writing request from writing requests sent by a host;
splitting large block data to be written corresponding to the large block data writing request into a plurality of small block data by using a DM module and mounting the small block data to a linked list;
sending a write request to an LKM module by a DM module, wherein the content of the write request comprises an LBA (logical block addressing) first address of large block data to be written and the total number of small block data;
receiving the write request by the LKM module, performing cache allocation to generate a plurality of write request results, and returning the write request results to the DM module, wherein each request result comprises a write cache address and the number of continuous small block data from the write cache address, and the sum of the number of continuous small block data in all the request results is equal to the total number of small block data;
the DM module searches the context content of each small block of data from the mounted linked list by taking the small block of data as a unit according to the received request results and initiates a data moving request to the DMA module;
and releasing the DM module resource and the LKM module resource in response to the DMA module completing the data movement.
In some embodiments, the step of screening large block data write requests from write requests sent by the host comprises:
the method comprises the steps that a writing request issued by a host is received by ZNS solid state disk firmware, and the data size corresponding to the writing request is obtained;
and responding to the data size corresponding to the write request which is more than or equal to the preset value, and taking the write request issued by the host as a large-block data write request.
In some embodiments, the step of splitting, by the DM module, the to-be-written large block data corresponding to the large block data write request into a plurality of small block data and mounting the small block data to the linked list includes:
splitting the large block data into a plurality of small blocks of data by taking 4KB as a unit by utilizing a DM module;
apply for the context of each 4KB small block of data and mount to the linked list.
In some embodiments, in response to the DMA module completing the data movement, the step of releasing the DM module resource and the LKM module resource includes:
responding to DMA to move a small block of data from the host to the write cache, and sending a move completion message back to the DM module;
the DM module sends a write completion message to the LKM module;
responding to the LKM receiving the writing completion message, releasing resources of the LKM corresponding to a certain small data block which completes data movement;
and the DM module releases the resources corresponding to the small data blocks which have completed the data movement.
In some embodiments, the step of the DM module retrieving context content of each small block of data from the mounted linked list in units of small blocks of data according to the result of receiving several requests and initiating a data moving request to the DMA module includes:
in response to that the DM receives that the LKM returns a certain request result, retrieving the context of the LBA first address from the mounted linked list according to the LBA first address and the continuous 4KB stroke number starting from the LBA first address, and deleting the corresponding element from the linked list;
a move request is sent by the DM to the DMA module.
In some embodiments, the method further comprises:
and in response to the fact that the context of all the small block data of the DM module is completely released, returning the completion of writing the large block data to be written to the host.
In some embodiments, the size of the large block of data to be written is 128 KB. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A solid state disk big data writing processing method is characterized by comprising the following steps:
screening a large-block data writing request from writing requests sent by a host;
splitting large block data to be written corresponding to the large block data writing request into a plurality of small block data by using a DM module and mounting the small block data to a linked list;
sending a write request to an LKM module by a DM module, wherein the content of the write request comprises the LBA first address of the large block data to be written and the total number of small block data;
receiving the write request by the LKM module, performing cache allocation to generate a plurality of write request results, and returning the write request results to the DM module, wherein each request result comprises a write cache address and the number of continuous small block data from the write cache address, and the sum of the number of continuous small block data in all the request results is equal to the total number of small block data;
the DM module searches the context content of each small block of data from the mounted linked list by taking the small block of data as a unit according to the received request results and initiates a data moving request to the DMA module;
and releasing the DM module resource and the LKM module resource in response to the DMA module completing the data movement.
2. The method for processing big block data writing of the solid state disk of claim 1, wherein the step of screening the big block data writing requests from the writing requests sent by the host comprises:
the method comprises the steps that a writing request issued by a host is received by ZNS solid state disk firmware, and the data size corresponding to the writing request is obtained;
and responding to the data size corresponding to the write request which is more than or equal to the preset value, and taking the write request issued by the host as a large-block data write request.
3. The method for processing big block data written on a solid state disk according to claim 1, wherein the step of splitting the big block data to be written corresponding to the big block data writing request into a plurality of small block data by using the DM module and mounting the small block data to the linked list comprises:
splitting the large block data into a plurality of small blocks of data by taking 4KB as a unit by utilizing a DM module;
apply for the context of each 4KB small block of data and mount to the linked list.
4. The method for processing big data writing of the solid state disk according to claim 3, wherein the step of releasing the DM module resource and the LKM module resource in response to the DMA module completing the data movement comprises:
responding to DMA to move a small block of data from the host to the write cache, and sending a move completion message back to the DM module;
the DM module sends a write completion message to the LKM module;
responding to the LKM receiving the writing completion message, releasing resources of the LKM corresponding to a certain small data block which completes data movement;
and the DM module releases the resources corresponding to the small data blocks which have completed the data movement.
5. The method for processing big block data writing of solid state disk according to claim 4, wherein the step of the DM module retrieving context content of each small block data from the mounted linked list in units of small block data according to the result of receiving a plurality of requests and initiating a data move request to the DMA module comprises:
in response to that the DM receives that the LKM returns a certain request result, retrieving the context of the LBA first address from the mounted linked list according to the LBA first address and the continuous 4KB stroke number starting from the LBA first address, and deleting the corresponding element from the linked list;
a move request is sent by the DM to the DMA module.
6. The solid state disk big block writing data processing method according to claim 4, further comprising:
and in response to the fact that the context of all the small block data of the DM module is completely released, returning the completion of writing the large block data to be written to the host.
7. The solid state disk write chunk data processing method of any one of claims 1 to 6, wherein the size of the to-be-written chunk data is 128 KB.
8. A solid state disk big data writing processing device is characterized by comprising:
the request screening unit is used for screening the large block data writing requests from the writing requests sent by the host;
the splitting unit is used for splitting the large block data to be written corresponding to the large block data writing request into a plurality of small block data by using the DM module and mounting the small block data to the linked list;
a write request sending unit, configured to send a write request to an LKM module by a DM module, where the content of the write request includes an LBA header address of a large block of data to be written and a total number of small block of data;
the writing cache allocation unit is used for receiving the writing request by the LKM module, performing cache allocation to generate a plurality of writing request results and returning the writing request results to the DM module, wherein each request result comprises a writing cache address and the number of continuous small block data from the writing cache address, and the sum of the number of continuous small block data of all the request results is equal to the total number of small block data;
a moving request unit, which is used for the DM module to retrieve the context content of each small block data from the mounted linked list by taking the small block data as a unit according to the received request results and to initiate a data moving request to the DMA module;
and the resource release unit is used for releasing the DM module resource and the LKM module resource after the DMA module finishes data transfer.
9. A computer device, comprising:
at least one processor; and
a memory storing a computer program operable in the processor, the processor executing the program to perform the solid state disk block writing data processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to execute the solid state disk large block data writing processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110627597.2A CN113448517B (en) | 2021-06-04 | 2021-06-04 | Solid state disk big data writing processing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110627597.2A CN113448517B (en) | 2021-06-04 | 2021-06-04 | Solid state disk big data writing processing method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113448517A CN113448517A (en) | 2021-09-28 |
| CN113448517B true CN113448517B (en) | 2022-08-09 |
Family
ID=77810883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110627597.2A Active CN113448517B (en) | 2021-06-04 | 2021-06-04 | Solid state disk big data writing processing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113448517B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107850983B (en) * | 2015-09-11 | 2020-11-03 | 株式会社日立制作所 | Computer system, storage device and data management method |
| CN107908573B (en) * | 2017-11-09 | 2020-05-19 | 苏州浪潮智能科技有限公司 | Data caching method and device |
| CN110647476B (en) * | 2019-09-29 | 2021-10-15 | 苏州浪潮智能科技有限公司 | Method, device and equipment for writing data in solid state disk and storage medium |
| CN110764711B (en) * | 2019-10-29 | 2022-03-22 | 北京浪潮数据技术有限公司 | IO data classification deleting method and device and computer readable storage medium |
| CN111552540B (en) * | 2020-04-23 | 2023-07-04 | 华云数据控股集团有限公司 | Resource synchronization method based on VMware cloud platform and super-fusion cloud platform |
| CN111694515B (en) * | 2020-05-23 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Zone writing distribution method and system based on ZNS solid state disk |
-
2021
- 2021-06-04 CN CN202110627597.2A patent/CN113448517B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113448517A (en) | 2021-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11243701B2 (en) | Data write method and solid-state drive array | |
| CN110096221B (en) | Memory system and control method thereof | |
| CN113918101B (en) | Method, system, equipment and storage medium for writing data cache | |
| CN112000287B (en) | IO request processing device, method, equipment and readable storage medium | |
| US10789170B2 (en) | Storage management method, electronic device and computer readable medium | |
| CN113760560A (en) | Inter-process communication method and inter-process communication device | |
| CN114327137A (en) | Touch method and device based on multiple vehicle-mounted operating systems and computer equipment | |
| CN115964319A (en) | Data processing method for remote direct memory access and related product | |
| US20220253252A1 (en) | Data processing method and apparatus | |
| CN112506823A (en) | FPGA data reading and writing method, device, equipment and readable storage medium | |
| CN115470156A (en) | RDMA-based memory use method, system, electronic device and storage medium | |
| CN113986773A (en) | Write amplification optimization method and device based on solid state disk and computer equipment | |
| CN116886719B (en) | Data processing method and device of storage system, equipment and medium | |
| CN113448517B (en) | Solid state disk big data writing processing method, device, equipment and medium | |
| US11972110B2 (en) | Storage device and storage system | |
| CN116340198B (en) | Data writing method and device of solid state disk and solid state disk | |
| CN113704027B (en) | File aggregation compatible method and device, computer equipment and storage medium | |
| CN116010093A (en) | Data processing method, apparatus, computer device and readable storage medium | |
| CN115686782A (en) | Resource scheduling method and device based on solid state disk, electronic equipment and storage medium | |
| CN112000289B (en) | Data management method for full flash storage server system and related components | |
| CN112835520A (en) | Mapping table dynamic loading-based read request processing method and device | |
| CN115712581A (en) | Data access method, storage system and storage node | |
| EP3819771B1 (en) | Data processing method and device, apparatus, and system | |
| CN114296640B (en) | Data driving method, apparatus, device and storage medium for accelerating computation | |
| WO2022262623A1 (en) | Data exchange method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |