CN111782136A - Multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device - Google Patents
Multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device Download PDFInfo
- Publication number
- CN111782136A CN111782136A CN202010433140.3A CN202010433140A CN111782136A CN 111782136 A CN111782136 A CN 111782136A CN 202010433140 A CN202010433140 A CN 202010433140A CN 111782136 A CN111782136 A CN 111782136A
- Authority
- CN
- China
- Prior art keywords
- stored
- application program
- access
- read
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0625—Power saving in storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Abstract
The invention discloses a multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and a device, wherein the method comprises the steps of obtaining an access request received within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored; extracting an access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored; obtaining access characteristics corresponding to each application program to be stored according to the number of access requests corresponding to each application program to be stored, the number of all the access requests and the read-write characteristic information in the access requests; inputting the access characteristics of each application program to be stored into a channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the method and the device can accurately predict the number of the channels corresponding to the application program and improve the utilization rate of the flash memory solid-state disk storage resources.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device.
Background
Flash memory solid state disks have the advantages of low latency, high bandwidth, shock resistance, and low power consumption compared to conventional magnetic disks, and are therefore widely used in high performance computing environments. The flash memory solid-state disk is composed of a plurality of channels, a plurality of flash memory chips are hung on each channel, each flash memory chip is used for storing data, and if the flash memory channels can be reasonably used, the parallelism of flash memory storage can be favorably exerted, and the storage performance of the flash memory solid-state disk is improved.
With the development of 3D stacking and multi-level cell technology, the capacity of flash solid-state disks has increased year by year, enabling multiple applications to share the same flash solid-state disk. However, the conventional channel allocation method for the flash solid-state disk is to store data of all the applications uniformly in each channel, which will cause access conflict among the applications, thereby sacrificing the read/write performance of the flash solid-state disk.
In the prior art, the open flash solid-state disk can directly expose the bottom layer characteristics of the solid-state disk to an application program, and the access conflict between the application programs is avoided to a certain extent by allocating a specific number of channels to the application program to realize hardware isolation. However, the existing flash memory solid-state disk channel allocation method allocates the corresponding channel number to each application program by means of human experience, and there is a case that the channel number allocated to the application program is not matched with the channel number actually required by the application program, if the channel number actually required by the application program is greater than the allocated channel number, the storage space corresponding to the application program is too small, so that more data cannot be stored, and if the channel number actually required by the application program is less than the allocated channel number, the storage space corresponding to the application program is too large, so that the storage resources are wasted, and further the waste of the open flash memory solid-state disk storage resources is caused.
Disclosure of Invention
The technical problem to be solved by the embodiments of the present invention is to provide a method and an apparatus for allocating channels of a multi-application-oriented adaptive flash memory solid-state disk, which can accurately predict the number of channels required by each application, and improve the utilization rate of flash memory solid-state disk storage resources.
In order to solve the above technical problem, an embodiment of the present invention provides a method for allocating channels of a multi-application-oriented adaptive flash memory solid-state disk, including:
acquiring N access requests received by a flash memory solid-state disk within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored, and N > 0;
extracting an access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored;
obtaining access characteristics corresponding to each application program to be stored according to the number of access requests corresponding to each application program to be stored, the number of all the access requests received by the flash memory solid-state disk in the preset time and the read-write characteristic information in the access requests;
inputting the access characteristics of each application program to be stored into a preset channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the channel quantity prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises an access feature and a label for identifying the channel quantity corresponding to the access feature.
Further, the access characteristics comprise access density and read-write characteristics; then, the obtaining, according to the number of access requests corresponding to each application program to be stored, the number of all the access requests received by the flash solid-state disk within the preset time, and the read-write characteristic information in the access requests, an access characteristic corresponding to each application program to be stored specifically includes:
respectively calculating the ratio of the number of the access requests corresponding to each application program to be stored to the number of all the access requests received by the flash memory solid-state disk in the preset time, and obtaining the access density corresponding to each application program to be stored;
respectively calculating the ratio of the number of the access requests corresponding to the read characteristics in each application program to be stored to the number of the access requests corresponding to the application program to be stored according to the read-write characteristic information in the access requests, and obtaining the ratio of one-to-one correspondence of each application program to be stored;
and respectively comparing each ratio with a preset ratio threshold, acquiring the read-write characteristics of the application program to be stored corresponding to the ratio according to the comparison result, and acquiring the read-write characteristics corresponding to each application program to be stored.
Further, the comparing each ratio with a preset ratio threshold, and obtaining the read-write characteristics of the to-be-stored application program corresponding to the ratio according to the comparison result, to obtain the read-write characteristics corresponding to each to-be-stored application program specifically includes:
for any one of the ratios, comparing the ratio with the preset ratio threshold;
if the ratio is greater than or equal to the preset ratio threshold, the read-write characteristic of the application program to be stored corresponding to the ratio is a read characteristic;
and if the ratio is smaller than the preset ratio threshold, the read-write characteristic of the application program to be stored corresponding to the ratio is a write characteristic.
Further, after obtaining the number of channels corresponding to each of the applications to be stored, the method further includes:
and according to the read-write characteristics in the access characteristics of each application program to be stored, using a corresponding page distribution mode in a channel corresponding to each application program to be stored.
Further, the using a corresponding page allocation mode in a channel corresponding to each application program to be stored according to the read-write characteristics in the access characteristics of each application program to be stored specifically includes:
for any one application program to be stored, if the read-write characteristic of the application program to be stored is a read characteristic, a static page allocation mode is used in a channel corresponding to the application data to be stored;
and if the read-write characteristic of the application program to be stored is a write characteristic, using a dynamic page allocation mode in a channel corresponding to the application data to be stored.
Further, the method for constructing the channel number prediction model specifically includes:
acquiring and obtaining M channel allocation strategies according to the total channel number of the flash memory solid-state disk and the number of the application programs to be tested; the total channel number of the flash memory solid-state disk is greater than or equal to the number of the application programs to be tested;
acquiring K access requests to be tested; each access request to be tested comprises identification information and read-write characteristic information of an application program to be tested, and K is larger than 1000000;
extracting a to-be-tested access request corresponding to each to-be-tested application program from all to-be-tested access requests according to the identification information of all to-be-stored application programs;
obtaining the access characteristics to be trained corresponding to each application program to be tested according to the number of the access requests to be tested corresponding to each application program to be tested, the number of all the access requests to be tested and the read-write characteristic information in the access requests to be tested;
obtaining all the access characteristics to be trained and the delay time of each channel distribution strategy through an SSD-Sim simulator, and obtaining M delay times;
extracting a channel distribution strategy corresponding to the shortest delay time from all the delay times, and obtaining the number of channels corresponding to each access feature to be trained according to the channel distribution strategy corresponding to the shortest delay time;
and inputting the access features to be trained and the number of channels corresponding to the access features to be trained into a machine learning model for training to obtain the channel number prediction model.
Further, the identification information of the application program to be stored is a name of the application program to be stored or a number of the application program to be stored.
Correspondingly, the invention also provides a multi-application-oriented adaptive flash memory solid-state disk channel distribution device, which comprises:
the acquisition module is used for acquiring N access requests received by the flash memory solid-state disk within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored, and N > 0;
the extraction module is used for extracting the access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored;
the computing module is used for obtaining the access characteristics corresponding to each application program to be stored according to the number of the access requests corresponding to each application program to be stored, the number of all the access requests received by the flash memory solid-state disk within the preset time and the read-write characteristic information in the access requests;
the prediction module is used for inputting the access characteristics of each application program to be stored into a preset channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the channel quantity prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises an access feature and a label for identifying the channel quantity corresponding to the access feature.
Further, the multi-application-program-oriented adaptive flash memory solid-state disk channel allocation device further comprises a page allocation module, which is used for using a corresponding page allocation mode for the channel corresponding to each application program to be stored according to the read-write characteristics in the access characteristics of each application program to be stored.
Further, the page allocation module further includes:
the first page allocation unit is used for using a static page allocation mode in a channel corresponding to the application data to be stored if the read-write characteristic of the application program to be stored is a read characteristic;
and the second page allocation unit is used for using a dynamic page allocation mode in a channel corresponding to the application data to be stored if the read-write characteristic of the application program to be stored is a write characteristic.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method, which comprises the steps of obtaining an access request received within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored; extracting an access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored; obtaining access characteristics corresponding to each application program to be stored according to the number of access requests corresponding to each application program to be stored, the number of all the access requests and the read-write characteristic information in the access requests; inputting the access characteristics of each application program to be stored into a channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; compared with the existing channel allocation method of the flash memory solid-state disk, the channel allocation method of the flash memory solid-state disk is judged by a tester according to experience, the channel number corresponding to an application program is accurately predicted through the channel number prediction model, waste of flash memory solid-state disk storage resources is reduced, and the utilization rate of the flash memory solid-state disk storage resources is improved.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of a multi-application-oriented adaptive flash solid state disk channel allocation method according to the present invention;
FIG. 2 is a flow chart of another preferred embodiment of a multi-application-oriented adaptive flash solid state disk channel allocation method according to the present invention;
FIG. 3 is a flow chart of a method for constructing a channel number prediction model according to the present invention;
fig. 4 is a block diagram of a preferred embodiment of an adaptive flash memory solid-state disk channel allocation apparatus for multiple applications according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without any inventive step, are within the scope of the present invention.
The Flash memory solid-state disk provided by the embodiment of the present invention is similar to a conventional Flash memory solid-state disk, and besides basic functions such as garbage collection, wear leveling, address mapping, etc., the embodiment of the present invention also provides a multi-application-oriented adaptive Flash memory solid-state disk channel allocation method, which is located in a Flash Translation Layer (FTL) of the Flash memory solid-state disk, and a specific method flow can be shown in fig. 1, where the method includes steps S11 to S14:
step S11, acquiring N access requests received by the flash memory solid-state disk within a preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored, and N > 0;
step S12, extracting an access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored;
step S13, obtaining access characteristics corresponding to each application program to be stored according to the number of access requests corresponding to each application program to be stored, the number of all the access requests received by the flash memory solid-state disk within the preset time, and the read-write characteristic information in the access requests;
step S14, inputting the access characteristics of each application program to be stored into a preset channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the channel quantity prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises an access feature and a label for identifying the channel quantity corresponding to the access feature.
Specifically, a piece of computer equipment is acquired in advance; the computer equipment comprises necessary hardware equipment such as a CPU (central processing unit) and the like, and can run an operating system based on Linux kernel, an SSD-Sim simulator and an access request corresponding to a generated application program; recording an access request received by a flash memory solid-state disk within preset time; acquiring an access request corresponding to each application program to be stored according to the identification information of the application program to be stored in the access request; generating access characteristics corresponding to each application program to be stored according to an access request corresponding to each application program to be stored, all the access requests received within preset time and the read-write characteristic information in the access requests; finally, inputting the access characteristics corresponding to each application program to be stored into a channel quantity prediction model, and outputting the channel quantity corresponding to each application program to be stored; compared with the existing channel allocation method of the flash memory solid-state disk, the channel allocation method of the flash memory solid-state disk is judged by a tester according to experience, and the channel quantity prediction model is obtained through training of multiple groups of training data, so that the accuracy of the channel quantity corresponding to each application program to be stored is higher.
In another preferred embodiment, the access characteristics include access density and read-write characteristics; then, the obtaining, according to the number of access requests corresponding to each application program to be stored, the number of all the access requests received by the flash solid-state disk within the preset time, and the read-write characteristic information in the access requests, an access characteristic corresponding to each application program to be stored specifically includes:
respectively calculating the ratio of the number of the access requests corresponding to each application program to be stored to the number of all the access requests received by the flash memory solid-state disk in the preset time, and obtaining the access density corresponding to each application program to be stored;
respectively calculating the ratio of the number of the access requests corresponding to the read characteristics in each application program to be stored to the number of the access requests corresponding to the application program to be stored according to the read-write characteristic information in the access requests, and obtaining the ratio of one-to-one correspondence of each application program to be stored;
comparing each ratio with a preset ratio threshold, and acquiring the read-write characteristics of the application program to be stored corresponding to the ratio according to the comparison result to acquire the read-write characteristics corresponding to each application program to be stored; it should be noted that the read-write characteristic information is read characteristic information, which indicates behavior information of the application program to be stored for extracting data from the flash memory solid-state disk; and if the read-write characteristic information is write characteristic information, the read-write characteristic information represents behavior information of the application program to be stored for storing data in the flash memory solid-state disk.
For example, if the number of access requests of the ith application program to be stored is 8, and the data of all the access requests received by the flash solid-state disk within the preset time is 40, the access density corresponding to the ith application program to be stored is 20%, and if the number of access requests corresponding to the read characteristics in the ith application program to be stored is 2 according to the read-write characteristic information in the access requests, the ratio corresponding to the ith application program to be stored is 25%.
In this embodiment, the format of the access request is as follows: I/O trace format [ arr _ t, APP _ ID, LPN, size, R/W ]; the I/O trace format is an access request, arr _ t represents the arrival time of the access request, APP _ ID represents identification information of an application program to be stored which sends the access request, LPN represents the initial logical address of the access request, size represents the sector number of the access request, and R/W represents read-write characteristic information of the access request.
In this embodiment, the inventor finds that the read-write characteristics and the access density of an application program are main factors that affect the overall read-write delay of a flash memory solid-state disk, so that the read-write characteristics and the access density are used as the input of a model, so that the accuracy of a prediction result of the model is higher, and in order to improve the running speed of a computer, the access characteristics of all application programs to be stored are expressed by using an R-dimensional vector format, for example, four application programs to be stored have the following overall access characteristic format: access pattern: [ total _ I, APP1_ RW, APP2_ RW, APP3_ RW, APP4_ RW, APP1_ I, APP2_ I, APP3_ I, APP4_ I ]; wherein, the Access pattern is an integral Access characteristic; total _ I represents the number of all access requests received in preset time; the APPx _ RW represents read-write characteristic information of an application program to be stored with the number of x; APPx _ I indicates the number of access requests corresponding to the application to be stored, which is numbered x.
In another preferred embodiment, the comparing each ratio with a preset ratio threshold, and obtaining the read-write characteristics of the to-be-stored application program corresponding to the ratio according to the comparison result, to obtain the read-write characteristics corresponding to each to-be-stored application program specifically includes:
for any one of the ratios, comparing the ratio with the preset ratio threshold;
if the ratio is greater than or equal to the preset ratio threshold, the read-write characteristic of the application program to be stored corresponding to the ratio is a read characteristic;
and if the ratio is smaller than the preset ratio threshold, the read-write characteristic of the application program to be stored corresponding to the ratio is a write characteristic.
In this embodiment, since the interaction between the application program to be stored and the flash memory solid-state disk is only read operation and write operation, the preset ratio is 0.5, and when the ratio is greater than 0.5, it indicates that the read operation behavior of the application program to be stored is greater than the write operation behavior, so that the read-write characteristic of the application program to be stored is a read characteristic.
In another preferred embodiment, referring to fig. 2, after obtaining the number of channels corresponding to each of the applications to be stored, the method further includes:
and according to the read-write characteristics in the access characteristics of each application program to be stored, using a corresponding page distribution mode in a channel corresponding to each application program to be stored.
In another preferred embodiment, the using a corresponding page allocation mode in a channel corresponding to each application program to be stored according to a read-write characteristic in the access characteristic of each application program to be stored specifically includes:
for any one application program to be stored, if the read-write characteristic of the application program to be stored is a read characteristic, a static page allocation mode is used in a channel corresponding to the application data to be stored;
and if the read-write characteristic of the application program to be stored is a write characteristic, using a dynamic page allocation mode in a channel corresponding to the application data to be stored.
It should be noted that the conventional flash solid-state disk can only adopt one page allocation mode (static or dynamic), and usually the write-based application has good performance in the dynamic page allocation mode, and the read-based application tends to adopt the static page allocation mode. When multiple applications with different read and write characteristics share the same flash solid-state disk, a single page allocation mode may result in overall performance degradation of the applications. According to the method and the device, the page allocation mode of the application program to be stored is determined according to the read-write characteristics of each application program to be stored, and the performance of the flash memory solid-state disk is improved.
In another preferred embodiment, the method for constructing the channel number prediction model, which can be described with reference to fig. 3, specifically includes:
acquiring and obtaining M channel allocation strategies according to the total channel number of the flash memory solid-state disk and the number of the application programs to be tested; the total channel number of the flash memory solid-state disk is greater than or equal to the number of the application programs to be tested;
acquiring K access requests to be tested; each access request to be tested comprises identification information and read-write characteristic information of an application program to be tested, and K is larger than 1000000;
extracting a to-be-tested access request corresponding to each to-be-tested application program from all to-be-tested access requests according to the identification information of all to-be-stored application programs;
obtaining the access characteristics to be trained corresponding to each application program to be tested according to the number of the access requests to be tested corresponding to each application program to be tested, the number of all the access requests to be tested and the read-write characteristic information in the access requests to be tested;
obtaining all the access characteristics to be trained and the delay time of each channel distribution strategy through an SSD-Sim simulator, and obtaining M delay times;
extracting a channel distribution strategy corresponding to the shortest delay time from all the delay times, and obtaining the number of channels corresponding to each access feature to be trained according to the channel distribution strategy corresponding to the shortest delay time;
and inputting the access features to be trained and the number of channels corresponding to the access features to be trained into a machine learning model for training to obtain the channel number prediction model.
Specifically, assuming that there are 4 applications to be tested and the flash memory solid-state disk has 8 channels, there are 336 channel allocation strategies, for example, the channel allocation strategy is {1:1:1:5}, which means that the first application to be tested allocates 1 channel, the second application to be tested allocates 1 channel, the third application to be tested allocates 1 channel, and the fourth application to be tested allocates 5 channels, because the channel allocation strategies are of various types and it is not known which allocation strategy is the best, the overall access delay corresponding to each channel allocation strategy needs to be tested, the access characteristics to be trained and the channel allocation strategies of the 4 applications to be tested are input into the SSD-Sim simulator, the SSD-Sim outputs the delay time corresponding to each channel allocation strategy, and extracts the channel allocation strategy corresponding to the shortest delay time from all the delay times, obtaining the number of channels corresponding to each access feature to be trained according to the channel distribution strategy corresponding to the shortest delay time; taking the access features to be trained and the corresponding channel number as a group of data, obtaining a plurality of groups of data, dividing all the groups of data into a training set and a test set according to a ratio of 7:3, and training a machine learning model by using the training set; the training method comprises forward propagation, backward propagation and parameter updating; it should be noted that the access delay represents the time taken by 4 applications to be tested to read and write data in all channel allocation methods.
In another preferred embodiment, the identification information of the application program to be stored is a name of the application program to be stored or a number of the application program to be stored.
The embodiment of the present invention further provides a multi-application-oriented adaptive flash memory solid-state disk channel allocation apparatus, which is capable of implementing all the processes of the multi-application-oriented adaptive flash memory solid-state disk channel allocation method described in any of the above embodiments, and the functions and implemented technical effects of each module and unit in the apparatus are respectively the same as those of the multi-application-oriented adaptive flash memory solid-state disk channel allocation method described in the above embodiments and implemented technical correlation, and are not described herein again.
Referring to fig. 4, it is a block diagram of a preferred embodiment of a multi-application-oriented adaptive flash memory solid-state disk channel allocation apparatus provided in the present invention, where the apparatus includes:
the acquisition module is used for acquiring N access requests received by the flash memory solid-state disk within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored, and N > 0;
the extraction module is used for extracting the access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored;
the computing module is used for obtaining the access characteristics corresponding to each application program to be stored according to the number of the access requests corresponding to each application program to be stored, the number of all the access requests received by the flash memory solid-state disk within the preset time and the read-write characteristic information in the access requests;
the prediction module is used for inputting the access characteristics of each application program to be stored into a preset channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the channel quantity prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises an access feature and a label for identifying the channel quantity corresponding to the access feature.
Further, the multi-application-program-oriented adaptive flash memory solid-state disk channel allocation device further comprises a page allocation module, which is used for using a corresponding page allocation mode for the channel corresponding to each application program to be stored according to the read-write characteristics in the access characteristics of each application program to be stored.
Further, the page allocation module further includes:
the first page allocation unit is used for using a static page allocation mode in a channel corresponding to the application data to be stored if the read-write characteristic of the application program to be stored is a read characteristic;
and the second page allocation unit is used for using a dynamic page allocation mode in a channel corresponding to the application data to be stored if the read-write characteristic of the application program to be stored is a write characteristic.
Therefore, compared with the existing channel allocation method of the flash memory solid-state disk, the channel allocation method of the flash memory solid-state disk is judged by a tester according to experience, and the channel number corresponding to the application program is accurately predicted through the channel number prediction model, so that the waste of the flash memory solid-state disk storage resources is reduced, and the utilization rate of the flash memory solid-state disk storage resources is improved.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A multi-application-oriented adaptive flash memory solid-state disk channel allocation method is characterized by comprising the following steps:
acquiring N access requests received by a flash memory solid-state disk within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored, and N > 0;
extracting an access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored;
obtaining access characteristics corresponding to each application program to be stored according to the number of access requests corresponding to each application program to be stored, the number of all the access requests received by the flash memory solid-state disk in the preset time and the read-write characteristic information in the access requests;
inputting the access characteristics of each application program to be stored into a preset channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the channel quantity prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises an access feature and a label for identifying the channel quantity corresponding to the access feature.
2. The multi-application-oriented adaptive flash memory solid-state disk channel allocation method according to claim 1, wherein the access characteristics include access density and read-write characteristics; then, the obtaining, according to the number of access requests corresponding to each application program to be stored, the number of all the access requests received by the flash solid-state disk within the preset time, and the read-write characteristic information in the access requests, an access characteristic corresponding to each application program to be stored specifically includes:
respectively calculating the ratio of the number of the access requests corresponding to each application program to be stored to the number of all the access requests received by the flash memory solid-state disk in the preset time, and obtaining the access density corresponding to each application program to be stored;
respectively calculating the ratio of the number of the access requests corresponding to the read characteristics in each application program to be stored to the number of the access requests corresponding to the application program to be stored according to the read-write characteristic information in the access requests, and obtaining the ratio of one-to-one correspondence of each application program to be stored;
and respectively comparing each ratio with a preset ratio threshold, acquiring the read-write characteristics of the application program to be stored corresponding to the ratio according to the comparison result, and acquiring the read-write characteristics corresponding to each application program to be stored.
3. The multi-application-program-oriented adaptive flash memory solid-state disk channel allocation method according to claim 2, wherein the comparing of each ratio with a preset ratio threshold and the obtaining of the read-write characteristics of the to-be-stored application program corresponding to the ratio according to the comparison result, and the obtaining of the read-write characteristics corresponding to each to-be-stored application program specifically include:
for any one of the ratios, comparing the ratio with the preset ratio threshold;
if the ratio is greater than or equal to the preset ratio threshold, the read-write characteristic of the application program to be stored corresponding to the ratio is a read characteristic;
and if the ratio is smaller than the preset ratio threshold, the read-write characteristic of the application program to be stored corresponding to the ratio is a write characteristic.
4. The multi-application-oriented adaptive flash solid-state disk channel allocation method according to claim 3, wherein after the obtaining of the number of channels corresponding to each of the applications to be stored, the method further comprises:
and according to the read-write characteristics in the access characteristics of each application program to be stored, using a corresponding page distribution mode in a channel corresponding to each application program to be stored.
5. The multi-application-program-oriented adaptive flash memory solid-state disk channel allocation method according to claim 4, wherein the using of a corresponding page allocation mode in a channel corresponding to each of the to-be-stored application programs according to a read-write characteristic in an access characteristic of each of the to-be-stored application programs specifically includes:
for any one application program to be stored, if the read-write characteristic of the application program to be stored is a read characteristic, a static page allocation mode is used in a channel corresponding to the application data to be stored;
and if the read-write characteristic of the application program to be stored is a write characteristic, using a dynamic page allocation mode in a channel corresponding to the application data to be stored.
6. The multi-application-oriented adaptive flash memory solid-state disk channel allocation method according to any one of claims 1 to 5, wherein the method for constructing the channel number prediction model specifically comprises:
acquiring and obtaining M channel allocation strategies according to the total channel number of the flash memory solid-state disk and the number of the application programs to be tested; the total channel number of the flash memory solid-state disk is greater than or equal to the number of the application programs to be tested;
acquiring K access requests to be tested; each access request to be tested comprises identification information and read-write characteristic information of an application program to be tested, and K is larger than 1000000;
extracting a to-be-tested access request corresponding to each to-be-tested application program from all to-be-tested access requests according to the identification information of all to-be-stored application programs;
obtaining the access characteristics to be trained corresponding to each application program to be tested according to the number of the access requests to be tested corresponding to each application program to be tested, the number of all the access requests to be tested and the read-write characteristic information in the access requests to be tested;
obtaining all the access characteristics to be trained and the delay time of each channel distribution strategy through an SSD-Sim simulator, and obtaining M delay times;
extracting a channel distribution strategy corresponding to the shortest delay time from all the delay times, and obtaining the number of channels corresponding to each access feature to be trained according to the channel distribution strategy corresponding to the shortest delay time;
and inputting the access features to be trained and the number of channels corresponding to the access features to be trained into a machine learning model for training to obtain the channel number prediction model.
7. The multi-application-program-oriented adaptive flash memory solid-state disk channel allocation method according to any one of claims 1 to 5, wherein the identification information of the application program to be stored is a name of the application program to be stored or a number of the application program to be stored.
8. An adaptive flash memory solid state disk channel allocation device oriented to multiple applications, comprising:
the acquisition module is used for acquiring N access requests received by the flash memory solid-state disk within preset time; each access request comprises identification information and read-write characteristic information of an application program to be stored, and N > 0;
the extraction module is used for extracting the access request corresponding to each application program to be stored from all the access requests according to the identification information of all the application programs to be stored;
the computing module is used for obtaining the access characteristics corresponding to each application program to be stored according to the number of the access requests corresponding to each application program to be stored, the number of all the access requests received by the flash memory solid-state disk within the preset time and the read-write characteristic information in the access requests;
the prediction module is used for inputting the access characteristics of each application program to be stored into a preset channel quantity prediction model to obtain the channel quantity corresponding to each application program to be stored; the channel quantity prediction model is obtained by using multiple groups of data through machine learning training, and each group of data in the multiple groups of data comprises an access feature and a label for identifying the channel quantity corresponding to the access feature.
9. The multi-application-program-oriented adaptive flash memory solid-state disk channel allocation method according to claim 8, further comprising a page allocation module configured to use a corresponding page allocation mode for a channel corresponding to each of the applications to be stored according to a read-write characteristic in the access characteristic of each of the applications to be stored.
10. The multi-application-oriented adaptive flash solid state disk channel allocation method of claim 9, wherein the page allocation module further comprises:
the first page allocation unit is used for using a static page allocation mode in a channel corresponding to the application data to be stored if the read-write characteristic of the application program to be stored is a read characteristic;
and the second page allocation unit is used for using a dynamic page allocation mode in a channel corresponding to the application data to be stored if the read-write characteristic of the application program to be stored is a write characteristic.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010433140.3A CN111782136B (en) | 2020-05-20 | 2020-05-20 | Multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010433140.3A CN111782136B (en) | 2020-05-20 | 2020-05-20 | Multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782136A true CN111782136A (en) | 2020-10-16 |
| CN111782136B CN111782136B (en) | 2021-07-30 |
Family
ID=72754333
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010433140.3A Active CN111782136B (en) | 2020-05-20 | 2020-05-20 | Multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782136B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104239233A (en) * | 2014-09-19 | 2014-12-24 | 华为技术有限公司 | Cache managing method, cache managing device and cache managing equipment |
| CN105512051A (en) * | 2015-12-16 | 2016-04-20 | 鸿秦（北京）科技有限公司 | Self-learning type intelligent solid-state hard disk cache management method and device |
| US20170090450A1 (en) * | 2015-09-24 | 2017-03-30 | Intel Corporation | Contextual clothing system |
| CN109471762A (en) * | 2018-10-30 | 2019-03-15 | 新华三技术有限公司 | Solid-state hard disk SSD performance test methods and device |
| WO2019060325A1 (en) * | 2017-09-19 | 2019-03-28 | Justin Herz | Consumer intelligence for automatic real time message decisions and selection |
| CN109947667A (en) * | 2017-12-21 | 2019-06-28 | 华为技术有限公司 | Data access prediction technique and device |
| CN110214305A (en) * | 2017-12-29 | 2019-09-06 | 华为技术有限公司 | A kind of data access method and storage array |
| CN110414260A (en) * | 2018-11-20 | 2019-11-05 | 腾讯科技（深圳）有限公司 | A kind of data access method, device, system and storage medium |
| CN110543509A (en) * | 2019-09-04 | 2019-12-06 | 北京奇艺世纪科技有限公司 | Monitoring system, method and device for user access data and electronic equipment |
| WO2019137562A9 (en) * | 2019-04-25 | 2020-02-13 | Alibaba Group Holding Limited | Identifying entities in electronic medical records |
| CN111090401A (en) * | 2020-03-23 | 2020-05-01 | 上海飞旗网络技术股份有限公司 | Storage device performance prediction method and device |
-
2020
- 2020-05-20 CN CN202010433140.3A patent/CN111782136B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104239233A (en) * | 2014-09-19 | 2014-12-24 | 华为技术有限公司 | Cache managing method, cache managing device and cache managing equipment |
| US20170090450A1 (en) * | 2015-09-24 | 2017-03-30 | Intel Corporation | Contextual clothing system |
| CN105512051A (en) * | 2015-12-16 | 2016-04-20 | 鸿秦（北京）科技有限公司 | Self-learning type intelligent solid-state hard disk cache management method and device |
| WO2019060325A1 (en) * | 2017-09-19 | 2019-03-28 | Justin Herz | Consumer intelligence for automatic real time message decisions and selection |
| CN109947667A (en) * | 2017-12-21 | 2019-06-28 | 华为技术有限公司 | Data access prediction technique and device |
| CN110214305A (en) * | 2017-12-29 | 2019-09-06 | 华为技术有限公司 | A kind of data access method and storage array |
| CN109471762A (en) * | 2018-10-30 | 2019-03-15 | 新华三技术有限公司 | Solid-state hard disk SSD performance test methods and device |
| CN110414260A (en) * | 2018-11-20 | 2019-11-05 | 腾讯科技（深圳）有限公司 | A kind of data access method, device, system and storage medium |
| WO2019137562A9 (en) * | 2019-04-25 | 2020-02-13 | Alibaba Group Holding Limited | Identifying entities in electronic medical records |
| CN110543509A (en) * | 2019-09-04 | 2019-12-06 | 北京奇艺世纪科技有限公司 | Monitoring system, method and device for user access data and electronic equipment |
| CN111090401A (en) * | 2020-03-23 | 2020-05-01 | 上海飞旗网络技术股份有限公司 | Storage device performance prediction method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782136B (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11461246B2 (en) | Multiple memory type memory module systems and methods | |
| US10120588B2 (en) | Sliding-window multi-class striping | |
| CN107220185A (en) | Date storage method, device and flash chip based on flash memory | |
| CN103020174A (en) | Similarity analysis method, device and system | |
| CN115422098B (en) | GPU access self-adaptive optimization method and device based on extended page table | |
| CN112286461A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN108829346A (en) | A kind of user's write request processing method for the solid state hard disk adapting to Hash memory pages difference | |
| CN109460406A (en) | A kind of data processing method and device | |
| CN114830098A (en) | Memory operation considering wear leveling | |
| CN110362277B (en) | Data classification storage method based on hybrid storage system | |
| CN110888592B (en) | Request scheduling method and system based on delay SSD system intelligent parallel resource utilization | |
| CN107203341A (en) | Date storage method, device and flash chip based on flash memory | |
| CN111782136B (en) | Multi-application-program-oriented self-adaptive flash memory solid-state disk channel allocation method and device | |
| CN111984425B (en) | Memory management method, device and equipment for operating system | |
| CN112148226A (en) | Data storage method and related device | |
| CN110618872B (en) | Hybrid memory dynamic scheduling method and system | |
| KR20150142250A (en) | Method for operating controller and method for operating device including the same | |
| US20220365876A1 (en) | Method of cache management based on file attributes, and cache management device operating based on file attributes | |
| EP3296878B1 (en) | Electronic device and page merging method therefor | |
| CN112732182A (en) | NAND data writing method and related device | |
| WO2016095451A1 (en) | Method and apparatus for improving memory access efficiency, and storage medium | |
| CN112015672A (en) | Data processing method, device, equipment and storage medium in storage system | |
| CN101470660A (en) | Method for system test through subsystem efficiency reference generator | |
| CN116909825B (en) | Automatic testing method for solid state disk performance test pretreatment and steady-state verification | |
| CN116450055B (en) | Method and system for distributing storage area between multi-processing cards |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |