CN111208944B - Data prejudging method, device and readable storage medium - Google Patents
Data prejudging method, device and readable storage medium Download PDFInfo
- Publication number
- CN111208944B CN111208944B CN201911400215.1A CN201911400215A CN111208944B CN 111208944 B CN111208944 B CN 111208944B CN 201911400215 A CN201911400215 A CN 201911400215A CN 111208944 B CN111208944 B CN 111208944B
- Authority
- CN
- China
- Prior art keywords
- data access
- data
- access request
- operations
- sequential
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data prejudging method, a device and a readable storage medium, wherein the data prejudging method comprises the steps of determining whether a data access request designates the length of target access data or not when the data access request is received; when the data access request does not specify the length of target access data, searching a data access log pointed by a logic address carried by the data access request; judging the operation type of the data access request according to the data access log; and executing corresponding operation according to the judging result. The invention solves the problem that the operation type is difficult to judge under the condition that the data access request does not specify the data length, and improves the prejudgment accuracy of the data access request sent by the host, thereby improving the capability of the FTL for distinguishing and processing different types of data access requests.
Description
Technical Field
The present invention relates to the field of computer storage technologies, and in particular, to a data pre-judging method, a device and a readable storage medium.
Background
With the advent of the big data age, the requirement of the whole computer system on the storage system is higher and higher, and compared with the traditional magnetic disk, nand Flash (Flash) storage equipment is widely applied due to the advantages of low storage delay, low power consumption, higher storage density, better anti-seismic model, low noise and the like. However, nand Flash has the characteristics of inconsistent read-write erasing units and P/E times limitation per block, so that the Nand Flash cannot be directly used by simple interface conversion. Therefore, a management software FTL (Flash Translation Layer Flash translation layer) is added on Nand Flash to manage, the upper layer application accesses the storage device through the logical address, the FTL maps different logical addresses to different physical addresses of the Nand Flash, so that data access of the upper layer application to the Nand Flash is realized, and the data access process does not need to burden the logic of Nand Flash operation, so that the read-write operation efficiency is improved.
Therefore, the fast response capability of the FTL serving as a management system of the solid state disk directly affects the read-write operation efficiency of the upper layer application, wherein the response speed of the operation request sent by the upper layer application is particularly important. For this, a more common manner is to pre-determine the type of the operation request by obtaining the length of the data to be accessed included in the operation request, so as to implement the corresponding operation according to the pre-determination result, to reduce the waiting time, for example, shorten the flow for random operation, and perform the cache operation for sequential operation. However, since some devices issue operation requests with unknown data length, the method for predicting operation requests based on known data length obviously has the problem of low probability of correct prediction.
Disclosure of Invention
The invention mainly aims to provide a data prejudging method, which aims to solve the problem that in the prior art, when an operation request is met and the length of data to be accessed is not specified, the method for judging the operation type based on the specified data length is difficult to apply.
In order to achieve the above object, the present invention provides a data pre-judging method, which includes:
when a data access request is received, determining whether the data access request specifies the length of target access data;
when the data access request does not specify the length of target access data, searching a data access log pointed by a logic address carried by the data access request;
determining the operation type of the data access request according to the data access log, wherein the data access log comprises the number of random operations and the number of sequential operations executed by the logic address;
and executing corresponding operation according to the judging result.
Preferably, the determining the operation type of the data access request according to the data access log includes:
calculating the ratio of the times of each operation type to the total operation times in the data access log;
if the ratio of the number of times of one operation type to the total operation number is larger than a preset value, the data access request is judged to be the operation type.
Preferably, the number of random operations includes a number of random write operations and a number of random read operations; the number of sequential operations includes a number of sequential write operations and a number of sequential read operations, and the calculating a ratio of the number of each operation type to a total number of operations in the data access log includes:
judging the data access request as a read operation or a write operation;
when the data access request is a write operation, calculating the ratio of the number of sequential write operations to the total number of write operations and the ratio of the number of random write operations to the total number of write operations respectively, wherein the total number of write operations is the sum of the number of sequential write operations and the number of random write operations;
when the data access request is a read operation, calculating the ratio of the number of sequential read operations to the total number of read operations and the ratio of the number of random read operations to the total number of read operations, wherein the total number of read operations is the sum of the number of sequential read operations and the number of random read operations.
Preferably, if the ratio of the number of times of one operation type to the total number of operations is greater than a preset value, determining that the data access request is the operation type includes:
if the ratio of the number of the sequential write operations to the total number of the write operations is greater than a preset value, determining that the data access request is a sequential write operation;
if the ratio of the number of random write operations to the total number of write operations is greater than a preset value, determining that the data access request is a random write operation;
or if the ratio of the number of times of one operation type to the total number of operations is greater than a preset value, determining that the data access request is the operation type includes:
if the ratio of the number of the sequential read operations to the total number of the read operations is greater than a preset value, determining that the data access request is a sequential read operation;
and if the ratio of the number of random read operations to the total number of read operations is greater than a preset value, determining that the data access request is a random read operation.
Preferably, the performing the corresponding operation according to the determination result includes:
if the data access request is judged to be a sequential write operation, executing a write cache;
if the data access request is judged to be a sequential read operation, executing pre-reading and reading the cache memory;
and if the data access request is judged to be random write operation or random read operation, executing normal read/write operation and performing short processing.
Preferably, the data prejudging method further comprises:
when the data access request specifies the length of the target access data, determining the operation type of the data access request according to the length of the target access data.
Preferably, after the corresponding operation is performed according to the determination result, the data pre-determining method further includes:
and recording the operation type of the operation to the data access log.
The invention also provides a data prejudging device, which comprises:
a length determining module, configured to determine, when a data access request is received, whether the data access request specifies a length of target access data;
the log access module is used for searching a data access log pointed by a logic address carried by the data access request if the length of target access data is not specified by the data access request;
a first determining module, configured to determine an operation type of the data access request according to the data access log, where the data access log includes a number of random operations and a number of sequential operations that are performed by the logical address;
and the execution module is used for executing corresponding operation according to the judging result.
Preferably, the first determining module includes:
the probability calculation unit is used for calculating the ratio of the times of each operation type to the total operation times in the data access log;
and the first judging unit is used for judging that the data access request is of one operation type when the ratio of the number of times of the operation type to the total operation number is larger than a preset value.
The invention also provides a data prejudging device, which comprises:
a memory for storing a computer program;
the processor is used for realizing the steps of the data pre-judging method when executing the computer program, and the data pre-judging method at least comprises the following steps: when a data access request is received, determining whether the data access request specifies the length of target access data;
when the data access request does not specify the length of target access data, searching a data access log pointed by a logic address carried by the data access request;
determining the operation type of the data access request according to the data access log, wherein the data access log comprises the number of random operations and the number of sequential operations executed by the logic address;
and executing corresponding operation according to the judging result.
The present invention also provides a readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data pre-determination method as described above.
Compared with the prior art, the embodiment of the invention has the beneficial effects that: under the condition that the data access request does not specify the length of target access data, the operation type of the data access request is judged by searching the data access log pointed by the logic address carried by the data access request and according to the content recorded in the log, the problem that the operation type is difficult to judge under the condition that the data access request does not specify the length of the data is solved, and the prejudging accuracy rate of the data access request sent by a host end is improved, so that the distinguishing processing capacity of the FTL on the data access requests of different types is improved.
Drawings
FIG. 1 is a flow chart of the steps of the data pre-judging method of the present invention;
FIG. 2 is a flowchart of a preferred embodiment of the data pre-judging method according to the present invention, wherein the step is performed by "judging the operation type of the data access request according to the data access log";
FIG. 3 is a flowchart of a preferred embodiment of the data pre-judging method according to the present invention, wherein the step of judging the operation type of the data access request according to the data access log and the step of executing the corresponding operation according to the judging result are executed;
FIG. 4 is a functional block diagram of a data pre-determining apparatus according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements or elements having the same function throughout. The embodiments described below are exemplary and intended to illustrate the present invention and should not be construed as limiting the invention, and all other embodiments, based on the embodiments of the present invention, which may be obtained by persons of ordinary skill in the art without making any inventive effort, are intended to be within the scope of the present invention.
The present invention proposes a data pre-judging method, in an embodiment, as shown in fig. 1, the data pre-judging method includes:
step S100: when a data access request is received, it is determined whether the data access request specifies a length of target access data.
The method is implemented by FTL (Flash Translation Layer flash translation layer), and the sender of the data access request is the host (upper layer application). When the FTL receives the data access request, it is first determined whether the data access request specifies the length of the target access data, if so, the operation type of the data access request may be determined directly according to the length, for example, if the specified length of the target access data is not less than 32K, the operation type of the data access request may be determined to be a sequential operation (i.e., a continuous operation), and if the specified length of the target access data is less than 32K, the operation type of the data access request may be determined to be a random operation. The aim of this step is to start with the simplest and rapid determination method and to accelerate the pre-determination speed.
Step S200: when the data access request does not specify the length of the target access data, searching a data access log pointed by a logic address carried by the data access request.
In this step, since the data access request does not specify the length of the target access data, the operation type of the data access request cannot be judged by the data length. The data access log is stored in the header of the mapping table from the logical address to the physical address, which is specified by the data access request, that is, a section of area is added to the header of the mapping table as the header, so as to store the data access log. The data access log records the logs of the operations of various types of logical addresses, specifically the times of sequential operations and random operations, and the like.
Step S300: the operation type of the data access request is determined according to the data access log, and the data access log contains the number of random operations and the number of sequential operations of which the logical address is executed.
For special purposes, or for writing in whole file form, some logical spaces are often operated in a fixed type, such as: a certain logical space holds a movie file, which in most cases will be subjected to sequential read operations; some block of logical space holds small files, then random operations will be performed in most cases. The purpose of this step is to improve the pre-judgment probability by estimating the operation type of the present data access request based on the operation history of the logical address specified by the data access request. For example, if the data access log has a majority of sequential operations, it can be inferred that the operation type of the data access request is sequential operation.
Step S400: executing corresponding operation according to the judging result, for example, executing short-flow processing if the judging result is random operation, and improving the processing speed; and if the sequential operation is judged, the pre-reading is executed, so that the execution waiting time is reduced. Different processing flows are executed aiming at different data access requests, unnecessary operation steps are reduced, and operation efficiency is improved.
According to the method and the device, the operation type of the data access request with the unspecified target access data length is judged according to the content recorded in the data access log pointed by the logical address carried by the data access request, the problem that the operation type is difficult to judge under the condition that the data access request has unspecified data length is solved, the prejudging accuracy of the data access request sent by the host is improved, and therefore the distinguishing processing capacity of the FTL for different types of data access requests is improved.
In a preferred embodiment, as shown in FIG. 2, step 300 (determining the type of operation for a data access request based on a data access log) includes:
step 310: the ratio of the number of each operation type to the total number of operations in the data access log is calculated.
In this step, the number of times of each operation type refers to the number of times of random operation or the number of times of sequential operation, and the total number of operations refers to the sum of the number of times of random operation and the number of times of sequential operation. The ratio of the number of times of each operation type to the total operation number is calculated, and the purpose is to obtain various types of historical operation probabilities of the logical address designated by the data access request, and through the probabilities, the operation type of the logical address accessed at the time, namely the operation type of the data access request, can be prejudged.
Step 320: if the ratio of the number of one operation type to the total operation number is larger than a preset value, the data access request is judged to be the operation type.
The method comprises the following steps: and if the ratio of the number of sequential operations to the total number of operations in the data access log is greater than a preset value, determining that the data access request is sequential operation, and if the ratio of the number of random operations to the total number of operations in the data access log is greater than a preset value, determining that the data access request is random operation. When none of the above are coincident, the data access request is not processed.
In a preferred embodiment, as shown in FIG. 3, the number of random operations includes the number of random write operations and the number of random read operations; the number of sequential operations includes the number of sequential write operations and the number of sequential read operations; step 310 (calculating the ratio of the number of times of each operation type to the total number of operations in the data access log) includes:
step 311: the data access request is judged to be a read operation or a write operation.
Step 312: when the data access request is a write operation, calculating the ratio of the number of sequential write operations to the total number of write operations and the ratio of the number of random write operations to the total number of write operations respectively, wherein the total number of write operations is the sum of the number of sequential write operations and the number of random write operations;
step 313: when the data access request is a read operation, calculating a ratio of the number of sequential read operations to the total number of read operations and a ratio of the number of random read operations to the total number of read operations, respectively, wherein the total number of read operations is the sum of the number of sequential read operations and the number of random read operations.
A single access request, which may be a sequential read operation, a sequential write operation, a random read operation, or a random write operation, corresponds to a type of operation that may be performed by a piece of logical space, and also includes a sequential read operation, a sequential write operation, a random read operation, and a random write operation, that is, a data access log describing a history that the logical space is accessed, where contents that may be described include a sequential read operation, a sequential write operation, a random read operation, and a random write operation. For this case, the present embodiment determines whether the data access request is a read operation or a write operation in advance, and if the data access request is a read operation, only the probability of a sequential read operation and the probability of a random read operation need to be calculated, that is, the ratio of the number of times the logical address is executed in the sequential read operation to the total number of times the logical address is executed is calculated, and, similarly to the previous embodiment, it can be predetermined whether the data access request is a sequential read operation or a random read operation according to the ratio. Similarly, if the data access request is a write operation, only the probability of the sequential write operation and the probability of the random write operation are calculated, namely, the ratio of the number of times the logic address is executed in the sequential write operation to the total number of times of the write operation is calculated, and the specific data access request is the sequential write operation or the random write operation can be prejudged according to the ratio. If the two are not in conformity, the processing is not performed.
In a preferred embodiment, as shown in fig. 3, step 320 (determining that the data access request is of one operation type if the ratio of the number of operations of the one operation type to the total number of operations is greater than a predetermined value) includes:
step 321: if the ratio of the number of sequential write operations to the total number of write operations is greater than a preset value, determining that the data access request is a sequential write operation;
step 322: if the ratio of the number of random write operations to the total number of write operations is greater than a preset value, determining that the data access request is a random write operation; or alternatively, the first and second heat exchangers may be,
step 323: if the ratio of the number of sequential read operations to the total number of read operations is greater than a preset value, determining that the data access request is a sequential read operation;
step 324: if the ratio of the number of random read operations to the total number of read operations is greater than a preset value, the data access request is determined to be a random read operation.
The present embodiment aims at performing type determination for both read and write data access requests, respectively, and the preset value as a determination criterion is preferably 80%.
In a preferred embodiment, as shown in fig. 3, step S400 (performing a corresponding operation according to the determination result) includes:
step S410: if the data access request is judged to be a sequential write operation, the write cache memory is executed, namely target access data is written into the cache memory in advance, so that the writing speed of the subsequent write physical address is improved.
Step S420: if the data access request is determined to be a sequential read operation, a read-ahead and read-cache are performed.
Because of the performance difference between the memory and the disk, in order to improve the system access efficiency, a copy of part of data needs to be cached in the memory, when data access is performed, the cache is firstly tried to be accessed, and if the cache is not hit, the data is accessed from the rear-end disk. In this embodiment, when it is determined that the current data access request is a sequential read operation, operations of pre-reading and cache reading are performed, where the idea of pre-reading is to put the data that may be accessed subsequently into the cache in advance, so that subsequent data access may directly read the cache without accessing a back-end disk, thereby improving access efficiency.
Step S430: if it is determined that the data access request is a random write operation or a random read operation, performing a normal read/write operation and performing a short process, that is, directly reading/writing data from the disk, without performing additional special operations such as pre-reading, writing or reading a cache, short process, and the like, and reducing unnecessary operations, such as searching only the first one when searching the table entry, and checking whether there is a continuation or not later, so as to reduce the running time of the flow.
In a preferred embodiment, as shown in fig. 1, the data pre-determining method further includes:
step 500: when the data access request specifies the length of the target access data, the operation type of the data access request is predicted according to the length of the target access data.
When the data access request specifies the length of the target access data, the operation type of the data access request may be determined directly according to the data length, for example, if the specified length of the target access data is not less than 32K, the operation type of the data access request may be determined to be a sequential operation (i.e., a continuous operation), and if the specified length of the target access data is less than 32K, the operation type of the data access request may be determined to be a random operation.
When a certain section of logical address is accessed for the first time, after the operation type of the data access request is judged, the judgment result can be directly recorded into a data access log pointed by the logical address carried by the data access request, so as to be used as an operation pre-judgment reference when the subsequent logical address is accessed for data access; or after completing the data access according to the data access request, recording the actual operation process. The latter is preferred in the case of non-first accesses.
In a preferred embodiment, as shown in fig. 1, after step 400, the data pre-determining method further includes:
step 600: the operation type of the operation is recorded to a data access log.
In this embodiment, the data access log records the actual operation process after the data access is completed, which is more favorable for the subsequent reference of operation pre-judgment.
Based on the method for predicting random or sequential operation of flash memory proposed in the foregoing embodiment, the present invention further proposes a data predicting device, in an embodiment, as shown in fig. 4, the data predicting device includes:
a length determining module 10 for determining, when a data access request is received, whether the data access request specifies a length of target access data;
the log access module 20 is configured to search a data access log pointed by a logical address carried by the data access request when the data access request does not specify a length of target access data;
a first determining module 30 for determining an operation type of the data access request according to a data access log including the number of random operations and the number of sequential operations in which the logical address is executed;
and an execution module 40, configured to execute a corresponding operation according to the determination result.
In a preferred embodiment, as shown in fig. 4, the first determining module 30 includes:
a probability calculation unit 31 for calculating a ratio of the number of times of each operation type to the total number of operations in the data access log;
a first determining unit 32, configured to determine that the data access request is of one operation type when the ratio of the number of operations of the operation type to the total number of operations is greater than a preset value.
In a preferred embodiment, the number of random operations includes a number of random write operations and a number of random read operations; the number of sequential operations includes the number of sequential write operations and the number of sequential read operations; the probability calculation unit includes:
the read-write judging unit is used for judging the data access request as a read operation or a write operation;
the first probability calculation unit is used for calculating the ratio of the number of sequential write operations to the total number of write operations and the ratio of the number of random write operations to the total number of write operations respectively when the data access request is a write operation, wherein the total number of write operations is the sum of the number of sequential write operations and the number of random write operations;
and the second probability calculation unit is used for respectively calculating the ratio of the number of sequential read operations to the total number of read operations and the ratio of the number of random read operations to the total number of read operations when the data access request is the read operation, wherein the total number of read operations is the sum of the number of sequential read operations and the number of random read operations.
In a preferred embodiment, the first determination module includes:
a sequential write operation determination unit configured to determine that the data access request is a sequential write operation when a ratio of a number of sequential write operations to a total number of write operations is greater than a preset value;
a random write operation judgment unit for judging that the data access request is random write operation when the ratio of the number of random write operations to the total number of write operations is greater than a preset value; or alternatively, the first and second heat exchangers may be,
a sequential read operation judging unit for judging that the data access request is a sequential read operation when the ratio of the number of sequential read operations to the total number of read operations is greater than a preset value;
and the random read operation judging unit is used for judging that the data access request is random read operation when the ratio of the number of random read operations to the total number of read operations is larger than a preset value.
In a preferred embodiment, the execution module comprises:
a first execution unit configured to execute writing to the cache memory when it is determined that the data access request is a sequential writing operation;
a second execution unit for executing the pre-reading and the reading of the cache memory when the data access request is determined to be a sequential read operation;
and a third execution unit for executing a normal read/write operation and performing a short process when it is determined that the data access request is a random write operation or a random read operation.
In a preferred embodiment, as shown in fig. 4, the data pre-determining apparatus further includes:
the second determining module 50 is configured to determine an operation type of the data access request according to the length of the target access data when the data access request specifies the length of the target access data.
In a preferred embodiment, as shown in fig. 4, the data pre-determining apparatus further includes:
a recording module 60, configured to record the operation type of the operation to the data access log.
The invention also provides a data prejudging device, which comprises:
a memory for storing a computer program;
and a processor, configured to implement the steps of the data pre-judging method in the foregoing embodiment when executing the computer program. The data prejudging method at least comprises the following steps:
when a data access request is received, determining whether the data access request specifies the length of target access data;
when the data access request does not specify the length of target access data, searching a data access log pointed by a logic address carried by the data access request;
judging the operation type of the data access request according to a data access log, wherein the data access log comprises the number of random operations and the number of sequential operations of which the logic address is executed;
and executing corresponding operation according to the judging result.
The present invention also proposes a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a data pre-determination method as described above, the steps of the data pre-determination method comprising at least:
when a data access request is received, determining whether the data access request specifies the length of target access data;
when the data access request does not specify the length of target access data, searching a data access log pointed by a logic address carried by the data access request;
determining the operation type of the data access request according to the data access log, wherein the data access log comprises the number of random operations and the number of sequential operations of which the logic address is executed;
and executing corresponding operation according to the judging result.
In the several embodiments provided in the present application, it should be understood that the disclosed methods and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules.
The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above description and drawings should not be taken as limiting the scope of the invention in any way, but rather should be understood to cover all modifications, structural equivalents, or direct/indirect applications of the invention in the light of the general principles of the present invention which may be employed in the present invention and illustrated by the accompanying drawings.
Claims (9)
1. A data prejudging method, comprising:
when a data access request is received, determining whether the data access request specifies the length of target access data;
when the length of target access data is not specified by the data access request, searching a data access log pointed by a logic address carried by the data access request;
determining the operation type of the data access request according to the data access log, wherein the data access log comprises the number of random operations and the number of sequential operations executed by the logic address;
executing corresponding operation according to the judging result;
the determining the operation type of the data access request according to the data access log comprises the following steps:
calculating the ratio of the times of each operation type to the total operation times in the data access log;
if the ratio of the number of times of one operation type to the total operation number is larger than a preset value, the data access request is judged to be the operation type.
2. The data prejudging method according to claim 1, wherein the number of random operations includes a number of random write operations and a number of random read operations; the number of sequential operations includes a number of sequential write operations and a number of sequential read operations; the calculating the ratio of the number of times of each operation type in the data access log to the total number of operations comprises:
judging the data access request as a read operation or a write operation;
when the data access request is a write operation, calculating the ratio of the number of sequential write operations to the total number of write operations and the ratio of the number of random write operations to the total number of write operations respectively, wherein the total number of write operations is the sum of the number of sequential write operations and the number of random write operations;
when the data access request is a read operation, calculating the ratio of the number of sequential read operations to the total number of read operations and the ratio of the number of random read operations to the total number of read operations, wherein the total number of read operations is the sum of the number of sequential read operations and the number of random read operations.
3. The method of claim 2, wherein if the ratio of the number of operations of one of the operation types to the total number of operations is greater than a preset value, determining that the data access request is the operation type comprises:
if the ratio of the number of the sequential write operations to the total number of the write operations is greater than a preset value, determining that the data access request is a sequential write operation;
if the ratio of the number of random write operations to the total number of write operations is greater than a preset value, determining that the data access request is a random write operation;
or if the ratio of the number of times of one operation type to the total number of operations is greater than a preset value, determining that the data access request is the operation type includes:
if the ratio of the number of the sequential read operations to the total number of the read operations is greater than a preset value, determining that the data access request is a sequential read operation;
and if the ratio of the number of random read operations to the total number of read operations is greater than a preset value, determining that the data access request is a random read operation.
4. The data prejudging method according to claim 3, wherein the performing the corresponding operation according to the judgment result comprises:
if the data access request is judged to be a sequential write operation, executing a write cache;
if the data access request is judged to be a sequential read operation, executing pre-reading and reading the cache memory;
and if the data access request is judged to be random write operation or random read operation, executing normal read/write operation and performing short processing.
5. The data prejudging method according to claim 1, further comprising:
and when the data access request specifies the length of the target access data, judging the operation type of the data access request according to the length of the target access data.
6. The data prejudging method according to claim 1, further comprising, after the corresponding operation is performed according to the judgment result:
and recording the operation type of the operation to the data access log.
7. A data prejudging device, comprising:
a length determining module, configured to determine, when a data access request is received, whether the data access request specifies a length of target access data;
the log access module is used for searching a data access log pointed by a logic address carried by the data access request when the length of target access data is not specified by the data access request;
a first determining module, configured to determine an operation type of the data access request according to the data access log, where the data access log includes a number of random operations and a number of sequential operations that are performed by the logical address;
the execution module is used for executing corresponding operation according to the judgment result;
the first determination module includes:
the probability calculation unit is used for calculating the ratio of the times of each operation type to the total operation times in the data access log;
and the first judging unit is used for judging that the data access request is of one operation type when the ratio of the number of times of the operation type to the total operation number is larger than a preset value.
8. A data prejudging device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data pre-determination method according to any one of claims 1 to 6 when executing the computer program.
9. A readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the data pre-determination method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911400215.1A CN111208944B (en) | 2019-12-30 | 2019-12-30 | Data prejudging method, device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911400215.1A CN111208944B (en) | 2019-12-30 | 2019-12-30 | Data prejudging method, device and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111208944A CN111208944A (en) | 2020-05-29 |
| CN111208944B true CN111208944B (en) | 2023-06-27 |
Family
ID=70786443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911400215.1A Active CN111208944B (en) | 2019-12-30 | 2019-12-30 | Data prejudging method, device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111208944B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011107983A (en) * | 2009-11-17 | 2011-06-02 | Nec Corp | System, method, and program for storing information |
| CN104239233A (en) * | 2014-09-19 | 2014-12-24 | 华为技术有限公司 | Cache managing method, cache managing device and cache managing equipment |
| CN105224240A (en) * | 2014-05-30 | 2016-01-06 | 北京忆恒创源科技有限公司 | By the method for data write storage device, device and memory device |
| CN107750355A (en) * | 2015-08-31 | 2018-03-02 | 邻客音公司 | Transparent blended data storage device |
| CN107967306A (en) * | 2017-11-13 | 2018-04-27 | 华中科技大学 | The Fast Mining Algorithm of associated block in a kind of storage system |
| CN109684236A (en) * | 2018-12-25 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of data write buffer control method, device, electronic equipment and storage medium |
-
2019
- 2019-12-30 CN CN201911400215.1A patent/CN111208944B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011107983A (en) * | 2009-11-17 | 2011-06-02 | Nec Corp | System, method, and program for storing information |
| CN105224240A (en) * | 2014-05-30 | 2016-01-06 | 北京忆恒创源科技有限公司 | By the method for data write storage device, device and memory device |
| CN104239233A (en) * | 2014-09-19 | 2014-12-24 | 华为技术有限公司 | Cache managing method, cache managing device and cache managing equipment |
| CN107750355A (en) * | 2015-08-31 | 2018-03-02 | 邻客音公司 | Transparent blended data storage device |
| CN107967306A (en) * | 2017-11-13 | 2018-04-27 | 华中科技大学 | The Fast Mining Algorithm of associated block in a kind of storage system |
| CN109684236A (en) * | 2018-12-25 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of data write buffer control method, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111208944A (en) | 2020-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106547476B (en) | Method and apparatus for data storage system | |
| US9767140B2 (en) | Deduplicating storage with enhanced frequent-block detection | |
| KR101422557B1 (en) | Predictive data-loader | |
| CN100362462C (en) | Method for managing magnetic disk array buffer storage | |
| US9052826B2 (en) | Selecting storage locations for storing data based on storage location attributes and data usage statistics | |
| US7613870B2 (en) | Efficient memory usage in systems including volatile and high-density memories | |
| US20090089501A1 (en) | Method of prefetching data in hard disk drive, recording medium including program to execute the method, and apparatus to perform the method | |
| US6848034B2 (en) | Dense server environment that shares an IDE drive | |
| JPS58155464A (en) | Detection of sequential data stream | |
| US20170004087A1 (en) | Adaptive cache management method according to access characteristics of user application in distributed environment | |
| US20220164316A1 (en) | Deduplication method and apparatus | |
| CN113835614A (en) | SSD intelligent caching method and system based on distributed file storage client | |
| CN111309262B (en) | Distributed storage cache reading and writing method | |
| US8732404B2 (en) | Method and apparatus for managing buffer cache to perform page replacement by using reference time information regarding time at which page is referred to | |
| CN112347044B (en) | Object storage optimization method based on SPDK | |
| CN112181870B (en) | Memory page management method, device and equipment and readable storage medium | |
| CN111208944B (en) | Data prejudging method, device and readable storage medium | |
| CN108052296B (en) | Data reading method and device and computer storage medium | |
| US11836092B2 (en) | Non-volatile storage controller with partial logical-to-physical (L2P) address translation table | |
| CN116820342A (en) | Data processing method and device of disk array and disk array | |
| CN109189696B (en) | SSD (solid State disk) caching system and caching method | |
| JPH08263380A (en) | Disk cache control system | |
| CN112084121B (en) | Hard disk pre-reading method and device, computer readable storage medium and electronic equipment | |
| CN114461681A (en) | Cache content management method and device, electronic equipment and storage medium | |
| CN116069752A (en) | Mixed prefetching method for distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000 floors 1-3 and 4 of buildings 4 and 8, zone 2, Zhongguan honghualing Industrial South Zone, No. 1213 Liuxian Avenue, Pingshan community, Taoyuan Street, Nanshan District, Shenzhen, Guangdong Applicant after: BIWIN STORAGE TECHNOLOGY Co.,Ltd. Address before: 518000 1st, 2nd, 4th and 5th floors of No.4 factory building, tongfuyu industrial town, Taoyuan Street, Nanshan District, Shenzhen City, Guangdong Province Applicant before: BIWIN STORAGE TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |