CN111399767A - IO request processing method, system, equipment and computer readable storage medium - Google Patents
IO request processing method, system, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111399767A CN111399767A CN202010105640.4A CN202010105640A CN111399767A CN 111399767 A CN111399767 A CN 111399767A CN 202010105640 A CN202010105640 A CN 202010105640A CN 111399767 A CN111399767 A CN 111399767A
- Authority
- CN
- China
- Prior art keywords
- request
- block
- virtual block
- data migration
- creating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000013508 migration Methods 0.000 claims abstract description 71
- 230000005012 migration Effects 0.000 claims abstract description 71
- 238000013507 mapping Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 8
- 230000001960 triggered effect Effects 0.000 claims description 5
- 230000000875 corresponding effect Effects 0.000 abstract 4
- 238000007726 management method Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 7
- 239000010410 layer Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0667—Virtualisation aspects at data level, e.g. file, record or object virtualisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses an IO request processing method, which is applied to a host at least comprising an Optane SSD and comprises the following steps: creating virtual block equipment, and establishing an initial mapping relation between each block address of the virtual block equipment and the block address of corresponding physical equipment; adjusting the mapping relation according to the execution object of the data migration when the data migration is carried out; when an IO request sent by a user is received, determining address information of data corresponding to the IO request in corresponding physical equipment according to the IO request and the current mapping relation, and executing the IO request based on the address information. By applying the scheme of the application, when the high performance of the Optane SSD is effectively utilized to store the hot data, the influence of frequent migration of the data on IO can be reduced. The application also discloses a system and a device for processing the IO request and a computer readable storage medium, and the system and the device have corresponding effects.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a method, a system, a device, and a computer-readable storage medium for processing an IO request.
Background
Intel's Optane technology combines the most advanced hardware medium and software scheme in present Intel's storage research, wherein the hardware medium 3D XPoint is the core of the whole Optane technology, can not only serve as hard disk cache to speed up for original mechanical hard disk or solid state hard disk at present, but also can directly serve as nonvolatile storage medium to save data. The read-write speed and the service life of the 3D XPoint are 1000 times of those of the NAND flash memory, the delay is one thousandth of that of the NAND flash memory, and the delay is 10 times of that of a DRAM (dynamic random access memory); the storage density is 10 times of the memory.
Therefore, at present, the application of the method is mainly divided into two categories, one is that the method is used as an SSD cache for storing hot data and improving data access capability. Write IO requests and missed IO read requests are usually submitted directly to a data disk of general performance, read the hit IO requests, and when hot. The IO will be redirected to the Optane SSD cache. However, this has the disadvantage that frequent and large data migration tends to affect the performance of the system. And secondly, the storage is layered according to the storage performance, and the Optane SSD with high performance is used as a single layer to specially store data with high heat. After the disk types are divided into a plurality of layers, it is understood that data with higher heat values are put into layers with higher performance. For example, three layers 0, 1 and 2, assuming that 0 is the fastest and 2 is the slowest. The data is typically written to layer 1 first, and if it is hot data, it is migrated to layer 0, and if it is cold data, it is migrated to layer 1. The disadvantage of such a scheme is that the mechanism is complex, and IO needs to be interrupted during the data migration process, which is easy to affect other applications, or damage the data integrity during the migration process.
In summary, how to effectively utilize the high performance of the Optane SSD to store hot data can reduce the impact of frequent migration of data on IO is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide an IO request processing method, an IO request processing system, an IO request processing device and a computer readable storage medium, so that the influence of frequent migration of data on IO can be reduced when hot data is stored by effectively utilizing the high performance of Optane SSD.
In order to solve the technical problems, the invention provides the following technical scheme:
an IO request processing method is applied to a host at least comprising one Optane SSD, and comprises the following steps:
creating virtual block equipment, and establishing an initial mapping relation between each block address of the virtual block equipment and the block address of corresponding physical equipment;
adjusting the mapping relation according to an execution object of data migration when data migration is carried out;
when an IO request sent by a user is received, determining address information of data corresponding to the IO request in corresponding physical equipment according to the IO request and a current mapping relation, and executing the IO request based on the address information.
Preferably, the creating a virtual block device includes:
registering the management equipment;
and after receiving a virtual block device creation request sent by the management device, creating a virtual block device.
Preferably, before the creating the virtual block device, the method further includes:
and receiving the verification parameters of the preset items, and creating the virtual block equipment after all the verification parameters are verified.
Preferably, the operation of performing data migration includes:
determining and updating the heat of each block of each physical device according to a preset statistical rule;
and when a preset data migration condition is triggered, performing data migration based on the heat of each block.
Preferably, the statistical rule includes:
for each block of each physical device, when any sector in the block is requested to be accessed by an IO (input/output) request, and the accessed IO request does not belong to a predefined large IO or a sequential IO, increasing the heat of the block;
and aiming at each block of each physical device, reducing the heat of the block at intervals of a preset first time length.
Preferably, the data migration condition includes:
and at every preset second time, or receiving a migration instruction sent by a user.
An IO request processing system, applied to a host including at least one Optane SSD, includes:
the virtual block device creating module is used for creating virtual block devices and establishing an initial mapping relation between each block address of the virtual block devices and the block address of the corresponding physical device;
a mapping relation adjusting module, configured to adjust the mapping relation according to an execution object of data migration each time data migration is performed;
and the IO request execution module is used for determining the address information of the data corresponding to the IO request in the corresponding physical equipment according to the IO request and the current mapping relation when the IO request sent by the user is received, and executing the IO request based on the address information.
Preferably, the virtual block device creating module is specifically configured to:
registering the management equipment;
after receiving a virtual block device creation request of the management device, creating a virtual block device, and establishing an initial mapping relationship between each block address of the virtual block device and a block address of a corresponding physical device.
An IO request processing device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the IO request processing method described in any one of the above.
A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of processing an IO request as set forth in any one of the above.
By applying the technical scheme provided by the embodiment of the invention, the host at least comprises one Optane SSD, so that the scheme of the application can utilize the high performance of the Optane SSD to store the hot data. Further, the virtual block device is created, an initial mapping relationship between each block address of the virtual block device and the block address of the corresponding physical device is established, and the mapping relationship is adjusted according to an execution object of data migration each time data migration is performed. Therefore, when the scheme of the application performs data migration, the operation of the virtual block device is not affected, that is, IO does not need to be interrupted, and the situation that in the conventional scheme, because IO needs to be interrupted in the data migration process, other applications are easily affected or the integrity of data is damaged does not occur. Therefore, when the high performance of Optane SSD is effectively utilized to store hot data, the influence of frequent migration of data on IO can be reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating an embodiment of a method for processing an IO request according to the present invention;
FIG. 2 is a schematic structural diagram of an IO request processing system according to the present invention;
fig. 3 is a schematic structural diagram of an IO request processing device according to the present invention.
Detailed Description
The core of the invention is to provide an IO request processing method, which can reduce the influence of frequent migration of data on IO when the high performance of Optane SSD is effectively used for storing hot data.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, it is a flowchart of an IO request processing method in the present invention, where the IO request processing method may be applied to a host including at least one Optane SSD, and may include the following steps:
step S101: and creating virtual block devices, and establishing an initial mapping relation between each block address of the virtual block devices and the block address of the corresponding physical device.
Specifically, the host may be generally connected to a management device, and the virtual block device may be created based on the management device. Namely, creating a virtual block device described in step S101 may include the following two steps:
the method comprises the following steps: registering the management equipment;
step two: after receiving a virtual block device creation request sent by the management device, the virtual block device is created.
The user may input a virtual block device creation request to the management device, and the management device may further initiate the virtual block device creation request to the host to implement the creation of the virtual block device.
In addition, since the virtual block device is created by using the management device in this embodiment, when the host is restarted, the virtual block device can be automatically reconstructed, and of course, the mapping relationship between each block address of the virtual block device and the block address of the corresponding physical device needs to be stored, and a certain backup policy is usually adopted to prevent a data loss situation caused by power failure or disk damage.
Further, in a specific embodiment of the present invention, after initiating the virtual block device creation request, before creating the virtual block device, the method may further include:
and receiving the verification parameters of the preset items, and creating the virtual block equipment after all the verification parameters are verified.
For example, the check parameters of the preset item may include all parameters of each physical device, such as device name, disk size, block size of the virtual disk, and so on. After receiving the check parameters of the preset items, the host may check, and if all the check parameters are verified, may create a virtual block device, for example, call register _ blkdev to create a new block device, that is, create a virtual block device.
After the virtual block devices are created, an initial mapping relationship between each block address of the virtual block devices and the block address of the corresponding physical device may be established.
The virtual block devices and the physical devices are divided into blocks of fixed size, and of course, the user may modify the block size according to actual needs. One Block is the minimum unit for hot statistics and data migration, and each Block usually contains several 512Byte sectors.
In addition, the block size of the virtual block device and the block size of the physical device may be the same or different, which is convenient for operation and is usually set to the same size. For example, there are two physical devices in the host, referred to as DEV1 and DEV2, DEV1 is an SSD, and DEV2 is a higher performance Optane SSD. Also, DEV1 may comprise X blocks, for example, and DEV2 may comprise Y blocks, for example, then the block size of the virtual block device may be generally the same as the block size of each physical device, in which case the virtual block device may comprise X + Y blocks.
Step S102: and adjusting the mapping relation according to the execution object of the data migration each time the data migration is carried out.
Data migration can be performed automatically periodically, and due to the data migration, addresses of partial data in the physical device are changed, so that a mapping relationship needs to be adjusted according to an execution object of the data migration, that is, according to information of the data migration, a mapping relationship needs to be adjusted accordingly. For example, before a certain data migration, block5 of a virtual block device corresponds to block3 of DEV1, block9 of the virtual block device corresponds to block7 of DEV2, after the data migration, data in block3 of DEV1 is exchanged with data storage positions in block7 of DEV2, the mapping relationship needs to be adjusted accordingly, after the data migration, block5 of the virtual block device corresponds to block7 of DEV2, and block9 of the virtual block device corresponds to block3 of DEV 1.
Step S103: when an IO request sent by a user is received, determining address information of data corresponding to the IO request in corresponding physical equipment according to the IO request and the current mapping relation, and executing the IO request based on the address information.
Specifically, the Block of the virtual Block device corresponding to the IO request and the offset in the Block may be found out from two pieces of information, namely bi sector and bi size, carried in the IO request.
After determining the Block of the virtual Block device corresponding to the IO request and the offset in the Block according to the information in the IO request, the mapping relationship can be queried, so that the Block of the corresponding physical device is found, the offset in the Block of the physical device is correspondingly read, and the IO request is executed.
In a specific embodiment of the present invention, the performing data migration may include:
determining and updating the heat of each block of each physical device according to a preset statistical rule;
and when a preset data migration condition is triggered, performing data migration based on the heat of each block.
The preset statistical rules can be set and adjusted according to actual needs, and it can be understood that the heat of a block can be increased when data in the block is frequently utilized, and conversely, the heat of the block can be decreased when the data in the block is rarely utilized.
Further, the applicant considers that, for large IO and sequential IO, the performance of the large IO and sequential IO is not particularly different between the conventional hard disk and the high performance hard disks such as SSD and Optane SSD, so that when performing the thermal statistics, the large IO and sequential IO can be considered to be excluded, thereby further improving the host performance. That is, in one embodiment of the present invention, the statistical rule may include:
for each block of each physical device, when any sector in the block is requested to be accessed by an IO (input/output) request, and the accessed IO request does not belong to a predefined large IO or a sequential IO, increasing the heat of the block;
and aiming at each block of each physical device, reducing the heat of the block at intervals of a preset first time length.
After the heat of each block is determined and updated according to the preset statistical rule, when the preset data migration condition is triggered, data migration can be carried out based on the heat of each block.
The preset data migration condition may be generally set to periodically perform data migration. Further, in a specific embodiment of the present invention, a user may also be supported to manually trigger data migration, which improves the flexibility of the scheme of the present application. That is, the data migration conditions may specifically include:
and at every preset second time, or receiving a migration instruction sent by a user.
And it should be emphasized that, the received migration instruction sent by the user may also include two forms, one is that the migration instruction sent by the user may not carry specific migration requirements, that is, it plays a role of a trigger condition, and after the migration instruction sent by the user is received, data migration is performed immediately according to the heat of each current block. And the other is that the migration instruction carries a specific migration requirement, which is more common, and a user can specify that a certain block of data is migrated to a high-performance storage device or a low-performance storage device.
By applying the technical scheme provided by the embodiment of the invention, the host at least comprises one Optane SSD, so that the scheme of the application can utilize the high performance of the Optane SSD to store the hot data. Further, the virtual block device is created, an initial mapping relationship between each block address of the virtual block device and the block address of the corresponding physical device is established, and the mapping relationship is adjusted according to an execution object of data migration each time data migration is performed. Therefore, when the scheme of the application performs data migration, the operation of the virtual block device is not affected, that is, IO does not need to be interrupted, and the situation that in the conventional scheme, because IO needs to be interrupted in the data migration process, other applications are easily affected or the integrity of data is damaged does not occur. Therefore, when the high performance of Optane SSD is effectively utilized to store hot data, the influence of frequent migration of data on IO can be reduced.
Corresponding to the above method embodiment, an embodiment of the present invention further provides a system for processing an IO request, which can be referred to in correspondence with the above.
Referring to fig. 2, a schematic structural diagram of an IO request processing system applied to a host including at least one optanesd includes:
a virtual block device creating module 201, configured to create a virtual block device, and create an initial mapping relationship between each block address of the virtual block device and a block address of a corresponding physical device;
a mapping relation adjusting module 202, configured to adjust a mapping relation according to an execution object of data migration each time data migration is performed;
the IO request execution module 203 is configured to, when an IO request sent by a user is received, determine address information of data in corresponding physical equipment, to which the IO request is directed, according to the IO request and a current mapping relationship, and execute the IO request based on the address information.
In a specific embodiment of the present invention, the virtual block device creating module 201 is specifically configured to:
registering the management equipment;
after receiving a virtual block device creation request of a management device, creating virtual block devices, and establishing an initial mapping relationship between each block address of the virtual block devices and a block address of a corresponding physical device.
In a specific embodiment of the present invention, before creating the virtual block device, the virtual block device creating module 201 is further configured to:
and receiving the verification parameters of the preset items, and creating the virtual block equipment after all the verification parameters are verified.
In a specific embodiment of the present invention, the operation of the mapping relation adjusting module 202 for data migration includes:
determining and updating the heat of each block of each physical device according to a preset statistical rule;
and when a preset data migration condition is triggered, performing data migration based on the heat of each block.
In one embodiment of the present invention, the statistical rule comprises:
for each block of each physical device, when any sector in the block is requested to be accessed by an IO (input/output) request, and the accessed IO request does not belong to a predefined large IO or a sequential IO, increasing the heat of the block;
and aiming at each block of each physical device, reducing the heat of the block at intervals of a preset first time length.
In one embodiment of the present invention, the data migration condition includes:
and at every preset second time, or receiving a migration instruction sent by a user.
Corresponding to the above method and system embodiments, the present invention further provides an IO request processing device and a computer readable storage medium, where a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the IO request processing method in any of the above embodiments may be implemented. A computer-readable storage medium as referred to herein may include Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Referring to fig. 3, a schematic structural diagram of an IO request processing device is shown, including:
a memory 301 for storing a computer program;
a processor 302 for executing a computer program to implement the steps of the method for processing an IO request in any of the above embodiments.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. The principle and the implementation of the present invention are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. An IO request processing method is applied to a host including at least one Optane SSD, and includes:
creating virtual block equipment, and establishing an initial mapping relation between each block address of the virtual block equipment and the block address of corresponding physical equipment;
adjusting the mapping relation according to an execution object of data migration when data migration is carried out;
when an IO request sent by a user is received, determining address information of data corresponding to the IO request in corresponding physical equipment according to the IO request and a current mapping relation, and executing the IO request based on the address information.
2. The method for processing an IO request according to claim 1, wherein the creating a virtual block device includes:
registering the management equipment;
and after receiving a virtual block device creation request sent by the management device, creating a virtual block device.
3. The IO request processing method according to claim 2, further comprising, before the creating a virtual block device:
and receiving the verification parameters of the preset items, and creating the virtual block equipment after all the verification parameters are verified.
4. The method for processing the IO request according to claim 1, wherein the performing the data migration operation includes:
determining and updating the heat of each block of each physical device according to a preset statistical rule;
and when a preset data migration condition is triggered, performing data migration based on the heat of each block.
5. The IO request processing method according to claim 4, wherein the statistical rule includes:
for each block of each physical device, when any sector in the block is requested to be accessed by an IO (input/output) request, and the accessed IO request does not belong to a predefined large IO or a sequential IO, increasing the heat of the block;
and aiming at each block of each physical device, reducing the heat of the block at intervals of a preset first time length.
6. The IO request processing method according to claim 4, wherein the data migration condition includes:
and at every preset second time, or receiving a migration instruction sent by a user.
7. An IO request processing system applied to a host including at least one Optane SSD, comprising:
the virtual block device creating module is used for creating virtual block devices and establishing an initial mapping relation between each block address of the virtual block devices and the block address of the corresponding physical device;
a mapping relation adjusting module, configured to adjust the mapping relation according to an execution object of data migration each time data migration is performed;
and the IO request execution module is used for determining the address information of the data corresponding to the IO request in the corresponding physical equipment according to the IO request and the current mapping relation when the IO request sent by the user is received, and executing the IO request based on the address information.
8. The IO request processing system according to claim 7, wherein the virtual block device creating module is specifically configured to:
registering the management equipment;
after receiving a virtual block device creation request of the management device, creating a virtual block device, and establishing an initial mapping relationship between each block address of the virtual block device and a block address of a corresponding physical device.
9. An IO request processing device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the method of processing an IO request according to any one of claims 1 to 6.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of processing an IO request according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010105640.4A CN111399767A (en) | 2020-02-21 | 2020-02-21 | IO request processing method, system, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010105640.4A CN111399767A (en) | 2020-02-21 | 2020-02-21 | IO request processing method, system, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111399767A true CN111399767A (en) | 2020-07-10 |
Family
ID=71430361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010105640.4A Pending CN111399767A (en) | 2020-02-21 | 2020-02-21 | IO request processing method, system, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111399767A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463047A (en) * | 2020-11-25 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Data migration and recovery method and related device |
| CN115048227A (en) * | 2022-08-15 | 2022-09-13 | 阿里巴巴（中国）有限公司 | Data processing method, system and storage medium |
| CN116027990A (en) * | 2023-03-29 | 2023-04-28 | 苏州浪潮智能科技有限公司 | RAID card, data access method, system and storage medium thereof |
| WO2023165567A1 (en) * | 2022-03-03 | 2023-09-07 | 中兴通讯股份有限公司 | File processing method, electronic device, and computer storage medium |
| CN118093100A (en) * | 2024-04-17 | 2024-05-28 | 苏州元脑智能科技有限公司 | Semi-virtual block equipment control method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104486110A (en) * | 2014-12-09 | 2015-04-01 | 上海爱数软件有限公司 | Unified storage resource management method based on RPC (remote procedure call) protocols |
| CN107678981A (en) * | 2017-08-24 | 2018-02-09 | 北京盛和大地数据科技有限公司 | Data processing method and device |
-
2020
- 2020-02-21 CN CN202010105640.4A patent/CN111399767A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104486110A (en) * | 2014-12-09 | 2015-04-01 | 上海爱数软件有限公司 | Unified storage resource management method based on RPC (remote procedure call) protocols |
| CN107678981A (en) * | 2017-08-24 | 2018-02-09 | 北京盛和大地数据科技有限公司 | Data processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 工程师3: "基于Linux 的两种分层存储实现方案", 《电子发烧友》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463047A (en) * | 2020-11-25 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Data migration and recovery method and related device |
| WO2023165567A1 (en) * | 2022-03-03 | 2023-09-07 | 中兴通讯股份有限公司 | File processing method, electronic device, and computer storage medium |
| CN115048227A (en) * | 2022-08-15 | 2022-09-13 | 阿里巴巴（中国）有限公司 | Data processing method, system and storage medium |
| CN116027990A (en) * | 2023-03-29 | 2023-04-28 | 苏州浪潮智能科技有限公司 | RAID card, data access method, system and storage medium thereof |
| CN118093100A (en) * | 2024-04-17 | 2024-05-28 | 苏州元脑智能科技有限公司 | Semi-virtual block equipment control method and device, electronic equipment and storage medium |
| CN118093100B (en) * | 2024-04-17 | 2024-08-02 | 苏州元脑智能科技有限公司 | Semi-virtual block equipment control method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111399767A (en) | IO request processing method, system, equipment and computer readable storage medium | |
| US9507732B1 (en) | System and method for cache management | |
| US9996542B2 (en) | Cache management in a computerized system | |
| US9547591B1 (en) | System and method for cache management | |
| US9798655B2 (en) | Managing a cache on storage devices supporting compression | |
| US9128855B1 (en) | Flash cache partitioning | |
| TWI494761B (en) | Method of partitioning physical block and memory system thereof | |
| WO2021218038A1 (en) | Storage system, memory management method, and management node | |
| JP2017123177A (en) | Method and storage device for collecting garbage data | |
| CN109344094B (en) | Address mapping relation feedback method, device and equipment and readable storage medium | |
| WO2017148242A1 (en) | Method for accessing shingled magnetic recording (smr) hard disk, and server | |
| US11436159B2 (en) | Using multi-tiered cache to satisfy input/output requests | |
| CN115686385B (en) | Data storage method and device, computer equipment and storage medium | |
| US20100082904A1 (en) | Apparatus and method to harden computer system | |
| EP4012547B1 (en) | Storage method and apparatus for key value (kv) and storage device | |
| CN109240943B (en) | Address mapping relation feedback method, device and equipment and readable storage medium | |
| CN110502188A (en) | A kind of date storage method and device based on data base read-write performance | |
| US11861175B2 (en) | Latency in data storage systems | |
| US8930626B1 (en) | Cache management system and method | |
| CN111639037A (en) | Dynamic cache allocation method and device and DRAM-Less solid state disk | |
| CN112346658B (en) | Improving data heat trace resolution in a storage device having a cache architecture | |
| JP2013210846A (en) | Storage control device, storage system and storage control method | |
| KR102589609B1 (en) | Snapshot management in partitioned storage | |
| US20170052899A1 (en) | Buffer cache device method for managing the same and applying system thereof | |
| US11662949B2 (en) | Storage server, a method of operating the same storage server and a data center including the same storage server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200710 |