CN116048412A - Method, device and medium for processing disk read-write request - Google Patents

Method, device and medium for processing disk read-write request Download PDF

Info

Publication number
CN116048412A
CN116048412A CN202310076375.5A CN202310076375A CN116048412A CN 116048412 A CN116048412 A CN 116048412A CN 202310076375 A CN202310076375 A CN 202310076375A CN 116048412 A CN116048412 A CN 116048412A
Authority
CN
China
Prior art keywords
read
write
request
disk
pool
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310076375.5A
Other languages
Chinese (zh)
Inventor
张克涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jinan Inspur Data Technology Co Ltd
Original Assignee
Jinan Inspur Data Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jinan Inspur Data Technology Co Ltd filed Critical Jinan Inspur Data Technology Co Ltd
Priority to CN202310076375.5A priority Critical patent/CN116048412A/en
Publication of CN116048412A publication Critical patent/CN116048412A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • G06F3/0611Improving I/O performance in relation to response time
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to the field of cloud storage, and discloses a disk read-write request processing method, a device and a medium, wherein the method comprises the following steps: judging whether each IO pool meets preset conditions so as to facilitate the merging and processing of a plurality of IO read-write requests, wherein the IO pools are used for storing IO read-write requests acquired by a system according to IO information distribution. If the preset condition is met, determining an IO basic management unit corresponding to the IO pool so as to facilitate data interaction with the disk through the IO basic management unit, merging IO read-write requests in the IO pool to obtain a merged IO request, and operating the IO basic management unit according to the merged IO request. According to the method and the device, the IO read-write requests are stored in different IO pools in advance according to the IO information, and when the IO pools meet the preset conditions, the IO read-write requests are combined and processed, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and the disk data read-write speed and the working efficiency of the cloud storage system are improved.

Description

Method, device and medium for processing disk read-write request
Technical Field
The present invention relates to the field of cloud storage, and in particular, to a method, an apparatus, and a medium for processing a disk read-write request.
Background
Cloud storage is a mode of online storage on-line, i.e., storing data in large data centers typically provided by third party hosting companies. The data center operator prepares the virtualized resources at the back end according to the requirements of the client and provides the virtualized resources in a storage resource pool mode, and the client can use the storage resource pool to store files or objects. In practice, these resources may be distributed across numerous server hosts. When a user needs to read data or write data into the disk, the cloud storage platform calls a data read-write interface provided by the system and sends a write request to the disk so as to write the data into the disk.
At present, a bottom storage engine (Bluestone) is commonly used for managing read-write requests of a disk, the Bluestone manages read-write Data of the disk through an IO basic management unit (OBJ), and FIG. 1 is a schematic diagram of an OBJ unit provided in an embodiment of the present application, and as shown in FIG. 1, one OBJ unit includes a Data portion for storing Data and a Meta portion for storing Data information. When reading and writing data from and to the magnetic disk, it is necessary to lock the OBJ unit and write the data into the OBJ unit. The cloud storage system needs to provide data storage service for a large number of clients, so that a large number of data read-write requests exist in the system, and the system needs to frequently call a data read-write interface and read-write disk data, so that a large amount of time is wasted, and the working efficiency of the cloud storage system is affected.
Therefore, how to provide a method for processing a disk read-write request to improve the processing efficiency of the disk read-write request in the cloud storage system is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The purpose of the application is to provide a method, a device and a medium for processing a disk read-write request, so that the disk data read-write speed in a cloud storage system is improved, and the working efficiency of the cloud storage system is improved.
In order to solve the above technical problems, the present application provides a method for processing a disk read-write request, including:
judging whether each IO pool meets preset conditions, wherein the IO pool is a resource pool for storing IO read-write requests acquired by a system according to IO information distribution;
if the preset condition is met, determining an IO basic management unit corresponding to the IO pool;
merging IO read-write requests in the IO pool to obtain a merged IO request;
and operating the IO basic management unit according to the combined IO request.
Preferably, the determining whether each IO pool meets a preset condition includes:
judging whether the number of IO read-write requests in the IO pool is larger than a number threshold;
or whether the IO pool satisfies an IO merge period.
Preferably, when each of the IO read-write requests is a write IO, the merging the IO read-write requests in the IO pool to obtain a merged IO request includes:
judging whether the disk area corresponding to each IO read-write request is a continuous area or not;
and if the IO read-write request is the continuous area, merging the continuous IO read-write requests into an IO write request.
Preferably, when each of the IO read-write requests is a read IO, the merging the IO read-write requests in the IO pool to obtain a merged IO request includes:
judging whether a disk area corresponding to the IO read-write request is an overlapping area or not;
if the IO read-write request is an overlapping area, converting each IO read-write request into an independent read request;
if the disk area is not the overlapping area, judging whether the disk area corresponding to the IO read-write request is a continuous area or not;
and if the IO read-write request is the continuous area, merging the continuous IO read-write requests into an IO read request.
Preferably, the IO information includes: IO read-write attribute information and an OBJ identifier of the IO read-write request;
after the step of distributing the IO read-write request to the IO pool according to the IO information, the method further includes:
and ordering the IO read-write requests according to the disk areas corresponding to the IO read-write requests.
Preferably, the operating the IO basic management unit according to the merged IO request includes:
locking the IO basic management unit and determining Meta data corresponding to the IO read-write request in the IO basic management unit;
and carrying out data exchange with a disk according to the combined IO request and the Meta data information.
Preferably, when the IO read-write request length is a lower case IO request:
determining each memory block corresponding to each lower case IO request;
generating a cache area according to the disk data of each memory block and the lower case IO request;
updating the disk data of each memory block and the Meta data of the IO basic management unit according to the cache region;
the lower case IO request is a request that a disk area corresponding to the IO read-write request is not an integer multiple of the memory block, and the memory block is a minimum read-write unit of the IO basic management unit.
In order to solve the above technical problem, the present application further provides a disk read-write request processing device, including:
the judging module is used for judging whether each IO pool meets preset conditions or not, wherein the IO pool is a resource pool for storing IO read-write requests acquired by a system distributed according to IO information;
the determining module is used for determining an IO basic management unit corresponding to the IO pool if the preset condition is met;
the merging module is used for merging the IO read-write requests in the IO pool to obtain a merged IO request;
and the operation module is used for operating the IO basic management unit according to the combined IO request.
In order to solve the technical problem, the application also provides a disk read-write request processing device, which comprises a memory for storing a computer program;
and the processor is used for realizing the steps of the disk read-write request processing method when executing the computer program.
In order to solve the above technical problem, the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the steps of the method for processing a disc read/write request.
The application provides a disk read-write request processing method, which comprises the following steps: judging whether each IO pool meets preset conditions so as to facilitate the merging of a plurality of IO read-write requests, wherein the IO pool is a resource pool for storing the IO read-write requests acquired by a system according to IO information distribution. If the preset condition is met, determining an IO basic management unit corresponding to the IO pool so as to facilitate data interaction with the disk through the IO basic management unit, merging IO read-write requests in the IO pool to obtain a merged IO request, and operating the IO basic management unit according to the merged IO request. Therefore, according to the technical scheme provided by the application, the IO read-write requests are stored in different IO pools in advance according to the IO information, and the IO read-write requests in the IO pools are combined and processed when the IO pools meet the preset conditions, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and the disk data read-write speed and the working efficiency of the cloud storage system are further improved.
In addition, the application also provides a disk read-write request processing device and a medium, comprising: judging whether each IO pool meets preset conditions so as to facilitate the merging of a plurality of IO read-write requests, wherein the IO pool is a resource pool for storing the IO read-write requests acquired by a system according to IO information distribution. If the preset condition is met, determining an IO basic management unit corresponding to the IO pool so as to facilitate data interaction with the disk through the IO basic management unit, merging IO read-write requests in the IO pool to obtain a merged IO request, and operating the IO basic management unit according to the merged IO request. Therefore, according to the technical scheme provided by the application, the IO read-write requests are stored in different IO pools in advance according to the IO information, and the IO read-write requests in the IO pools are combined and processed when the IO pools meet the preset conditions, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and the disk data read-write speed and the working efficiency of the cloud storage system are further improved.
Drawings
For a clearer description of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an OBJ unit according to an embodiment of the present disclosure;
FIG. 2 is a flowchart of a method for processing a disk read/write request according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a write IO merging method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a read/IO merging method according to an embodiment of the present disclosure;
FIG. 5 is a flowchart of a method for processing a lower case IO request according to an embodiment of the present disclosure;
FIG. 6 is a block diagram of a disk read/write request processing apparatus according to an embodiment of the present application;
fig. 7 is a block diagram of another disc read/write request processing apparatus according to an embodiment of the present application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments herein without making any inventive effort are intended to fall within the scope of the present application.
The core of the application is to provide a method, a device and a medium for processing a disk read-write request, so that the disk data read-write speed in a cloud storage system is improved, and the working efficiency of the cloud storage system is improved.
For easy understanding, a hardware architecture to which the technical solution of the present invention is applicable will be described below. In specific implementation, when the cloud storage server receives an IO read-write request sent by a user, the IO read-write request is distributed to different IO pools for storage according to attribute information of the IO read-write request and an IO basic management unit identifier. When the IO pool meets the preset requirement (for example, the number of IO requests in the IO pool is larger than a number threshold or the time interval between the IO requests and the operation of executing the merging IO with the last IO pool is larger than a time threshold), merging IO read-write requests in the IO pool to generate a merging IO request, and realizing data interaction with a disk according to the merging IO request. As shown in fig. 1, one OBJ unit includes a Data portion for storing Data and a Meta portion for storing Data information. When a large number of IO read-write requests exist in the cloud storage system, the system frequently executes the steps of calling the disk interface and performing data interaction with the disk, wherein the data interaction with the disk takes a large amount of time, so that the working efficiency of the cloud storage system is reduced. In order to solve the problem, the application provides a disk read-write request processing method, which is characterized in that IO read-write requests are stored in different IO pools in advance according to IO information, and when the IO pools meet preset conditions, the IO read-write requests are combined and processed, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and further the disk data read-write speed and the working efficiency of a cloud storage system are improved.
In order to provide a better understanding of the present application, those skilled in the art will now make further details of the present application with reference to the drawings and detailed description.
Fig. 2 is a flowchart of a method for processing a disc read/write request according to an embodiment of the present application, where, as shown in fig. 2, the method includes:
s10: judging whether each IO pool meets preset conditions, wherein the IO pool is a resource pool for storing IO read-write requests acquired by a system distributed according to IO information.
The preset condition may be whether the number of the IO read-write requests in the IO pool is greater than a threshold, whether the existence time of the IO read-write requests in the IO pool is greater than a threshold, and the like, which is not limited herein. It should be noted that, according to the technical scheme provided by the application, the plurality of IO read-write requests are combined, so that the number of times of data interaction with the locking IO basic management unit and the disk in the execution process of the IO read-write requests is reduced, and the working efficiency of the cloud storage system is improved. To achieve this, the number of merged IO read and write requests is particularly important. When the number of IO read-write requests executed in a merging way is too small, the aim of reducing the number of data interaction with the disk and thus reducing IO processing time may not be achieved. When there are many IO read-write requests, the waiting time of the IO read-write requests entering the IO pool in the early stage is too long, so that the waiting time of the user is too long, and the use experience of the user is affected. Therefore, the number of IO requests of the IO pool or the limiting time needs to be reasonably set according to the service pressure of the cloud storage system. For example: when the service pressure of the cloud storage system is too low, the preset condition of the IO pool can be properly reduced, so that the IO read-write request merging operation is easier to trigger.
S11: if the preset condition is met, determining an IO basic management unit corresponding to the IO pool.
When the IO pool meets the preset condition, processing the IO read-write request in the IO pool is started. Because the IO pools are allocated according to IO read-write attributes and the OBJ identifier of the IO read-write request when the IO pools are allocated for the IO read-write request, the IO basic management units corresponding to the IO read-write requests in each IO pool are identical. Therefore, firstly, an IO basic management unit corresponding to the IO pool is determined, a corresponding OBJ is obtained, and the OBJ is locked to avoid the fault of the cloud storage system caused by concurrent reading and writing.
It should be noted that, in order to prevent data uncertainty generated during read-write, only read IOs or only write IOs exist in each IO pool.
S12: and merging the IO read-write requests in the IO pool to obtain a merged IO request.
S13: and operating the IO basic management unit according to the combined IO request.
In specific implementation, the IO read-write requests in the IO pool meeting the preset conditions are combined to generate a combined IO request, and the IO basic management unit is operated according to the combined IO request. The IO processing flow provided by the application comprises the following steps: receiving an IO read-write request sent by a user; merging IO read-write requests under the same OBJ; and reading and writing Data of the OBJ unit and reading and writing Meta Data according to the combined IO request.
It should be noted that, the operation of merging the IO read-write requests in the IO pool to obtain the merged IO request is only an operation process of the partial IO read-write requests meeting the requirements (for example, the disk area corresponding to the IO read-write requests is a continuous area), and the IO read-write requests not meeting the requirements are not merged. Therefore, the combined IO request not only comprises the combined IO read-write request, but also comprises the original IO read-write request which does not meet the requirement and cannot be combined.
Further, in order to prevent data read-write collision, in the process of merging IO read-write requests, the read IO request and the write IO request need to be merged respectively.
The embodiment provides a disk read-write request processing method, which comprises the following steps: judging whether each IO pool meets preset conditions so as to facilitate the merging of a plurality of IO read-write requests, wherein the IO pool is a resource pool for storing the IO read-write requests acquired by a system according to IO information distribution. If the preset condition is met, determining an IO basic management unit corresponding to the IO pool so as to facilitate data interaction with the disk through the IO basic management unit, merging IO read-write requests in the IO pool to obtain a merged IO request, and operating the IO basic management unit according to the merged IO request. Therefore, according to the technical scheme provided by the application, the IO read-write requests are stored in different IO pools in advance according to the IO information, and the IO read-write requests in the IO pools are combined and processed when the IO pools meet the preset conditions, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and the disk data read-write speed and the working efficiency of the cloud storage system are further improved.
In a specific implementation, the selection of the preset condition directly affects the number of IO read-write requests for executing the merging operation once, if the implementation of the preset condition is too simple, the number of IO read-write requests for executing the merging operation once may be too small, so that the time for establishing a data read-write channel with a disk cannot be saved; if the implementation of the preset condition is too difficult, a large number of IO read-write requests are accumulated in the IO pool, so that the response efficiency of the cloud storage system is too low, and the use experience of a user is affected.
On the basis of the above embodiment, determining whether each IO pool meets the preset condition includes:
judging whether the number of IO read-write requests in the IO pool is larger than a number threshold; or whether the IO pool meets the IO merging period; or there is an IO read-write request to access the same area of the disk.
It should be noted that the number threshold and the merging period may be preset values, or may be values determined by the system according to the service pressure of the system, which are not limited herein.
In particular implementations, to prevent data read-write collisions due to data uncertainty, IO read requests and IO write requests need to be handled separately.
As a preferred embodiment, when each IO read-write request is a write IO, merging the IO read-write requests in the IO pool to obtain a merged IO request includes: judging whether the disk area corresponding to each IO read-write request is a continuous area or not; if the continuous area is the continuous area, the continuous IO read-write request is merged into the IO write request.
In a specific implementation, if each IO read request in the IO pool is an IO write request, traversing all IO requests in the IO pool. Fig. 3 is a schematic diagram of a write IO merging manner provided in the embodiment of the present application, as shown in fig. 3, merging continuous IO read-write requests of a corresponding disk area into an IO write request, and generating different disk write requests according to discontinuous IO requests. And updating the respective Meta data according to the original IO read-write request. And after the completion, writing all Data into the disk Data area according to the Meta Data. And finally, writing the updated latest Meta data into a disk Meta area, thereby completing all write IO requests in the pool.
As a preferred embodiment, when each IO read-write request is a read IO, merging the IO read-write requests in the IO pool to obtain a merged IO request includes: judging whether a disk area corresponding to the IO read-write request is an overlapping area or not; if the IO read-write request is an overlapping area, converting each IO read-write request into an independent read request; if the disk area is not the overlapping area, judging whether the disk area corresponding to the IO read-write request is a continuous area or not; if the continuous area is the continuous area, the continuous IO read-write request is merged into the IO read request. Fig. 4 is a schematic diagram of a read/write manner provided in the embodiment of the present application, as shown in fig. 4, for the IO read/write requests in the IO pool, the IO read/write requests with continuous read positions are first merged, similar to the merging of write IOs, and after the continuous IO read/write requests are merged into 1 independent continuous IO read request, a single read request is issued to the disk. Or converting the read requests with overlapped read positions into a single read request, and finally issuing the combined IO read request to a disk to read data, and respectively acquiring parts of the respective requests from the read data according to the original request positions and the lengths after the reading is finished, so as to finish the IO read-write request.
As a preferred embodiment, the IO information includes: IO read-write attribute information and an OBJ identifier of an IO read-write request; in addition, the IO read-write request can be divided according to the disk area corresponding to the IO read-write request. Further, after the step of distributing the IO read-write request to the IO pool according to the IO information, the method further includes: and ordering the IO read-write requests according to the disk areas corresponding to the IO read-write requests.
After receiving the IO read-write request, distributing the request to a corresponding IO pool according to the IO read-write attribute and the OBJ identification information, and sequencing the IO read-write request according to the position of the request. All IO read-write requests are stored in a fixed order so that a plurality of continuous IO read-write requests are combined into one combined IO request for processing.
It may be understood that the IO read-write requests may be ordered according to the sequential data entering the IO pool, or may be ordered according to the disk space corresponding to the IO read-write requests, which is not limited herein.
As a preferred embodiment, operating the IO basic management unit in accordance with the consolidated IO request includes: locking the IO basic management unit and determining Meta data corresponding to the IO read-write request in the IO basic management unit; and carrying out data exchange with the disk according to the combined IO request and the Meta data information.
Further, since Bluestone must read and write to the underlying disk, it is in units of buffers (blocks). So each time a Bluestone issues an IO read-write request to disk, its read-write location and length must be an integer multiple of Block. When the IO read-write request meets the integral multiple of the Block size, the IO read-write request is called as a high-write IO request, otherwise, the IO read-write request is called as a low-write IO request.
In the prior art, a read-before-write scheme is adopted for processing the lower case IO request, data in a Data area of each IO basic management unit is read into a memory, then old Data and new Data which are requested to be written are replaced and combined in the memory, and finally updated whole Data are written into a disk. However, the proposal also has the problem of excessive times of establishing the read-write channel of the magnetic disk. To solve this problem, on the basis of the above embodiment, when the IO read-write request length is a lower-case IO request: determining each memory block corresponding to each lower case IO request; generating a buffer area according to the disk data of each memory block and the lower case IO request; updating the disk data of each memory block and the Meta data of the IO basic management unit according to the cache region; the lower-case IO request is a request that a disk area corresponding to the IO read-write request is not an integer multiple of a memory block, and the memory block is a minimum read-write unit of the IO basic management unit. Based on the embodiment, the lower case IO request is processed in a multiparty IO splicing mode. Fig. 5 is a flowchart of a method for processing a lower case IO request according to an embodiment of the present application. As shown in fig. 5, taking a Block size of 4k as an example, two lower case IO requests, a merging process of lower case IOs is shown. Traversing all blocks related to lowercase IO, and firstly acquiring data on a disk of the Block through read IO. After the completion, a buffer with the size of Block is constructed, and then corresponding data is filled in different positions of the buffer from the starting position of the Block according to the IO request. And then updating Meta data of the corresponding OBJ in a disk issued by the buffer data, and finishing the IO of the current merging.
In the foregoing embodiments, a method for processing a disc read/write request is described in detail, and the present application further provides a corresponding embodiment of a disc read/write request processing apparatus. It should be noted that the present application describes an embodiment of the device portion from two angles, one based on the angle of the functional module and the other based on the angle of the hardware.
Fig. 6 is a block diagram of a disk read-write request processing apparatus according to an embodiment of the present application, as shown in fig. 6, where the apparatus includes:
the judging module 10 is configured to judge whether each IO pool meets a preset condition, where the IO pool is a resource pool for storing an IO read-write request acquired by a system according to IO information allocation;
the determining module 11 is configured to determine an IO basic management unit corresponding to the IO pool if a preset condition is met;
the merging module 12 is configured to merge the IO read-write requests in the IO pool to obtain a merged IO request;
and the operation module 13 is used for operating the IO basic management unit according to the combined IO request.
Since the embodiments of the apparatus portion and the embodiments of the method portion correspond to each other, the embodiments of the apparatus portion are referred to the description of the embodiments of the method portion, and are not repeated herein.
The embodiment provides a disk read-write request processing device, which comprises: judging whether each IO pool meets preset conditions so as to facilitate the merging of a plurality of IO read-write requests, wherein the IO pool is a resource pool for storing the IO read-write requests acquired by a system according to IO information distribution. If the preset condition is met, determining an IO basic management unit corresponding to the IO pool so as to facilitate data interaction with the disk through the IO basic management unit, merging IO read-write requests in the IO pool to obtain a merged IO request, and operating the IO basic management unit according to the merged IO request. Therefore, according to the technical scheme provided by the application, the IO read-write requests are stored in different IO pools in advance according to the IO information, and the IO read-write requests in the IO pools are combined and processed when the IO pools meet the preset conditions, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and the disk data read-write speed and the working efficiency of the cloud storage system are further improved.
Fig. 7 is a block diagram of another disc read/write request processing apparatus according to an embodiment of the present application, and as shown in fig. 7, the disc read/write request processing apparatus includes: a memory 20 for storing a computer program;
the processor 21 is configured to implement the steps of the disk read/write request processing method according to the above embodiment when executing the computer program.
The smart device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.
Processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in hardware in at least one of a digital signal processor (Digital Signal Processor, DSP), a Field programmable gate array (Field-Programmable Gate Array, FPGA), a programmable logic array (Programmable Logic Array, PLA). The processor 21 may also comprise a main processor, which is a processor for processing data in an awake state, also called central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with an image processor (Graphics Processing Unit, GPU) for taking care of rendering and rendering of the content that the display screen is required to display. In some embodiments, the processor 21 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.
Memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing a computer program 201, where the computer program, when loaded and executed by the processor 21, can implement the relevant steps of the disk read/write request processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may further include an operating system 202, data 203, and the like, where the storage manner may be transient storage or permanent storage. The operating system 202 may include Windows, unix, linux, among others. The data 203 may include, but is not limited to, IO pools, IO read-write requests, and the like.
In some embodiments, the disk read-write request processing device may further include a display 22, an input-output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
It will be appreciated by those skilled in the art that the structure shown in FIG. 7 is not limiting of the disk read/write request processing apparatus and may include more or fewer components than shown.
The disk read-write request processing device provided by the embodiment of the application comprises a memory and a processor, wherein the processor can realize the following method when executing a program stored in the memory:
judging whether each IO pool meets preset conditions, wherein the IO pool is a resource pool for storing IO read-write requests acquired by a system distributed according to IO information;
if the preset condition is met, determining an IO basic management unit corresponding to the IO pool;
merging IO read-write requests in the IO pool to obtain a merged IO request;
and operating the IO basic management unit according to the combined IO request.
The embodiment provides a disk read-write request processing device, which comprises: judging whether each IO pool meets preset conditions so as to facilitate the merging of a plurality of IO read-write requests, wherein the IO pool is a resource pool for storing the IO read-write requests acquired by a system according to IO information distribution. If the preset condition is met, determining an IO basic management unit corresponding to the IO pool so as to facilitate data interaction with the disk through the IO basic management unit, merging IO read-write requests in the IO pool to obtain a merged IO request, and operating the IO basic management unit according to the merged IO request. Therefore, according to the technical scheme provided by the application, the IO read-write requests are stored in different IO pools in advance according to the IO information, and the IO read-write requests in the IO pools are combined and processed when the IO pools meet the preset conditions, so that the number of times of establishing a data read-write channel with a disk in the IO read-write process is reduced, and the disk data read-write speed and the working efficiency of the cloud storage system are further improved.
Finally, the present application also provides a corresponding embodiment of the computer readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps as described in the method embodiments above.
It will be appreciated that the methods of the above embodiments, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored on a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution contributing to the prior art, or in a software product stored in a storage medium, performing all or part of the steps of the methods of the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The method, the device and the medium for processing the disk read-write request provided by the application are described in detail. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present application, and such improvements and modifications fall within the scope of the claims of the present application.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

Claims (10)

1. A method for processing a disk read-write request, comprising:
judging whether each IO pool meets preset conditions, wherein the IO pool is a resource pool for storing IO read-write requests acquired by a system according to IO information distribution;
if the preset condition is met, determining an IO basic management unit corresponding to the IO pool;
merging IO read-write requests in the IO pool to obtain a merged IO request;
and operating the IO basic management unit according to the combined IO request.
2. The method for processing a disc read-write request according to claim 1, wherein said determining whether each of the IO pools satisfies a preset condition comprises:
judging whether the number of IO read-write requests in the IO pool is larger than a number threshold;
or whether the IO pool satisfies an IO merge period.
3. The method according to claim 2, wherein when each of the IO read/write requests is a write IO, the merging the IO read/write requests in the IO pool to obtain a merged IO request includes:
judging whether the disk area corresponding to each IO read-write request is a continuous area or not;
and if the IO read-write request is the continuous area, merging the continuous IO read-write requests into an IO write request.
4. The method according to claim 2, wherein when each of the IO read/write requests is a read IO, the merging the IO read/write requests in the IO pool to obtain a merged IO request includes:
judging whether a disk area corresponding to the IO read-write request is an overlapping area or not;
if the IO read-write request is an overlapping area, converting each IO read-write request into an independent read request;
if the disk area is not the overlapping area, judging whether the disk area corresponding to the IO read-write request is a continuous area or not;
and if the IO read-write request is the continuous area, merging the continuous IO read-write requests into an IO read request.
5. The disk read-write request processing method according to any one of claims 1 to 4, wherein the IO information includes: IO read-write attribute information and an OBJ identifier of the IO read-write request;
after the step of distributing the IO read-write request to the IO pool according to the IO information, the method further includes:
and ordering the IO read-write requests according to the disk areas corresponding to the IO read-write requests.
6. The method of claim 5, wherein the operating the IO basic management unit according to the merged IO request comprises:
locking the IO basic management unit and determining Meta data corresponding to the IO read-write request in the IO basic management unit;
and carrying out data exchange with a disk according to the combined IO request and the Meta data information.
7. The disk read-write request processing method according to claim 6, wherein when the IO read-write request length is a lower case IO request:
determining each memory block corresponding to each lower case IO request;
generating a cache area according to the disk data of each memory block and the lower case IO request;
updating the disk data of each memory block and the Meta data of the IO basic management unit according to the cache region;
the lower case IO request is a request that a disk area corresponding to the IO read-write request is not an integer multiple of the memory block, and the memory block is a minimum read-write unit of the IO basic management unit.
8. A disk read-write request processing apparatus, comprising:
the judging module is used for judging whether each IO pool meets preset conditions or not, wherein the IO pool is a resource pool for storing IO read-write requests acquired by a system distributed according to IO information;
the determining module is used for determining an IO basic management unit corresponding to the IO pool if the preset condition is met;
the merging module is used for merging the IO read-write requests in the IO pool to obtain a merged IO request;
and the operation module is used for operating the IO basic management unit according to the combined IO request.
9. A disk read-write request processing apparatus, comprising a memory for storing a computer program;
a processor for implementing the steps of the disk read-write request processing method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, which when executed by a processor, implements the steps of the disk read-write request processing method according to any one of claims 1 to 7.
CN202310076375.5A 2023-01-30 2023-01-30 Method, device and medium for processing disk read-write request Pending CN116048412A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310076375.5A CN116048412A (en) 2023-01-30 2023-01-30 Method, device and medium for processing disk read-write request

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310076375.5A CN116048412A (en) 2023-01-30 2023-01-30 Method, device and medium for processing disk read-write request

Publications (1)

Publication Number Publication Date
CN116048412A true CN116048412A (en) 2023-05-02

Family

ID=86127728

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310076375.5A Pending CN116048412A (en) 2023-01-30 2023-01-30 Method, device and medium for processing disk read-write request

Country Status (1)

Country Link
CN (1) CN116048412A (en)

Similar Documents

Publication Publication Date Title
EP3796150B1 (en) Storage volume creation method and apparatus, server, and storage medium
US20130311988A1 (en) Migrating virtual machines between networked computing environments based on resource utilization
EP3594798A1 (en) Data storage method and device
CN104461744A (en) Resource allocation method and device
US9971808B2 (en) Fast query processing in columnar databases with GPUs
WO2018054200A1 (en) Method and device for reading file
US20150112934A1 (en) Parallel scanners for log based replication
CN110223216B (en) Data processing method and device based on parallel PLB and computer storage medium
CN108475201B (en) Data acquisition method in virtual machine starting process and cloud computing system
CN114327946A (en) Shared memory access control method and device, electronic equipment and automatic driving vehicle
TW315449B (en) Graphics system
CN109684270A (en) Database filing method, apparatus, system, equipment and readable storage medium storing program for executing
CN114327917A (en) Memory management method, computing device and readable storage medium
CN113535087B (en) Data processing method, server and storage system in data migration process
US20150163111A1 (en) Managing resources in a distributed computing environment
US20220147260A1 (en) Method, device and computer program product for managing storage system
CN116483274A (en) Online migration method, device, equipment and medium for distributed block storage volume
CN110223215B (en) Control method and device of graphics rendering pipeline and computer storage medium
JP3844933B2 (en) Database server processing method
CN116048412A (en) Method, device and medium for processing disk read-write request
JP6851350B2 (en) Storage system and storage control method
US10423348B1 (en) Increasing serial read performance
CN110781137A (en) Directory reading method and device for distributed system, server and storage medium
CN115016735A (en) Control method, device and medium for distributed cache system
CN110659312B (en) Data processing method, device, equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination