CN117742616A - Small IO random read-write processing method, device and computer readable medium - Google Patents
Small IO random read-write processing method, device and computer readable medium Download PDFInfo
- Publication number
- CN117742616A CN117742616A CN202410015570.1A CN202410015570A CN117742616A CN 117742616 A CN117742616 A CN 117742616A CN 202410015570 A CN202410015570 A CN 202410015570A CN 117742616 A CN117742616 A CN 117742616A
- Authority
- CN
- China
- Prior art keywords
- small
- random read
- data
- write
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 11
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 34
- 230000002776 aggregation Effects 0.000 claims description 20
- 238000004220 aggregation Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 18
- 238000004806 packaging method and process Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The method realizes more efficient processing of small IO random read-write requests, a plurality of small IO random read-write requests are not packed into individual data packets to be transmitted separately between a client side and a server side of network block equipment, but the discrete small IO random read-write requests are aggregated into a data packet with larger data volume and transmitted to the server side at one time, and then the data packet is split and reconstructed by the server side and then is submitted to a back-end storage device to execute read-write operation, so that the data transmission efficiency is greatly improved, and the performance bottleneck problem of qemu network block equipment in a small IO random read-write scene is solved.
Description
Technical Field
The present disclosure relates to the field of information technologies, and in particular, to a method and an apparatus for random read-write processing of small IOs, and a computer readable medium.
Background
qemu (Quick Emulator) virtualization software is widely used in modern data centers as one of the most important underlying foundations for cloud computing. qemu provides a variety of block device drivers for emulating physical hard disk, optical drive, etc., storage devices and mapping them for use in virtual machines. The qemu block device has wide application scenes including data storage, data migration, data backup and recovery, virtual machine snapshot and the like in the virtual machine, so that the requirements of different application scenes are met, users are helped to conveniently manage and share data, and the efficiency and reliability of the virtualization environment are improved.
The qemu virtualization software is provided with various block device drivers, and different storage configurations are provided for users. The main standard for measuring the storage performance in the industry is the performance of large IO (input/output) sequential read-write and small IO random read-write, and for a large IO sequential read-write scene, qemu has excellent driving performance of self-contained block equipment, has extremely low virtualization overhead and is close to the large IO sequential read-write performance of the storage medium. For the small IO random read-write scene, the storage optimization difficulty is always the same, the qemu performance is not satisfactory in this block, especially, network block equipment, besides the virtualization cost, additional network transmission cost is added, and the small IO random read-write performance problem of the qemu network block equipment is more prominent due to the fact that the small IO random read-write performance problem is limited by the performance bottleneck of small data packet transmission per se across the network.
Disclosure of Invention
An object of the application is to provide a small IO random read-write processing method, device and computer readable medium of qemu network block device.
In order to achieve the above objective, the present application provides a method for processing random read-write of small IO, where the method is applied to a client of a network block device, and includes:
when a plurality of small IO random read-write requests need to be sent, request metadata about the plurality of small IO random read-write requests are constructed, wherein the request metadata comprise the number of the small IO random read-write requests which need to be sent at this time and parameter information of each small IO random read-write request;
combining the data blocks corresponding to the plurality of small IO random read-write requests and the metadata into aggregated data;
packaging the aggregated data, adding an aggregate IO mark, sending the data packet to a server side of the network block equipment through a network, enabling the server side to check whether the data packet carries the aggregate IO mark or not, acquiring the aggregated data in the data packet when determining that the data packet carries the aggregate IO mark, reconstructing each small IO random read-write request contained in the data packet according to the aggregated data, and sequentially sending the small IO random read-write requests to a rear-end storage equipment for processing.
Further, the parameter information includes a data offset value and a data length value of each small IO random read-write request.
Further, the small IO random read-write request is a random read-write request with the read-write data quantity smaller than a preset value.
Based on another aspect of the present application, another method for random read-write processing of small IOs is also provided, where the method is applied to a server side of a network block device, and includes:
receiving a data packet from a client of the network block device through a network;
checking whether an aggregation IO mark is carried in the data packet, if so, acquiring aggregation data in the data packet, wherein the aggregation data comprises data blocks corresponding to a plurality of small IO random read-write requests and request metadata, and the request metadata is constructed by the client when the client needs to send the small IO random read-write requests, and comprises the request number of the small IO random read-write requests to be sent at this time and parameter information of each small IO random read-write request;
reconstructing each small IO random read-write request contained in the data packet according to the aggregated data;
and sequentially sending the small IO random read-write request to the back-end storage equipment for processing.
Further, the method further comprises:
and if the aggregation IO mark is not carried, acquiring an IO read-write request contained in the data packet, and sending the IO read-write request to the back-end storage equipment for processing.
Further, reconstructing each small IO random read-write request contained in the data packet according to the aggregate data, including:
analyzing request metadata in the aggregated data, and acquiring the request number and parameter information of each small IO random read-write request contained in the data packet;
and reconstructing each small IO random read-write request according to the request number, the parameter information and the data block corresponding to each small IO random read-write request in the aggregated data.
Further, the parameter information includes a data offset value and a data length value of each small IO random read-write request.
Further, the small IO random read-write request is a random read-write request with the read-write data quantity smaller than a preset value.
Based on another aspect of the application, there is also provided a small IO random read-write processing device, the device including a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein the computer program instructions, when executed by the processor, trigger the device to execute the small IO random read-write processing method.
Embodiments of the present application also provide a computer readable medium having stored thereon computer program instructions executable by a processor to implement the small IO random read-write processing method.
Compared with the prior art, the small IO random read-write processing scheme is provided, more efficient processing of small IO random read-write requests is achieved between a client side and a server side of network block equipment, when a plurality of small IO random read-write requests are required to be sent, the client side builds request metadata related to the small IO random read-write requests, data blocks corresponding to the small IO random read-write requests and the metadata are combined into aggregated data, then the aggregated data are packaged and an aggregate IO mark is added, the data package is sent to the server side of the network block equipment through a network, so that the server side can acquire the aggregated data in the data package when detecting that the data package carries the aggregate IO mark, reconstruct all the small IO random read-write requests contained in the data package according to the aggregated data, and sequentially send the small IO random read-write requests to a back-end storage device for processing. Therefore, a plurality of small IO random read-write requests are not packed into individual data packets to be transmitted separately between a client side and a server side of the network block device, but are aggregated into a data packet with larger data quantity by discrete small IO random read-write requests, the data packet is transmitted to the server side at one time, and then the data packet is split and reconstructed by the server side and then is submitted to a back-end storage device to execute read-write operation, so that the data transmission efficiency is greatly improved, and the problem of performance bottleneck of qemu network block devices in a small IO random read-write scene is solved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings, in which:
FIG. 1 is a schematic diagram of a framework of qemu network block devices;
fig. 2 is a schematic architecture diagram of a small IO random read-write processing method according to an embodiment of the present application;
fig. 3 is a process flow of a client of the network block device in the embodiment of the present application when performing a small IO random read-write process;
fig. 4 is a process flow of a server side of a network block device in the embodiment of the present application when performing a small IO random read-write process;
FIG. 5 is a schematic diagram of the memory principle of the DMA technique;
the same or similar reference numbers in the drawings refer to the same or similar parts.
Detailed Description
The present application is described in further detail below with reference to the accompanying drawings.
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
In a typical configuration of the present application, the terminals, the devices of the services network each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer-readable media include both permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computing device.
The embodiment of the application provides a small IO random read-write processing method, which is used for realizing more efficient processing of small IO random read-write requests between a client side and a server side of network block equipment, constructing request metadata related to the small IO random read-write requests by the client side when the small IO random read-write requests are required to be sent, combining data blocks corresponding to the small IO random read-write requests and the metadata into aggregated data, packaging the aggregated data, adding an aggregated IO mark, sending a data packet to the server side of the network block equipment through a network, so that the server side can acquire the aggregated data in the data packet when checking that the data packet carries the aggregated IO mark, reconstructing each small IO random read-write request contained in the data packet according to the aggregated data, and sequentially sending the small IO random read-write requests to a back-end storage device for processing. Therefore, a plurality of small IO random read-write requests are not packed into individual data packets to be transmitted separately between a client side and a server side of the network block device, but are aggregated into a data packet with larger data quantity by discrete small IO random read-write requests, the data packet is transmitted to the server side at one time, and then the data packet is split and reconstructed by the server side and then is submitted to a back-end storage device to execute read-write operation, so that the data transmission efficiency is greatly improved, and the problem of performance bottleneck of qemu network block devices in a small IO random read-write scene is solved.
In a practical scenario, the specific implementation of the client and the server for executing the method may be a device formed by integrating user equipment, network equipment or user equipment and network equipment through a network, or may also be an application program running on the device. The user equipment comprises, but is not limited to, various terminal equipment such as computers, mobile phones, tablet computers and the like; the network device includes, but is not limited to, an implementation such as a network host, a single network server, a set of multiple network servers, or a set of computers based on cloud computing. Here, the Cloud is composed of a large number of hosts or web servers based on Cloud Computing (Cloud Computing), which is a kind of distributed Computing, one virtual computer composed of a group of loosely coupled computer sets.
Fig. 1 shows a schematic diagram of a qemu network block device, where nbd client is a client of the qemu network block device and nbd server is a server of the qemu network block device, and the two are in communication through a network (typically an ethernet network). When needing to initiate an IO request, the NBDclient packages the IO request into a data packet, then transmits the data packet to the NBDServer through a network, and after receiving the IO request, the Server end sends an IO read-write request to the storage equipment at the rear end of the Server end, and the storage equipment at the rear end performs reading or writing operation. For a plurality of IO requests which are sequentially read and written, as the related data blocks are physically continuous, the IO requests can be processed by being aggregated together through an vector mechanism, so that the IO read and write performance is greatly improved. However, for discrete IO random read-write requests, since the related data blocks are physically discontinuous, the offset (data offset value)/length (data length value) of each IO request corresponding to the target storage is dispersed, and cannot be aggregated to form a large IO data block to be processed together, and only can be packaged into individual small data packets to be transmitted separately, so that performance bottleneck of transmitting each small data packet by a network is limited, and the IO read-write performance is poor.
Fig. 2 is a schematic architecture diagram of a small IO random read-write processing method according to an embodiment of the present application, in which a new request type is introduced, where the request type is used to implement efficient processing on small IO random read-write. The scheme realizes the support of the new request type between the client side and the server side of the network block equipment, and for discrete small IO random read-write requests, the discrete small IO random read-write requests are not packed into individual small data packets to be transmitted to the server side separately, but are aggregated into a large data packet and then transmitted through the network once, so that the data transmission efficiency is greatly improved.
The processing flow in the client nbd client of the network block device is shown in fig. 3, and at least includes the following processing steps:
in step S301, when a plurality of small IO random read-write requests need to be sent, request metadata about the plurality of small IO random read-write requests is constructed. The small IO random read-write request refers to a random read-write request with a data amount of read-write smaller than a preset value, for example, in this embodiment, the preset value may be set to m, and the unit is KB, and then the random read-write request with a data amount of write-in or read-out smaller than mKB may be determined as the small IO random read-write request.
Taking the scenario of the embodiment of the present application as an example, if 3 small IO random read-write requests need to be sent at this time, i.e. the IO request 1, the IO request 2 and the IO request 3, respectively, then request metadata about the three small IO random read-write requests can be constructed. In this embodiment, the request metadata may be named as a scanner/gateway metadata, where the request metadata may at least include the number of small IO random read/write requests that need to be sent this time and parameter information of each small IO random read/write request. Taking the foregoing 3 small IO random read-write requests as an example, in the corresponding scanner/gateway metadata, the number of the small IO random read-write requests that need to be sent at this time is 3. The parameter information of each small IO random read/write request may include an offset (data offset value) and a length (data length value) corresponding to each small IO random read/write request, which can be used to determine a specific location in storage of a data block read or written by the small IO random read/write request.
In step S302, after the request metadata is built, the client may combine the data blocks corresponding to the plurality of small IO random read-write requests and the metadata into aggregated data. Taking 3 small IO random read-write requests as an example, the aggregate data is the data buffer in the figure, and because three data blocks corresponding to the 3 small IO random read-write requests are the data block 1, the data block 2 and the data block 3 respectively, the aggregate data as shown in fig. 2 can be obtained after the corresponding scanner/gateway metadata are combined with the three data blocks.
In step S303, the client packages the aggregate data and adds an aggregate IO flag, and sends the data package to the server side of the network block device through the network.
After the client packages the aggregate data, a data packet including a plurality of small IO random read-write requests may be obtained, where the aggregate IO flag may be represented by a specific flag bit in the data packet, for example, in this embodiment, a scan-other flag bit may be preset, where when a value corresponding to the scan-other flag bit is set to 1, it indicates that the aggregate IO flag exists, and otherwise, when a value corresponding to the scan-other flag bit is 0, it indicates that there is no aggregate IO flag. After the packaging of the aggregate data is completed, the data packet can be sent to the server side of the network block device through the network.
The processing flow in the server side NBDserver of the network block device is shown in fig. 4, and at least includes the following processing steps:
in step S401, the server receives a data packet from the client of the network block device through the network. The data packet is packed and generated by the client device, if the data packet carries an aggregation IO mark, the data packet indicates that the read-write processing is based on a new request type, and the aggregation processing of a plurality of discrete small IO random read-write requests is realized. If the data packet does not carry the aggregate IO mark, the read-write processing adopts conventional processing logic to process the IO request in the data packet. Therefore, the server side needs to check the aggregation IO mark of the data packet.
In step S402, the server side checks whether the data packet carries an aggregate IO flag. Taking the foregoing scenario as an example, if the aggregate IO flag is represented by a scan-other flag bit in the data packet, the current value of the scan-other flag bit may be checked, if the aggregate IO flag bit is 1, it indicates that the data packet carries an aggregate IO flag, otherwise if the scan-other flag bit is 0, it indicates that the data packet carries an unpolymerized IO flag.
Step S403, if the aggregation IO mark is carried, the server side obtains the aggregation data in the data packet. When the data packet carries the aggregation IO mark, the data packet is formed by packaging aggregation data, and the aggregation data comprises a plurality of data blocks corresponding to the small IO random read-write requests and request metadata. The request metadata is constructed by the client when a plurality of small IO random read-write requests need to be sent, and comprises the request number of the small IO random read-write requests which need to be sent this time and parameter information of each small IO random read-write request.
Taking the scenario of the foregoing 3 small IO random read-write requests as an example, aggregate data in a data packet is generated by combining the scan-gather metadata with the data block 1, the data block 2 and the data block 3, where the scan-gather metadata includes request data (3) and parameter information of the IO request 1, the IO request 2 and the IO request 3. The parameter information may include an offset and a length corresponding to the three small IO random read-write requests.
In step S404, the server reconstructs each small IO random read-write request included in the data packet according to the aggregate data.
When reconstruction is carried out, the server side can firstly analyze the request metadata in the aggregated data and acquire the request number and parameter information of each small IO random read-write request contained in the data packet. For example, in this embodiment, the number of the small IO random read/write requests obtained by parsing is 3, and the parameter information includes the offset and length corresponding to the 3 small IO random read/write requests respectively. Therefore, each small IO random read-write request can be reconstructed further according to the request number, the parameter information and the data block corresponding to each small IO random read-write request in the aggregated data. For example, in this embodiment, the data blocks corresponding to the 3 small IO random read-write requests are the data block 1, the data block 2 and the data block 3, respectively, and then three small IO random read-write requests can be reconstructed by combining the parameter information, which are the IO request 1, the IO request 2 and the IO request 3, as shown in fig. 2.
In step S405, after the reconstruction is completed, the server side may sequentially send the reconstructed small IO random read-write requests to the back-end storage device for processing, and the back-end storage device responds to the small IO random read-write requests to perform a disc reading (reading) or disc dropping (writing) operation. As shown in fig. 2, the back-end storage device writes or reads corresponding data blocks into or from storage locations determined based on the parameter information offset and length.
In addition, if the checking result in step S402 does not carry the aggregation IO flag, it indicates that the received data packet is not a regular data packet formed by packaging aggregation data, but is not aggregated by the scheme of the present application. Therefore, the server side only needs to process the data packet by adopting conventional processing logic, and at the moment, the server side can acquire the IO read-write request contained in the data packet and send the IO read-write request to the back-end storage equipment for processing.
The scheme of the application is similar to the SCATER-gather (scatter-gather) of DMA (Direct Memory Access) in characteristic concept, but the application fields and implementation mechanisms of the two are quite different. The DMA's scanner-gather mainly solves the performance problem when the physical device accesses the memory operation, and it implements the aggregation function of discrete IO through hardware. The solution provided by the embodiment of the application solves the problem of storage performance of qemu, and the aggregation of discrete IO is realized by modifying qemu virtualization software. As shown in fig. 5, the conventional DMA technique can only transfer one physically contiguous block of data at a time, and an interrupt is initiated after the transfer is completed. The scanner-other DMA allows a plurality of physically discontinuous data blocks to be transmitted at a time, and only one interrupt is initiated after the transmission is completed, so that the number of times of interrupt is greatly reduced, and the efficiency of data transmission is improved. The scan-other DMA may be implemented by setting a descriptor table in the memory, each descriptor containing the address and length of a memory block. The DMA controller may collect data from the descriptor table and transfer it to the target device or memory. The method can reduce the intervention of a CPU and improve the efficiency and the speed of data transmission. However, the DMA technology, even the inter-network RDMA, can only be used for exchanging data between the memory and the physical device, which greatly improves the data transmission efficiency by means of the hardware characteristic of the DMA, while the scene of exchanging data between the non-physical device and the memory belongs to different application fields, and cannot be applied to the IO read-write process of the qemu network block device storage to be solved in the application.
By introducing a new request type in the IO processing process of qemu, the scheme provided by the embodiment of the application is specially used for realizing the efficient processing of small IO random reading and writing. The method is characterized in that a plurality of small IO random read-write requests are not packed into individual data packets to be transmitted separately between a client side and a server side of the network block device, the discrete small IO random read-write requests are aggregated into a data packet with larger data quantity and transmitted to the server side at one time, and then the data packet is split and reconstructed by the server side and then is submitted to a back-end storage device to execute read-write operation, so that the data transmission efficiency is greatly improved, and the problem of performance bottleneck of qemu network block devices in a small IO random read-write scene is solved.
Based on the same inventive concept, the embodiment of the application also provides a small IO random read-write processing device, the corresponding method of the device is the small IO random read-write processing method in the previous embodiment, and the principle of solving the problem is similar to that of the method. The small IO random read-write processing device provided by the embodiment of the application comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein the computer program instructions, when executed by the processor, trigger the device to realize the method and/or the technical scheme of the embodiments of the application.
The specific implementation of the device may be a device formed by integrating the user device, the network device or the user device and the network device through a network, or may be an application program running on the device. The user equipment comprises, but is not limited to, various terminal equipment such as computers, mobile phones, tablet computers and the like; the network device includes, but is not limited to, an implementation such as a network host, a single network server, a set of multiple network servers, or a set of computers based on cloud computing. Here, the Cloud is composed of a large number of hosts or web servers based on Cloud Computing (Cloud Computing), which is a kind of distributed Computing, one virtual computer composed of a group of loosely coupled computer sets.
In particular, the methods and/or embodiments of the present application may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. The above-described functions defined in the method of the present application are performed when the computer program is executed by a processing unit.
It should be noted that, the computer readable medium described in the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowchart or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As another aspect, the present application also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more computer program instructions executable by a processor to implement the methods and/or aspects of the various embodiments of the present application described above.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, using Application Specific Integrated Circuits (ASIC), a general purpose computer or any other similar hardware device. In some embodiments, the software programs of the present application may be executed by a processor to implement the above steps or functions. Likewise, the software programs of the present application (including associated data structures) may be stored on a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. In addition, some steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order. The numerical sequence of the sequence numbers corresponding to the steps does not represent any specific execution sequence, and each step can be executed in any sequence combination on the premise of conforming to the execution logic.
Claims (10)
1. The small IO random read-write processing method is characterized by being applied to a client of network block equipment and comprising the following steps of:
when a plurality of small IO random read-write requests need to be sent, request metadata about the plurality of small IO random read-write requests are constructed, wherein the request metadata comprise the number of the small IO random read-write requests which need to be sent at this time and parameter information of each small IO random read-write request;
combining the data blocks corresponding to the plurality of small IO random read-write requests and the metadata into aggregated data;
packaging the aggregated data, adding an aggregate IO mark, sending the data packet to a server side of the network block equipment through a network, enabling the server side to check whether the data packet carries the aggregate IO mark or not, acquiring the aggregated data in the data packet when determining that the data packet carries the aggregate IO mark, reconstructing each small IO random read-write request contained in the data packet according to the aggregated data, and sequentially sending the small IO random read-write requests to a rear-end storage equipment for processing.
2. The method of claim 1, wherein the parameter information comprises a data offset and a data length value for each small IO random read-write request.
3. The method of claim 1, wherein the small IO random read-write request is a random read-write request with a read-write data amount less than a preset value.
4. The small IO random read-write processing method is characterized by being applied to a server side of network block equipment and comprising the following steps:
receiving a data packet from a client of the network block device through a network;
checking whether an aggregation IO mark is carried in the data packet, if so, acquiring aggregation data in the data packet, wherein the aggregation data comprises data blocks corresponding to a plurality of small IO random read-write requests and request metadata, and the request metadata is constructed by the client when the client needs to send the small IO random read-write requests, and comprises the request number of the small IO random read-write requests to be sent at this time and parameter information of each small IO random read-write request;
reconstructing each small IO random read-write request contained in the data packet according to the aggregated data;
and sequentially sending the small IO random read-write request to the back-end storage equipment for processing.
5. The method according to claim 4, wherein the method further comprises:
and if the aggregation IO mark is not carried, acquiring an IO read-write request contained in the data packet, and sending the IO read-write request to the back-end storage equipment for processing.
6. The method of claim 4, wherein reconstructing each small IO random read-write request contained in the data packet from the aggregate data comprises:
analyzing request metadata in the aggregated data, and acquiring the request number and parameter information of each small IO random read-write request contained in the data packet;
and reconstructing each small IO random read-write request according to the request number, the parameter information and the data block corresponding to each small IO random read-write request in the aggregated data.
7. The method of claim 4, wherein the parameter information comprises a data offset and a data length value for each small IO random read-write request.
8. The method of claim 4, wherein the small IO random read-write request is a random read-write request with a read-write data amount less than a preset value.
9. A small IO random read-write processing device comprising a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein the computer program instructions, when executed by the processor, trigger the device to perform the method of any one of claims 1 to 8.
10. A computer readable medium having stored thereon computer program instructions executable by a processor to implement the method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410015570.1A CN117742616A (en) | 2024-01-04 | 2024-01-04 | Small IO random read-write processing method, device and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410015570.1A CN117742616A (en) | 2024-01-04 | 2024-01-04 | Small IO random read-write processing method, device and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117742616A true CN117742616A (en) | 2024-03-22 |
Family
ID=90256508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410015570.1A Pending CN117742616A (en) | 2024-01-04 | 2024-01-04 | Small IO random read-write processing method, device and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117742616A (en) |
-
2024
- 2024-01-04 CN CN202410015570.1A patent/CN117742616A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Cheriton et al. | The distributed V kernel and its performance for diskless workstations | |
| US9864538B1 (en) | Data size reduction | |
| TW201220197A (en) | for improving the safety and reliability of data storage in a virtual machine based on cloud calculation and distributed storage environment | |
| CN114201421B (en) | Data stream processing method, storage control node and readable storage medium | |
| US20220405173A1 (en) | Variable number and size of put operations for storage backup | |
| US20130054841A1 (en) | Multiple i/o request processing in a storage system | |
| CN103605630B (en) | Virtual server system and data reading-writing method thereof | |
| CN109564502B (en) | Processing method and device applied to access request in storage device | |
| EP4141641A2 (en) | Method and apparatus for writing data in append mode, device and storage medium | |
| Stuedi et al. | jverbs: Ultra-low latency for data center applications | |
| CN109478171B (en) | Improving throughput in openfabics environment | |
| CN109857545A (en) | A kind of data transmission method and device | |
| EP4369171A1 (en) | Method and apparatus for processing access request, and storage device and storage medium | |
| CN114721995A (en) | Data transmission method applied to virtual database and RDMA-based database virtualization method | |
| CN109274721B (en) | LAN-free transmission method and system based on virtual disk mapping | |
| US10606776B2 (en) | Adding dummy requests to a submission queue to manage processing queued requests according to priorities of the queued requests | |
| CN110798366B (en) | Task logic processing method, device and equipment | |
| Nguyen et al. | Reducing data copies between gpus and nics | |
| CN111435323B (en) | Information transmission method, device, terminal, server and storage medium | |
| CN117742616A (en) | Small IO random read-write processing method, device and computer readable medium | |
| US11659070B2 (en) | Interface circuit for providing extension packet and processor including the same | |
| Anderssen et al. | Optimization | |
| CN113422792A (en) | Data transmission method and device, electronic equipment and computer storage medium | |
| CN111600943A (en) | Method and equipment for acquiring target data | |
| JP6277276B2 (en) | Descriptive objects and descriptors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |