CN112269538A - Data reading method, device, equipment and medium - Google Patents
Data reading method, device, equipment and medium Download PDFInfo
- Publication number
- CN112269538A CN112269538A CN202011111783.2A CN202011111783A CN112269538A CN 112269538 A CN112269538 A CN 112269538A CN 202011111783 A CN202011111783 A CN 202011111783A CN 112269538 A CN112269538 A CN 112269538A
- Authority
- CN
- China
- Prior art keywords
- reading
- data
- request
- direct
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000001360 synchronised effect Effects 0.000 claims abstract description 67
- 230000006870 function Effects 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 12
- 230000008901 benefit Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The application discloses a method, a device, equipment and a medium for reading data of a file system kernel client, wherein the method comprises the following steps: acquiring a data reading request initiated by a user mode data reading system; judging whether a reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark; if the reading mode corresponding to the data reading request is synchronous direct reading, establishing a direct reading IO structure and a reading request working thread corresponding to the data reading request, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request; and linking each direct-reading IO request into a read request list of a direct-reading IO structure, and sending the data read request to a storage disk so that a callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel to obtain target data corresponding to the data read request. Therefore, the data reading performance of the file system kernel client can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a medium for reading data of a file system kernel client.
Background
With the rapid development of the information age, the data storage performance of the server is required to be higher and higher. In order to meet the requirement of a user on the data storage performance of the server, the file storage performance needs to be improved, and the advantage of improving the kernel performance is obvious in the process of improving the file storage performance of the server. At present, a synchronous direct reading process of a kernel client of a file system is as follows, after the kernel client in the file system receives a data reading request sent by a user mode data reading system, according to the number of data segments required to be read in the data reading request, a direct reading IO request corresponding to a first data segment required to be read is sent to a storage disk, after the first data segment required to be read is obtained, a direct reading IO request corresponding to a second data segment required to be read is sent to the storage disk until each required data segment is read, and the data reading method in this way makes the whole data reading process consume a long time, that is, the data reading performance is low. Therefore, how to improve the data reading performance of the file system kernel client is a main problem to be solved by those skilled in the art.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, a device, and a medium for reading data of a kernel client of a file system, which can improve data reading performance of the kernel client of the file system, exert advantages of the kernel client of the file system, and meet requirements for reading high-performance files. The specific scheme is as follows:
in a first aspect, the present application discloses a method for reading data of a kernel client of a file system, which is applied to the kernel client and includes:
acquiring a data reading request initiated by a user mode data reading system;
judging whether a reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark, wherein the synchronous direct reading is synchronous data reading from a storage disc;
if the reading mode corresponding to the data reading request is synchronous direct reading, establishing a direct reading IO structure and a reading request working thread corresponding to the data reading request, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request;
and linking each direct-reading IO request into a read request list of the direct-reading IO structure, and sending the data read request to the storage disk, so that a callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, and target data corresponding to the data read request is obtained.
Optionally, before sending the data read request to the storage disk, the method further includes:
storing the head node of each direct reading IO request into a space corresponding to a private pointer of a storage disk read request structure corresponding to the storage disk;
and putting the read request working thread into a callback pointer of the storage disk read request structure.
Optionally, after determining whether the reading mode corresponding to the data reading request is synchronous direct reading according to a preset reading mark, the method further includes:
if the reading mode corresponding to the data reading request is asynchronous reading, acquiring target data corresponding to the data reading request from a preset cache page;
and if the target data is not hit in the preset cache page, acquiring the target data from a preset cache memory, wherein the preset cache memory is a preset memory for storing pre-read data when the storage space of the preset cache page is smaller than a preset storage space threshold value.
Optionally, after the target data is acquired from the preset cache memory, the method further includes:
and if the target data is not hit in the preset cache memory, synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, and returning the target data to the user mode data reading system.
Optionally, in the process of synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, the method further includes:
and if the storage space in the preset cache page is smaller than the preset storage space threshold, storing the target data read after the storage space in the preset cache page is smaller than the preset storage space threshold into the preset cache memory.
Optionally, after synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, the method further includes:
when the target data synchronous pre-reading is finished, setting a synchronous pre-reading identifier at the position where the target data synchronous pre-reading is finished in the storage disc;
and starting asynchronous pre-reading data from a position corresponding to the synchronous pre-reading identifier to the preset cache page or the preset cache memory, wherein the data volume of the pre-reading data is determined based on the size of the preset asynchronous pre-reading data, and the size of the preset asynchronous pre-reading data is the data size preset in the kernel client.
Optionally, the method for reading data by a file system kernel client further includes:
if the data in the preset cache page is not read beyond the preset time interval, the cache page data which is not read beyond the preset time interval is transferred to the preset cache memory.
In a second aspect, the present application discloses a device for reading data of a kernel client of a file system, which is applied to a kernel client, and includes:
the request acquisition module is used for acquiring a data reading request initiated by a user mode data reading system;
the data reading mode judging module is used for judging whether a reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark, wherein the synchronous direct reading is synchronous data reading from a storage disc;
the first data reading processing module is used for creating a direct reading IO structure and a reading request working thread corresponding to the data reading request when the reading mode corresponding to the data reading request is synchronous direct reading, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request;
and the second data reading processing module is configured to link each direct-reading IO request into a read request list of the direct-reading IO structure, and send the data read request to the storage disk, so that a callback function of the storage disk calls the read request worker thread to execute each direct-reading IO request in the read request list in parallel, and target data corresponding to the data read request is obtained.
In a third aspect, the present application discloses an electronic device, comprising:
a memory and a processor;
wherein the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the file system kernel client data reading method disclosed above.
In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the file system kernel client data reading method disclosed above.
Therefore, the method and the device firstly acquire the data reading request initiated by the user mode data reading system, then judge whether the reading mode corresponding to the data reading request is synchronous direct reading according to the preset reading mark, wherein the synchronous direct reading is synchronous reading of data from a storage disk, if the reading mode corresponding to the data reading request is synchronous direct reading, creating a direct-reading IO structure and a read request working thread corresponding to the data read request, dividing the data read request into different direct-reading IO requests according to the number of read segments in the data read request, linking each direct-reading IO request into a read request list of the direct-reading IO structure, and sending the data read request to the storage disk, and calling the read request working thread by the callback function of the storage disk so as to execute each direct-reading IO request in the read request list in parallel to obtain target data corresponding to the data read request. Therefore, when the kernel client side performs synchronous direct reading of data, a data reading request sent by a user mode reading system is divided into different direct reading IO requests, each direct reading IO request is linked into a reading request list of a direct reading IO structure, and after the data reading request is sent to a storage disk, a callback function of the storage disk calls a reading request working thread to execute each direct reading IO request in the reading request list in parallel to obtain target data corresponding to the data reading request, so that compared with the prior art that the kernel client side needs to send a plurality of direct reading IO requests corresponding to the same data reading request to the storage disk in sequence and needs to wait for the problems of long consumed time and low data reading performance caused after each direct reading IO request is sent, the method only needs to link each direct reading IO request corresponding to the data reading request into the reading request list first, the data reading request is sent to the storage disk, the callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, the time consumed by reading data is saved, the data reading performance of the kernel client of the file system is improved, the advantages of the kernel client of the file system are brought into play, and the reading requirement of high-performance files is met.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of a method for reading data from a file system kernel client disclosed in the present application;
FIG. 2 is a flowchart of a specific file system kernel client data reading method disclosed herein;
fig. 3 is a schematic structural diagram of an apparatus for a file system kernel client data reading method disclosed in the present application;
fig. 4 is a schematic structural diagram of an electronic device disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, an embodiment of the present application discloses a data reading method for a kernel client of a file system, which is applied to the kernel client, and the method includes:
step S11: and acquiring a data reading request initiated by a user mode data reading system.
In a specific implementation process, for a kernel client, a data read request initiated by a user-mode data read (read) system needs to be obtained first, where the data read request may include a read request for multiple pieces of data.
Step S12: and judging whether the reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark, wherein the synchronous direct reading is synchronous data reading from a storage disk.
After the data read request is acquired, whether a read mode corresponding to the data read request is synchronous direct reading or not needs to be judged according to a preset read mark, wherein the synchronous direct reading is synchronous data reading from a storage disk. The kernel client comprises a preset reading flag (flag), wherein the preset reading flag represents a reading mode when the kernel client reads data, and the reading mode comprises synchronous direct reading from a storage disc and asynchronous reading from a buffer page.
Step S13: and if the reading mode corresponding to the data reading request is synchronous direct reading, establishing a direct reading IO structure and a reading request working thread corresponding to the data reading request, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request.
And if the reading mode corresponding to the data reading request is synchronous direct reading, establishing a direct reading IO structure and a reading request working thread corresponding to the data reading request, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request.
Specifically, if the read mode corresponding to the data read request is synchronous direct reading, a kernel direct reading IO structure (read _ IO) is defined, where the direct reading IO structure includes a read request list implementation, a request number, and a kernel IO structure (kiocb); defining a read request work thread (read _ work), wherein the read request work thread executes a request in a read request list according to a work queue; and dividing the data reading request into different direct reading IO requests according to the number of reading sections in the data reading request.
Step S14: and linking each direct-reading IO request into a read request list of the direct-reading IO structure, and sending the data read request to the storage disk, so that a callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, and target data corresponding to the data read request is obtained.
After dividing the data read request into different direct-reading IO requests according to the number of reads in the data read request, it is further required to link each direct-reading IO request into a read request list of the direct-reading IO structure, that is, each direct-reading IO request is added into a read request list of the direct-reading IO result.
The data read request is further required to be sent to the storage disk first, and after the storage disk receives the data read request, the callback function takes effect to call back the read request working thread and execute each direct-read IO request in the read request list in parallel, so as to obtain target data corresponding to the data read request. And when the read request working thread executes the direct-reading IO requests in the read request list in parallel, reading the direct-reading IO requests from the read request list according to a working queue mode.
In practical applications, before sending the data read request to the storage disk, the method further includes: storing the head node of each direct reading IO request into a space corresponding to a private pointer of a storage disk read request structure corresponding to the storage disk; and putting the read request working thread into a callback pointer of the storage disk read request structure.
Therefore, the data read request is only needed to be sent to the storage disk once, after the storage disk receives the data read request, the storage disk can call a read request working thread to execute each direct-reading IO request corresponding to the data read request in the read request list in parallel, time for sending the direct-reading IO request to the storage disk for multiple times and waiting is saved, the direct-reading IO requests in the read request list can be executed in parallel, time consumed by data reading is further shortened, and data reading performance of a file system kernel client is improved.
Therefore, the method and the device firstly acquire the data reading request initiated by the user mode data reading system, then judge whether the reading mode corresponding to the data reading request is synchronous direct reading according to the preset reading mark, wherein the synchronous direct reading is synchronous reading of data from a storage disk, if the reading mode corresponding to the data reading request is synchronous direct reading, creating a direct-reading IO structure and a read request working thread corresponding to the data read request, dividing the data read request into different direct-reading IO requests according to the number of read segments in the data read request, linking each direct-reading IO request into a read request list of the direct-reading IO structure, and sending the data read request to the storage disk, and calling the read request working thread by the callback function of the storage disk so as to execute each direct-reading IO request in the read request list in parallel to obtain target data corresponding to the data read request. Therefore, when the kernel client side performs synchronous direct reading of data, a data reading request sent by a user mode reading system is divided into different direct reading IO requests, each direct reading IO request is linked into a reading request list of a direct reading IO structure, and after the data reading request is sent to a storage disk, a callback function of the storage disk calls a reading request working thread to execute each direct reading IO request in the reading request list in parallel to obtain target data corresponding to the data reading request, so that compared with the prior art that the kernel client side needs to send a plurality of direct reading IO requests corresponding to the same data reading request to the storage disk in sequence and needs to wait for the problems of long consumed time and low data reading performance caused after each direct reading IO request is sent, the method only needs to link each direct reading IO request corresponding to the data reading request into the reading request list first, the data reading request is sent to the storage disk, the callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, the time consumed by reading data is saved, the data reading performance of the kernel client of the file system is improved, the advantages of the kernel client of the file system are brought into play, and the reading requirement of high-performance files is met.
Referring to fig. 2, an embodiment of the present application discloses a specific file system kernel client data reading method, which is applied to a kernel client, and the method includes:
step S21: and acquiring a data reading request initiated by a user mode data reading system.
Step S22: and judging whether the reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark, wherein the synchronous direct reading is synchronous data reading from a storage disk.
The specific implementation of step S21 and step S22 can refer to the disclosure in the foregoing embodiments, and will not be described herein again.
Step S23: and if the reading mode corresponding to the data reading request is asynchronous reading, acquiring the target data corresponding to the data reading request from a preset cache page.
After judging whether the reading mode corresponding to the data reading request is synchronous direct reading according to a preset reading mark, if the reading mode corresponding to the data reading request is asynchronous reading, the target data corresponding to the data reading request is obtained from a preset cache page. That is, in the data reading mode, a cache page is provided inside the system to store data pre-read from the storage disk, so that when a new data reading request is obtained, the target data corresponding to the data reading request can be obtained from the cache page first.
Step S24: and if the target data is not hit in the preset cache page, acquiring the target data from a preset cache memory, wherein the preset cache memory is a preset memory for storing pre-read data when the storage space of the preset cache page is smaller than a preset storage space threshold value.
It can be understood that, if the target data is not hit in the preset cache page, the target data is obtained from a preset cache memory, where the preset cache memory is a preset memory for storing pre-read data when a storage space of the preset cache page is smaller than a preset storage space threshold.
Specifically, the data stored in the preset cache page is only the pre-read data corresponding to the data read request acquired before the data read request is acquired, so that when the target data corresponding to the data read request is acquired from the preset cache page, the target data may not be hit, and when the target data is not hit in the preset cache page, the target data needs to be acquired from a preset cache memory. The kernel client realizes a single preset cache memory (fs _ read _ cache), and packages and realizes a related operation interface, and creates a kernel working thread to maintain the fs _ read _ cache.
And if the target data are not hit in the preset cache memory, synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, and returning the target data to the user mode data reading system.
That is, if the target data is not hit in the preset cache memory, it indicates that the target data needs to be read from a storage disk first, and the target data is synchronously pre-read into the preset cache page, and the target data is returned to the user-mode data reading system.
In the process of synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, the method further includes: and if the storage space in the target cache page is smaller than the preset storage space threshold, storing the target data read after the storage space in the preset cache page is smaller than the preset storage space threshold into the preset cache memory.
That is, starting from the condition that the storage space in the preset cache page is smaller than the preset storage space threshold, the data which is read out in advance is stored in the preset cache memory.
After synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, the method further includes: when the target data synchronous pre-reading is finished, setting a synchronous pre-reading identifier at the position where the target data synchronous pre-reading is finished in the storage disc; and starting asynchronous pre-reading data from a position corresponding to the synchronous pre-reading identifier to the preset cache page or the preset cache memory, wherein the data volume of the pre-reading data is determined based on the size of the preset asynchronous pre-reading data, and the size of the preset asynchronous pre-reading data is the data size preset in the kernel client. Specifically, the data size of the pre-read data may be 2 times the size of the preset asynchronous pre-read data.
After the target data corresponding to the data reading request is pre-read, starting asynchronous pre-reading of certain pre-read data from a position where the target data is preset to the preset cache page or the preset cache memory, wherein the pre-read data is preferentially stored in the preset cache page, and when the storage space of the preset cache page is smaller than a preset storage space threshold, the pre-read data is stored in the preset cache memory.
And after the target data is read in advance, establishing an asynchronous read-ahead working thread, performing asynchronous read-ahead by the asynchronous read-ahead working thread to read-ahead data with a certain size, and filling the preset page cache by using the data volume of the read-ahead data with the size specified by the mount of the kernel client. And adding fields such as a preset asynchronous pre-reading data size and the like in the struct file _ ra _ state.
In a specific implementation process, if the data in the preset cache page is not read beyond a preset time interval, the cache page data which is not read beyond the preset time interval is transferred to the preset cache memory.
Therefore, the kernel client can define the size of the preset asynchronous pre-reading data in the asynchronous pre-reading process so as to achieve the effect of adjusting the pre-reading performance according to the service. The pre-reading of the page cache is assisted by adding a preset cache memory caching mechanism in a pre-reading mechanism of the kernel client, so that the effect of improving the efficiency is achieved. When the page cache is insufficient, the data can be pre-read into the preset cache memory, synchronous pre-reading and asynchronous pre-reading can be performed, page cache pages can be eliminated into the preset cache memory for temporary storage, more available space in the page cache can be ensured, the probability of hitting the data corresponding to the data reading request in the page cache is increased, the data reading time is saved, and the data reading performance is improved.
Referring to fig. 3, an embodiment of the present application discloses a data reading apparatus for a kernel client of a file system, which is applied to a kernel client, and includes:
a request obtaining module 11, configured to obtain a data reading request initiated by a user-mode data reading system;
a data reading mode determining module 12, configured to determine, according to a preset reading flag, whether a reading mode corresponding to the data reading request is synchronous direct reading, where the synchronous direct reading is synchronous reading of data from a storage disk;
the first data reading processing module 13 is configured to create a direct-reading IO structure and a read request working thread corresponding to the data read request when the read mode corresponding to the data read request is synchronous direct reading, and divide the data read request into different direct-reading IO requests according to the number of read segments in the data read request;
the second data reading processing module 14 is configured to link each direct-reading IO request into a read request list of the direct-reading IO structure, and send the data read request to the storage disk, so that a callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, and target data corresponding to the data read request is obtained.
Therefore, the method and the device firstly acquire the data reading request initiated by the user mode data reading system, then judge whether the reading mode corresponding to the data reading request is synchronous direct reading according to the preset reading mark, wherein the synchronous direct reading is synchronous reading of data from a storage disk, if the reading mode corresponding to the data reading request is synchronous direct reading, creating a direct-reading IO structure and a read request working thread corresponding to the data read request, dividing the data read request into different direct-reading IO requests according to the number of read segments in the data read request, linking each direct-reading IO request into a read request list of the direct-reading IO structure, and sending the data read request to the storage disk, and calling the read request working thread by the callback function of the storage disk so as to execute each direct-reading IO request in the read request list in parallel to obtain target data corresponding to the data read request. Therefore, when the kernel client side performs synchronous direct reading of data, a data reading request sent by a user mode reading system is divided into different direct reading IO requests, each direct reading IO request is linked into a reading request list of a direct reading IO structure, and after the data reading request is sent to a storage disk, a callback function of the storage disk calls a reading request working thread to execute each direct reading IO request in the reading request list in parallel to obtain target data corresponding to the data reading request, so that compared with the prior art that the kernel client side needs to send a plurality of direct reading IO requests corresponding to the same data reading request to the storage disk in sequence and needs to wait for the problems of long consumed time and low data reading performance caused after each direct reading IO request is sent, the method only needs to link each direct reading IO request corresponding to the data reading request into the reading request list first, the data reading request is sent to the storage disk, the callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, the time consumed by reading data is saved, the data reading performance of the kernel client of the file system is improved, the advantages of the kernel client of the file system are brought into play, and the reading requirement of high-performance files is met.
Referring to fig. 4, a schematic structural diagram of an electronic device 20 provided in the embodiment of the present application is shown, where the electronic device 20 may implement the steps of the file system kernel client data reading method disclosed in the foregoing embodiment.
In general, the electronic device 20 in the present embodiment includes: a processor 21 and a memory 22.
The processor 21 may include one or more processing cores, such as a four-core processor, an eight-core processor, and so on. The processor 21 may be implemented by at least one hardware of a DSP (digital signal processing), an FPGA (field-programmable gate array), and a PLA (programmable logic array). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (graphics processing unit) which is responsible for rendering and drawing images to be displayed on the display screen. In some embodiments, the processor 21 may include an AI (artificial intelligence) processor for processing computing operations related to machine learning.
Memory 22 may include one or more computer-readable storage media, which may be non-transitory. Memory 22 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 22 is at least used for storing the following computer program 221, wherein after being loaded and executed by the processor 21, the computer program can implement the file system client data reading method steps disclosed in any of the foregoing embodiments.
In some embodiments, the electronic device 20 may further include a display 33, an input/output interface 34, a communication interface 35, a sensor 36, a power source 37, and a communication bus 38.
Those skilled in the art will appreciate that the configuration shown in FIG. 4 is not limiting to electronic device 20 and may include more or fewer components than those shown.
Further, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, where the computer program is executed by a processor to implement the file system kernel client data reading method disclosed in any of the foregoing embodiments.
For a specific process of the file system kernel client data reading method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of other elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The method, the device, the equipment and the medium for reading the data of the file system kernel client provided by the application are introduced in detail, a specific example is applied in the description to explain the principle and the implementation mode of the application, and the description of the embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A data reading method for a file system kernel client is characterized by being applied to the kernel client and comprising the following steps:
acquiring a data reading request initiated by a user mode data reading system;
judging whether a reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark, wherein the synchronous direct reading is synchronous data reading from a storage disc;
if the reading mode corresponding to the data reading request is synchronous direct reading, establishing a direct reading IO structure and a reading request working thread corresponding to the data reading request, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request;
and linking each direct-reading IO request into a read request list of the direct-reading IO structure, and sending the data read request to the storage disk, so that a callback function of the storage disk calls the read request working thread to execute each direct-reading IO request in the read request list in parallel, and target data corresponding to the data read request is obtained.
2. The file system kernel client data reading method according to claim 1, wherein before sending the data read request to a storage disk, further comprising:
storing the head node of each direct reading IO request into a space corresponding to a private pointer of a storage disk read request structure corresponding to the storage disk;
and putting the read request working thread into a callback pointer of the storage disk read request structure.
3. The method for reading data of a file system kernel client according to claim 1 or 2, wherein after determining whether a reading mode corresponding to the data reading request is synchronous direct reading according to a preset reading flag, the method further comprises:
if the reading mode corresponding to the data reading request is asynchronous reading, acquiring the target data corresponding to the data reading request from a preset cache page;
and if the target data is not hit in the preset cache page, acquiring the target data from a preset cache memory, wherein the preset cache memory is a preset memory for storing pre-read data when the storage space of the preset cache page is smaller than a preset storage space threshold value.
4. The method for reading the data of the file system kernel client according to claim 3, wherein after the target data is obtained from a preset cache memory, the method further comprises:
and if the target data is not hit in the preset cache memory, synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, and returning the target data to the user mode data reading system.
5. The method for reading data of a file system kernel client according to claim 4, wherein in the process of synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, the method further comprises:
and if the storage space in the preset cache page is smaller than the preset storage space threshold, storing the target data read after the storage space in the preset cache page is smaller than the preset storage space threshold into the preset cache memory.
6. The method for reading data of a file system kernel client according to claim 4, wherein after synchronously pre-reading the target data corresponding to the data reading request from the storage disk into the preset cache page, the method further comprises:
when the target data synchronous pre-reading is finished, setting a synchronous pre-reading identifier at the position where the target data synchronous pre-reading is finished in the storage disc;
and starting asynchronous pre-reading data from a position corresponding to the synchronous pre-reading identifier to the preset cache page or the preset cache memory, wherein the data volume of the pre-reading data is determined based on the size of the preset asynchronous pre-reading data, and the size of the preset asynchronous pre-reading data is the data size preset in the kernel client.
7. The file system kernel client data reading method according to claim 3, further comprising:
if the data in the preset cache page is not read beyond the preset time interval, the cache page data which is not read beyond the preset time interval is transferred to the preset cache memory.
8. A data reading device for a file system kernel client is applied to a kernel client, and comprises:
the request acquisition module is used for acquiring a data reading request initiated by a user mode data reading system;
the data reading mode judging module is used for judging whether a reading mode corresponding to the data reading request is synchronous direct reading or not according to a preset reading mark, wherein the synchronous direct reading is synchronous data reading from a storage disc;
the first data reading processing module is used for creating a direct reading IO structure and a reading request working thread corresponding to the data reading request when the reading mode corresponding to the data reading request is synchronous direct reading, and dividing the data reading request into different direct reading IO requests according to the number of reading segments in the data reading request;
and the second data reading processing module is configured to link each direct-reading IO request into a read request list of the direct-reading IO structure, and send the data read request to the storage disk, so that a callback function of the storage disk calls the read request worker thread to execute each direct-reading IO request in the read request list in parallel, and target data corresponding to the data read request is obtained.
9. An electronic device, comprising:
a memory and a processor;
wherein the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the file system kernel client data reading method according to any one of claims 1 to 7.
10. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the method of accessing a file system kernel client data read according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011111783.2A CN112269538B (en) | 2020-10-16 | 2020-10-16 | Data reading method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011111783.2A CN112269538B (en) | 2020-10-16 | 2020-10-16 | Data reading method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112269538A true CN112269538A (en) | 2021-01-26 |
| CN112269538B CN112269538B (en) | 2022-11-25 |
Family
ID=74337238
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011111783.2A Active CN112269538B (en) | 2020-10-16 | 2020-10-16 | Data reading method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269538B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177079A (en) * | 2019-12-30 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Network file system data reading method, device, equipment and medium |
-
2020
- 2020-10-16 CN CN202011111783.2A patent/CN112269538B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177079A (en) * | 2019-12-30 | 2020-05-19 | 浪潮电子信息产业股份有限公司 | Network file system data reading method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112269538B (en) | 2022-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109194960B (en) | Image frame rendering method and device and electronic equipment | |
| US20200213091A1 (en) | Circuit and method for overcoming memory bottleneck of asic-resistant cryptographic algorithms | |
| WO2022134428A1 (en) | Applet page rendering method and apparatus, electronic device, and storage medium | |
| US20180300250A1 (en) | Method and apparatus for storing data | |
| CN111813713B (en) | Data acceleration operation processing method and device and computer readable storage medium | |
| CN108897630B (en) | OpenCL-based global memory caching method, system and device | |
| CN113704301B (en) | Data processing method, device, system, equipment and medium of heterogeneous computing platform | |
| CN112286459A (en) | Data processing method, device, equipment and medium | |
| CN114816263A (en) | Storage access method and intelligent processing device | |
| CN112506823A (en) | FPGA data reading and writing method, device, equipment and readable storage medium | |
| US20200371827A1 (en) | Method, Apparatus, Device and Medium for Processing Data | |
| CN111177079A (en) | Network file system data reading method, device, equipment and medium | |
| US10817183B2 (en) | Information processing apparatus and information processing system | |
| CN111061429B (en) | Data access method, device, equipment and medium | |
| CN111159124B (en) | Asynchronous write caching method, device and medium for Linux kernel file system | |
| CN112269538B (en) | Data reading method, device, equipment and medium | |
| CN105989012B (en) | page display method, device, mobile terminal and system | |
| WO2019114044A1 (en) | Image processing method and device, electronic apparatus, and computer readable storage medium | |
| US9405470B2 (en) | Data processing system and data processing method | |
| CN111090633A (en) | Small file aggregation method, device and equipment of distributed file system | |
| CN110825652B (en) | Method, device and equipment for eliminating cache data on disk block | |
| CN112748854B (en) | Optimized access to a fast storage device | |
| CN111625192B (en) | Metadata object access method, device, equipment and medium | |
| CN107291628B (en) | Method and apparatus for accessing data storage device | |
| US11593355B2 (en) | Method, device and computer program product for processing metadata |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |