CN113312323B - IO (input/output) request scheduling method and system for reducing access delay in parallel file system - Google Patents
IO (input/output) request scheduling method and system for reducing access delay in parallel file system Download PDFInfo
- Publication number
- CN113312323B CN113312323B CN202110620133.9A CN202110620133A CN113312323B CN 113312323 B CN113312323 B CN 113312323B CN 202110620133 A CN202110620133 A CN 202110620133A CN 113312323 B CN113312323 B CN 113312323B
- Authority
- CN
- China
- Prior art keywords
- request
- delay
- scheduling
- file system
- requests
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an IO request scheduling method and system for reducing access delay in a parallel file system, wherein the method comprises the steps of calculating IO requests of client access server storage equipment of the parallel file system and marking delay sensitivity; under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay. The invention aims to reduce the waiting time Tw of partial IO requests under the IO request competition condition, thereby reducing the IO response delay sensed by client application and being capable of improving the comprehensive processing performance of a typical workload of various computer systems using a parallel file system for mixed requirements of high bandwidth and low IO delay, including large-scale high-performance computer systems.
Description
Technical Field
The invention relates to the field of computer operating systems, in particular to an IO request scheduling method and an IO request scheduling system for reducing access delay in a parallel file system.
Background
The parallel file system is an important component of a high-performance computer, manages a large number of storage devices, and is responsible for scheduling a large number of concurrent IO requests, the concurrent IO requests are sequentially sent to the storage devices, the requests are sequentially executed, different execution orders of the IO requests can show different performance performances, although the storage devices are usually provided with request queues, the IO queue depth of the storage devices is limited, and the IO requests are inevitably queued in a software layer firstly under a large-scale concurrent condition, so an IO request scheduler of the parallel file system layer has a large influence on IO response time of a user program and throughput rate of the storage devices.
The IO request scheduler of the system software is responsible for ordering the execution sequence of IO requests. For example, the Linux operating system is provided with a plurality of scheduling strategies, the scheduling optimization is respectively implemented from the aspects of optimizing performance, fairness, preventing starvation and the like, and a user selects according to respective workload characteristics. The Lustre parallel file system is a typical representative of the parallel file system in the high-performance computing field, and a request scheduler of the Lustre parallel file system also has a plurality of selectable scheduling strategies, such as bandwidth priority and the like. Different schedulers operate according to respective scheduling policies, with a scheduling policy generally being valid for a particular workload and invalid for other workloads. When multiple workload modes coexist, how IO requests are scheduled becomes an important and complex issue.
In a typical workload scenario for high performance computing, a parallel file system faces a mix of workload patterns. Traditional scientific computing application programs are sensitive to IO bandwidth, and novel applications such as intensive data processing and artificial intelligence are sensitive to IO delay. The mixed demand mode of high bandwidth and low IO latency becomes a typical workload under the condition that high performance computing systems are used by multi-domain application sharing competition. Under the new load mode, delay sensitive IO requests and bandwidth sensitive IO requests are queued in a queue of the parallel file system, and when the number of requests is large, competition is generated, which inevitably causes the waiting and scheduling time of the IO requests to be prolonged, thereby affecting the performance of the delay sensitive application program. Under the condition of using a traditional disk medium, because the latency of disk access is large, the IO request queuing time is low, and this problem is not obvious, but when a new storage medium (for example, a new-type nonvolatile storage medium NVM) starts to be widely used, because the access latency is several orders of magnitude lower than the access latency of the disk, the occupancy of the IO request queuing time in the whole IO access flow is rapidly increased, and the influence on the application program performance is obvious.
Disclosure of Invention
The parallel file system generally adopts a multi-client and multi-server architecture, IO requests simultaneously sent by various types of application programs at a client compete for storage equipment of a server, when the number of the IO requests exceeds the IO processing capacity of the storage equipment of the server, the IO requests queue in a queue of the server, and the waiting time Tw becomes a component of IO request delay. The technical problem to be solved by the invention is as follows: the method and the system aim at reducing the waiting time Tw of partial IO requests under the IO request competition condition, thereby reducing the IO response delay sensed by client application and being capable of improving the comprehensive processing performance of various computer systems using the parallel file system aiming at the typical workload of the mixed requirements of high bandwidth and low IO delay, including large-scale high-performance computer systems.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
an IO request scheduling method for reducing access delay in a parallel file system includes
1) Calculating and marking delay sensitivity to IO (input/output) requests of clients of the parallel file system for accessing storage equipment of a server;
2) Under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay.
Optionally, the step of calculating the delay sensitivity in step 1) includes: judging whether the IO request isThe IO request buffered by the write-back buffer WBC is judged to be a delay-sensitive IO request if the IO request buffered by the write-back buffer WBC is detected, otherwise, the IO request is judged to be a delay-insensitive IO request, and the delay-insensitive IO request R is aimed atdioTaking a preset fixed value as the delay sensitivity obtained by calculation; for delay sensitive IO requests RwbcThen the delay sensitivity of the IO request is calculated based on the correlation of the write-back buffered WBCs and the IO.
Optionally, the calculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO refers to: the available space ratio P of the write-back buffer WBC is obtained, and 1-P is taken as the calculated delay sensitivity of the IO request.
Optionally, when the delay sensitivity is marked in step 1), a field for adding a delay sensitivity to the IO request is included, and the field includes the calculated delay sensitivity S.
Optionally, when the delay sensitivity is marked in step 1), the method further includes adding a type field to the IO request, where the type field includes a delay-insensitive IO request RdioOr delay sensitive IO request RwbcThe type of (c) identifies T.
Optionally, the method further includes the following steps that a server of the parallel file system performs IO request processing:
s1) initializing the maximum waiting time Tdeadline；
S2) receiving the IO request R and recording the timestamp TSr；
S3) resolving the IO request R to obtain the delay sensitivity S and according to Tw＝Tdeadline(1-S) calculating latency Tw；
S4) determining the waiting time TwIf it is 0, if the waiting time T iswIf the number is 0, activating the scheduling execution workflow of the IO request R; otherwise, the IO request R is put into a scheduling waiting queue, and a queuing waiting workflow of the IO request R is activated.
Optionally, the processing step after the schedule execution workflow of the IO request is activated in step S4) includes:
S4.1A) receives IO request R, fetch timeStamp TSr；
S4.2A) to traverse from the wait queue to find an IO request R that originates from the same write-back buffered WBC as the IO request Ri；
S4.3A) fetch IO request RiTime stamp TS ofi；
S4.4A) determines the IO request RiTime stamp TS ofiTimestamp TS less than IO request RrIf yes, remove IO request R from wait queue iAnd request IO for RiThe send-in execution queue;
S4.5A) determining whether the IO request in the waiting queue and the IO request R originated from the same WBC, if not, continuously traversing the IO request R from the waiting queue and the IO request R originated from the same WBCiJump to execute step S4.3A); otherwise, skipping to execute the next step;
S4.6A) removes the IO request R from the wait queue and enqueues the IO request R into the execution queue, executing all IO requests enqueued in turn.
Optionally, the processing step after the queue-waiting workflow of the IO request is activated in step S4) includes:
S4.1B) fetching an IO request R from the dispatch wait queue traversal;
S4.2B) wait time T for IO request RwSubtracting a preset scheduling time slice T, and judging the waiting time T after subtracting the preset scheduling time slice TwIf the number of the IO request R is zero, activating a scheduling execution workflow of the IO request R; otherwise, judging whether the scheduling waiting queue is completely traversed, if not, continuously traversing from the scheduling waiting queue to take out an IO request R, and skipping to the step S4.2B); otherwise, skipping to execute the next step;
S4.3B) whether any of the following conditions is satisfied: condition 1: the time slice expires; condition 2: a new IO request enters a scheduling waiting queue; if either is satisfied, then the jump is performed to step S4.1B), otherwise wait to be activated.
In addition, the invention also provides an IO request scheduling system for reducing access delay in a parallel file system, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the steps of the IO request scheduling method for reducing access delay in the parallel file system.
Furthermore, the present invention also provides a computer-readable storage medium having stored therein a computer program programmed or configured to execute an IO request scheduling method for reducing access latency in the parallel file system.
Compared with the prior art, the invention mainly has the following advantages:
1. calculating and marking delay sensitivity to IO (input/output) requests of clients of a parallel file system for accessing storage equipment of a server; under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are preferentially scheduled to reduce the queuing time under the condition of competition congestion and achieve the purpose of reducing the delay.
2. The parallel file system generally adopts a Write-Back buffer technology (WBC, Write Back Cache) on a client, namely, IO call sent by an application program temporarily stores data in a buffer area of the client and then returns immediately, the WBC selects a proper time and then sends the data to a storage device at a rear end, and the WBC can reassemble discrete IO requests into continuous IO requests, so that the access bandwidth is improved.
3. The invention is suitable for a general parallel file system framework, can be used in series with a request scheduling system of an original system, and in a request sequence output after scheduling by the embodiment, delay sensitive requests are arranged in front, thereby being beneficial to the prior scheduling execution of a subsequent scheduling system.
Drawings
FIG. 1 is a core flow diagram of the method according to the embodiment of the present invention.
Fig. 2 is a schematic diagram of a system structure in an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a process of calculating and labeling delay sensitivity according to an embodiment of the present invention.
Fig. 4 is a schematic processing flow diagram of the server according to the embodiment of the present invention.
Detailed Description
As shown in fig. 1, the IO request scheduling method for reducing access delay in a parallel file system in this embodiment includes:
1) calculating and marking delay sensitivity to IO (input/output) requests of clients of the parallel file system for accessing storage equipment of a server;
2) under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay.
The IO request scheduling method for reducing access delay in the parallel file system of the embodiment endows each IO request with a delay sensitivity, the IO request with high delay sensitivity is scheduled and executed preferentially, queuing time of the IO request under the condition of competitive congestion is reduced, and the purpose of reducing delay is achieved.
Referring to fig. 2, as an optional implementation manner, in this embodiment, based on the correlation between WBCs and IO requests, each IO request is given a delay sensitivity, and IO requests with high delay sensitivities are scheduled and executed preferentially, so that the queuing time of the IO requests under the condition of contention congestion is reduced, and the purpose of reducing delay is achieved.
Referring to fig. 3, the step of calculating the delay sensitivity in step 1) of the present embodiment includes: judging whether the IO request is an IO request buffered by the write-back buffer WBC, if so, judging the IO request to be a delay-sensitive IO request, otherwise, judging the IO request to be a delay-insensitive IO request, and aiming at the delay-insensitive IO request RdioTaking a preset fixed value as the delay sensitivity obtained by calculation; for delay sensitive IO requests RwbcThen the delay sensitivity of the IO request is calculated based on the dependencies of the write-back buffered WBC and the IO. In this embodiment, the IO requests of the application are classified into 2 types: WBC buffered requests (hereinafter referred to as R)wbc) And requests that bypass WBC buffering (hereinafter referred to as R)dio). According to the WBC principle, RwbcAfter the carried data is written into the cache, the application program is informed that the IO is completed, so that the application program does not need to wait for the data to be really written into the storage equipment, and the data is sent to the storage equipment by the file system at a background selection time. The application need not wait for R wbcTrue completion, insensitive to its completion time, so this type of request is defined as a delay-insensitive request. For RdioThe application must wait until the IO request operation is completed at the storage device. Thus, R is showndioIs directly influencing the execution time of the application and is therefore defined as a delay-sensitive IO request. In this embodiment, whether an IO request enters a write-back buffer WBC is not controlled, the IO request is only discriminated at a client of a parallel file system, a field describing delay sensitivity is added to the two types of IO requests, the delay sensitive request is endowed with relatively high delay sensitivity, and the delay is not sensitiveThe sensory request gives relatively low delay sensitivity. The calculation rule for the delay sensitivity is as follows: the delay sensitivity of the Rdio request takes a fixed maximum value, and the delay sensitivity of the Rwbc request needs to be calculated according to the space availability of the WBC, so as to prevent the Rwbc request from being excessively delayed and scheduled to cause the data in the WBC not to be drained smoothly.
Referring to fig. 3, in the present embodiment, calculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO refers to: the available space ratio P of the write-back buffer WBC is obtained, and 1-P is taken as the calculated delay sensitivity of the IO request.
In this embodiment, when the delay sensitivity is marked in step 1), a field for adding a delay sensitivity to the IO request is included, and the field includes the calculated delay sensitivity S.
In this embodiment, when the delay sensitivity is marked in step 1), the method further includes adding a type field to the IO request, where the type field includes a delay-insensitive IO request RdioOr delay sensitive IO request RwbcThe type of (d) identifies T.
As shown in fig. 4, sub-diagram (a), this embodiment further includes the following steps of performing IO request processing by the server of the parallel file system:
s1) initializing the maximum waiting time Tdeadline；
S2) receives the IO request R and records the timestamp TSr；
S3) analyzing the IO request R to obtain the delay sensitivity S and analyzing the delay sensitivity according to Tw＝Tdeadline(1-S) calculating the waiting time Tw(ii) a I.e. the smaller S, the smaller TwThe larger the ratio, the more inverse linear relationship, i.e. when S is 0, T is correspondedw＝TdeadlineThe waiting time is longest; when S is 1, corresponding to Tw0, no waiting is needed;
s4) determining the waiting time TwIf it is 0, if waiting time TwIf the number is 0, activating the scheduling execution workflow of the IO request R; otherwise, the IO request R is put into a scheduling waiting queue, and a queuing waiting workflow of the IO request R is activated.
The present embodiment, step S1), will wait for the maximum time TdeadlineThe initialization setting is the time for the WBC available capacity to drop from 100% to 0%, as may be found based on specific system measurements.
In addition, step S1) of this embodiment further includes a step of initializing and setting a scheduling time slice t, where the scheduling time slice t is used for processing IO requests in the scheduling wait queue in a loop traversal manner. In this embodiment, assuming that the precision of the field S is g, the value of the scheduling time slice t is initialized to t ═ tdeadine ═ g, that is, RwbcAnd requesting the minimum granularity of the waiting time, and activating the scheduling flow at intervals of time t by the server.
In this embodiment, a scheduling wait queue is set at the server, request scheduling is implemented according to the delay sensitivity of the received IO request, and a request with high delay sensitivity is scheduled and executed preferentially. In the case of two types of request contention, the scheduled execution of Rwbc is delayed and Tw thereof increases, but according to the working principle of WBC, when there is available space in WBC, the application program issuing the Rwbc request does not have to wait for the completion of the request, so the moderately delayed scheduling does not affect the normal execution of the application. The delay of Rwbc instead causes the Rdio request to be in a preferentially scheduled position in the queue, and therefore has a shorter latency Tw, thereby reducing the IO delay perceptible to the application issuing the Rdio request.
As shown in fig. 4, sub-diagram (c), the processing step after the scheduling execution workflow of the IO request is activated in step S4) includes:
S4.1A) receives IO request R, fetches time stamp TSr；
S4.2A) to traverse from the wait queue to find an IO request R that originated from the same write-back buffered WBC as the IO request Ri；
S4.3A) fetch IO request RiTime stamp TS ofi；
S4.4A) judges the IO request RiTime stamp TS ofiTimestamp TS less than IO request RrIf yes, remove IO request R from wait queueiAnd request IO for RiIs sent to the execution queue;
S4.5A) determining whether the IO request in the wait queue and the IO request R originated from the same write-back buffer WBC have been traversed, and if notAfter the completion of the operation, the IO request R from the same write-back buffer WBC as the IO request R is continuously searched from the waiting queueiJump to execute step S4.3A); otherwise, skipping to execute the next step;
S4.6A) removes the IO request R from the wait queue and enqueues the IO request R into the execution queue, executing all IO requests enqueued in turn.
Through the loop traversal of steps S4.3A) -S4.5A) in this embodiment, IO requests { R) originating from the same write-back buffer WBC as IO request R can be fetched separatelyi,…,RjTime stamp of { TS }i,…,TSjRequesting IO for { R }i,…,RjTime stamp of { TS }i,…,TSjAre respectively connected with TSrComparing, and making the time stamp earlier than TSrThe requests are removed from the waiting queue and are sent to the next stage of the file system in sequence for execution.
As shown in fig. 4, sub-diagram (b), the processing step after the queue-waiting workflow for activating the IO request in step S4) includes:
S4.1B) fetching an IO request R from the dispatch wait queue;
S4.2B) wait time T for IO request RwSubtracting the preset scheduling time slice T, and judging the waiting time T after subtracting the preset scheduling time slice TwWhether the current value is 0 or not, and if the current value is zero, activating a scheduling execution workflow of the IO request R; otherwise, judging whether the scheduling waiting queue is traversed or not, if not, continuously traversing the scheduling waiting queue to take out an IO request R, and jumping to execute the step S4.2B); otherwise, skipping to execute the next step;
S4.3B) whether any of the following conditions is satisfied: condition 1: the time slice expires; condition 2: a new IO request enters a scheduling waiting queue; if either is satisfied, then jump to execution step S4.1B), otherwise wait for activation.
As an optional implementation manner, in this embodiment, the flows corresponding to subgraphs (a) to (c) in fig. 4 are executed by using different threads, and the activation is to activate the corresponding thread.
In summary, the parallel file system oriented IO request scheduling method of the embodiment has the following advantages: aiming at the problem of how to reduce the IO request delay under the condition of the mixed load mode, the delay sensitivity is respectively calculated in the parallel file system according to the type of the IO request, and the queue order of the IO request is adjusted according to the delay sensitivity. The method is suitable for a general parallel file system framework, can be used in series with a request scheduling system of an original system, and is beneficial to a subsequent scheduling system to perform priority scheduling in the request sequence output after scheduling by the method of the embodiment because delay sensitive requests are arranged in front.
In addition, the embodiment also provides an IO request scheduling system for reducing access delay in a parallel file system, which includes a microprocessor and a memory connected to each other, where the microprocessor is programmed or configured to execute the steps of the IO request scheduling method for reducing access delay in a parallel file system.
Furthermore, the present embodiment also provides a computer-readable storage medium, in which a computer program programmed or configured to execute the IO request scheduling method for reducing access delay in the parallel file system is stored.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the scope of the present invention is not limited to the above embodiments, and all technical solutions that belong to the idea of the present invention belong to the scope of the present invention. It should be noted that modifications and adaptations to those skilled in the art without departing from the principles of the present invention should also be considered as within the scope of the present invention.
Claims (5)
1. An IO request scheduling method for reducing access delay in a parallel file system is characterized by comprising
1) Calculating and marking delay sensitivity to IO (input/output) requests of clients of the parallel file system for accessing storage equipment of a server;
2) under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially to reduce the queuing time under the condition of competition congestion and achieve the purpose of reducing delay;
the step of calculating the delay sensitivity in step 1) includes: judging whether the IO request is an IO request buffered by the write-back buffer WBC, if so, judging the IO request to be a delay-sensitive IO request, otherwise, judging the IO request to be a delay-insensitive IO request, and aiming at the delay-insensitive IO request R dioTaking a preset fixed value as the delay sensitivity obtained by calculation; for delay sensitive IO requests RwbcCalculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO, wherein calculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO means: obtaining an available space ratio P of a write-back buffer WBC, and taking 1-P as the calculated delay sensitivity of the IO request;
the method also comprises the following steps that the server side of the parallel file system carries out IO request processing:
s1) initializing the maximum waiting time Tdeadline；
S2) receives the IO request R and records the timestamp TSr；
S3) analyzing the IO request R to obtain the delay sensitivity S and analyzing the delay sensitivity according to Tw=Tdeadline(1-S) calculating the waiting time Tw；
S4) determining the waiting time TwIf it is 0, if the waiting time T iswIf the number is 0, activating the scheduling execution workflow of the IO request R; otherwise, the IO request R is put into a scheduling waiting queue, and a queuing waiting work flow of the IO request R is activated;
the processing step after the scheduling execution workflow of the IO request is activated in step S4) includes:
S4.1A) receives the IO request R, fetches the timestamp TSr；
S4.2A) to traverse from the wait queue to find an IO request R that originates from the same write-back buffered WBC as the IO request R i；
S4.3A) fetch IO request RiTime stamp TS ofi；
S4.4A) judges the IO request RiWhen (2)Timestamp TSiTimestamp TS less than IO request RrIf yes, remove IO request R from wait queueiAnd request IO for RiThe send-in execution queue;
S4.5A) determining whether the IO request in the waiting queue and the IO request R originated from the same WBC are completely traversed, if not, continuously traversing and finding the IO request R originated from the same WBC from the waiting queue and the IO request RiJump to execute step S4.3A); otherwise, skipping to execute the next step;
S4.6A) removing the IO request R from the waiting queue, sending the IO request R into the execution queue, and sequentially executing all IO requests sent into the execution queue;
the processing step after the queue waiting workflow of the IO request is activated in step S4) includes:
S4.1B) fetching an IO request R from the dispatch wait queue traversal;
S4.2B) wait time T for IO request RwSubtracting a preset scheduling time slice T, and judging the waiting time T after subtracting the preset scheduling time slice TwIf the number of the IO request R is zero, activating a scheduling execution workflow of the IO request R; otherwise, judging whether the scheduling waiting queue is completely traversed, if not, continuously traversing from the scheduling waiting queue to take out an IO request R, and skipping to the step S4.2B); otherwise, skipping to execute the next step;
S4.3B) whether any of the following conditions is satisfied: condition 1: the time slice expires; condition 2: a new IO request enters a scheduling waiting queue; if either is satisfied, then jump to execution step S4.1B), otherwise wait for activation.
2. The IO request scheduling method for reducing access delay in a parallel file system according to claim 1, wherein when the delay sensitivity is marked in step 1), the method includes adding a field of the delay sensitivity to the IO request, and the field includes the calculated delay sensitivity S.
3. According to claim 2The IO request scheduling method for reducing access delay in the parallel file system is characterized in that when the delay sensitivity is marked in the step 1), a field of a type is added to the IO request, and the field contains a delay insensitive IO request RdioOr delay sensitive IO request RwbcThe type of (d) identifies T.
4. An IO request scheduling system for reducing access delay in a parallel file system, comprising a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the steps of the IO request scheduling method for reducing access delay in the parallel file system according to any one of claims 1 to 3.
5. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, the computer program being programmed or configured to perform an IO request scheduling method for reducing access delay in a parallel file system according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110620133.9A CN113312323B (en) | 2021-06-03 | 2021-06-03 | IO (input/output) request scheduling method and system for reducing access delay in parallel file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110620133.9A CN113312323B (en) | 2021-06-03 | 2021-06-03 | IO (input/output) request scheduling method and system for reducing access delay in parallel file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113312323A CN113312323A (en) | 2021-08-27 |
| CN113312323B true CN113312323B (en) | 2022-07-19 |
Family
ID=77377255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110620133.9A Active CN113312323B (en) | 2021-06-03 | 2021-06-03 | IO (input/output) request scheduling method and system for reducing access delay in parallel file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113312323B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114124830B (en) * | 2021-11-19 | 2024-04-30 | 南京大学 | RDMA service quality assurance method and system for multiple application scenes of data center |
| CN116737673B (en) * | 2022-09-13 | 2024-03-15 | 荣耀终端有限公司 | Scheduling method, equipment and storage medium of file system in embedded operating system |
| CN117453378B (en) * | 2023-12-25 | 2024-03-19 | 北京卡普拉科技有限公司 | Method, device, equipment and medium for scheduling I/O requests among multiple application programs |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104321758A (en) * | 2013-01-17 | 2015-01-28 | 英特尔公司 | Arbitrating memory access via a shared memory fabric |
| US8949489B1 (en) * | 2012-03-21 | 2015-02-03 | Google Inc. | Method for combining bulk and latency-sensitive input and output |
| CN107145388A (en) * | 2017-05-25 | 2017-09-08 | 深信服科技股份有限公司 | Method for scheduling task and system under a kind of multitask environment |
| CN107454017A (en) * | 2017-06-05 | 2017-12-08 | 上海交通大学 | Mixed data flow coordinated dispatching method in a kind of cloud data center network |
| CN107589997A (en) * | 2017-08-29 | 2018-01-16 | 山东师范大学 | Ensure delay-sensitive program QoS dynamic regulating method under data center environment |
| CN107707326A (en) * | 2017-11-10 | 2018-02-16 | 鹤壁天海电子信息系统有限公司 | A kind of TDMA two-stage time slot management methods of terminaloriented |
| CN107885667A (en) * | 2016-09-29 | 2018-04-06 | 北京忆恒创源科技有限公司 | Reduce the method and apparatus of read command processing delay |
| CN109347974A (en) * | 2018-11-16 | 2019-02-15 | 北京航空航天大学 | A kind of online offline mixed scheduling system improving online service quality and cluster resource utilization |
| CN110716797A (en) * | 2019-09-10 | 2020-01-21 | 无锡江南计算技术研究所 | DDR4 performance balance scheduling structure and method for multiple request sources |
| CN111328148A (en) * | 2020-03-11 | 2020-06-23 | 展讯通信（上海）有限公司 | Data transmission method and device |
| CN111444012A (en) * | 2020-03-03 | 2020-07-24 | 中国科学院计算技术研究所 | Dynamic resource regulation and control method and system for guaranteeing delay sensitive application delay S L O |
| CN111782355A (en) * | 2020-06-03 | 2020-10-16 | 上海交通大学 | Cloud computing task scheduling method and system based on mixed load |
| CN112463044A (en) * | 2020-11-23 | 2021-03-09 | 中国科学院计算技术研究所 | Method and system for ensuring tail reading delay of server side of distributed storage system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10452572B2 (en) * | 2016-10-06 | 2019-10-22 | Vmware, Inc. | Automatic system service resource management for virtualizing low-latency workloads that are input/output intensive |
-
2021
- 2021-06-03 CN CN202110620133.9A patent/CN113312323B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8949489B1 (en) * | 2012-03-21 | 2015-02-03 | Google Inc. | Method for combining bulk and latency-sensitive input and output |
| CN104321758A (en) * | 2013-01-17 | 2015-01-28 | 英特尔公司 | Arbitrating memory access via a shared memory fabric |
| CN107885667A (en) * | 2016-09-29 | 2018-04-06 | 北京忆恒创源科技有限公司 | Reduce the method and apparatus of read command processing delay |
| CN107145388A (en) * | 2017-05-25 | 2017-09-08 | 深信服科技股份有限公司 | Method for scheduling task and system under a kind of multitask environment |
| CN107454017A (en) * | 2017-06-05 | 2017-12-08 | 上海交通大学 | Mixed data flow coordinated dispatching method in a kind of cloud data center network |
| CN107589997A (en) * | 2017-08-29 | 2018-01-16 | 山东师范大学 | Ensure delay-sensitive program QoS dynamic regulating method under data center environment |
| CN107707326A (en) * | 2017-11-10 | 2018-02-16 | 鹤壁天海电子信息系统有限公司 | A kind of TDMA two-stage time slot management methods of terminaloriented |
| CN109347974A (en) * | 2018-11-16 | 2019-02-15 | 北京航空航天大学 | A kind of online offline mixed scheduling system improving online service quality and cluster resource utilization |
| CN110716797A (en) * | 2019-09-10 | 2020-01-21 | 无锡江南计算技术研究所 | DDR4 performance balance scheduling structure and method for multiple request sources |
| CN111444012A (en) * | 2020-03-03 | 2020-07-24 | 中国科学院计算技术研究所 | Dynamic resource regulation and control method and system for guaranteeing delay sensitive application delay S L O |
| CN111328148A (en) * | 2020-03-11 | 2020-06-23 | 展讯通信（上海）有限公司 | Data transmission method and device |
| CN111782355A (en) * | 2020-06-03 | 2020-10-16 | 上海交通大学 | Cloud computing task scheduling method and system based on mixed load |
| CN112463044A (en) * | 2020-11-23 | 2021-03-09 | 中国科学院计算技术研究所 | Method and system for ensuring tail reading delay of server side of distributed storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113312323A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113312323B (en) | IO (input/output) request scheduling method and system for reducing access delay in parallel file system | |
| US10089142B2 (en) | Dynamic task prioritization for in-memory databases | |
| US8046758B2 (en) | Adaptive spin-then-block mutual exclusion in multi-threaded processing | |
| US5790851A (en) | Method of sequencing lock call requests to an O/S to avoid spinlock contention within a multi-processor environment | |
| US10133602B2 (en) | Adaptive contention-aware thread placement for parallel runtime systems | |
| Reda et al. | Rein: Taming tail latency in key-value stores via multiget scheduling | |
| US20060037017A1 (en) | System, apparatus and method of reducing adverse performance impact due to migration of processes from one CPU to another | |
| US9875141B2 (en) | Managing pools of dynamic resources | |
| JP2009541848A (en) | Method, system and apparatus for scheduling computer microjobs to run uninterrupted | |
| JP2003044295A (en) | Sleep queue management | |
| US20060037021A1 (en) | System, apparatus and method of adaptively queueing processes for execution scheduling | |
| US20230127112A1 (en) | Sub-idle thread priority class | |
| US10409640B1 (en) | Methods and apparatus for data request scheduling in performing parallel IO operations | |
| WO2005048009A2 (en) | Method and system for multithreaded processing using errands | |
| JP2009541851A (en) | Resource-based scheduler | |
| Mutlu et al. | Parallelism-aware batch scheduling: Enabling high-performance and fair shared memory controllers | |
| Pang et al. | Efficient CUDA stream management for multi-DNN real-time inference on embedded GPUs | |
| CN111597044A (en) | Task scheduling method and device, storage medium and electronic equipment | |
| EP2840513B1 (en) | Dynamic task prioritization for in-memory databases | |
| US10713089B2 (en) | Method and apparatus for load balancing of jobs scheduled for processing | |
| US11061724B2 (en) | Programmable hardware scheduler for digital processing systems | |
| US11113101B2 (en) | Method and apparatus for scheduling arbitration among a plurality of service requestors | |
| CN110837415A (en) | Thread scheduling method and device based on RISC-V multi-core processor | |
| JP5299869B2 (en) | Computer micro job | |
| US9043353B2 (en) | Query stream execution using priority gradient multiprogramming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |