CN113312323A - IO (input/output) request scheduling method and system for reducing access delay in parallel file system - Google Patents

IO (input/output) request scheduling method and system for reducing access delay in parallel file system Download PDF

Info

Publication number
CN113312323A
CN113312323A CN202110620133.9A CN202110620133A CN113312323A CN 113312323 A CN113312323 A CN 113312323A CN 202110620133 A CN202110620133 A CN 202110620133A CN 113312323 A CN113312323 A CN 113312323A
Authority
CN
China
Prior art keywords
request
delay
file system
scheduling
parallel file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110620133.9A
Other languages
Chinese (zh)
Other versions
CN113312323B (en
Inventor
周恩强
董勇
张伟
谢旻
迟万庆
朱清华
邬会军
张文喆
李佳鑫
吴振伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202110620133.9A priority Critical patent/CN113312323B/en
Publication of CN113312323A publication Critical patent/CN113312323A/en
Application granted granted Critical
Publication of CN113312323B publication Critical patent/CN113312323B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses an IO request scheduling method and system for reducing access delay in a parallel file system, wherein the method comprises the steps of calculating IO requests of client access server storage equipment of the parallel file system and marking delay sensitivity; under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay. The invention aims to reduce the waiting time Tw of partial IO requests under the IO request competition condition, thereby reducing the IO response delay sensed by the client application, and being capable of improving the comprehensive processing performance of various computer systems using parallel file systems, including large-scale high-performance computer systems, aiming at the typical workload with mixed requirements of high bandwidth and low IO delay.

Description

IO (input/output) request scheduling method and system for reducing access delay in parallel file system
Technical Field
The invention relates to the field of computer operating systems, in particular to an IO (input/output) request scheduling method and system for reducing access delay in a parallel file system.
Background
The parallel file system is an important component of a high-performance computer, manages a large number of storage devices, and is responsible for scheduling a large number of concurrent IO requests, the concurrent IO requests are sequentially sent to the storage devices, the requests are sequentially executed, different execution orders of the IO requests can show different performance performances, although the storage devices are usually provided with request queues, the IO queues of the storage devices are limited in depth, and the IO requests can be queued in a software layer firstly under a large-scale concurrent condition, so an IO request scheduler of the parallel file system layer has a large influence on IO response time of a user program and throughput rate of the storage devices.
The IO request scheduler of the system software is responsible for ordering the execution sequence of IO requests. For example, the Linux operating system is provided with various scheduling strategies, the requested scheduling optimization is implemented from the aspects of optimizing performance, fairness, preventing starvation and the like, and users select the scheduling strategies according to respective workload characteristics. The Lustre parallel file system is a typical representative of a parallel file system in the high-performance computing field, and a request scheduler of the Lustre parallel file system also has a plurality of selectable scheduling strategies, such as bandwidth priority and the like. Different schedulers operate according to respective scheduling policies, with a scheduling policy generally being valid for a particular workload and invalid for other workloads. When multiple workload patterns coexist, how IO requests are scheduled becomes an important and complex issue.
In a typical workload scenario for high performance computing, a parallel file system faces a mix of workload patterns. Traditional scientific computing application programs are sensitive to IO bandwidth, and novel applications such as intensive data processing and artificial intelligence are sensitive to IO delay. The mixed demand mode of high bandwidth and low IO latency becomes a typical workload under the condition that high performance computing systems are used by multi-domain application sharing competition. Under the new load mode, both the delay-sensitive IO requests and the bandwidth-sensitive IO requests are queued in a queue of the parallel file system, and when the number of the requests is large, competition occurs, which inevitably results in a long time for the IO requests to wait for scheduling, which affects the performance of the delay-sensitive application program. Under the condition of using the traditional disk medium, because the latency of the disk access is large, the IO request queuing time is low, and this problem is not obvious, but when a new storage medium (for example, a new-type nonvolatile storage medium NVM (Non-Volatile Memory) starts to be widely used, because the access latency is lower than the access latency of the disk by several orders of magnitude, the latency of the IO request queuing time in the whole IO access flow is rapidly increased, and the influence on the performance of the application program is obvious.
Disclosure of Invention
Parallel file systems generally adopt a multi-client and multi-server architecture, IO requests sent by multiple types of application programs at the client at the same time compete for storage devices of the server, when the number of IO requests exceeds the IO processing capacity of the storage devices of the server, the IO requests queue in a queue of the server, and the waiting time Tw becomes a component of IO request delay. The technical problem to be solved by the invention is as follows: the method and the system aim to reduce the waiting time Tw of partial IO requests under the IO request competition condition, thereby reducing the IO response delay sensed by client application, and being capable of improving the comprehensive processing performance of various computer systems using the parallel file system aiming at the typical workload with mixed requirements of high bandwidth and low IO delay, including large-scale high-performance computer systems.
In order to solve the technical problems, the invention adopts the technical scheme that:
an IO request scheduling method for reducing access delay in a parallel file system comprises
1) Calculating and marking delay sensitivity to IO (input/output) requests of clients of the parallel file system for accessing storage equipment of a server;
2) under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay.
Optionally, the step of calculating the delay sensitivity in step 1) includes: judging whether the IO request is an IO request buffered by a write-back buffer WBC, if so, judging the IO request to be a delay-sensitive IO request, otherwise, judging the IO request to be a delay-insensitive IO request, and aiming at the delay-insensitive IO request RdioTaking a preset fixed value as the delay sensitivity obtained by calculation; for delay sensitive IO requests RwbcThen the delay sensitivity of the IO request is calculated based on the correlation of the write-back buffered WBCs and the IO.
Optionally, the calculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO refers to: the available space ratio P of the write-back buffer WBC is obtained, and 1-P is taken as the calculated delay sensitivity of the IO request.
Optionally, when the delay sensitivity is marked in step 1), a field for adding a delay sensitivity to the IO request is included, and the field includes the calculated delay sensitivity S.
Optionally, when the delay sensitivity is marked in step 1), the method further includes adding a type field to the IO request, and the field includes a delay-insensitive IO request RdioOr delay sensitive IO request RwbcThe type of (d) identifies T.
Optionally, the method further includes the following steps that a server of the parallel file system performs IO request processing:
s1) initializing the maximum waiting time Tdeadline；
S2) receives the IO request R andrecording time stamp TSr；
S3) resolving the IO request R to obtain the delay sensitivity S and according to Tw＝Tdeadline(1-S) calculating the waiting time Tw；
S4) determining the waiting time TwIf it is 0, if the waiting time T iswIf the number is 0, activating the scheduling execution workflow of the IO request R; otherwise, the IO request R is put into a scheduling waiting queue, and a queuing waiting workflow of the IO request R is activated.
Optionally, the processing step after the schedule execution workflow of the IO request is activated in step S4) includes:
S4.1A) receives the IO request R, fetches the timestamp TSr；
S4.2A) to traverse from the wait queue to find an IO request R that originates from the same write-back buffered WBC as the IO request Ri；
S4.3A) fetch IO request RiTime stamp TS ofi；
S4.4A) determines the IO request RiTime stamp TS ofiTimestamp TS less than IO request RrIf yes, remove IO request R from wait queueiAnd request IO for RiThe send-in execution queue;
S4.5A) determining whether the IO request in the waiting queue and the IO request R originated from the same WBC are completely traversed, if not, continuously traversing and finding the IO request R originated from the same WBC from the waiting queue and the IO request RiJump to execute step S4.3A); otherwise, skipping to execute the next step;
S4.6A) removes the IO request R from the wait queue and enqueues the IO request R into the execution queue, executing all IO requests enqueued in turn.
Optionally, the processing step after the queue-waiting workflow of the IO request is activated in step S4) includes:
S4.1B) fetching an IO request R from the dispatch wait queue traversal;
S4.2B) wait time T for IO request RwSubtracting the preset scheduling time slice t, and judging the value obtained after subtracting the preset scheduling time slice tWaiting time TwIf the number of the IO request R is zero, activating a scheduling execution workflow of the IO request R; otherwise, judging whether the scheduling waiting queue is completely traversed, if not, continuously traversing from the scheduling waiting queue to take out an IO request R, and skipping to the step S4.2B); otherwise, skipping to execute the next step;
S4.3B) whether any of the following conditions is satisfied: condition 1: the time slice expires; condition 2: a new IO request enters a scheduling waiting queue; if either is satisfied, then jump to execution step S4.1B), otherwise wait for activation.
In addition, the invention also provides an IO request scheduling system for reducing access delay in a parallel file system, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the steps of the IO request scheduling method for reducing access delay in the parallel file system.
Furthermore, the present invention also provides a computer-readable storage medium having stored therein a computer program programmed or configured to execute an IO request scheduling method for reducing access latency in the parallel file system.
Compared with the prior art, the invention mainly has the following advantages:
1. calculating and marking delay sensitivity to IO (input/output) requests of clients of a parallel file system for accessing storage equipment of a server; under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are preferentially scheduled to reduce the queuing time under the condition of competition congestion and achieve the purpose of reducing the delay.
2. The invention is suitable for large-scale high-performance computer systems, such systems generally adopt a parallel file system to provide IO access service, the parallel file system usually adopts a Write-Back buffer technology (WBC, Write Back Cache) on a client, namely, IO calls sent by an application program temporarily store data in a buffer area of the client, then return the data immediately, the WBC selects a proper time and sends the data to a storage device at the rear end, and the WBC can reassemble discrete IO requests into continuous IO requests, thereby improving the access bandwidth.
3. The invention is suitable for a general parallel file system framework, can be used in series with a request scheduling system of an original system, and in a request sequence output after scheduling by the embodiment, delay sensitive requests are arranged in front, thereby being beneficial to the prior scheduling execution of a subsequent scheduling system.
Drawings
FIG. 1 is a core flow diagram of a method according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a system according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a process of calculating and labeling delay sensitivity according to an embodiment of the present invention.
Fig. 4 is a schematic processing flow diagram of the server according to the embodiment of the present invention.
Detailed Description
As shown in fig. 1, the IO request scheduling method for reducing access delay in a parallel file system in this embodiment includes:
1) calculating and marking delay sensitivity to IO (input/output) requests of clients of the parallel file system for accessing storage equipment of a server;
2) under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay.
The IO request scheduling method for reducing access delay in the parallel file system of this embodiment assigns a delay sensitivity to each IO request, and the IO request with high delay sensitivity is preferentially scheduled and executed, thereby reducing queuing time under contention congestion, and achieving the purpose of reducing delay.
Referring to fig. 2, as an optional implementation manner, in this embodiment, each IO request is given a delay sensitivity based on the correlation between the WBC and the IO request in the client, and the IO request with high delay sensitivity is preferentially scheduled and executed, so that the queuing time of the IO request under the contention congestion condition is reduced, and the purpose of reducing the delay is achieved.
Referring to fig. 3, the step of calculating the delay sensitivity in step 1) of the present embodiment includes: judging whether the IO request is an IO request buffered by a write-back buffer WBC, if so, judging the IO request to be a delay-sensitive IO request, otherwise, judging the IO request to be a delay-insensitive IO request, and aiming at the delay-insensitive IO request RdioTaking a preset fixed value as the delay sensitivity obtained by calculation; for delay sensitive IO requests RwbcThen the delay sensitivity of the IO request is calculated based on the correlation of the write-back buffered WBCs and the IO. In this embodiment, the IO requests of the application program are classified into 2 types: WBC buffered requests (hereinafter referred to as R)wbc) And requests that bypass WBC buffering (hereinafter referred to as R)dio). According to the WBC principle, RwbcAfter the carried data is written into the cache, the application program is informed that the IO is finished, so that the application program does not need to wait for the data to be really written into the storage equipment, and the data is selected by the file system in the backgroundThe machine sends to the storage device. The application need not wait for RwbcTrue completion is not sensitive to its completion time, so this type of request is defined as a delay insensitive request. For RdioThe application must wait until the IO request operation is completed at the storage device. Thus, R can be seendioIs directly influencing the execution time of the application and is therefore defined as a delay sensitive IO request. In the embodiment, whether the IO request enters the write-back buffer WBC is not controlled, the IO request is only discriminated at the client of the parallel file system, and a field describing delay sensitivity is added to the two types of IO requests, so that the delay-sensitive request is endowed with relatively high delay sensitivity, and the delay-insensitive request is endowed with relatively low delay sensitivity. The calculation rule of the delay sensitivity is as follows: the delay sensitivity of the Rdio request takes a fixed maximum value, and the delay sensitivity of the Rwbc request needs to be calculated according to the space availability of the WBC, so as to prevent the Rwbc request from being excessively delayed and scheduled to cause the data in the WBC not to be emptied smoothly.
Referring to fig. 3, in the present embodiment, calculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO refers to: the available space ratio P of the write-back buffer WBC is obtained, and 1-P is taken as the calculated delay sensitivity of the IO request.
In this embodiment, when the delay sensitivity is marked in step 1), a field for adding a delay sensitivity to the IO request is included, and the field includes the calculated delay sensitivity S.
In this embodiment, when the delay sensitivity is marked in step 1), the method further includes adding a type field to the IO request, where the type field includes a delay-insensitive IO request RdioOr delay sensitive IO request RwbcThe type of (d) identifies T.
As shown in fig. 4, sub-diagram (a), this embodiment further includes the following steps of performing IO request processing by the server of the parallel file system:
s1) initializing the maximum waiting time Tdeadline；
S2) receives the IO request R and records the timestamp TSr；
S3) resolution from IO request RObtaining a delay sensitivity S according to Tw＝Tdeadline(1-S) calculating the waiting time Tw(ii) a I.e. the smaller S, the smaller TwThe larger the ratio, the more inverse linear relationship, i.e. when S is 0, T is correspondedw＝TdeadlineThe waiting time is longest; when S is 1, corresponding to Tw0, no waiting is needed;
s4) determining the waiting time TwIf it is 0, if the waiting time T iswIf the number is 0, activating the scheduling execution workflow of the IO request R; otherwise, the IO request R is put into a scheduling waiting queue, and a queuing waiting workflow of the IO request R is activated.
The present embodiment, step S1), will wait for the maximum time TdeadlineThe initialization setting is the time for the WBC available capacity to drop from 100% to 0%, as may be found based on specific system measurements.
In addition, step S1) of this embodiment further includes a step of initializing and setting a scheduling time slice t, where the scheduling time slice t is used for processing IO requests in the scheduling wait queue in a loop traversal manner. In this embodiment, assuming that the precision of the field S is g, the value of the scheduling time slice t is initialized to t ═ Tdeadline ×, i.e., RwbcAnd requesting the minimum granularity of the waiting time, and activating the scheduling flow at intervals of time t by the server.
In this embodiment, a scheduling wait queue is set at the server, request scheduling is implemented according to the delay sensitivity of the received IO request, and a request with high delay sensitivity is scheduled and executed preferentially. In the case of two types of request contention, the scheduled execution of Rwbc is delayed and Tw thereof increases, but according to the working principle of WBC, when there is available space in WBC, the application program issuing the Rwbc request does not have to wait for the completion of the request, so the moderately delayed scheduling does not affect the normal execution of the application. The delay of Rwbc instead causes the Rdio request to be in a preferentially scheduled position in the queue, and therefore has a shorter latency Tw, thereby reducing the IO delay perceptible to the application issuing the Rdio request.
As shown in fig. 4, sub-diagram (c), the processing step after the scheduling execution workflow of the IO request is activated in step S4) includes:
S4.1A) receives the IO request R, fetches the timestamp TSr；
S4.2A) to traverse from the wait queue to find an IO request R that originates from the same write-back buffered WBC as the IO request Ri；
S4.3A) fetch IO request RiTime stamp TS ofi；
S4.4A) determines the IO request RiTime stamp TS ofiTimestamp TS less than IO request RrIf yes, remove IO request R from wait queueiAnd request IO for RiThe send-in execution queue;
S4.5A) determining whether the IO request in the waiting queue and the IO request R originated from the same WBC are completely traversed, if not, continuously traversing and finding the IO request R originated from the same WBC from the waiting queue and the IO request RiJump to execute step S4.3A); otherwise, skipping to execute the next step;
S4.6A) removes the IO request R from the wait queue and enqueues the IO request R into the execution queue, executing all IO requests enqueued in turn.
Through the loop traversal of steps S4.3A) -S4.5A) in this embodiment, IO requests { R) originating from the same write-back buffer WBC as IO request R can be fetched separatelyi,…,RjTime stamp of { TS }i,…,TSjRequesting IO for { R }i,…,RjTime stamp of { TS }i,…,TSjAre respectively connected with TSrComparing, and making the time stamp earlier than TSrThe requests are removed from the waiting queue and are sent to the next stage of the file system in sequence for execution.
As shown in fig. 4, sub-diagram (b), the processing steps after the queue-waiting workflow for activating the IO request in step S4) include:
S4.1B) fetching an IO request R from the dispatch wait queue traversal;
S4.2B) wait time T for IO request RwSubtracting a preset scheduling time slice T, and judging the waiting time T after subtracting the preset scheduling time slice TwIf the number of the IO request R is zero, activating a scheduling execution workflow of the IO request R; otherwise, judging whether the scheduling waiting queue is traversed completely or not, and if not, judging whether the scheduling waiting queue is traversed completely or notContinuously taking out an IO request R from the scheduling waiting queue in a traversing way after the completion of the process, and jumping to execute the step S4.2B); otherwise, skipping to execute the next step;
S4.3B) whether any of the following conditions is satisfied: condition 1: the time slice expires; condition 2: a new IO request enters a scheduling waiting queue; if either is satisfied, then jump to execution step S4.1B), otherwise wait for activation.
As an optional implementation manner, in this embodiment, the flows corresponding to subgraphs (a) to (c) in fig. 4 are executed by using different threads, and the activation is to activate the corresponding thread.
In summary, the parallel file system oriented IO request scheduling method of the embodiment has the following advantages: aiming at the problem of how to reduce the IO request delay under the condition of the mixed load mode, the delay sensitivity is respectively calculated in the parallel file system according to the type of the IO request, and the queue order of the IO request is adjusted according to the delay sensitivity. The method is suitable for a general parallel file system framework, can be used in series with a request scheduling system of an original system, and is beneficial to a subsequent scheduling system to perform priority scheduling in the request sequence output after scheduling by the method of the embodiment because delay sensitive requests are arranged in front.
In addition, the present embodiment also provides an IO request scheduling system for reducing access delay in a parallel file system, including a microprocessor and a memory, which are connected to each other, and the microprocessor is programmed or configured to execute the steps of the IO request scheduling method for reducing access delay in the parallel file system.
Furthermore, the present embodiment also provides a computer-readable storage medium, in which a computer program is stored, the computer program being programmed or configured to execute the IO request scheduling method for reducing access latency in the parallel file system.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.

Claims (10)

1. An IO request scheduling method for reducing access delay in a parallel file system is characterized by comprising
1) Calculating and marking delay sensitivity to IO (input/output) requests of clients of the parallel file system for accessing storage equipment of a server;
2) under the condition that a plurality of IO requests are in competition, the IO requests with higher delay sensitivity are scheduled preferentially so as to reduce the queuing time of the IO requests under the condition of competition congestion and achieve the purpose of reducing delay.
2. The IO request scheduling method for reducing access delay in a parallel file system according to claim 1, wherein the step of calculating the delay sensitivity in step 1) comprises: judging whether the IO request is an IO request buffered by a write-back buffer WBC, if so, judging the IO request to be a delay-sensitive IO request, otherwise, judging the IO request to be a delay-insensitive IO request, and aiming at the delay-insensitive IO request RdioTaking a preset fixed value as the delay sensitivity obtained by calculation; for delay sensitive IO requests RwbcThen the delay sensitivity of the IO request is calculated based on the correlation of the write-back buffered WBCs and the IO.
3. The IO request scheduling method for reducing access delay in a parallel file system according to claim 2, wherein the calculating the delay sensitivity of the IO request based on the correlation between the write-back buffer WBC and the IO is: the available space ratio P of the write-back buffer WBC is obtained, and 1-P is taken as the calculated delay sensitivity of the IO request.
4. The IO request scheduling method for reducing access delay in a parallel file system according to claim 3, wherein when the delay sensitivity is marked in step 1), the method includes adding a field of the delay sensitivity to the IO request, and the field includes the calculated delay sensitivity S.
5. The IO request scheduling method for reducing access delay in a parallel file system according to claim 4, wherein when the delay sensitivity is marked in step 1), the method further comprises adding a type field to the IO request, and the type field contains a delay-insensitive IO request RdioOr delay sensitive IO request RwbcThe type of (d) identifies T.
6. The IO request scheduling method for reducing the access delay in the parallel file system according to any one of claims 1 to 5, further comprising the following steps of performing IO request processing by a server of the parallel file system:
s1) initializing the maximum waiting time Tdeadline；
S2) receives the IO request R and records the timestamp TSr；
S3) resolving the IO request R to obtain the delay sensitivity S and according to Tw＝Tdeadline(1-S) calculating the waiting time Tw；
S4) determining the waiting time TwIf it is 0, if the waiting time T iswIf the number is 0, activating the scheduling execution workflow of the IO request R; otherwise, the IO request R is put into a scheduling waiting queue, and a queuing waiting workflow of the IO request R is activated.
7. The IO request scheduling method for reducing access delay in a parallel file system according to claim 6, wherein the processing step after the scheduling execution workflow of the IO request is activated in step S4) includes:
S4.1A) receives the IO request R, fetches the timestamp TSr；
S4.2A) to traverse from the wait queue to find an IO request R that originates from the same write-back buffered WBC as the IO request Ri；
S4.3A) fetch IO request RiTime stamp TS ofi；
S4.4A) determines the IO request RiTime stamp TS ofiTimestamp TS less than IO request RrIf yes, remove IO request R from wait queueiAnd request IO for RiThe send-in execution queue;
S4.5A) determining whether the IO request in the waiting queue and the IO request R originated from the same WBC are completely traversed, if not, continuously traversing and finding the IO request R originated from the same WBC from the waiting queue and the IO request RiJump to execute step S4.3A); otherwise, skipping to execute the next step;
S4.6A) removes the IO request R from the wait queue and enqueues the IO request R into the execution queue, executing all IO requests enqueued in turn.
8. The IO request scheduling method for reducing access delay in a parallel file system according to claim 7, wherein the processing step after the queue-waiting workflow for activating the IO request in step S4) includes:
S4.1B) fetching an IO request R from the dispatch wait queue traversal;
S4.2B) wait time T for IO request RwSubtracting a preset scheduling time slice T, and judging the waiting time T after subtracting the preset scheduling time slice TwIf the number of the IO request R is zero, activating a scheduling execution workflow of the IO request R; otherwise, judging whether the scheduling waiting queue is completely traversed, if not, continuously traversing from the scheduling waiting queue to take out an IO request R, and skipping to the step S4.2B); otherwise, skipping to execute the next step;
S4.3B) whether any of the following conditions is satisfied: condition 1: the time slice expires; condition 2: a new IO request enters a scheduling waiting queue; if either is satisfied, then jump to execution step S4.1B), otherwise wait for activation.
9. An IO request scheduling system for reducing access delay in a parallel file system, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the steps of the IO request scheduling method for reducing access delay in a parallel file system according to any one of claims 1 to 8.
10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, the computer program being programmed or configured to perform an IO request scheduling method for reducing access latency in a parallel file system according to any one of claims 1 to 8.
CN202110620133.9A 2021-06-03 2021-06-03 IO (input/output) request scheduling method and system for reducing access delay in parallel file system Active CN113312323B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110620133.9A CN113312323B (en) 2021-06-03 2021-06-03 IO (input/output) request scheduling method and system for reducing access delay in parallel file system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110620133.9A CN113312323B (en) 2021-06-03 2021-06-03 IO (input/output) request scheduling method and system for reducing access delay in parallel file system

Publications (2)

Publication Number Publication Date
CN113312323A true CN113312323A (en) 2021-08-27
CN113312323B CN113312323B (en) 2022-07-19

Family

ID=77377255

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110620133.9A Active CN113312323B (en) 2021-06-03 2021-06-03 IO (input/output) request scheduling method and system for reducing access delay in parallel file system

Country Status (1)

Country Link
CN (1) CN113312323B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114124830A (en) * 2021-11-19 2022-03-01 南京大学 RDMA service quality assurance method and system for multiple application scenes of data center
CN116737673A (en) * 2022-09-13 2023-09-12 荣耀终端有限公司 Scheduling method, equipment and storage medium of file system in embedded operating system
CN117453378A (en) * 2023-12-25 2024-01-26 北京卡普拉科技有限公司 Method, device, equipment and medium for scheduling I/O requests among multiple application programs

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104321758A (en) * 2013-01-17 2015-01-28 英特尔公司 Arbitrating memory access via a shared memory fabric
US8949489B1 (en) * 2012-03-21 2015-02-03 Google Inc. Method for combining bulk and latency-sensitive input and output
CN107145388A (en) * 2017-05-25 2017-09-08 深信服科技股份有限公司 Method for scheduling task and system under a kind of multitask environment
CN107454017A (en) * 2017-06-05 2017-12-08 上海交通大学 Mixed data flow coordinated dispatching method in a kind of cloud data center network
CN107589997A (en) * 2017-08-29 2018-01-16 山东师范大学 Ensure delay-sensitive program QoS dynamic regulating method under data center environment
CN107707326A (en) * 2017-11-10 2018-02-16 鹤壁天海电子信息系统有限公司 A kind of TDMA two-stage time slot management methods of terminaloriented
CN107885667A (en) * 2016-09-29 2018-04-06 北京忆恒创源科技有限公司 Reduce the method and apparatus of read command processing delay
US20180101486A1 (en) * 2016-10-06 2018-04-12 Vmware, Inc. Automatic System Service Resource Management for Virtualizing Low-Latency Workloads that are Input/Output Intensive
CN109347974A (en) * 2018-11-16 2019-02-15 北京航空航天大学 A kind of online offline mixed scheduling system improving online service quality and cluster resource utilization
CN110716797A (en) * 2019-09-10 2020-01-21 无锡江南计算技术研究所 DDR4 performance balance scheduling structure and method for multiple request sources
CN111328148A (en) * 2020-03-11 2020-06-23 展讯通信（上海）有限公司 Data transmission method and device
CN111444012A (en) * 2020-03-03 2020-07-24 中国科学院计算技术研究所 Dynamic resource regulation and control method and system for guaranteeing delay sensitive application delay S L O
CN111782355A (en) * 2020-06-03 2020-10-16 上海交通大学 Cloud computing task scheduling method and system based on mixed load
CN112463044A (en) * 2020-11-23 2021-03-09 中国科学院计算技术研究所 Method and system for ensuring tail reading delay of server side of distributed storage system

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8949489B1 (en) * 2012-03-21 2015-02-03 Google Inc. Method for combining bulk and latency-sensitive input and output
CN104321758A (en) * 2013-01-17 2015-01-28 英特尔公司 Arbitrating memory access via a shared memory fabric
CN107885667A (en) * 2016-09-29 2018-04-06 北京忆恒创源科技有限公司 Reduce the method and apparatus of read command processing delay
US20180101486A1 (en) * 2016-10-06 2018-04-12 Vmware, Inc. Automatic System Service Resource Management for Virtualizing Low-Latency Workloads that are Input/Output Intensive
CN107145388A (en) * 2017-05-25 2017-09-08 深信服科技股份有限公司 Method for scheduling task and system under a kind of multitask environment
CN107454017A (en) * 2017-06-05 2017-12-08 上海交通大学 Mixed data flow coordinated dispatching method in a kind of cloud data center network
CN107589997A (en) * 2017-08-29 2018-01-16 山东师范大学 Ensure delay-sensitive program QoS dynamic regulating method under data center environment
CN107707326A (en) * 2017-11-10 2018-02-16 鹤壁天海电子信息系统有限公司 A kind of TDMA two-stage time slot management methods of terminaloriented
CN109347974A (en) * 2018-11-16 2019-02-15 北京航空航天大学 A kind of online offline mixed scheduling system improving online service quality and cluster resource utilization
CN110716797A (en) * 2019-09-10 2020-01-21 无锡江南计算技术研究所 DDR4 performance balance scheduling structure and method for multiple request sources
CN111444012A (en) * 2020-03-03 2020-07-24 中国科学院计算技术研究所 Dynamic resource regulation and control method and system for guaranteeing delay sensitive application delay S L O
CN111328148A (en) * 2020-03-11 2020-06-23 展讯通信（上海）有限公司 Data transmission method and device
CN111782355A (en) * 2020-06-03 2020-10-16 上海交通大学 Cloud computing task scheduling method and system based on mixed load
CN112463044A (en) * 2020-11-23 2021-03-09 中国科学院计算技术研究所 Method and system for ensuring tail reading delay of server side of distributed storage system

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114124830A (en) * 2021-11-19 2022-03-01 南京大学 RDMA service quality assurance method and system for multiple application scenes of data center
CN114124830B (en) * 2021-11-19 2024-04-30 南京大学 RDMA service quality assurance method and system for multiple application scenes of data center
CN116737673A (en) * 2022-09-13 2023-09-12 荣耀终端有限公司 Scheduling method, equipment and storage medium of file system in embedded operating system
CN116737673B (en) * 2022-09-13 2024-03-15 荣耀终端有限公司 Scheduling method, equipment and storage medium of file system in embedded operating system
CN117453378A (en) * 2023-12-25 2024-01-26 北京卡普拉科技有限公司 Method, device, equipment and medium for scheduling I/O requests among multiple application programs
CN117453378B (en) * 2023-12-25 2024-03-19 北京卡普拉科技有限公司 Method, device, equipment and medium for scheduling I/O requests among multiple application programs

Also Published As

Publication number Publication date
CN113312323B (en) 2022-07-19

Similar Documents

Publication Publication Date Title
US10089142B2 (en) Dynamic task prioritization for in-memory databases
CN113312323B (en) IO (input/output) request scheduling method and system for reducing access delay in parallel file system
US5790851A (en) Method of sequencing lock call requests to an O/S to avoid spinlock contention within a multi-processor environment
US9208116B2 (en) Maintaining I/O priority and I/O sorting
US8397236B2 (en) Credit based performance managment of computer systems
US8392633B2 (en) Scheduling requesters of a shared storage resource
US8108571B1 (en) Multithreaded DMA controller
US8141089B2 (en) Method and apparatus for reducing contention for computer system resources using soft locks
US20090328053A1 (en) Adaptive spin-then-block mutual exclusion in multi-threaded processing
US20060037017A1 (en) System, apparatus and method of reducing adverse performance impact due to migration of processes from one CPU to another
JP2013232207A (en) Method, system, and apparatus for scheduling computer micro-jobs for execution without disruption
US8397234B2 (en) Determining a priority value for a thread for execution on a multithreading processor system
KR20050020942A (en) Continuous media priority aware storage scheduler
AU2011213795A1 (en) Efficient cache reuse through application determined scheduling
US20060037021A1 (en) System, apparatus and method of adaptively queueing processes for execution scheduling
US10409640B1 (en) Methods and apparatus for data request scheduling in performing parallel IO operations
Mutlu et al. Parallelism-aware batch scheduling: Enabling high-performance and fair shared memory controllers
CN111597044A (en) Task scheduling method and device, storage medium and electronic equipment
EP2840513B1 (en) Dynamic task prioritization for in-memory databases
US10713089B2 (en) Method and apparatus for load balancing of jobs scheduled for processing
US20180349180A1 (en) Method and apparatus for scheduling arbitration among a plurality of service requestors
JP5299869B2 (en) Computer micro job
US8245229B2 (en) Temporal batching of I/O jobs
US11372649B2 (en) Flow control for multi-threaded access to contentious resource(s)
US9043353B2 (en) Query stream execution using priority gradient multiprogramming

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant