CN115499513A - Data request processing method and device, computer equipment and storage medium - Google Patents
Data request processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115499513A CN115499513A CN202211008577.8A CN202211008577A CN115499513A CN 115499513 A CN115499513 A CN 115499513A CN 202211008577 A CN202211008577 A CN 202211008577A CN 115499513 A CN115499513 A CN 115499513A
- Authority
- CN
- China
- Prior art keywords
- queue
- depth
- waiting
- processing
- waiting queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/56—Queue scheduling implementing delay-aware scheduling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
- H04L47/6275—Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a data request processing method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring depth information of a waiting queue corresponding to each request type; respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type; aiming at each waiting queue, calculating according to the current idle request number of the processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests; and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing a data request, a computer device, and a storage medium.
Background
In a computer system, the requirements of processing aging for various data requests needing to be processed in the computer system are dynamically changed according to different application scenes. In order to guarantee the processing priority and timeliness of each data request, the data requests need to be managed in a unified manner.
In the related art, the priority of each data request is generally predetermined, and the data requests are processed according to the priority order of the data requests, so that the timeliness of the data requests with high priority can be ensured. However, this results in a backlog of low priority data requests that cannot be processed in a timely manner.
Disclosure of Invention
In view of the above, it is necessary to provide a data request processing method, an apparatus, a computer device, and a storage medium, which can guarantee processing timeliness of data requests of different priorities, in order to solve the above technical problems.
In a first aspect, the present application provides a method for processing a data request. The method comprises the following steps:
acquiring depth information of a waiting queue corresponding to each request type;
respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
aiming at each waiting queue, calculating according to the current idle request number of a processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests;
and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
The data request processing method provided by the embodiment of the application can dynamically calculate the current weight corresponding to each waiting queue pair, can ensure the balance between data requests with different priority weights in the processing queue, improve the processing capacity of low-priority data requests, avoid backlog of low-priority data requests, shorten the response time of high-priority data requests, improve the adaptive processing capacity of a service system to the data requests of each request type, and further improve the user experience.
In one embodiment, the depth information includes real-time queue depth;
the calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type respectively comprises:
for each waiting queue, determining the maximum priority weight in the priority weights of the waiting queues;
determining the maximum real-time queue depth in the real-time queue depths of the waiting queues, and calculating the queue depth weight of the waiting queues according to the real-time queue depth of the waiting queues, the maximum real-time queue depth and the maximum priority weight;
and determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue.
The data request processing method provided by the embodiment of the application can determine the trend of the processing pressure of the data requests of each request type, and calculate the current weight of the waiting queue corresponding to each request type according to the trend, so as to realize the balance of the current weights of the waiting queues of each request type with different priority weights.
In one embodiment, the depth information further includes a remaining queue depth;
determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue, wherein the determining comprises the following steps:
obtaining the remaining queue depth of each waiting queue in the previous period of the current processing period, and determining the maximum remaining queue depth in each remaining queue depth;
calculating backlog depth weight of the waiting queue according to the remaining queue depth of the waiting queue, the maximum remaining queue depth and the priority weight;
calculating a first sum of a priority weight of a waiting queue, a queue depth weight of the waiting queue and a backlog depth weight of the waiting queue, and taking the first sum as a current weight of the waiting queue.
The method for processing the data request provided by the embodiment of the application can dynamically adjust the weight of the waiting queue of each request type according to the number of the requests of each request type in the current processing period and the number of the remaining requests in the last period, thereby realizing the balance between the processing amount of the data request with high priority and the processing amount of the data request with low priority, improving the average response time of the data request with high priority, avoiding the backlog of the data request with low priority and improving the user experience.
In one embodiment, the depth information further includes a remaining queue depth;
determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue, wherein the determining comprises the following steps:
and calculating a second sum of the priority weight of the waiting queue and the queue depth weight of the waiting queue, and taking the first sum as the current weight of the waiting queue.
The data request processing method provided by the embodiment of the application can dynamically adjust the weight of the waiting queue of each request type according to the request number of each request type in the current processing period, thereby realizing the balance between the processing amount of the data request with high priority and the processing amount of the data request with low priority, improving the average response time of the data request with high priority, avoiding the backlog of the data request with low priority and improving the user experience.
In one embodiment, the method further comprises:
acquiring the real-time queue depth of the processing queue;
and calculating the current idle request number of the processing queue according to the preset maximum queue depth of the processing queue and the real-time queue depth.
According to the data request processing method provided by the embodiment of the application, the number of the current idle requests of the processing queue can be accurately obtained through the preset maximum queue depth and the real-time queue depth of the processing queue determined based on the actual application scene, and the backlog of the data requests is avoided.
In one embodiment, the method further comprises:
and under the condition that the real-time queue depth of the processing queue is less than or equal to a preset queue depth threshold value, executing the step of acquiring the depth information of the waiting queue corresponding to each request type.
The data request processing method provided by the embodiment of the application can ensure the continuity of processing the data requests in the processing queue, avoid the additional time consumption of waiting for the data requests to be added to the processing queue, and reduce the resource consumption of dynamically adjusting the current weight of each request type and extracting the data requests from the waiting queue.
In one embodiment, the method further comprises:
receiving a plurality of said data requests;
and under the condition that the data request processing link is determined to meet the preset congestion condition, determining the request type of each data request respectively, and adding each data request to a waiting queue corresponding to the request type of each data request.
According to the data request processing method provided by the embodiment of the application, under the condition that the data request processing link is determined to meet the preset congestion condition, the data requests are extracted from the waiting queues and added to the processing queues, the trend of the processing pressure of each request type in a certain time range can be accurately identified, and the dynamic adjustment of the weight of the waiting queues of each request type is realized.
In a second aspect, the present application further provides a device for processing a data request. The device comprises:
the acquisition module is used for acquiring depth information of the waiting queue corresponding to each request type;
the first calculation module is used for respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
the second calculation module is used for calculating according to the current idle request number of a processing queue and the current weight of the waiting queue aiming at each waiting queue to obtain the target data request number corresponding to the waiting queue, and the processing queue comprises a plurality of data requests;
and the adding module is used for extracting the data requests with the target data request number from the waiting queue and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
In one embodiment, the depth information includes real-time queue depth;
the first calculation module is specifically configured to:
determining the maximum priority weight in the priority weights of the waiting queues, and determining the maximum real-time queue depth in the real-time queue depths of the waiting queues;
aiming at each waiting queue, calculating the queue depth weight of the waiting queue according to the real-time queue depth of the waiting queue, the maximum real-time queue depth and the maximum priority weight;
and determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue.
In one embodiment, the depth information further includes a remaining queue depth;
the first calculation module is specifically configured to:
acquiring the remaining queue depth of each waiting queue in the previous period of the current processing period, and determining the maximum remaining queue depth in each remaining queue depth;
calculating backlog depth weight of the waiting queue according to the remaining queue depth of the waiting queue, the maximum remaining queue depth and the priority weight;
calculating a first sum of a priority weight of a waiting queue, a queue depth weight of the waiting queue and a backlog depth weight of the waiting queue, and taking the first sum as a current weight of the waiting queue.
In one embodiment, the depth information further includes a remaining queue depth;
the first calculation module is specifically configured to:
and calculating a second sum of the priority weight of the waiting queue and the queue depth weight of the waiting queue, and taking the second sum as the current weight of the waiting queue.
In one embodiment, the apparatus further comprises:
the real-time queue depth acquisition module is used for acquiring the real-time queue depth of the processing queue;
and the current idle request number calculation module is used for calculating the current idle request number of the processing queue according to the preset maximum queue depth of the processing queue and the real-time queue depth.
In one embodiment, the apparatus further comprises:
and the execution module is used for executing the step of acquiring the depth information of the waiting queue corresponding to each request type under the condition that the real-time queue depth of the processing queue is less than or equal to a preset queue depth threshold value.
In one embodiment, the apparatus further comprises:
a receiving module, configured to receive a plurality of the data requests;
and the adding module is used for respectively determining the request type of each data request and adding each data request to a waiting queue corresponding to the request type of each data request under the condition that the data request processing link is determined to meet the preset congestion condition.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the following steps when executing the computer program:
acquiring depth information of a waiting queue corresponding to each request type;
respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
aiming at each waiting queue, calculating according to the current idle request number of a processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests;
and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring depth information of a waiting queue corresponding to each request type;
respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
aiming at each waiting queue, calculating according to the current idle request number of a processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests;
and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which when executed by a processor performs the steps of:
acquiring depth information of a waiting queue corresponding to each request type;
respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
aiming at each waiting queue, calculating according to the current idle request number of a processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests;
and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
The data request processing method, the data request processing device, the computer equipment and the storage medium comprise the following steps: acquiring depth information of a waiting queue corresponding to each request type; respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type; aiming at each waiting queue, calculating according to the current idle request number of the processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests; and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests. By adopting the invention, the current weight corresponding to each waiting queue can be calculated according to the dynamic state, the balance among the data requests with different priority weights in the queue can be ensured, the processing amount of the data requests with low priority can be improved, the backlog of the data requests with low priority can be avoided, the response time of the data requests with high priority can be shortened, the self-adaptive processing capability of the service system to the data requests with various request types can be improved, and the user experience can be further improved.
Drawings
FIG. 1 is a diagram of an application environment of a method for processing a data request in one embodiment;
FIG. 2 is a flow diagram that illustrates a method for processing a data request in one embodiment;
FIG. 3 is a schematic flow chart of the current weight determination step in one embodiment;
FIG. 4 is a flowchart illustrating the current weight determining step in one embodiment;
FIG. 5 is a schematic flow chart diagram illustrating the current weight determination step in one embodiment;
FIG. 6 is a flowchart of the step of determining the current number of idle requests in one embodiment;
FIG. 7 is a flow diagram illustrating the partitioning step in one embodiment;
FIG. 8 is a flowchart of a method of processing a data request in another embodiment;
FIG. 9 is a block diagram showing an example of a data request processing apparatus according to the present invention;
fig. 10 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The data request processing method provided by the embodiment of the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104, or may be located on the cloud or other network server. The server 104 may be a server corresponding to a distributed storage system, or may be a server corresponding to another computer system. The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and the like. The server 104 may be implemented as a stand-alone server or as a server cluster comprised of multiple servers.
As shown in fig. 2, in this embodiment, the method for processing a data request includes the following steps:
step 202, obtaining depth information of the waiting queue corresponding to each request type.
Wherein the data request may be a data request generated in response to a triggering request of a user. In one example, the data request may be an io (input output) request generated by the distributed storage system in response to a data modification operation of the user device, or may be other systems in the computer device. The data request may include a variety of types, and may include, for example, a user read data request, an internal complement read data request, a read ahead data request, a pass through write data request, a flush back write data request, and so forth. The wait queue may include multiple data requests, and the same wait queue contains multiple data requests of the same type. The depth information of the wait queue is information related to the number of data requests contained in the wait queue.
In implementation, the distributed storage system generates a data request in response to a trigger operation of a user, and transmits the data request to the terminal. When the terminal receives at least one data request, the at least one data request may be divided based on the request type to obtain a waiting queue corresponding to each request type, and the waiting queue may be used for queuing each data request when the data request processing link meets the preset congestion condition. In this way, the terminal may perform statistics on a plurality of data requests included in the waiting queue corresponding to each request type to obtain the depth information of the waiting queue.
And step 204, respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type.
The priority weight of each request type may be configured for each request type in advance according to an actual application scenario.
In an implementation, for a waiting queue of multiple request types, a terminal may first obtain a priority level configured in advance for the waiting queue of each request type. The terminal can determine the priority weight of each request type according to the priority of each request type, so that the terminal can respectively calculate the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type.
And step 206, aiming at each waiting queue, calculating according to the current idle request number of the processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue.
Wherein the processing queue comprises a plurality of data requests to be processed. The current free number of requests for a processing queue is the number of requests for data requests that can also be added to the processing queue in the current situation,
in implementation, for the waiting queue corresponding to each request type, the terminal may perform weighted calculation based on the current weight of the waiting queue and the current idle request number of the processing queue to obtain the target data request number corresponding to the request type. The current weight of the waiting queue corresponding to each request type may be a ratio of the number of data requests in the waiting queue of the request type to the current number of idle requests in the processing queue.
Step 208, extracting the data requests with the target number of data requests from the waiting queue, and adding the data requests with the target number of data requests to the processing queue, so that the processing queue processes the data requests.
In implementation, for each waiting queue, the terminal may calculate, based on the steps in the foregoing embodiment, a target data request number corresponding to the waiting queue, and extract, in the waiting queue, data requests of the target data request number according to a time sequence in which the data requests enter the waiting queue. In this way, the terminal can add the extracted target data request number of data requests to the processing queue. Based on the above scheme, the processing queue may send each data request in the processing queue to the data request processing link according to a preset processing order, so that the data request processing link processes the data request. In the case that the processing queue is a FIFO (First In First Out) queue, the preset processing order may be a time order.
In the data request processing method, depth information of the waiting queue corresponding to each request type is acquired. And respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type. And aiming at each waiting queue, calculating according to the current idle request number of the processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests. And extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests. By adopting the invention, the current weight corresponding to each waiting queue can be calculated according to the dynamic state, the balance among the data requests with different priority weights in the queue can be ensured, the processing amount of the data requests with low priority can be improved, the backlog of the data requests with low priority can be avoided, the response time of the data requests with high priority can be shortened, the self-adaptive processing capability of the service system to the data requests with various request types can be improved, and the user experience can be further improved.
In one embodiment, the depth information includes real-time queue depth.
In particular, the real-time queue depth of the wait queue represents the number of data requests that the wait queue contains in the present case.
Accordingly, as shown in fig. 3, a specific process of step 204 "calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type" includes:
step 302, determining the maximum priority weight in the priority weights of the waiting queues and determining the maximum real-time queue depth in the real-time queue depths of the waiting queues;
in an implementation, the terminal may obtain priority weights (which may be denoted as Tw) of the plurality of waiting queues, and perform filtering among the priority weights of the plurality of waiting queues, and use the largest priority weight as the largest priority weight. The terminal can also obtain the real-time queue depth corresponding to each request type, and the maximum real-time queue depth is taken as the maximum real-time queue depth in the real-time queue depths corresponding to each request type.
Step 304, for each waiting queue, calculating the queue depth weight of the waiting queue according to the real-time queue depth, the maximum real-time queue depth and the maximum priority weight of the waiting queue.
In an implementation, for each waiting queue, the terminal may calculate a first ratio of the real-time queue depth of the waiting queue to the maximum real-time queue depth, based on which the terminal may calculate a first product of the maximum priority weight and the first ratio, and use the first product as a queue depth weight (which may be denoted as Qw) of the waiting queue. The queue depth weight of the waiting queue indicates that under the current condition, under the backlog condition of the data request in the waiting queue, and under the condition that the backlog condition of the waiting queue is serious, the queue depth weight calculated based on the scheme is correspondingly increased.
In one example, the terminal may calculate the queue depth weight of the waiting queue by the following formula:
wherein que depth Representing the real-time queue depth of the wait queue.
Step 306, determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue.
In an implementation, the terminal may determine the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue.
In this embodiment, the trend of the processing pressure of the data request of each request type may be determined, and the current weight of the waiting queue corresponding to each request type may be calculated according to the trend, so as to achieve the balance of the current weights of the waiting queues of each request type having different priority weights.
In one embodiment, the depth information further includes a remaining queue depth.
Specifically, the terminal may extract a certain number of data requests from the waiting queue of each request type to add to the processing queue based on the calculated current weight of each request type. The remaining queue depth of the waiting queue represents the remaining number of data requests in the waiting queue after the terminal extracts a certain number of data requests from the waiting queue of each request type and adds the data requests to the waiting queue.
Accordingly, as shown in fig. 4, the specific process of step 306 "determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue" includes:
step 402, obtaining the remaining queue depth of each waiting queue in the previous cycle of the current processing cycle, and determining the maximum remaining queue depth in each remaining queue depth.
In implementation, the terminal may perform multiple rounds of adjustment on the processing queue, where each round of adjustment corresponds to a processing cycle. Thus, the terminal needs to determine the corresponding current processing cycle in the current situation. In the case where the current processing cycle (which may be denoted as the ith processing cycle) is a non-initial processing cycle, the terminal needs to determine a cycle (which may be denoted as the (i-1) th processing cycle) that is previous to the current processing cycle.
In the (i-1) th processing cycle, the terminal may determine that each waiting queue calculates the current weight of each waiting queue and calculate the number of target data requests corresponding to the waiting queue based on the method of the above embodiment. Based on this, the terminal can extract the data requests of the target number of data requests from the waiting queue and count the number of data requests remaining in the waiting queue, i.e. the remaining queue depth of the waiting queue of the i-1 th processing cycle.
In the ith processing cycle, the terminal may obtain the remaining queue depth of each waiting queue corresponding to each request type in the (i-1) th processing cycle, and determine the maximum remaining queue depth from the remaining queue depth of each waiting queue in the (i-1) th processing cycle.
Step 404, calculating backlog depth weight of the waiting queue according to the remaining queue depth, the maximum remaining queue depth and the priority weight of the waiting queue.
In an implementation, for each waiting queue, the terminal may calculate a second ratio of the remaining queue depth of the waiting queue to the maximum remaining queue depth, based on which the terminal may calculate a second product of the maximum priority weight and the second ratio, the second product being a backlog depth weight (which may be denoted as Pw) of the waiting queue. The backlog depth weight of the waiting queue indicates the backlog situation of the remaining data requests in the waiting queue under the current situation, and the backlog depth weight calculated based on the above scheme is correspondingly increased under the condition that the backlog situation of the remaining data requests of the waiting queue is serious.
In one example, the terminal may calculate the backlog depth weight for the waiting queue by the following formula:
where Round _ legacy represents the remaining queue depth of the waiting queue in the previous cycle of the current processing cycle.
Step 406, calculating a first sum of the priority weight of the waiting queue, the queue depth weight of the waiting queue and the backlog depth weight of the waiting queue, and using the first sum as the current weight of the waiting queue.
Thus, the terminal can sum the priority weight of the waiting queue, the queue depth weight of the waiting queue and the backlog depth weight of the waiting queue to obtain a first sum value. In this way, the terminal may use the obtained first sum value as the current weight of the waiting queue, i.e. the weight at the i-th processing cycle.
In this embodiment, the dynamic adjustment of the weight of the waiting queue of each request type can be realized according to the number of requests of each request type in the current processing cycle and the number of remaining requests in the previous cycle, so that the balance between the throughput of the data request with high priority and the throughput of the data request with low priority is realized, the average response time of the data request with high priority is improved, the backlog of the data request with low priority is avoided, and the user experience is improved.
In one embodiment, the current processing cycle includes a plurality of conditions, and accordingly, the determination of the current weight of the wait queue includes a plurality of conditions. As shown in fig. 5, the specific process of step 306 "determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue" includes:
step 502, calculating a second sum of the priority weight of the waiting queue and the queue depth weight of the waiting queue, and using the second sum as the current weight of the waiting queue.
In implementation, the terminal may perform multiple rounds of adjustment on the processing queue, where each round of adjustment corresponds to a processing cycle. Thus, the terminal needs to determine the corresponding current processing cycle in the current situation. When the current processing cycle is the initial processing cycle (which may be referred to as the 1 st processing cycle), the terminal may determine that, in the current situation, the remaining queue depth of the waiting queue corresponding to each request type is zero. Thus, the terminal can perform summation processing on the priority weight of the waiting queue and the queue depth weight of the waiting queue to obtain a second sum value. In this way, the terminal may use the obtained second sum as the current weight of the waiting queue, that is, the weight of the waiting queue in the 1 st processing cycle.
In this embodiment, the dynamic adjustment of the weight of the waiting queue of each request type can be realized according to the number of requests of each request type in the current processing cycle, so that the balance between the throughput of the data request with high priority and the throughput of the data request with low priority is realized, the average response time of the data request with high priority is improved, the backlog of the data request with low priority is avoided, and the user experience is improved.
In one embodiment, as shown in fig. 6, the method for processing the data request further includes:
step 602, obtain a real-time queue depth of a processing queue.
Step 604, calculating the current idle request number of the processing queue according to the preset maximum queue depth and the real-time queue depth of the processing queue.
The preset maximum queue depth of the processing queue may be configured in advance, and may also be determined by those skilled in the art according to an actual application scenario, which is not limited by the present disclosure.
In implementation, the terminal may obtain the number of data requests contained in the processing queue (real-time queue depth of the processing queue) under the current situation and a preset maximum queue depth configured for the processing queue in advance. Based on this, the terminal can perform difference calculation on the preset maximum queue depth and the real-time queue depth of the processing queue, and the obtained first difference value is used as the current idle request number of the processing queue.
In one example, the terminal may also predetermine the preset number of buffers, which may be 2, for example, for timely processing of the data requests held in the processing queue. Therefore, the terminal can perform difference processing on the preset maximum queue depth and the real-time queue depth of the processing queue to obtain a first difference value. The terminal may further perform difference processing on the first difference value and a preset buffer number to obtain a second difference value, and use the second difference value as the current idle request number of the processing queue.
In this embodiment, the number of current idle requests of the accurate processing queue can be obtained by the preset maximum queue depth and the real-time queue depth of the processing queue determined based on the actual application scenario, so as to avoid backlog of data requests.
In one embodiment, the method for processing the data request further includes:
and under the condition that the real-time queue depth of the processing queue is less than or equal to a preset queue depth threshold value, executing the step of acquiring the depth information of the waiting queue corresponding to each request type.
In implementation, the terminal may monitor a real-time queue depth of the processing queue, and when it is monitored that the real-time queue depth of the processing queue is less than or equal to a preset queue depth threshold, the terminal may determine that a certain number of data requests need to be extracted from waiting queues respectively corresponding to request types under the current situation, and add the extracted data requests to the processing queue.
The preset queue depth threshold may be determined according to a data request processing capability of a data request processing link corresponding to the processing queue. The data request processing link may be a system for processing the data request, and the data request processing capability of the data request processing link may be processing time information for the data request processing link to process the data request of each request type. Based on this, the terminal can determine the preset queue depth threshold value with the processing time information of the data request processing link for processing the data request of each request type.
In the embodiment, the continuity of processing the data requests in the processing queue can be ensured, the extra time consumption of waiting for the data requests to be added to the processing queue is avoided, and the resource consumption of dynamically adjusting the current weight of each request type and extracting the data requests from the waiting queue is reduced.
In one embodiment, as shown in fig. 7, the method for processing the data request further includes:
at step 702, a plurality of data requests are received.
Step 704, in a case that it is determined that the data request processing link satisfies the preset congestion condition, determining a request type of each data request, and adding each data request to a waiting queue corresponding to the request type of each data request.
In implementation, the terminal may obtain at least one monitoring index of the data request processing link, and if a value of the at least one monitoring index is greater than or equal to a monitoring threshold corresponding to the monitoring index, the terminal may determine that the data request processing link satisfies a preset congestion condition. Under the condition that a data request processing link meets a preset congestion condition, each data request received by the terminal needs to be queued to be processed, so that the terminal can classify each data request based on the request type and add the data request to a waiting queue corresponding to the request type.
The monitoring index of the data request processing link may include processing delay, amount of remaining resources, and the like.
Optionally, after the step of adding the data request to the waiting queue corresponding to the request type by the terminal, the terminal may further perform a driving operation on the processing queue, and the processing queue may send the data request to the data request processing link in response to the driving operation.
In this embodiment, under the condition that it is determined that the data request processing link satisfies the preset congestion condition, the data request is extracted from each waiting queue and added to the processing queue, so that the trend of the processing pressure of each request type within a certain time range can be accurately identified, and the dynamic adjustment of the weight of each waiting queue of each request type is realized.
In one embodiment, after the step of receiving a plurality of data requests, the method of processing the data requests further comprises:
and comparing the size of the data request with a preset size threshold, and splitting the data request under the condition that the size of the data request is larger than the preset size threshold to obtain a plurality of split data requests.
In an implementation, after receiving a plurality of data requests, the terminal may preprocess each data request. A possible preprocessing process may be that the terminal compares the size of the data request with a preset size threshold, and performs splitting processing on the data request to obtain multiple split data requests when the size of the data request is greater than the preset size threshold.
In this embodiment, by preprocessing the data request, the processing speed of the data request can be increased, and the experience of the user who sends the data request can be improved.
The following describes in detail a specific implementation process of the data request processing method provided by the present invention with reference to fig. 8:
the terminal can split a data request queue corresponding to the data request processing link into a waiting queue and a processing queue (FIFO queue), wherein the waiting queue is used for queuing data requests (IO requests) corresponding to each request type under the condition that the data request processing link meets a preset congestion condition. The processing queue is used for sequentially sending the data requests to the data request processing link according to a time sequence in a preset time period, and each request type can comprise a user data reading request, an internal data supplementing and reading request, a pre-reading data request, a through data writing request, a back-flushing data writing request and the like.
Under the condition that the data request processing link does not meet the preset congestion condition, the terminal can directly send the data request to the data request processing link so that the data request processing link processes the data request.
Under the condition that the data request processing link meets the preset congestion condition, the terminal can classify each data request based on the request type and add the data request to the waiting queue corresponding to the request type. Based on this, the terminal can perform a driving operation on the processing queue, and determine whether the data request in the processing queue can be sent to the data request processing link.
The processed process in the processing queue may include: if the processing queue contains data requests, a plurality of data requests contained therein are processed in chronological order. If the real-time queue depth corresponding to the processing queue is less than or equal to the preset queue depth threshold, the terminal may calculate the current weight of each request type according to the method in the foregoing embodiment, and extract a certain number of data requests from the waiting queue of each request type to add to the processing queue.
Alternatively, the preset queue depth threshold may be zero, so that, in a case that the terminal determines that the processing queue is an empty queue, the terminal may calculate the current weight of each request type according to the method in the foregoing embodiment, and extract a certain number of data requests from the waiting queue of each request type to add to the processing queue.
The terminal may calculate the current weight of each request type through SWRR (Smooth Weighted round robin ) in a specific process, which may include: the terminal may assign fixed and different priority weights to each request type according to the actual application scenario and the priority of each request type, so that the terminal may determine the priority weight (which may be denoted as Tw) of the waiting queue corresponding to each request type. The terminal may calculate a queue depth weight (which may be denoted as Qw) of the waiting queue corresponding to each request type according to the real-time queue depth (real-time arrangement) of each request type.
The terminal can process the processing queue for multiple times, and multiple processing cycles are corresponded to the processing queue. Therefore, the terminal can obtain the remaining queue depth corresponding to the waiting queue of each request type in the last processing period, and calculate the backlog depth weight of the waiting queue corresponding to each request type based on the remaining queue depth of the waiting queue corresponding to each request type. Therefore, the response speed of processing the data request in the waiting queue of which the data request in the last processing period is not emptied can be improved by increasing the weight, and the backlog situation is avoided.
In this way, the terminal can calculate the current weight of the waiting queue (which can be noted as W) based on the sum of the priority weight, the queue depth weight, and the backlog depth weight (which can be noted as Pw).
The processing queue may be a FIFO queue that may guarantee the temporal order of data requests therein. When the terminal determines that the real-time queue Depth (which can be recorded as Depth) of the FIFO queue is smaller than a preset queue Depth threshold, the terminal calculates the current weight of the waiting queue corresponding to each request type under the current condition, and extracts the data request from the waiting queue corresponding to each request type to the processing queue based on the current weight of the waiting queue corresponding to each request type, so that the processing queue sends the data request to the data request processing link according to the time sequence.
The processing method of the data request provided by the embodiment of the invention can dynamically adjust the weight of each request type, thereby balancing the processing amount of the data request with high and low priority on a data request processing link and improving the average response time of the data request with high priority; the occurrence of backlog caused by the fact that low-priority data requests cannot be obtained for a long time is avoided, the self-adaption capability of the distributed storage system is further improved, and better experience and higher comprehensive performance are provided for users.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.
Based on the same inventive concept, the embodiment of the present application further provides a data request processing apparatus for implementing the above-mentioned data request processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so specific limitations in the following embodiments of one or more data request processing devices may refer to the above limitations on the data request processing method, and details are not described herein again.
In one embodiment, as shown in fig. 9, there is provided a data request processing apparatus 900, including: an obtaining module 901, a first calculating module 902, a second calculating module 903, and an adding module 904, wherein:
an obtaining module 901, configured to obtain depth information of a waiting queue corresponding to each request type.
The first calculating module 902 is configured to calculate a current weight of each waiting queue according to the priority weight of each request type and depth information of the waiting queue corresponding to each request type.
A second calculating module 903, configured to calculate, for each waiting queue, according to the current idle request number of the processing queue and the current weight of the waiting queue, to obtain a target data request number corresponding to the waiting queue, where the processing queue includes multiple data requests.
And an adding module 904, configured to extract the data requests of the target number of data requests from the wait queue, and add the data requests of the target number of data requests to the processing queue, so that the processing queue processes the data requests.
In one embodiment, the depth information includes real-time queue depth;
the first calculation module is specifically configured to:
determining the maximum priority weight in the priority weights of the waiting queues, and determining the maximum real-time queue depth in the real-time queue depths of the waiting queues;
aiming at each waiting queue, calculating the queue depth weight of the waiting queue according to the real-time queue depth of the waiting queue, the maximum real-time queue depth and the maximum priority weight;
and determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue.
In one embodiment, the depth information further includes a remaining queue depth;
the first calculating module is specifically configured to:
acquiring the remaining queue depth of each waiting queue in the previous period of the current processing period, and determining the maximum remaining queue depth in each remaining queue depth;
calculating backlog depth weight of the waiting queue according to the remaining queue depth of the waiting queue, the maximum remaining queue depth and the priority weight;
calculating a first sum of a priority weight of a waiting queue, a queue depth weight of the waiting queue and a backlog depth weight of the waiting queue, and taking the first sum as a current weight of the waiting queue.
In one embodiment, the depth information further includes a remaining queue depth;
the first calculation module is specifically configured to:
and calculating a second sum of the priority weight of the waiting queue and the queue depth weight of the waiting queue, and taking the second sum as the current weight of the waiting queue.
In one embodiment, the apparatus further comprises:
the real-time queue depth acquisition module is used for acquiring the real-time queue depth of the processing queue;
and the current idle request number calculation module is used for calculating the current idle request number of the processing queue according to the preset maximum queue depth of the processing queue and the real-time queue depth.
In one embodiment, the apparatus further comprises:
and the execution module is used for executing the step of acquiring the depth information of the waiting queue corresponding to each request type under the condition that the real-time queue depth of the processing queue is less than or equal to a preset queue depth threshold value.
In one embodiment, the apparatus further comprises:
a receiving module, configured to receive a plurality of data requests;
and the adding module is used for respectively determining the request type of each data request and adding each data request to a waiting queue corresponding to the request type of each data request under the condition that the data request processing link is determined to meet the preset congestion condition.
The various modules in the data request processing device 900 described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. 10. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data request data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of processing a data request.
Those skilled in the art will appreciate that the architecture shown in fig. 10 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In an embodiment, a computer program product is provided, comprising a computer program which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, databases, or other media used in the embodiments provided herein can include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), magnetic Random Access Memory (MRAM), ferroelectric Random Access Memory (FRAM), phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases involved in the embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.
All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. A method for processing a data request, the method comprising:
acquiring depth information of a waiting queue corresponding to each request type;
respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
aiming at each waiting queue, calculating according to the current idle request number of a processing queue and the current weight of the waiting queue to obtain the target data request number corresponding to the waiting queue, wherein the processing queue comprises a plurality of data requests;
and extracting the data requests with the target data request number from the waiting queue, and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
2. The method of claim 1, wherein the depth information comprises a real-time queue depth;
the calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type respectively comprises:
determining the maximum priority weight in the priority weights of the waiting queues, and determining the maximum real-time queue depth in the real-time queue depths of the waiting queues;
aiming at each waiting queue, calculating the queue depth weight of the waiting queue according to the real-time queue depth of the waiting queue, the maximum real-time queue depth and the maximum priority weight;
and determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue.
3. The method of claim 2, wherein the depth information further comprises a remaining queue depth;
determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue, wherein the determining comprises the following steps:
acquiring the remaining queue depth of each waiting queue in the previous period of the current processing period, and determining the maximum remaining queue depth in each remaining queue depth;
calculating backlog depth weight of the waiting queue according to the remaining queue depth of the waiting queue, the maximum remaining queue depth and the priority weight;
calculating a first sum of a priority weight of a waiting queue, a queue depth weight of the waiting queue and a backlog depth weight of the waiting queue, and taking the first sum as a current weight of the waiting queue.
4. The method of claim 2, wherein the depth information further comprises a remaining queue depth;
determining the current weight of the waiting queue according to the priority weight of the waiting queue and the queue depth weight of the waiting queue, wherein the determining comprises the following steps:
and calculating a second sum of the priority weight of the waiting queue and the queue depth weight of the waiting queue, and taking the second sum as the current weight of the waiting queue.
5. The method of claim 1, further comprising:
acquiring the real-time queue depth of the processing queue;
and calculating the current idle request number of the processing queue according to the preset maximum queue depth of the processing queue and the real-time queue depth.
6. The method of claim 5, further comprising:
and under the condition that the real-time queue depth of the processing queue is less than or equal to a preset queue depth threshold value, executing the step of acquiring the depth information of the waiting queue corresponding to each request type.
7. The method of claim 1, further comprising:
receiving a plurality of said data requests;
and under the condition that the data request processing link is determined to meet the preset congestion condition, determining the request type of each data request respectively, and adding each data request to a waiting queue corresponding to the request type of each data request.
8. An apparatus for processing a data request, the apparatus comprising:
the acquisition module is used for acquiring depth information of the waiting queue corresponding to each request type;
the first calculation module is used for respectively calculating the current weight of each waiting queue according to the priority weight of each request type and the depth information of the waiting queue corresponding to each request type;
the second calculation module is used for calculating according to the current idle request number of a processing queue and the current weight of the waiting queue aiming at each waiting queue to obtain the target data request number corresponding to the waiting queue, and the processing queue comprises a plurality of data requests;
and the adding module is used for extracting the data requests with the target data request number from the waiting queue and adding the data requests with the target data request number to the processing queue so that the processing queue processes the data requests.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211008577.8A CN115499513A (en) | 2022-08-22 | 2022-08-22 | Data request processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211008577.8A CN115499513A (en) | 2022-08-22 | 2022-08-22 | Data request processing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115499513A true CN115499513A (en) | 2022-12-20 |
Family
ID=84466605
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211008577.8A Pending CN115499513A (en) | 2022-08-22 | 2022-08-22 | Data request processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115499513A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116353664A (en) * | 2023-02-28 | 2023-06-30 | 西门子交通技术(北京)有限公司 | Automatic rail train protection system and readable storage medium |
-
2022
- 2022-08-22 CN CN202211008577.8A patent/CN115499513A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116353664A (en) * | 2023-02-28 | 2023-06-30 | 西门子交通技术(北京)有限公司 | Automatic rail train protection system and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20150317188A1 (en) | Service resource allocation | |
| CN109005130B (en) | Network resource allocation scheduling method and device | |
| CN112866136B (en) | Service data processing method and device | |
| US11928580B2 (en) | Interleaving memory requests to accelerate memory accesses | |
| CN108874324B (en) | Access request processing method, device, equipment and readable storage medium | |
| CN112783807B (en) | Model calculation method and system | |
| CN111367651A (en) | Service current limiting system, method and device and electronic equipment | |
| CN115348222B (en) | Message distribution method, device, server side and storage medium | |
| CN115334082A (en) | Load balancing method, device, computer equipment, storage medium and product | |
| CN115499513A (en) | Data request processing method and device, computer equipment and storage medium | |
| CN114461384A (en) | Task execution method and device, computer equipment and storage medium | |
| CN111190541B (en) | Flow control method of storage system and computer readable storage medium | |
| CN116089477B (en) | Distributed training method and system | |
| CN117376645A (en) | Video preloading method, device, computer equipment and storage medium | |
| CN116248699B (en) | Data reading method, device, equipment and storage medium in multi-copy scene | |
| CN113992586A (en) | Flow control method and device, computer equipment and storage medium | |
| CN116737373A (en) | Load balancing method, device, computer equipment and storage medium | |
| CN116204293A (en) | Resource scheduling method, device, computer equipment and storage medium | |
| CN112631757B (en) | DDR4 multi-user access scheduling method and device | |
| CN114003370A (en) | Computing power scheduling method and related device | |
| CN109005052B (en) | Network task prediction method and device | |
| CN113329078A (en) | Data storage method and device | |
| CN112148483A (en) | Container migration method and related device | |
| CN115442432B (en) | Control method, device, equipment and storage medium | |
| CN114546279B (en) | IO request prediction method and device, storage node and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |