CN115840632A - Distributed scheduling management method and system based on time sequence database - Google Patents
Distributed scheduling management method and system based on time sequence database Download PDFInfo
- Publication number
- CN115840632A CN115840632A CN202310139421.1A CN202310139421A CN115840632A CN 115840632 A CN115840632 A CN 115840632A CN 202310139421 A CN202310139421 A CN 202310139421A CN 115840632 A CN115840632 A CN 115840632A
- Authority
- CN
- China
- Prior art keywords
- time
- priority queue
- task
- distributed node
- obtaining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to the field of scheduling management of a time sequence database, in particular to a distributed scheduling management method and a distributed scheduling management system based on the time sequence database, which comprise the following steps: establishing tasks corresponding to different monomer applications in a time sequence database, and acquiring each distributed node; obtaining a basic time slice of a highest priority queue and a basic time slice of a lowest priority queue according to the average predicted processing time and each predicted processing time of each task, and further obtaining a basic time slice of each priority queue; obtaining a server load adjusting coefficient of each distributed node according to the server load degree, the first load degree and the second load degree of each distributed node; and obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue, and further processing each task. The invention can improve the efficiency of distributed scheduling of the time sequence database.
Description
Technical Field
The invention relates to the field of scheduling management of a time sequence database, in particular to a distributed scheduling management method and a distributed scheduling management system based on the time sequence database.
Background
The distributed scheduling management of the time sequence database refers to that a distributed environment, namely a plurality of distributed nodes, is utilized to enable a plurality of tasks in the time sequence database to be scheduled and processed according to a certain scheduling strategy, and a multi-stage feedback wheel algorithm is a common method for scheduling and processing tasks of different distributed nodes of the time sequence database at present.
When a multi-level feedback queue scheduling algorithm is used for scheduling tasks of each distributed node of a time sequence database, the time slice size of queues with different priorities is set too fixedly, so that when the tasks in each priority queue are processed, the time slice is too short, the tasks are switched too frequently, and the computing resources of each distributed node are wasted; the longer time slice causes each task to need longer waiting time during task processing, so the length of the time slice has a more serious influence on the distributed scheduling management efficiency of the time sequence database.
Therefore, when the distributed scheduling of the time sequence database is carried out, a multi-level feedback queue scheduling algorithm which can correspond to each distributed node is designed, the time slices of queues with different priorities are adaptively adjusted, and the scheduling efficiency of the whole time sequence database is improved.
Disclosure of Invention
The invention provides a distributed scheduling management method and a distributed scheduling management system based on a time sequence database, which are used for solving the existing problems.
The distributed scheduling management method and the distributed scheduling management system based on the time sequence database adopt the following technical scheme:
one embodiment of the invention provides a distributed scheduling management method based on a time sequence database, which comprises the following steps:
establishing tasks corresponding to different monomer applications in a time sequence database, and building a distributed framework of the time sequence database according to each task to obtain each distributed node of the time sequence database;
acquiring each task to be queued in a time sequence database; obtaining average predicted processing time according to the predicted processing time of each task; obtaining each first-class task and each second-class task according to the average predicted processing time and each predicted processing time of each task; obtaining the basic time slice of the highest priority queue according to the average predicted processing time and the predicted processing time of each class of tasks; obtaining a basic time slice of the lowest priority queue according to the average predicted processing time and the predicted processing time of each second-class task; obtaining the basic time slices of each priority queue according to the basic time slice of the highest priority queue and the basic time slice of the lowest priority queue;
any one distributed node is taken as a target distributed node, a first load degree of the target distributed node is obtained according to the server load degrees of all distributed nodes in the time sequence database, and a second load degree of the target distributed node is obtained according to the server load degrees of the rest distributed nodes in the time sequence database; obtaining a server load adjusting coefficient of a target distributed node according to the server load degree, the first load degree and the second load degree of the target distributed node, and obtaining the server load adjusting coefficient of each distributed node by taking each distributed node as the target distributed node;
and obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue, and processing each task according to the final time slice of each priority queue.
Preferably, the method for acquiring each first-class task and each second-class task includes:
and recording the tasks with the predicted processing time less than the average predicted processing time as the first-class tasks, and recording the tasks with the predicted processing time more than or equal to the average predicted processing time as the second-class tasks.
Preferably, the method for acquiring the basic time slice of the highest priority queue includes:
calculating the difference between the average predicted processing time and the predicted processing time of each class of tasks, calculating the average value of each difference, and taking the difference between the average predicted processing time and the average value as the base time slice of the highest priority queue.
Preferably, the method for acquiring the basic time slice of the lowest priority queue comprises:
calculating the difference between the predicted processing time and the average predicted processing time of each of the two types of tasks, calculating the average value of the obtained differences, and taking the sum of the average predicted processing time and the obtained average value as the basic time slice of the lowest priority queue.
Preferably, the obtaining expression of the basic time slice of each priority queue is as follows:
in the formula (I), the compound is shown in the specification,
a base time slice for the nth priority queue; />
A base time slice for the highest priority queue;
a base time slice for the lowest priority queue; and N is the total number of the priority queues.
Preferably, the step of obtaining the server load adjustment coefficient of the target distribution node includes:
calculating a difference value between the server load degree of the target distributed node and the second load degree of the target distributed node, calculating a ratio between the obtained difference value and the first load degree of the target distributed node, and taking a sum result between a preset value and the obtained ratio as a server load adjustment coefficient of the target distributed node.
Another embodiment of the present invention provides a distributed scheduling management system based on a time sequence database, the system includes a task creating module, a basic time slice obtaining module, a server load adjustment coefficient obtaining module, and a task processing module, wherein:
the task creating module is used for creating tasks corresponding to different monomer applications in the time sequence database, and building a distributed framework of the time sequence database according to each task to obtain each distributed node of the time sequence database;
the basic time slice acquisition module is used for acquiring each task to be queued in the time sequence database; obtaining average predicted processing time according to the predicted processing time of each task; obtaining each first-class task and each second-class task according to the average predicted processing time and each predicted processing time of each task; obtaining the basic time slice of the highest priority queue according to the average predicted processing time and the predicted processing time of each class of tasks; obtaining a basic time slice of the lowest priority queue according to the average predicted processing time and the predicted processing time of each second-class task; obtaining the basic time slices of each priority queue according to the basic time slice of the highest priority queue and the basic time slice of the lowest priority queue;
the server load adjustment coefficient acquisition module is used for taking any one distributed node as a target distributed node, obtaining a first load degree of the target distributed node according to the server load degrees of all distributed nodes in the time sequence database, and obtaining a second load degree of the target distributed node according to the server load degrees of the rest distributed nodes in the time sequence database; obtaining a server load adjusting coefficient of a target distributed node according to the server load degree, the first load degree and the second load degree of the target distributed node, and obtaining the server load adjusting coefficient of each distributed node by taking each distributed node as the target distributed node;
and the task processing module is used for obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue and processing each task according to the final time slice of each priority queue.
The invention has the beneficial effects that: compared with the single fixed time slice in the queues with different priorities in the prior art, the method has the advantages that the basic time slices are set for the queues with different priorities according to the predicted processing time of different tasks, and then the processing capacity of each distributed node, namely the server load degree, is utilized to perform self-adaptive adjustment on the basic time slices of the queues with different priorities of each distributed node, so that when each distributed node performs task processing, frequent switching of processing tasks due to over-small time slices is avoided, resource waste of the distributed nodes is avoided, and longer waiting time is not needed when task processing is performed in each priority queue due to over-large time slices, so that the task processing efficiency is lower, and the distributed scheduling efficiency of the time sequence database is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart illustrating steps of a distributed scheduling management method based on a time series database according to the present invention;
fig. 2 is a block diagram of a distributed scheduling management system based on a time series database according to the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description, structures, features and effects of the distributed scheduling management method and system based on time series database according to the present invention are provided with the accompanying drawings and the preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following describes a specific scheme of the distributed scheduling management method based on the time-series database in detail with reference to the accompanying drawings.
Referring to fig. 1, a flowchart illustrating steps of a distributed scheduling management method based on a time series database according to an embodiment of the present invention is shown, where the method includes the following steps:
step S001: and establishing tasks corresponding to different monomer applications in the time sequence database, and building a distributed framework of the time sequence database according to each task to obtain each distributed node of the time sequence database.
In the time sequence database, a plurality of tasks of the single applications are often required to be created according to requirements, and in the process of distributed scheduling management, corresponding tasks such as a data backup task, a statistical form task and an order processing task need to be generated for different single applications, and the corresponding tasks are created according to actual requirements of the single applications in the time sequence database.
After each task in the time sequence database is established, establishing a distributed framework for all tasks, wherein the specific process for establishing the distributed framework is as follows: the tasks of each single application are backed up on all distributed nodes, each distributed node in the embodiment is a different server with task processing capability, and then the tasks of all the single applications on each distributed node are processed respectively, and the processing rule is as follows: when a certain task on a certain distributed node starts to be processed, the same tasks on the rest of the distributed nodes are not processed;
step S002: obtaining average predicted processing time; and obtaining the basic time slice of the highest priority queue and the basic time slice of the lowest priority queue according to the average predicted processing time and each predicted processing time of each task, and further obtaining the basic time slice of each priority queue.
When processing the task of each distributed node in the time sequence database, the existing method, such as a multi-stage feedback queue scheduling algorithm, artificially sets the time slice size of each priority corresponding to each priority queue according to an empirical value, wherein each priority corresponds to one priority queue, and the time slice size corresponding to each priority is artificially set, so that each task belonging to the same priority has the same time slice size; the higher the priority is, the smaller the corresponding time slice is, the lower the priority is, the larger the corresponding time slice is; if and only after all tasks in one priority queue are processed, the distributed node can process the tasks in the next priority queue;
however, this method is very subjective for tasks with different processing times, for example, if the time slice in a priority queue is set too large, that is, the processing time corresponding to the task in the priority queue is less than the set time slice, the priority queue may process the next task after the task is processed, and thus there is more waiting time, resulting in a great waste of time resources; if the time slice setting in the priority queue is too small, that is, the processing time of a task in the priority queue is longer than the set time slice, the task is not processed and completed after the time slice is consumed and completed, and at this time, the task needs to be queued again at the end of the next priority queue, so that when the task in the priority queue is processed, the task is switched too frequently, and the computing resources are wasted;
when the existing multistage feedback queue scheduling algorithm is used for processing the task in each distributed node, the efficiency is often not high enough when the task processing scheduling is performed due to the fact that the sizes of the time slices corresponding to the queues with different priorities are relatively fixed, the time slices of the queues with different priorities of each distributed node are adaptively adjusted, then the task of each distributed node is scheduled and processed by the multistage feedback queue scheduling algorithm through the adaptive time slices, the processing speed of each distributed node task in the distributed nodes of the time sequence database is increased, and the overall scheduling management efficiency is improved;
therefore, in this embodiment, the basic time slice is calculated according to the predicted processing time of the task queued for processing by each distributed node, and then the basic time slice adjustment coefficient of each distributed node is calculated according to the server load capacity of the distributed node, so that the basic time slices corresponding to different priority queues of each distributed node are adjusted to obtain the final time slice of each priority queue on each distributed node.
Wherein the total number of the priority queues is N, then for the second queue
The calculation process of the basic time slice size of each distributed node is as follows:
because the time slice of the highest priority queue is the smallest, which aims to process all tasks that can be processed and ended in a short time as quickly as possible, in the embodiment, on the premise of not considering the server load capacity of the distributed node, the task with shorter expected processing time in all queued tasks to be processed is used to obtain the basic time slice of the highest priority queue, wherein all queued tasks to be processed refer to all tasks to be processed contained in all priority queues, and each task with shorter expected processing time than the average expected processing time in all tasks to be processed is referred to as each class of task, so that the size of the basic time slice of the highest priority queue is the same as that of each class of tasks
The formula model was constructed as follows: />
Wherein the content of the first and second substances,
representing the average predicted processing time, wherein the average predicted processing time is the average value of the predicted processing times corresponding to all queued tasks to be processed; />
For the fifth or fifth of all tasks to be processed, a processing time which is expected to be less than the average expected processing time>
A task, i.e. the ^ th->
A class of tasks; />
The number of tasks with the estimated processing time smaller than the average estimated processing time in all the tasks to be processed is calculated; />
Indicates that the predicted processing time is less than the ^ th predicted processing time on average>
Predicted processing time corresponding to each task.
In the existing algorithm, when the time slice setting of the highest priority queue is inaccurate, tasks with short processing time cannot be processed in the highest priority queue at one time, and the tasks with short processing time need to be queued again at the tail of the next priority queue, so that in the processing process of each task in the highest priority queue, tasks to be processed need to be switched frequently, and part of the tasks need to be processed again;
therefore, the present embodiment first obtains the average value of the predicted processing times of all tasks, that is, the average predicted processing time, and then calculates the distribution concentration trend between the predicted processing times corresponding to all tasks smaller than the average predicted processing time
The average difference between the average predicted processing time and the average predicted processing time is characterized, and when the obtained average difference is larger, the difference between the predicted processing time corresponding to most tasks and the average predicted processing time is also larger, and then the difference between the average predicted processing time and the obtained average difference can reflect the predicted processing time corresponding to most tasks with predicted processing time smaller than the average predicted processing time, that is, most tasks with smaller predicted processing time are corresponding to most tasksThe expected processing time is relatively and intensively distributed near the obtained basic time slice of the highest priority queue, compared with the prior art, the task in the highest priority queue is the task with smaller processing time in all the queued tasks to be processed, and the size of the obtained basic time slice of the highest priority queue can represent the distribution concentration point of the processing time corresponding to the tasks, so that the obtained basic time slice can be used for realizing one-time processing of the tasks with smaller processing time as far as possible without excessive re-queuing and waiting processing.
Since the time slice in the lowest priority queue is the largest, which aims to complete the processing of the task with longer processing time at one time, the embodiment obtains the size of the basic time slice in the lowest priority queue by using the task with longer processing time in all queued tasks to be processed without considering the server load capacity of the distributed node; and (3) among all queued tasks to be processed, each task with predicted processing time greater than or equal to average predicted processing time is called as each second task, and the size of the basic time slice of the lowest priority queue
The formula model was constructed as follows:
wherein the content of the first and second substances,
representing the average predicted processing time, wherein the average predicted processing time is the average value of the predicted processing time corresponding to all queued tasks to be processed; />
For all queued tasks, the expected processing time is greater than or equal to the ^ th of the average expected processing time>
A task; />
The number of tasks with the estimated processing time more than or equal to the average estimated processing time in all the queued tasks to be processed; />
Indicates that the predicted processing time is greater than or equal to the ^ th predicted processing time on average>
Predicted processing time corresponding to each task.
The distribution concentration tendency between the predicted processing times corresponding to all the tasks equal to or more than the average predicted processing time is calculated, and the present embodiment uses
The average difference between all tasks with predicted processing time being larger than or equal to the average predicted processing time and the average predicted processing time is characterized, so that the basic time slice with the lowest priority is obtained, namely the predicted processing time corresponding to most tasks with larger predicted processing time is relatively and intensively distributed near the basic time slice of the obtained highest priority queue, and the basic time slice of the obtained highest priority queue can ensure that more tasks are processed at one time instead of frequent task switching when the tasks in the lowest priority queue are processed, so that the extra calculation burden and waiting time caused by frequent task switching are reduced.
The self-adaptation of the basic time slices of the highest priority queue and the lowest priority queue is carried out by utilizing the tasks with shorter expected processing time and the tasks with longer expected processing time, then the basic time slices of the highest priority queue and the lowest priority queue are utilized to obtain the basic time slices of each priority queue, each priority queue is arranged according to the priority from high to low, the sequenced result is numbered from 1, the sequence number of each priority queue is obtained, and then for the second priority queue, the second priority queue is used for carrying out the self-adaptation of the basic time slices of the highest priority queue and the lowest priority queue
A priority queue whose underlying time slice calculation formula model->
As follows:
the basic time slices of the highest priority queue are used as a basis, then the basic time slices of the highest priority queue and the basic time slices of the lowest priority queue are uniformly calculated and distributed, the calculation mode can ensure that queues with different priorities are fairer, the size difference of the basic time slices in each priority queue is equal, and when a task is processed, the task in the priority queue needs to be waited for a long time or switched frequently because the basic time slices in a certain priority queue are too long or too short;
for example, when N =5, note that
(ii) a When n =2, the corresponding base time slice is ≥>
(ii) a When n =3, the corresponding base time slice is ≥>
(ii) a When n =4, the corresponding base time slice is ≥>
(ii) a When n =5, the corresponding base time slice is ≥>
I.e. the base time slice of the highest priority->
。
Step S003: and obtaining the server load adjusting coefficient of each distributed node according to the server load degree, the first load degree and the second load degree of each distributed node.
The resulting base time slice for each priority queue is calculated without regard to the server load capacity of the mth distributed node. When the server load degree of the mth distributed node is smaller, the fast switching of the tasks in each priority queue can be completely met, namely, smaller time slices can be set for processing the tasks, the fast switching of the tasks is realized, the processing efficiency of each task is improved, more tasks with shorter processing time are processed quickly, so that other distributed nodes can process the tasks when the tasks of the mth distributed node stop processing, and the processing capacity of other distributed nodes is improved; when the server load degree of the mth distributed node is large, because the fast switching of the task cannot be satisfied, a larger time slice should be set to reduce the processing task due to frequent switching, thereby avoiding the waste of resources, so the embodiment quantizes the server load degree of each distributed node by using the server load degrees of all the distributed nodes, and adjusts the size of the time slice corresponding to each priority queue of each distributed node according to the quantization result; then to the first
A distributed node having a formula model of server load adjustment factors constructed >>
As follows: />
Wherein the content of the first and second substances,
the server load degree of the mth distributed node is represented, and the server load degree can be obtained by the prior art, which is not described herein any more; />
Represents the server load degree of the kth distributed node, and @>
；/>
Representing the server load degree of the t distributed node; m is the total number of distributed nodes in the time sequence database; a is a preset value, and a =1 in the embodiment.
Since the processing of all tasks is distributed processing, and the capability of all distributed nodes to process the tasks needs to be considered, an average value of the server load degrees of all distributed nodes of the time-series database, that is, an average value of the server load degrees is obtained first
Recording as a first load degree of the mth distributed node; then an average of the server load levels of all remaining distributed nodes, except the mth distributed node, is taken, i.e., ->
Recording as a second load degree of the mth distributed node; then calculating a difference value between the server load degree of the mth distributed node and the second load degree, and calculating a ratio between the obtained difference value and the first load degree, wherein the obtained ratio is the server load degree of the mth distributed node relative to other distributed nodes, when the obtained ratio is positive, the larger the ratio is, the higher the load degree of the distributed node in all distributed nodes is, at the moment, when the distributed node processes tasks, the time slice of each priority queue should be made longer, so that resource waste caused by frequent task switching when the distributed node processes each priority queue is reduced; conversely, when the obtained ratio is negative, the smaller the ratio is, the lower the load degree of the distributed node in all distributed nodes in the distribution is, and the time slice of each priority queue is smaller when the distributed node needs to process tasks, so that each priority queue is optimizedThe task in the priority queue is switched for multiple times during processing, which is more beneficial to quickly processing the task, so that the summation result of the ratio and the preset value a is used as the degree to be adjusted by each priority queue of the mth distributed node corresponding to the basic time slice.
Finally, the server load adjustment coefficient of the mth distributed node is used for adjusting the basic time slice in each priority queue of the mth distributed node, taking the nth priority queue as an example, and the calculation mode of the adjusted final time slice size is as follows:
wherein the content of the first and second substances,
the base slot for the nth priority queue in the mth distributed node.
When the load degree of the mth distributed node server is larger, the larger the load degree of the mth distributed node server is
The larger, the greater the number of the>
After the basic time slice corresponding to the nth priority queue of the mth distributed node is adjusted, the size of the obtained final time slice is larger than that of the basic time slice, and at the moment, when the task in the nth priority queue is processed, the frequency of task switching is correspondingly reduced, so that the computing resources of the distributed node are saved, otherwise, the task can be switched for many times, and the fast processing of the task is facilitated.
And adjusting the basic time slices corresponding to all the priority queues of the mth distributed node to obtain the final time slice size corresponding to all the priority queues of the mth distributed node.
And repeating the method to obtain the final time slice of each priority queue of each distributed node.
Step S004: and obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue, and processing each task according to the final time slice of each priority queue.
Acquiring time slice self-adaption of different priority queues on all distributed nodes, processing tasks on the distributed nodes according to time slices of the corresponding different priority queues of the different distributed nodes, and processing the task on the distributed node by using the currently processed second distributed node on the mth distributed node
The specific process of each task is as follows:
firstly to the first
Each task is processed in the queue with the highest priority and the processing time length is->
The size of each time slice corresponds to two cases, which are respectively as follows:
in the first case: one time slice length in the highest priority queue
The ith task is processed;
in the second case: one time slice length in the highest priority queue
In the method, the ith task is not processed and finished;
for the first condition, other tasks are continuously selected for processing; for the second case, then
The priority of each task is degraded to a next level, the ith task is queued at the end of the priority queue of the next level, and the processing time of the task is the time of the taskAnd (3) the time slice size corresponding to the priority queue of the first level is judged, whether the processing is finished or not is judged continuously, if the processing is finished, the processing and the task are processed, if the processing is not finished, the queuing waiting processing at the end of the priority degradation is continuously carried out, and the process is repeated until the processing of the task is finished.
Through the steps, distributed scheduling management of the time sequence database is completed.
Another embodiment of the present invention provides a distributed scheduling management system based on a time-series database, as shown in fig. 2, the system includes a task creation module, a basic time slice acquisition module, a server load adjustment coefficient acquisition module, and a task processing module, where:
the task creating module is used for creating tasks corresponding to different monomer applications in the time sequence database, and building a distributed framework of the time sequence database according to each task to obtain each distributed node of the time sequence database;
the basic time slice acquisition module is used for acquiring each task to be queued in the time sequence database; obtaining average predicted processing time according to the predicted processing time of each task; obtaining each first-class task and each second-class task according to the average predicted processing time and each predicted processing time of each task; obtaining the basic time slice of the highest priority queue according to the average predicted processing time and the predicted processing time of each class of tasks; obtaining a basic time slice of the lowest priority queue according to the average predicted processing time and the predicted processing time of each second-class task; obtaining the basic time slices of each priority queue according to the basic time slice of the highest priority queue and the basic time slice of the lowest priority queue;
the server load adjustment coefficient acquisition module is used for taking any one distributed node as a target distributed node, obtaining a first load degree of the target distributed node according to the server load degrees of all distributed nodes in the time sequence database, and obtaining a second load degree of the target distributed node according to the server load degrees of the rest distributed nodes in the time sequence database; obtaining a server load adjusting coefficient of a target distributed node according to the server load degree, the first load degree and the second load degree of the target distributed node, and obtaining the server load adjusting coefficient of each distributed node by taking each distributed node as the target distributed node;
and the task processing module is used for obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue and processing each task according to the final time slice of each priority queue.
Compared with the single fixed time slice in the queues with different priorities in the prior art, the method and the device have the advantages that the basic time slices are set for the queues with different priorities according to the predicted processing time of different tasks, and then the basic time slices of the queues with different priorities of each distributed node are adaptively adjusted by utilizing the processing capacity of each distributed node, namely the server load degree, so that when each distributed node performs task processing, frequent switching of processing tasks due to too small time slices is avoided, resource waste of the distributed nodes is avoided, and long waiting time is not needed when task processing is performed in each priority queue due to too large time slices, so that the task processing efficiency is low, and the distributed scheduling efficiency of the time sequence database is improved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (7)
1. The distributed scheduling management method based on the time sequence database is characterized by comprising the following steps:
establishing tasks corresponding to different monomer applications in a time sequence database, and building a distributed framework of the time sequence database according to each task to obtain each distributed node of the time sequence database;
acquiring each task to be queued in a time sequence database; obtaining average predicted processing time according to the predicted processing time of each task; obtaining each first-class task and each second-class task according to the average predicted processing time and each predicted processing time of each task; obtaining the basic time slice of the highest priority queue according to the average predicted processing time and the predicted processing time of each class of tasks; obtaining a basic time slice of the lowest priority queue according to the average predicted processing time and the predicted processing time of each second-class task; obtaining the basic time slice of each priority queue according to the basic time slice of the highest priority queue and the basic time slice of the lowest priority queue;
any one distributed node is taken as a target distributed node, a first load degree of the target distributed node is obtained according to the server load degrees of all distributed nodes in the time sequence database, and a second load degree of the target distributed node is obtained according to the server load degrees of the rest distributed nodes in the time sequence database; obtaining a server load adjusting coefficient of a target distributed node according to the server load degree, the first load degree and the second load degree of the target distributed node, and obtaining the server load adjusting coefficient of each distributed node by taking each distributed node as the target distributed node;
and obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue, and processing each task according to the final time slice of each priority queue.
2. The distributed scheduling management method based on the time series database according to claim 1, wherein the method for acquiring each first-class task and each second-class task comprises:
and recording the tasks with the predicted processing time less than the average predicted processing time as the first-class tasks, and recording the tasks with the predicted processing time more than or equal to the average predicted processing time as the second-class tasks.
3. The distributed scheduling management method based on time sequence database according to claim 1, wherein the method for obtaining the base time slice of the highest priority queue comprises:
calculating the difference between the average predicted processing time and the predicted processing time of each class of tasks, calculating the average value of each difference, and taking the difference between the average predicted processing time and the average value as the base time slice of the highest priority queue.
4. The distributed scheduling management method based on time sequence database according to claim 1, wherein the method for acquiring the basic time slice of the lowest priority queue comprises:
calculating the difference between the predicted processing time and the average predicted processing time of each of the two types of tasks, calculating the average value of the obtained differences, and taking the sum of the average predicted processing time and the obtained average value as the base time slice of the lowest priority queue.
5. The distributed scheduling management method based on time series database according to claim 1, wherein the obtaining expression of the basic time slice of each priority queue is:
in the formula (I), the compound is shown in the specification,
a base time slice for the nth priority queue;
a base time slice for the highest priority queue;
a base time slice for the lowest priority queue; and N is the total number of the priority queues.
6. The distributed scheduling management method based on time series database according to claim 1, wherein the step of obtaining the server load adjustment factor of the target distribution node comprises:
calculating a difference value between the server load degree of the target distributed node and the second load degree of the target distributed node, calculating a ratio between the obtained difference value and the first load degree of the target distributed node, and taking a sum result between a preset value and the obtained ratio as a server load adjustment coefficient of the target distributed node.
7. The distributed scheduling management system based on the time sequence database is characterized by comprising the following modules:
the task creating module is used for creating tasks corresponding to different monomer applications in the time sequence database, and building a distributed framework of the time sequence database according to each task to obtain each distributed node of the time sequence database;
the basic time slice acquisition module is used for acquiring each task to be queued in the time sequence database; obtaining average predicted processing time according to the predicted processing time of each task; obtaining each first-class task and each second-class task according to the average predicted processing time and each predicted processing time of each task; obtaining the basic time slice of the highest priority queue according to the average predicted processing time and the predicted processing time of each class of tasks; obtaining a basic time slice of the lowest priority queue according to the average predicted processing time and the predicted processing time of each second-class task; obtaining the basic time slices of each priority queue according to the basic time slice of the highest priority queue and the basic time slice of the lowest priority queue;
the server load adjustment coefficient acquisition module is used for taking any one distributed node as a target distributed node, obtaining a first load degree of the target distributed node according to the server load degrees of all distributed nodes in the time sequence database, and obtaining a second load degree of the target distributed node according to the server load degrees of the rest distributed nodes in the time sequence database; obtaining a server load adjusting coefficient of a target distributed node according to the server load degree, the first load degree and the second load degree of the target distributed node, and obtaining the server load adjusting coefficient of each distributed node by taking each distributed node as the target distributed node;
and the task processing module is used for obtaining the final time slice of each priority queue according to the server load adjustment coefficient of each distributed node and the basic time slice of each priority queue and processing each task according to the final time slice of each priority queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310139421.1A CN115840632B (en) | 2023-02-21 | 2023-02-21 | Distributed scheduling management method and system based on time sequence database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310139421.1A CN115840632B (en) | 2023-02-21 | 2023-02-21 | Distributed scheduling management method and system based on time sequence database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115840632A true CN115840632A (en) | 2023-03-24 |
| CN115840632B CN115840632B (en) | 2023-05-23 |
Family
ID=85579943
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310139421.1A Active CN115840632B (en) | 2023-02-21 | 2023-02-21 | Distributed scheduling management method and system based on time sequence database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115840632B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8151100B2 (en) * | 2008-02-06 | 2012-04-03 | Broadcom Corporation | Operating system for a computing device with handheld and extended computing units |
| CN106775947A (en) * | 2016-12-06 | 2017-05-31 | 中国电子科技集团公司第三十二研究所 | Large-scale virtual computing dynamic load balancing method based on openstack |
| CN108345501A (en) * | 2017-01-24 | 2018-07-31 | 全球能源互联网研究院 | A kind of distributed resource scheduling method and system |
| CN109947552A (en) * | 2019-03-28 | 2019-06-28 | 南京邮电大学 | Margin control dynamic task scheduling method based on process and thread scheduling |
| CN111464611A (en) * | 2020-03-30 | 2020-07-28 | 中科边缘智慧信息科技（苏州）有限公司 | Method for efficiently accessing service between fixed cloud and edge node in dynamic complex scene |
| CN114416785A (en) * | 2021-11-23 | 2022-04-29 | 北京基智科技有限公司 | Stream type enterprise big data processing method and storage medium |
| CN114860460A (en) * | 2022-07-05 | 2022-08-05 | 深圳市遇贤微电子有限公司 | Database acceleration method and device and computer equipment |
-
2023
- 2023-02-21 CN CN202310139421.1A patent/CN115840632B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8151100B2 (en) * | 2008-02-06 | 2012-04-03 | Broadcom Corporation | Operating system for a computing device with handheld and extended computing units |
| CN106775947A (en) * | 2016-12-06 | 2017-05-31 | 中国电子科技集团公司第三十二研究所 | Large-scale virtual computing dynamic load balancing method based on openstack |
| CN108345501A (en) * | 2017-01-24 | 2018-07-31 | 全球能源互联网研究院 | A kind of distributed resource scheduling method and system |
| CN109947552A (en) * | 2019-03-28 | 2019-06-28 | 南京邮电大学 | Margin control dynamic task scheduling method based on process and thread scheduling |
| CN111464611A (en) * | 2020-03-30 | 2020-07-28 | 中科边缘智慧信息科技（苏州）有限公司 | Method for efficiently accessing service between fixed cloud and edge node in dynamic complex scene |
| CN114416785A (en) * | 2021-11-23 | 2022-04-29 | 北京基智科技有限公司 | Stream type enterprise big data processing method and storage medium |
| CN114860460A (en) * | 2022-07-05 | 2022-08-05 | 深圳市遇贤微电子有限公司 | Database acceleration method and device and computer equipment |
Non-Patent Citations (3)
| Title |
|---|
| KENNETH E. HOGANSON: "Multi-core real-time scheduling in multilevel feedback queue with starvation mitigation (MLFQ-RT)" * |
| 李金方;张军;饶德胜;张艳丽;: "路由器自适应可变时间片数据包转发方法" * |
| 邱相存: "基于改进的多级反馈队列算法的任务调度研究" * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115840632B (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113950103B (en) | Multi-server complete computing unloading method and system under mobile edge environment | |
| CN108762896B (en) | Hadoop cluster-based task scheduling method and computer equipment | |
| CN107579926B (en) | QoS setting method of Ceph cloud storage system based on token bucket algorithm | |
| CN111400022A (en) | Resource scheduling method and device and electronic equipment | |
| CN109408215B (en) | Task scheduling method and device for computing node | |
| CN111694655A (en) | Multitask-oriented edge computing resource allocation method | |
| CN109992403B (en) | Optimization method and device for multi-tenant resource scheduling, terminal equipment and storage medium | |
| CN111163178B (en) | Game theory-based service deployment and task unloading method in edge computing | |
| CN109656713B (en) | Container scheduling method based on edge computing framework | |
| CN113641417B (en) | Vehicle security task unloading method based on branch-and-bound method | |
| CN115776524B (en) | Multi-level scheduling transmission system for mass data of Internet of things for intelligent manufacturing | |
| CN111585915B (en) | Long and short flow balanced transmission method and system, storage medium and cloud server | |
| CN112148454A (en) | Edge computing method supporting serial and parallel and electronic equipment | |
| CN114205353A (en) | Calculation unloading method based on hybrid action space reinforcement learning algorithm | |
| CN109491775B (en) | Task processing and scheduling method used in edge computing environment | |
| CN111597025B (en) | Edge calculation scheduling algorithm and system | |
| CN115840632A (en) | Distributed scheduling management method and system based on time sequence database | |
| US20050182747A1 (en) | Method and system for executing multiple tasks at adaptively controlled resource utilization rates to achieve equal QoS levels | |
| CN107678843B (en) | Process scheduling method adopting multistage feedback queue | |
| CN113448714A (en) | Computing resource control system based on cloud platform | |
| CN116302578B (en) | QoS (quality of service) constraint stream application delay ensuring method and system | |
| CN116089033A (en) | Task scheduling method based on multistage heterogeneous dynamic queue | |
| CN115858048A (en) | Hybrid key level task oriented dynamic edge arrival unloading method | |
| CN113419831B (en) | Sandbox task scheduling method and system | |
| CN115766612A (en) | Scheduling method based on weight conversion probability and corresponding device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |