CN115934300B - Cloud computing platform inspection task scheduling method and system - Google Patents
Cloud computing platform inspection task scheduling method and system Download PDFInfo
- Publication number
- CN115934300B CN115934300B CN202310215940.1A CN202310215940A CN115934300B CN 115934300 B CN115934300 B CN 115934300B CN 202310215940 A CN202310215940 A CN 202310215940A CN 115934300 B CN115934300 B CN 115934300B
- Authority
- CN
- China
- Prior art keywords
- tasks
- task
- cloud computing
- computing platform
- inspection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 123
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000001174 ascending effect Effects 0.000 claims abstract description 36
- 230000007306 turnover Effects 0.000 claims description 34
- 238000004364 calculation method Methods 0.000 claims description 26
- 238000001514 detection method Methods 0.000 claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 238000012417 linear regression Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 abstract description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 10
- 238000013515 script Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000010453 quartz Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention provides a cloud computing platform inspection task scheduling method and a cloud computing platform inspection task scheduling system, wherein the method comprises the following steps: detecting performance data of the cloud computing platform and calculating the number of submittable tasks; calculating the number of spare tasks, arranging the inspection tasks to be executed in ascending order, and selecting the submitting tasks to be executed; counting pre-completion data of the submitting task, calculating average weighted turnaround time of the submitting task, and determining upper and lower limits of corresponding adjustment index thresholds; iteratively adjusting the threshold range until the numerical value of the calculated number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range; and calculating the number of executable tasks according to the adjusted number of the submittable tasks and the current number of the tasks in execution, and scheduling the patrol tasks to be executed in ascending order arrangement. The method can ensure the execution efficiency of the inspection system task and simultaneously give consideration to the influence on the inspection object; and meanwhile, self-learning is introduced, and automatic iterative training is performed for tuning.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a cloud computing platform inspection task scheduling method and system.
Background
After the cloud computing center goes through the stage of scale development, various physical resources and virtual resources are integrated to form a unified logic resource pool, so that the resource utilization rate and management efficiency of the cloud computing center are effectively improved. Under the distributed architecture, the cloud computing center applies the distributed deployment of system function modules, the service system is finely divided in function and various in version, and meanwhile, the calling relationship among the modules is complex.
The inspection work is an indispensable work for guaranteeing stable and effective operation of the cloud computing platform system, and aims to discover potential hidden danger in the system in time. Different inspection scripts are defined through an inspection system in the industry, the inspection scripts are executed on target resources or remotely executed, the running states of the corresponding resources are obtained, potential hidden dangers in the system are found in time, the system risk is reduced, and the service continuity is improved.
When the existing scheme is used for inspecting more target resources, the problems of low efficiency, service stability damage and the like exist: when the automatic inspection system is executed in batches, different batches are executed in series every time of batch execution, and the existing efficiency is low; when tasks of each batch are more, performance pressure can be caused on the cloud computing platform, and normal operation of a service system on the cloud computing platform is affected.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a cloud computing platform inspection task scheduling method and system.
The embodiment of the invention provides a cloud computing platform inspection task scheduling method, which comprises the following steps:
detecting performance data of a cloud computing platform, and calculating the number of submittable tasks of the cloud computing platform according to the performance data, wherein the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and a threshold range corresponding to the performance index;
inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of submittable tasks, inquiring the historical execution time of the completed historical tasks in a database, and calculating the expected execution time of the inspection tasks to be executed by combining a unitary linear regression algorithm;
the inspection tasks to be executed are arranged in an ascending order based on the expected execution time, the first N inspection tasks to be executed after the ascending order are selected to be used as pre-completed submitting tasks of the inspection tasks to be executed, and N is the number of spare tasks;
counting pre-completion data of the submitting task after the submitting task is pre-completed, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time;
iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the calculated numerical value of the number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range;
and calculating the number of executable tasks of the cloud computing platform according to the adjusted number of submittable tasks and the number of tasks in current execution, and scheduling the inspection tasks to be executed in ascending order according to the number of executable tasks.
In one embodiment, in the iterative adjustment, a calculation formula for adjusting an upper limit and a lower limit of an index threshold includes:
and (3) adjusting a calculation formula of the index threshold lower limit:
wherein, the liquid crystal display device comprises a liquid crystal display device,
to adjust the index threshold lower limit->
For average weighted turnaround time,/->
For the average weighted turnaround time of the previous period in said iterative adjustment, the initial value is 0,/->
In order to adjust the adjustment factor of the index threshold,
,/>
wherein->
The number of the performance indexes is the number;
the calculation formula of the upper limit of the adjustment index threshold is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
to adjust the upper limit of the index threshold.
In one embodiment, the method further comprises:
when the task type in the inspection task to be executed is greater than 1 item, after calculating the number of executable tasks of the cloud computing platform, the method further includes:
calculating the task proportion of each type of classification task to the inspection task to be executed, calculating an adjustment step length based on a threshold range after the iterative adjustment of the classification task is completed, and calculating an adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classification task;
based on the adjusted task proportion, combining the number of submittable tasks, calculating the number of submittable tasks of the classified tasks, acquiring the number of tasks in the current execution of the classified tasks, and calculating the number of executable tasks of the classified tasks.
In one embodiment, the calculating formula for calculating the adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classification task includes:
In one embodiment, the task types include:
computing service type, storage service type, network service type, authentication service type.
In one embodiment, the method further comprises:
modifying the to-be-executed inspection task after scheduling from the to-be-executed state to an in-execution state in the cloud computing platform;
and when detecting that the inspection task in the state to be executed does not exist in the cloud computing platform, ending the task.
The embodiment of the invention provides a cloud computing platform inspection task scheduling system, which comprises the following steps:
the system comprises a detection module, a calculation module and a calculation module, wherein the detection module is used for detecting performance data of a cloud computing platform and calculating the number of submittable tasks of the cloud computing platform according to the performance data, and the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and a threshold range corresponding to the performance index;
the query module is used for querying the number of tasks in current execution, calculating the number of spare tasks by combining the number of submittable tasks, querying the historical execution time of the completed historical tasks in the database, and calculating the expected execution time of the patrol task to be executed by combining a unitary linear regression algorithm;
the arrangement module is used for carrying out ascending arrangement on the inspection tasks to be executed based on the expected execution time, selecting the first N inspection tasks to be executed after the ascending arrangement as pre-completed submitting tasks of the inspection tasks to be executed, wherein N is the number of spare tasks;
the statistics module is used for counting pre-completion data of the submitting task after the submitting task is pre-completed, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time;
the iteration adjustment module is used for carrying out iteration adjustment on the threshold range according to the upper limit and the lower limit of the adjustment index threshold until the numerical value of the number of the adjusted submittable tasks of the cloud computing platform obtained through calculation reaches the maximum based on the adjusted threshold range;
and the scheduling module is used for calculating the executable task number of the cloud computing platform according to the adjusted number of the submittable tasks and the current executing task number, and scheduling the routing inspection tasks to be executed in ascending order according to the executable task number.
In one embodiment, the system further comprises:
the calculation module is used for calculating the task proportion of each type of classification task to the to-be-executed inspection task when the task type in the to-be-executed inspection task is more than 1 item, calculating an adjustment step length based on a threshold range after the iterative adjustment of the classification task is completed, and calculating an adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classification task;
the second calculation module is used for calculating the number of the submittable tasks of the classified tasks based on the adjusted task proportion and combining the number of the submittable tasks, obtaining the number of the tasks in the current execution of the classified tasks and calculating the number of the executable tasks of the classified tasks.
The embodiment of the invention provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the steps of the cloud computing platform inspection task scheduling method are realized when the processor executes the program.
The embodiment of the invention provides a non-transitory computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the cloud computing platform inspection task scheduling method described above.
According to the cloud computing platform inspection task scheduling method and system, performance data of a cloud computing platform are detected, the number of the submittable tasks of the cloud computing platform is calculated according to the performance data, and the performance data comprise detection values of performance indexes of the cloud computing platform, maximum concurrency upper limits of inspection tasks, weight and threshold value ranges corresponding to the performance indexes; inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of tasks which can be submitted, inquiring the historical execution time of the completed historical tasks in the database, and calculating the expected execution time of the inspection task to be executed by combining a unitary linear regression algorithm; the method comprises the steps of carrying out ascending arrangement on the inspection tasks to be executed based on expected execution time, selecting the first N inspection tasks to be executed after the ascending arrangement as pre-completed submitting tasks of the inspection tasks to be executed, wherein N is the number of spare tasks; counting pre-completion data of the submitting task after the pre-completion of the submitting task, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time; iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the numerical value of the calculated number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range; and calculating the executable task number of the cloud computing platform according to the adjusted submittable task number and the current executing task number, and scheduling the inspection tasks to be executed in ascending order according to the executable task number. Therefore, the concurrent quantity of the patrol tasks can be adjusted in real time by counting the system load of the patrol objects and the execution condition of the patrol tasks, so that the influence on the patrol objects is considered while the execution efficiency of the patrol system tasks is ensured; and simultaneously, self-learning is introduced, automatic iterative training is carried out to correct input parameters affecting the concurrent number of tasks, and the execution efficiency of the inspection system is optimized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a cloud computing platform inspection task scheduling method in an embodiment of the invention;
FIG. 2 is a flowchart of a method for scheduling inspection tasks of a cloud computing platform according to another embodiment of the present invention;
FIG. 3 is a block diagram of a cloud computing platform inspection task scheduling system in an embodiment of the invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flow chart of a cloud computing platform inspection task scheduling method according to an embodiment of the present invention, and as shown in fig. 1, the embodiment of the present invention provides a cloud computing platform inspection task scheduling method, including:
step S101, detecting performance data of a cloud computing platform, and calculating the number of submittable tasks of the cloud computing platform according to the performance data, wherein the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and a threshold range corresponding to the performance index.
Specifically, cloud computing platform performance data is predefined, including: performance indexes such as host CPU, memory, disk IO, network throughput, core component request latency, etc., aggregate j= { J 1 ,J 2 ,…,J k Each performance index Ji corresponds to a weight w i And oneThreshold range d i ，
Using w= (W 1 ,w 2 ,...,w k ) Weight representing k performance indicators, d= (D 1 ,d 2 ,...,d k ) The threshold range of k performance indexes is represented, M represents the maximum concurrency upper limit of the allowable inspection task preset by the cloud computing platform, and C= (C) 1 ,c 2 ,...,c k ) And obtaining the performance index detection value of the cloud computing platform. And calculating the number of tasks which can be submitted currently by the cloud computing platform according to the following formula by combining the running condition of the current cloud computing platform:
wherein P is the number of submittable tasks of the cloud computing platform.
Step S102, inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of submittable tasks, inquiring the historical execution time of the completed historical tasks in a database, and calculating the expected execution time of the patrol task to be executed by combining a unitary linear regression algorithm.
Specifically, the method includes the steps of inquiring the number Q of tasks in current execution, calculating the number N=P-Q of spare tasks in combination with the number P of submittable tasks, then inquiring the historical execution time of completed historical tasks in a database, and calculating the expected execution time of the patrol task to be executed, wherein the method comprises the following steps:
the completed inspection task R= { R in the query history 1 ,r 2 ,..,r m }. Acquisition task r k Completed history execution time: t (T) k = (t k1 ,t k2 ,..,t kn ) The execution time v= (V) of each patrol task to be executed is predicted 1 ,v 2 ,...,v m )；
The prediction algorithm uses a unitary linear regression algorithm, which comprises the following steps:
1. the sample regression equation is set as
2. The parameters a and b in the regression equation are evaluated using the least squares method, with the aim of minimizing the objective function
Then:
wherein:
3. predicting the execution time of all tasks:
Step S103, ascending arrangement is carried out on the inspection tasks to be executed based on the expected execution time, the first N inspection tasks to be executed after ascending arrangement are selected to serve as pre-completed submitting tasks of the inspection tasks to be executed, and N is the number of spare tasks.
Specifically, the inspection tasks to be executed are arranged in an ascending order based on expected execution time, the first N inspection tasks to be executed with the shortest expected execution time after the ascending order are selected to be used as pre-completed submitting tasks of the inspection tasks to be executed, and N is the number of spare tasks, wherein the pre-completion is performed by submitting inspection scripts to a thread pool, executing the pre-completed tasks and waiting for script execution results. And acquiring and recording the running state of the cloud computing platform, and updating the task completion state and completion time to be used as a follow-up patrol record analysis.
Step S104, counting pre-completion data of the submitting task after the submitting task is pre-completed, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time.
Specifically, after the task is pre-completed, pre-completion data of the task is counted, and average weighted turnover time of the task is calculated according to the pre-completion data, wherein the pre-completion data may include relevant data such as a task completion state, a completion time and the like, and the task turnover time e of each task may be:
then, based on the average weighted turnaround time, determining the upper and lower limits of the corresponding adjustment index threshold, wherein the task turnaround time of the submitted task
The average weighted turnaround time is then calculated:
then in the iterative adjustment process, calculating through a calculation formula corresponding to the lower limit and the upper limit, including:
and (3) adjusting a calculation formula of the index threshold lower limit:
wherein, the liquid crystal display device comprises a liquid crystal display device,
to adjust the index threshold lower limit->
For average weighted turnaround time,/->
For the average weighted turnaround time of the previous period in said iterative adjustment, the initial value is 0,/->
In order to adjust the adjustment factor of the index threshold,
,/>
wherein->
The number of the performance indexes is the number;
the calculation formula of the upper limit of the adjustment index threshold is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
to adjust the upper limit of the index threshold.
And step 105, iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the calculated numerical value of the number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range.
The method comprises the steps of carrying out iterative adjustment on a threshold range according to the upper limit and the lower limit of an adjustment index threshold, namely, after the upper limit and the lower limit of the index threshold are adjusted, calculating the number of the submittable tasks of the cloud computing platform by combining other data of performance data, then carrying out adjustment until the numerical value of the calculated number of the submittable tasks of the cloud computing platform reaches the maximum value based on the adjusted threshold range, and completing iterative adjustment.
And step S106, calculating the number of executable tasks of the cloud computing platform according to the adjusted number of submittable tasks and the number of currently executing tasks, and scheduling the patrol tasks to be executed in ascending order according to the number of executable tasks.
Specifically, according to the adjusted number of submittable tasks and the number of tasks currently being executed, the number of executable tasks of the cloud computing platform is calculated, for example, the number of the submittable tasks is 100, the number of the tasks currently being executed is 40, the executable task data is 60, and the routing inspection tasks to be executed in ascending order are scheduled according to the number of the executable tasks.
In addition, when the task type in the inspection task to be executed is greater than 1 item, that is, the inspection task includes a plurality of task types, the task types may be as follows: computing service type, storage service type, network service type, authentication service type. When data calculation is performed, such as average weighted turnover time, the calculated average weighted turnover time corresponding to the current task type is calculated, and then the task proportion of each type of classified task to the inspection task to be executed is calculated
And calculate the adjustment step length based on the threshold range after finishing the adjustment of the classified task iteration, for example, the adjustment compensation may be as general as the adjustment factor of step S104, or may be calculated by using the reciprocal of the number of submittable tasks of the cloud computing platform, and then calculate the adjusted task proportion based on the task proportion, the adjustment step length, and the average weighted turnaround time corresponding to the classified task, including:
wherein, the liquid crystal display device comprises a liquid crystal display device,
task proportion corresponding to classification task, +.>
Initial value of task proportion in the iterative adjustment for classifying tasks, < >>
Average weighted turnaround time for classification tasks, +.>
The average weighted turnaround time of the classification task for the previous period in the iterative adjustment is 0, # in initial value>
The step size is adjusted for the classification task.
And then based on the adjusted task proportion, combining the number of submittable tasks, calculating the number of submittable tasks of the classified tasks, acquiring the number of tasks in the current execution of the classified tasks, and calculating the number of executable tasks of the classified tasks.
According to the cloud computing platform inspection task scheduling method and system, performance data of a cloud computing platform are detected, the number of the submittable tasks of the cloud computing platform is calculated according to the performance data, and the performance data comprise detection values of performance indexes of the cloud computing platform, maximum concurrency upper limits of inspection tasks, weight and threshold value ranges corresponding to the performance indexes; inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of tasks which can be submitted, inquiring the historical execution time of the completed historical tasks in the database, and calculating the expected execution time of the inspection task to be executed by combining a unitary linear regression algorithm; the method comprises the steps of carrying out ascending arrangement on the inspection tasks to be executed based on expected execution time, selecting the first N inspection tasks to be executed after the ascending arrangement as pre-completed submitting tasks of the inspection tasks to be executed, wherein N is the number of spare tasks; counting pre-completion data of the submitting task after the pre-completion of the submitting task, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time; iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the numerical value of the calculated number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range; and calculating the executable task number of the cloud computing platform according to the adjusted submittable task number and the current executing task number, and scheduling the inspection tasks to be executed in ascending order according to the executable task number. Therefore, the concurrent quantity of the patrol tasks can be adjusted in real time by counting the system load of the patrol objects and the execution condition of the patrol tasks, so that the influence on the patrol objects is considered while the execution efficiency of the patrol system tasks is ensured; and simultaneously, self-learning is introduced, automatic iterative training is carried out to correct input parameters affecting the concurrent number of tasks, and the execution efficiency of the inspection system is optimized.
In another embodiment, as shown in fig. 2, a method for scheduling a polling task of a cloud computing platform includes 3 modules:
the method comprises the steps of starting to checking whether a to-be-executed patrol task exists as a first module, specifically, introducing a Quartz timing task scheduling framework, creating a cloud platform performance detection timing task, associating scripts in a patrol script management module, checking the inquiry of the to-be-executed patrol task in a cloud computing platform database in a remote script execution mode, and detecting whether the to-be-executed patrol task exists;
the method comprises the steps of detecting cloud platform performance data, calculating the current executable inspection task, and calculating average weighted turnover time of submitting tasks by detecting cloud platform performance data and calculating historical task execution time, and then adjusting the upper limit and the lower limit of an index threshold by means of iterative adjustment, so that the number of executable tasks of the cloud computing platform is calculated.
The method comprises the steps of judging whether the schedulable routing inspection task data is more than 0 to the update routing inspection task to be executed is in the executing state, specifically, detecting whether the schedulable routing inspection task data of the cloud computing platform is more than 0, executing schedulable quantity of routing inspection tasks to be executed when the schedulable routing inspection task data is in the executing state, modifying the routing inspection task to be executed after scheduling in the cloud computing platform from the to-be-executed state to the executing state, and ending the task when detecting that the routing inspection task in the to-be-executed state does not exist in the cloud computing platform.
Fig. 3 is a schematic diagram of a cloud computing platform inspection task scheduling system according to an embodiment of the present invention, including: the device comprises a detection module S201, a query module S202, an arrangement module S203, a statistics module S204, an iteration adjustment module S205 and a scheduling module S206, wherein:
the detection module S201 is configured to detect performance data of a cloud computing platform, and calculate the number of submittable tasks of the cloud computing platform according to the performance data, where the performance data includes a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and a threshold range corresponding to the performance index.
The query module S202 is configured to query the number of tasks currently being executed, calculate the number of idle tasks in combination with the number of tasks that can be submitted, query the historical execution time of the completed historical tasks in the database, and calculate the expected execution time of the inspection task to be executed in combination with the unitary linear regression algorithm.
The arrangement module S203 is configured to arrange the inspection tasks to be executed in an ascending order based on the expected execution time, and select the first N inspection tasks to be executed after the ascending order, as the pre-completed submitted tasks of the inspection tasks to be executed, where N is the number of spare tasks.
And the statistics module S204 is used for counting pre-completion data of the submitted task after the pre-completion of the submitted task, calculating the average weighted turnover time of the submitted task according to the pre-completion data, and determining the upper and lower limits of the corresponding adjustment index threshold based on the average weighted turnover time.
And the iteration adjustment module S205 is configured to iteratively adjust the threshold range according to the upper and lower limits of the adjustment index threshold until the calculated number of the adjusted submittable tasks of the cloud computing platform reaches the maximum value based on the adjusted threshold range.
And the scheduling module S206 is used for calculating the number of executable tasks of the cloud computing platform according to the adjusted number of submittable tasks and the number of currently executing tasks, and scheduling the routing inspection tasks to be executed in ascending order according to the number of executable tasks.
In one embodiment, the system further comprises:
and the calculation module is used for calculating the task proportion of each type of classified task to the to-be-executed inspection task when the task type in the to-be-executed inspection task is more than 1 item, calculating an adjustment step length based on the threshold range after the iterative adjustment of the classified task is completed, and calculating the adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classified task.
The second calculation module is used for calculating the number of the submittable tasks of the classified tasks based on the adjusted task proportion and combining the number of the submittable tasks, obtaining the number of the tasks in the current execution of the classified tasks and calculating the number of the executable tasks of the classified tasks.
For specific limitation of the cloud computing platform inspection task scheduling system, reference may be made to the limitation of the cloud computing platform inspection task scheduling method hereinabove, and no further description is given here. All or part of each module in the cloud computing platform inspection task scheduling system can be realized by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
Fig. 4 illustrates a physical schematic diagram of an electronic device, as shown in fig. 4, which may include: a processor (processor) 301, a memory (memory) 302, a communication interface (Communications Interface) 303 and a communication bus 304, wherein the processor 301, the memory 302 and the communication interface 303 perform communication with each other through the communication bus 304. The processor 301 may call logic instructions in the memory 302 to perform the following method: detecting performance data of the cloud computing platform, and computing the number of submittable tasks of the cloud computing platform according to the performance data, wherein the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and threshold range corresponding to the performance index; inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of tasks which can be submitted, inquiring the historical execution time of the completed historical tasks in the database, and calculating the expected execution time of the inspection task to be executed by combining a unitary linear regression algorithm; the method comprises the steps of carrying out ascending arrangement on the inspection tasks to be executed based on expected execution time, selecting the first N inspection tasks to be executed after the ascending arrangement as pre-completed submitting tasks of the inspection tasks to be executed, wherein N is the number of spare tasks; counting pre-completion data of the submitting task after the pre-completion of the submitting task, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time; iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the numerical value of the calculated number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range; and calculating the executable task number of the cloud computing platform according to the adjusted submittable task number and the current executing task number, and scheduling the inspection tasks to be executed in ascending order according to the executable task number.
Further, the logic instructions in memory 302 described above may be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, embodiments of the present invention further provide a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the transmission method provided in the above embodiments, for example, including: detecting performance data of the cloud computing platform, and computing the number of submittable tasks of the cloud computing platform according to the performance data, wherein the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and threshold range corresponding to the performance index; inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of tasks which can be submitted, inquiring the historical execution time of the completed historical tasks in the database, and calculating the expected execution time of the inspection task to be executed by combining a unitary linear regression algorithm; the method comprises the steps of carrying out ascending arrangement on the inspection tasks to be executed based on expected execution time, selecting the first N inspection tasks to be executed after the ascending arrangement as pre-completed submitting tasks of the inspection tasks to be executed, wherein N is the number of spare tasks; counting pre-completion data of the submitting task after the pre-completion of the submitting task, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time; iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the numerical value of the calculated number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range; and calculating the executable task number of the cloud computing platform according to the adjusted submittable task number and the current executing task number, and scheduling the inspection tasks to be executed in ascending order according to the executable task number.
The system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. The cloud computing platform inspection task scheduling method is characterized by comprising the following steps of:
detecting performance data of a cloud computing platform, and calculating the number of submittable tasks of the cloud computing platform according to the performance data, wherein the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and a threshold range corresponding to the performance index;
inquiring the number of tasks in current execution, calculating the number of spare tasks by combining the number of submittable tasks, inquiring the historical execution time of the completed historical tasks in a database, and calculating the expected execution time of the inspection tasks to be executed by combining a unitary linear regression algorithm;
the inspection tasks to be executed are arranged in an ascending order based on the expected execution time, the first N inspection tasks to be executed after the ascending order are selected to be used as pre-completed submitting tasks of the inspection tasks to be executed, and N is the number of spare tasks;
counting pre-completion data of the submitting task after the submitting task is pre-completed, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time;
iteratively adjusting the threshold range according to the upper and lower limits of the adjustment index threshold until the calculated numerical value of the number of the adjusted submittable tasks of the cloud computing platform reaches the maximum based on the adjusted threshold range;
calculating the number of executable tasks of the cloud computing platform according to the adjusted number of submittable tasks and the number of tasks in current execution, and scheduling the inspection tasks to be executed in ascending order according to the number of executable tasks;
the performance data of the cloud computing platform is detected, the number of the submittable tasks of the cloud computing platform is calculated according to the performance data, and the number of the submittable tasks of the cloud computing platform is calculated according to the following formula:
wherein each performance index Ji corresponds to a weight w i And a threshold range d i ，d i ＝[d imin ，d imax ]，W＝(w 1 ,w 2 ,...,w k ) Weight representing k performance indicators, d= (D 1 ,d 2 ,...,d k ) The threshold range of k performance indexes is represented, M represents the maximum concurrency of the allowable inspection tasks preset by the cloud computing platformUpper limit, c= (C 1 ,c 2 ,...,c k ) The method comprises the steps of representing and acquiring performance index detection values of a cloud computing platform, wherein P is the number of submittable tasks of the cloud computing platform;
in the iterative adjustment, a calculation formula for adjusting the upper and lower limits of the index threshold comprises:
and (3) adjusting a calculation formula of the index threshold lower limit:
wherein d imin In order to adjust the index threshold lower limit,
for average weighted turnaround time,/->
The average weighted turnover time of the previous period in the iterative adjustment is 0 with the initial value of delta d i In order to adjust the adjustment factor of the index threshold,
wherein k is the number of the performance indexes;
the calculation formula of the upper limit of the adjustment index threshold is as follows:
wherein d imax To adjust the upper limit of the index threshold.
2. The cloud computing platform inspection task scheduling method of claim 1, further comprising:
when the task type in the inspection task to be executed is greater than 1 item, after calculating the number of executable tasks of the cloud computing platform, the method further includes:
calculating the task proportion of each type of classification task to the inspection task to be executed, calculating an adjustment step length based on a threshold range after the iterative adjustment of the classification task is completed, and calculating an adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classification task;
based on the adjusted task proportion, combining the number of submittable tasks, calculating the number of submittable tasks of the classified tasks, acquiring the number of tasks in the current execution of the classified tasks, and calculating the number of executable tasks of the classified tasks.
3. The cloud computing platform inspection task scheduling method according to claim 2, wherein a calculation formula for calculating the adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classified task comprises:
wherein U is min To classify the task proportion corresponding to the task, U 0 For classifying the initial value of the task proportion corresponding to the task in the iterative adjustment,
average weighted turnaround time for classification tasks, +.>
The average weighted turnaround time of the classification task for the previous period in the iterative adjustment is 0, # in initial value>
The step size is adjusted for the classification task.
4. The cloud computing platform inspection task scheduling method according to claim 2, wherein the task types include:
computing service type, storage service type, network service type, authentication service type.
5. The cloud computing platform inspection task scheduling method according to claim 1, wherein after the inspection tasks to be executed in the ascending order are scheduled according to the number of executable tasks, further comprising:
modifying the to-be-executed inspection task after scheduling from the to-be-executed state to an in-execution state in the cloud computing platform;
and when detecting that the inspection task in the state to be executed does not exist in the cloud computing platform, ending the task.
6. A cloud computing platform inspection task scheduling system, the system comprising:
the system comprises a detection module, a calculation module and a calculation module, wherein the detection module is used for detecting performance data of a cloud computing platform and calculating the number of submittable tasks of the cloud computing platform according to the performance data, and the performance data comprises a detection value of a performance index of the cloud computing platform, a maximum concurrency upper limit of a patrol task, and a weight and a threshold range corresponding to the performance index;
the query module is used for querying the number of tasks in current execution, calculating the number of spare tasks by combining the number of submittable tasks, querying the historical execution time of the completed historical tasks in the database, and calculating the expected execution time of the patrol task to be executed by combining a unitary linear regression algorithm;
the arrangement module is used for carrying out ascending arrangement on the inspection tasks to be executed based on the expected execution time, selecting the first N inspection tasks to be executed after the ascending arrangement as pre-completed submitting tasks of the inspection tasks to be executed, wherein N is the number of spare tasks;
the statistics module is used for counting pre-completion data of the submitting task after the submitting task is pre-completed, calculating average weighted turnover time of the submitting task according to the pre-completion data, and determining corresponding upper and lower limits of the adjustment index threshold based on the average weighted turnover time;
the iteration adjustment module is used for carrying out iteration adjustment on the threshold range according to the upper limit and the lower limit of the adjustment index threshold until the numerical value of the number of the adjusted submittable tasks of the cloud computing platform obtained through calculation reaches the maximum based on the adjusted threshold range;
the scheduling module is used for calculating the number of executable tasks of the cloud computing platform according to the adjusted number of the submittable tasks and the number of the tasks in current execution, and scheduling the routing inspection tasks to be executed in ascending order according to the number of the executable tasks;
the performance data of the cloud computing platform is detected, the number of the submittable tasks of the cloud computing platform is calculated according to the performance data, and the number of the submittable tasks of the cloud computing platform is calculated according to the following formula:
wherein each performance index Ji corresponds to a weight w i And a threshold range d i ，d i ＝[d imin ，d imax ]，W＝(w 1 ,w 2 ,...,w k ) Weight representing k performance indicators, d= (D 1 ,d 2 ,...,d k ) The threshold range of k performance indexes is represented, M represents the maximum concurrency upper limit of the allowable inspection task preset by the cloud computing platform, and C= (C) 1 ,c 2 ,...,c k ) The method comprises the steps of representing and acquiring performance index detection values of a cloud computing platform, wherein P is the number of submittable tasks of the cloud computing platform;
in the iterative adjustment, a calculation formula for adjusting the upper and lower limits of the index threshold comprises:
and (3) adjusting a calculation formula of the index threshold lower limit:
wherein d imin In order to adjust the index threshold lower limit,
for average weighted turnaround time,/->
The average weighted turnover time of the previous period in the iterative adjustment is 0 with the initial value of delta d i In order to adjust the adjustment factor of the index threshold,
wherein k is the number of the performance indexes;
the calculation formula of the upper limit of the adjustment index threshold is as follows:
wherein d imax To adjust the upper limit of the index threshold.
7. The cloud computing platform inspection task scheduling system of claim 6, said system further comprising:
the calculation module is used for calculating the task proportion of each type of classification task to the to-be-executed inspection task when the task type in the to-be-executed inspection task is more than 1 item, calculating an adjustment step length based on a threshold range after the iterative adjustment of the classification task is completed, and calculating an adjusted task proportion based on the task proportion, the adjustment step length and the average weighted turnover time corresponding to the classification task;
the second calculation module is used for calculating the number of the submittable tasks of the classified tasks based on the adjusted task proportion and combining the number of the submittable tasks, obtaining the number of the tasks in the current execution of the classified tasks and calculating the number of the executable tasks of the classified tasks.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor performs the steps of the cloud computing platform inspection task scheduling method of any one of claims 1 to 5 when the program is executed.
9. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the cloud computing platform inspection task scheduling method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310215940.1A CN115934300B (en) | 2023-03-08 | 2023-03-08 | Cloud computing platform inspection task scheduling method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310215940.1A CN115934300B (en) | 2023-03-08 | 2023-03-08 | Cloud computing platform inspection task scheduling method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115934300A CN115934300A (en) | 2023-04-07 |
| CN115934300B true CN115934300B (en) | 2023-06-23 |
Family
ID=86656170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310215940.1A Active CN115934300B (en) | 2023-03-08 | 2023-03-08 | Cloud computing platform inspection task scheduling method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115934300B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026602A (en) * | 2019-10-22 | 2020-04-17 | 烽火通信科技股份有限公司 | Health inspection scheduling management method and device of cloud platform and electronic equipment |
| CN115586961A (en) * | 2022-09-28 | 2023-01-10 | 苏州浪潮智能科技有限公司 | AI platform computing resource task scheduling method, device and medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018015779A1 (en) * | 2016-07-20 | 2018-01-25 | Worldline | Multi-criteria adaptive scheduling for a market-oriented hybrid cloud infrastructure |
| CN109714395B (en) * | 2018-12-10 | 2021-10-26 | 平安科技（深圳）有限公司 | Cloud platform resource use prediction method and terminal equipment |
| CN114579270A (en) * | 2022-02-24 | 2022-06-03 | 北京理工大学 | Task scheduling method and system based on resource demand prediction |
| CN114756328A (en) * | 2022-04-21 | 2022-07-15 | 光大科技有限公司 | Container cloud platform inspection method and device |
-
2023
- 2023-03-08 CN CN202310215940.1A patent/CN115934300B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111026602A (en) * | 2019-10-22 | 2020-04-17 | 烽火通信科技股份有限公司 | Health inspection scheduling management method and device of cloud platform and electronic equipment |
| CN115586961A (en) * | 2022-09-28 | 2023-01-10 | 苏州浪潮智能科技有限公司 | AI platform computing resource task scheduling method, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115934300A (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109242135B (en) | Model operation method, device and business server | |
| CN111881023B (en) | Software aging prediction method and device based on multi-model comparison | |
| CN110633977A (en) | Payment exception processing method and device and terminal equipment | |
| CN107688626B (en) | Slow query log processing method and device and electronic equipment | |
| US8832839B2 (en) | Assessing system performance impact of security attacks | |
| CN111381970A (en) | Cluster task resource allocation method and device, computer device and storage medium | |
| CN113837596A (en) | Fault determination method and device, electronic equipment and storage medium | |
| CN110377519B (en) | Performance capacity test method, device and equipment of big data system and storage medium | |
| CN106686619B (en) | Performance evaluation method and equipment | |
| CN115934300B (en) | Cloud computing platform inspection task scheduling method and system | |
| CN113918438A (en) | Method and device for detecting server abnormality, server and storage medium | |
| CN112035236B (en) | Task scheduling method, device and storage medium based on multi-factor cooperation | |
| CN117234859A (en) | Performance event monitoring method, device, equipment and storage medium | |
| CN114884813B (en) | Network architecture determining method and device, electronic equipment and storage medium | |
| CN116011677A (en) | Time sequence data prediction method and device, electronic equipment and storage medium | |
| CN110955587A (en) | Method and device for determining equipment to be replaced | |
| CN111190800B (en) | Method, system, device and storage medium for predicting batch operation duration of host | |
| CN114999665A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111722977A (en) | System inspection method and device and electronic equipment | |
| CN116049836B (en) | Method, device, equipment and storage medium for determining vehicle vulnerability priority | |
| CN116955504B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN117575654B (en) | Scheduling method and device for data processing job | |
| CN114615144B (en) | Network optimization method and system | |
| CN117573412A (en) | System fault early warning method and device, electronic equipment and storage medium | |
| CN115185649A (en) | Resource scheduling method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 313000 floor 2, building C, building 9, Huzhou multimedia Industrial Park, No. 999, Wuxing Avenue, Wuxing District, Huzhou City, Zhejiang Province Patentee after: Zhejiang Jiuzhou Future Information Technology Co.,Ltd. Country or region after: China Address before: 313000 floor 2, building C, building 9, Huzhou multimedia Industrial Park, No. 999, Wuxing Avenue, Wuxing District, Huzhou City, Zhejiang Province Patentee before: Zhejiang Jiuzhou cloud Mdt InfoTech Ltd. Country or region before: China |