CN114327925A - Power data real-time calculation scheduling optimization method and system - Google Patents
Power data real-time calculation scheduling optimization method and system Download PDFInfo
- Publication number
- CN114327925A CN114327925A CN202111161277.9A CN202111161277A CN114327925A CN 114327925 A CN114327925 A CN 114327925A CN 202111161277 A CN202111161277 A CN 202111161277A CN 114327925 A CN114327925 A CN 114327925A
- Authority
- CN
- China
- Prior art keywords
- scheduling
- task
- time
- algorithm
- power data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000005457 optimization Methods 0.000 title claims abstract description 26
- 238000004364 calculation method Methods 0.000 title claims description 28
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 56
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 4
- 230000006872 improvement Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 8
- 230000008447 perception Effects 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000012360 testing method Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- KDYFGRWQOYBRFD-UHFFFAOYSA-N succinic acid Chemical compound OC(=O)CCC(O)=O KDYFGRWQOYBRFD-UHFFFAOYSA-N 0.000 description 1
Images
Landscapes
- Supply And Distribution Of Alternating Current (AREA)
Abstract
The invention belongs to the technical field of intelligent power grids and cloud computing, and provides a real-time computing scheduling optimization method and system for power data. The method comprises the steps of obtaining a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled; introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm; performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result; and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
Description
Technical Field
The invention belongs to the technical field of intelligent power grids and cloud computing, and particularly relates to a real-time computing scheduling optimization method and system for power data.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
With the increase of the acquisition range and frequency of the power data, the real-time calculation amount of the power data also shows the increase of geometric grade, how to reasonably distribute the overall load executed by the data real-time calculation task to each task execution node, and avoid the processing capacity and the I/O capacity of each node from becoming the bottleneck of instruction execution, which is a problem to be solved urgently. At present, the most common method is to apply a cloud computing technology, adjust the load distribution condition on each node in a task load balancing manner, and perform load balancing among the nodes, so as to utilize the existing system resources to the maximum extent and realize maximization of user service and expansibility.
The load balancing technology needs to solve three problems of node load condition definition, node state acquisition, node load adjustment and the like. Bayesian classification is probabilistic reasoning based on Bayesian formulas, namely how to accomplish reasoning and decision making tasks under the condition that various uncertain factors exist in a sample or an event but occurrence probability of the sample or the event is clarified. A bayesian network is a commonly used mathematical model, and is a graphical network based on probabilistic inference, which uses a Directed Acyclic Graph (DAG) to represent dependency relationships between attributes, and uses a conditional probability table to describe dependency degrees or joint probability distributions between attributes. The current research results only improve the load balancing efficiency of a monotonicity unit to a certain extent aiming at the problems of high task scheduling algorithm complexity, unreasonable task allocation and the like of mass data in a cloud computing environment, and cannot adapt to the service scene of complex computing task associated operation under the condition of real-time power data computing.
Disclosure of Invention
In order to solve the technical problems existing in the background technology, the invention provides a method and a system for optimizing real-time calculation scheduling of power data, which are used for comprehensively evaluating the influence of the real-time calculation tasks of the power data on queue execution in the case of multi-priority task sorting execution and the case of single task execution, adopting a HEFT algorithm, introducing a task execution delay parameter to improve the feedback of the task execution delay parameter, returning the dynamic cost evaluation of the self-executed tasks to a multi-scheduler through service equipment, and controlling task scheduling according to the evaluation result. On the basis of the research, the Bayesian network model is used for learning the algorithm scheduling result to obtain the scheduling result, and the method has high practical application value for the research of the dynamic load balancing problem of the real-time calculation of the power data.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a real-time calculation scheduling optimization method for power data.
A real-time calculation scheduling optimization method for power data comprises the following steps:
acquiring a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled;
introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm;
performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result;
and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
Further, the introduced task execution delay parameter improvement scheduling algorithm includes: in the HEFT scheduling algorithm, the task with the largest rank value is scheduled preferentially, and the HEFT algorithm is improved by introducing a task execution delay parameter.
Further, the task execution delay parameter is (the sum of the execution time of the high-priority task of the device where the task is located and the time increment of the task after the current task)/the total time of the task.
Further, the scheduling policy for improving the scheduling algorithm includes:
step (11): calculating the current task execution time delay parameter of the equipment when the time occupancy rate of the current task is tau;
step (12) calculating the rank value of the task, performing descending order arrangement on the rank value, and outputting an arrangement result queue Q;
step (13) selecting the equipment which can make the first task finished earliest for the first task, and removing the task;
step (14) repeats step (13) until all tasks in Q have been removed.
Further, the load state feedback model is utilized to perform load balancing scheduling on the directed acyclic graph to obtain a first scheduling result; learning the first scheduling result by adopting a Bayesian network model, and obtaining a second scheduling result comprises the following steps:
step (21): carrying out load balancing scheduling on the directed acyclic graph tasks with the randomly generated number of tasks being N by using a load state feedback model, and recording a first scheduling result;
step (22): selecting processing time, sequencing and time delay parameters of the subtasks on each service device as characteristic values of a training set, carrying out label classification by using a first scheduling result, and constructing a data set D;
step (23): processing the data set D by using Bayesian decision to obtain the probability that each subtask is distributed to different service equipment;
step (24): adding the probability obtained in the step (23) as a new characteristic value into the original data set D to form a data set D';
step (25): calculating a prior probability of task assignment in D';
step (26): in the training set D', randomly acquiring subtasks, and calculating the posterior probability of the tasks to be dispatched to different service equipment;
step (27): predicting a first scheduling result using a bayesian network model;
step (28): repeating steps (26) and (27) until all subtasks are predicted to be completed;
step (29): and after the scheduling is finished, outputting a second scheduling result.
Further, the rule for performing load balancing scheduling on the directed acyclic graph by using the load state feedback model includes: and (3) using the current task execution time delay parameter concept of the equipment to minimize the occupied time of the task of the equipment, and distributing the tasks by the equipment according to a preset arrival sequence.
The second aspect of the invention provides a power data real-time calculation scheduling optimization system.
A power data real-time computation scheduling optimization system, comprising:
a data acquisition module configured to: acquiring a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled;
a load state feedback module configured to: introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm;
a load balancing module configured to: performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result;
an output module configured to: and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
Further, the system comprises: a data storage module configured to: and storing and verifying the data information of each stage.
A third aspect of the invention provides a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the real-time calculation schedule optimization method for power data as described in the first aspect above.
A fourth aspect of the invention provides a computer apparatus.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the real-time calculation schedule optimization method for power data as described in the first aspect when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
in the invention, a task execution delay parameter Occupanacy (O) is introduced to improve a popular HEFT scheduling algorithm.
The invention provides a Bayesian network-based dynamic load balancing algorithm, and by utilizing a Bayesian network-based directed acyclic graph structure, the relevance and the sequence scheduling order among tasks can be clearly described, and the problem of unreasonable task allocation is solved.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
Fig. 1 is a flowchart of a method for real-time calculation, scheduling and optimization of power data according to a first embodiment of the present invention;
fig. 2 is a flowchart of a scheduling policy of a load state feedback model according to a first embodiment of the present invention;
FIG. 3 is a flowchart of a Bayesian network model construction according to a first embodiment of the present invention;
FIG. 4 is a diagram of a Bayesian network according to a first embodiment of the present invention;
fig. 5 is a line loss calculation example according to the first embodiment of the present invention;
FIG. 6 is an example of an anti-theft analysis according to one embodiment of the present invention;
FIG. 7 is an example of intelligent diagnosis of power outage according to a first embodiment of the present invention;
FIG. 8 is a schematic diagram of a real-time calculation scheduling optimization system for power data according to the second embodiment of the present invention;
FIG. 9 is a graph comparing the efficiency of the algorithm according to the first embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
It is noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of the present invention. It should be noted that each block in the flowchart or block diagrams may represent a module, a segment, or a portion of code, which may comprise one or more executable instructions for implementing the logical function specified in the respective embodiment. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Example one
As shown in fig. 1, the present embodiment provides a method for optimizing real-time calculation scheduling of power data, and the present embodiment is illustrated by applying the method to a server, it is to be understood that the method may also be applied to a terminal, and may also be applied to a system including a terminal and a server, and is implemented by interaction between the terminal and the server. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network server, cloud communication, middleware service, a domain name service, a security service CDN, a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein. In this embodiment, the method includes the steps of:
acquiring a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled;
introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm;
performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result;
and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
The embodiment may feed back the second scheduling result to the corresponding terminal device.
Specifically, the technical solution of this embodiment can be implemented according to the following:
referring to fig. 1, 2, 3, 4, the method includes:
A. constructing a generic Directed Acyclic Graph (DAG) model based on tasks:
the DAG model task is denoted G ═ (V, E). G represents a task, and the set V is { V ═ V1,v2，…，vnWhere each element represents a subtask, | V | represents the number of subtasks in a DAG task at all times. D ═ D1，d2，…,dnRepresents the set of devices in the task to be processed, and | D | is the number of devices in the system.
B. HEFT algorithm improved by introducing task execution delay parameter
In the HEFT scheduling algorithm, the task with the largest rank value is scheduled preferentially, and the calculation formula is as follows:
in the above formula, succ (v)i) Is v isiAll successor tasks of, WiFor task viThe average execution time on different devices, c is the communication cost between tasks. When v isiWhen it is an egress node, ranki＝Wi. Order to
So as to obtain the compound with the characteristics of,
where d represents the number of devices available in the cloud computing environment. r (d)j，vi) Representing a task viAt the device djTime cost of execution. When the device djProcessing task viWhen a delay occurs, there is time (v)i)>r(dj，vi). At this time, we introduce the current task execution delay parameter Occupanacy (O) of the device, some
time(vi)＝(1+O)r(dj，vi)，O≥0 (4)
Task v capable of controlling value of time delay parameteriRank size of, representing device djThe task execution load state. Therefore, we now have
When the time delay parameter of the current task execution of the device is calculated, the embodiment uses the sum of the execution time of the high-priority task of the device where the task is located and the time increment of the task after the current task, thereby constructing the following formula.
Wherein f is the number of tasks with higher task execution priority than the current task, tiIs the execution time of the ith high-priority task, tau is the execution time of the current insertion task, and T is the total time of the tasks.
Substituting the formula (6) into the formula (5) to obtain,
the algorithm of the invention uses the concept of the current task execution time delay parameter of the equipment to minimize the occupied time of the task of the equipment, and by using the algorithm, the equipment can distribute the tasks according to a certain arrival sequence, and the specific scheduling strategy is as follows:
step (11): and calculating the current task execution delay parameter of the equipment. And calculating the current task execution time delay parameter of the equipment when the time occupancy rate of the current task is tau.
Step (12): and calculating the rank value of the task slice. And calculating rank values according to the formula, performing descending arrangement on the rank values, and outputting an arrangement result queue Q.
Step (13): the earliest finished equipment is selected. The first task is selected as the equipment that will make its earliest completion and the task is removed.
Step (14): and (5) repeating the step (13). Repeating step (13) until all tasks in Q are removed.
C. And learning the scheduling result of the improved HEFT algorithm by using a Bayesian network model.
The invention designs an interconnection perception dynamic load balancing algorithm based on a Bayesian network, and the algorithm learns the scheduling result of the improved HEFT algorithm by using a Bayesian network model.
The method for constructing the Bayesian network model comprises three steps of determining model variable nodes, determining a network structure and defining a probability table to calculate posterior probability, and is specifically shown in a flow chart 3.
(1) Firstly, designing a model;
(2) determining initial variables and preparing data;
(3) judging whether the data is reasonable, if so, entering (4); otherwise, return to (2)
(4) Constructing a Bayesian network model;
(5) obtaining prior distribution;
(6) constructing conditional probability distribution;
(7) judging whether the searching is finished or not, if so, entering (8); otherwise, adjusting the Bayesian network model in the step (4) according to expert knowledge/machine learning/formula definition;
(8) a posterior probability is generated.
The invention is based on the DAG structure of the Bayesian network, can clearly describe the relevance and the sequence scheduling order among the tasks, and the scheduling result of the previous task directly influences the scheduling of the subsequent task in the process of realizing the load balancing, and the network structure is shown in FIG. 4. The specific algorithm process is as follows:
step (21): and carrying out load balancing scheduling on DAG tasks with the number of the randomly generated tasks being N by utilizing an improved HEFT load state feedback model, and recording a scheduling result.
Step (22): and selecting the processing time, the sequencing and the time delay parameters of the subtasks on each service device as the characteristic values of the training set, carrying out label classification according to the scheduling result, and constructing a data set D.
Step (23): and processing the data set D by using Bayesian decision to obtain the probability that each subtask is distributed to different service devices.
Step (24): and (5) adding the probability obtained in the step (23) as a new characteristic value into the original data set D to form a data set D'.
Step (25): in D' a priori probabilities of task assignments are calculated.
Step (26): in the training set D', sub-tasks are randomly acquired, and the posterior probability of the task to be dispatched to different service devices is calculated.
Step (27): and predicting a scheduling result by using the Bayesian network model.
Step (28): and (5) repeating the step (26) and the step (27) until all subtasks are predicted to be completed.
Step (29): and after the scheduling is finished, outputting a scheduling result.
In order to test the practicability and the efficiency of the real-time calculation scheduling optimization method of the power data based on the improved HEFT algorithm and the Bayesian network, three real-time calculation tasks of line loss calculation, anti-electricity-stealing analysis and intelligent power failure study and judgment are adopted as templates in the experiment to perform simulation calculation, as shown in FIGS. 5, 6 and 7.
In the testing stage, the operating system of the computing equipment used for the experiment is Windows10, the CPU is i52.4GHz, the memory is 16GB, Python3.8 is used as a development language, Pycharm2020 is used as a program development IDE, and the testing examples and DAG tasks in the experiment are all generated randomly by python programs according to rules. To build the comprehensive data experiment, 1500 tasks were generated per task template for a total of 4500 tasks. And testing each template for three times, wherein tasks of three templates are mixed and executed each time. Each template executes 200 tasks for the first time, and the total number of the tasks is 600; the second time, each template executes 500 tasks, for a total of 1500 tasks; the third time 1500 tasks were performed per template for a total of 4500 tasks. And recording the execution time of each sub task and the overall execution time of the task during each execution. The task execution results are shown in tables 1, 2, and 3:
TABLE 1 average execution time table for line loss calculation template task
TABLE 2 average executive time table for anti-stealing electricity analysis task
TABLE 3 Intelligent average execution time table for power failure study and judgment
In the invention, three template tasks shown in fig. 5, fig. 6, fig. 7, and tables 1, 2, and 3 are selected as the tasks to be scheduled for load balancing scheduling in the experiment. According to the increase of the number of the tasks, in a certain range, the learning effect of the feedback model of the load state of the improved HEFT algorithm can be improved by the interconnection perception load balancing algorithm based on the Bayesian network along with the increase of the number of the tasks; meanwhile, when the number of tasks exceeds a certain numerical value, the simulation effect is not improved any more.
The experiment of the invention utilizes 4000 tasks to carry out load balancing scheduling, and the rest 500 tasks carry out the test of the algorithm execution effect. In the improved HEFT algorithm load state feedback model, T is 20, and comparison of the load balancing scheduling completion time of the algorithm and other popular algorithms is calculated. The experimental result is shown in fig. 9 by taking the interconnection perception dynamic load balancing algorithm based on the bayesian network as a reference and comparing the completion time with popular algorithms such as HEFT, HHDS, PCH and the like.
Experimental results show that the completion time of the interconnection perception load balancing algorithm based on the Bayesian network provided by the embodiment is obviously superior to popular HEFT, HHDS and PCH algorithms.
Example two
The embodiment provides a power data real-time calculation scheduling optimization system.
A power data real-time computation scheduling optimization system, comprising:
a data acquisition module configured to: acquiring a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled;
a load state feedback module configured to: introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm;
a load balancing module configured to: performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result;
an output module configured to: and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
Specifically, the technical solution of this embodiment can be implemented according to the following:
referring to fig. 8, the system includes: the data input module is used for acquiring load data and carrying out data preprocessing; the load state feedback module is used for calculating a task execution delay parameter and distributing tasks according to the rank value; the load balancing module is used for learning the scheduling result of the improved HEFT algorithm by using an interconnection perception dynamic load balancing algorithm based on the Bayesian network and outputting the scheduling result; and the data storage module stores and verifies the data information of each stage.
The data input module directly determines the learning effect of the Bayesian model on the improved HEFT scheduling algorithm provided by the invention through the selection and the quantity of the characteristics of the data set, so that the scheduling result is influenced, and when the attributes of the data set are selected, parameters which have important influence on the scheduling result are selected as far as possible. The algorithm has the advantages that the state of the equipment can be dynamically acquired, the rank value is further influenced, and the equipment competition of the task piece is reduced. The structure of the input data set designed by the invention is composed of the processing time of the subtask on each device, rank, rank', the delay parameter Occupanacy (O) of the current task execution of the device, the probability of the subtask being distributed to each device, and the classified label is the serial number of the device.
The load state feedback module comprises a general DAG model constructed based on tasks and task allocation by utilizing the load state feedback model based on the improved HEFT algorithm, the module uses the current task execution time delay parameter concept of the equipment to minimize the occupied time of the equipment tasks, and the equipment can allocate the tasks according to a certain arrival sequence.
Wherein, load balancing module includes: on the basis of a load state feedback model based on an improved HEFT algorithm, an interconnection perception dynamic load balancing algorithm based on a Bayesian network is designed, the scheduling result of the improved HEFT algorithm is learned, the module can clearly describe the relevance and the sequence scheduling sequence among tasks, the actual situation of interconnection perception dynamic load balancing scheduling is combined through cloud computing, construction and load balancing scheduling of the Bayesian network are carried out based on the dependency relationship, and the scheduling result is output.
Wherein, the data storage module includes: the data information of each stage is stored and verified, and the module can store a complete data processing process by using a data backtracking technology, so that the reliability of a scheduling result is ensured.
It should be noted here that the data acquisition module, the load state feedback module, the load balancing module, and the output module are the same as the example and the application scenario realized by the steps in the first embodiment, but are not limited to the disclosure of the first embodiment. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the real-time calculation scheduling optimization method for power data as described in the first embodiment above.
Example four
The embodiment provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in the real-time calculation scheduling optimization method for power data according to the first embodiment.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A real-time calculation scheduling optimization method for power data is characterized by comprising the following steps:
acquiring a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled;
introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm;
performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result;
and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
2. The power data real-time computation scheduling optimization method according to claim 1, wherein the introducing task execution delay parameter improvement scheduling algorithm comprises: in the HEFT scheduling algorithm, the task with the largest rank value is scheduled preferentially, and the HEFT algorithm is improved by introducing a task execution delay parameter.
3. The power data real-time calculation scheduling optimization method according to claim 1, wherein the task execution delay parameter (the execution time of the high-priority task of the device where the task is located and the sum of the time increment of the task after the current task)/the total time of the task.
4. The power data real-time computation scheduling optimization method of claim 1, wherein the scheduling strategy of the improved scheduling algorithm comprises:
step (11): calculating the current task execution time delay parameter of the equipment when the time occupancy rate of the current task is tau;
step (12) calculating the rank value of the task, performing descending order arrangement on the rank value, and outputting an arrangement result queue Q;
step (13) selecting the equipment which can make the first task finished earliest for the first task, and removing the task;
step (14) repeats step (13) until all tasks in Q have been removed.
5. The power data real-time computing scheduling optimization method according to claim 1, wherein the load balancing scheduling is performed on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result; learning the first scheduling result by adopting a Bayesian network model, and obtaining a second scheduling result comprises the following steps:
step (21): carrying out load balancing scheduling on the directed acyclic graph tasks with the randomly generated number of tasks being N by using a load state feedback model, and recording a first scheduling result;
step (22): selecting processing time, sequencing and time delay parameters of the subtasks on each service device as characteristic values of a training set, carrying out label classification by using a first scheduling result, and constructing a data set D;
step (23): processing the data set D by using Bayesian decision to obtain the probability that each subtask is distributed to different service equipment;
step (24): adding the probability obtained in the step (23) as a new characteristic value into the original data set D to form a data set D';
step (25): calculating a prior probability of task assignment in D';
step (26): in the training set D', randomly acquiring subtasks, and calculating the posterior probability of the tasks to be dispatched to different service equipment;
step (27): predicting a first scheduling result using a bayesian network model;
step (28): repeating steps (26) and (27) until all subtasks are predicted to be completed;
step (29): and after the scheduling is finished, outputting a second scheduling result.
6. The method according to claim 1, wherein the rule for performing load balancing scheduling on the directed acyclic graph by using the load state feedback model comprises: and (3) using the current task execution time delay parameter concept of the equipment to minimize the occupied time of the task of the equipment, and distributing the tasks by the equipment according to a preset arrival sequence.
7. A power data real-time computation scheduling optimization system, comprising:
a data acquisition module configured to: acquiring a task to be scheduled, and constructing a directed acyclic graph based on the task to be scheduled;
a load state feedback module configured to: introducing a task execution delay parameter to improve a scheduling algorithm; obtaining a load state feedback model based on an improved scheduling algorithm;
a load balancing module configured to: performing load balancing scheduling on the directed acyclic graph by using the load state feedback model to obtain a first scheduling result;
an output module configured to: and learning the first scheduling result by adopting a Bayesian network model to obtain a second scheduling result.
8. The power data real-time computation scheduling optimization system of claim 1, wherein the system comprises: a data storage module configured to: and storing and verifying the data information of each stage.
9. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the method for real-time calculation schedule optimization of power data according to any one of claims 1-6.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps in the power data real-time computation scheduling optimization method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111161277.9A CN114327925A (en) | 2021-09-30 | 2021-09-30 | Power data real-time calculation scheduling optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111161277.9A CN114327925A (en) | 2021-09-30 | 2021-09-30 | Power data real-time calculation scheduling optimization method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114327925A true CN114327925A (en) | 2022-04-12 |
Family
ID=81045049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111161277.9A Pending CN114327925A (en) | 2021-09-30 | 2021-09-30 | Power data real-time calculation scheduling optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114327925A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114756358A (en) * | 2022-06-15 | 2022-07-15 | 苏州浪潮智能科技有限公司 | DAG task scheduling method, device, equipment and storage medium |
-
2021
- 2021-09-30 CN CN202111161277.9A patent/CN114327925A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114756358A (en) * | 2022-06-15 | 2022-07-15 | 苏州浪潮智能科技有限公司 | DAG task scheduling method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200257968A1 (en) | Self-learning scheduler for application orchestration on shared compute cluster | |
| CN109992404A (en) | PC cluster resource regulating method, device, equipment and medium | |
| CN114756358B (en) | DAG task scheduling method, device, equipment and storage medium | |
| CN113064879A (en) | Database parameter adjusting method and device and computer readable storage medium | |
| CN103092683A (en) | Scheduling used for analyzing data and based on elicitation method | |
| Nguyen et al. | Scaling upf instances in 5g/6g core with deep reinforcement learning | |
| CN117271101B (en) | Operator fusion method and device, electronic equipment and storage medium | |
| Razavi et al. | FA2: Fast, accurate autoscaling for serving deep learning inference with SLA guarantees | |
| CN114675975B (en) | Job scheduling method, device and equipment based on reinforcement learning | |
| CN118396294A (en) | Cloud manufacturing scheduling method based on quantum multi-agent reinforcement learning | |
| CN112632615B (en) | Scientific workflow data layout method based on hybrid cloud environment | |
| CN114327925A (en) | Power data real-time calculation scheduling optimization method and system | |
| Jang et al. | Fast quality driven selection of composite Web services | |
| Malz et al. | Agent-based test case prioritization | |
| CN117687774A (en) | Task model training method for computing power scheduling and computing power scheduling method and system | |
| Vahidipour et al. | Priority assignment in queuing systems with unknown characteristics using learning automata and adaptive stochastic Petri nets | |
| JP7009971B2 (en) | Process scheduling device and process scheduling method | |
| CN106874215B (en) | Serialized storage optimization method based on Spark operator | |
| Prado et al. | On providing quality of service in grid computing through multi-objective swarm-based knowledge acquisition in fuzzy schedulers | |
| Jafarpour et al. | QoS-aware Selection of Web Service Compositions using Harmony Search Algorithm. | |
| Addis et al. | Function placement and acceleration for in-network federated learning services | |
| Liu et al. | Scheduling tasks with Markov-chain based constraints | |
| Lin et al. | Dynamic load balancing in cloud-based multimedia system using genetic algorithm | |
| CN118245234B (en) | Distributed load balancing method and system based on cloud computing | |
| CN116089021B (en) | Deep learning-oriented large-scale load mixed part scheduling method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |