CN103049330A - Method and system for scheduling trusteeship distribution task - Google Patents
Method and system for scheduling trusteeship distribution task Download PDFInfo
- Publication number
- CN103049330A CN103049330A CN2012105173898A CN201210517389A CN103049330A CN 103049330 A CN103049330 A CN 103049330A CN 2012105173898 A CN2012105173898 A CN 2012105173898A CN 201210517389 A CN201210517389 A CN 201210517389A CN 103049330 A CN103049330 A CN 103049330A
- Authority
- CN
- China
- Prior art keywords
- work
- task
- workstation
- computing
- finished
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method and system for scheduling a trusteeship distribution task. The method includes the following steps: S100, decomposing a certain decomposable task into a plurality of jobs finished by working stations; S200, distributing the plurality of jobs to a plurality of working stations involved in task operation randomly according to the quantity, and processing the jobs through the working stations; S300, in a process of job processing in the working stations, pre-estimating calculation capacity of jobs to be processed through an artificial neural network, and distributing jobs with large calculation quantity to the working stations with good calculating capacity preferentially; S400, when any one of the working stations involved in the operation finishes a job, conducting result processing on the job; and S500, when the plurality of working stations finish all jobs, collecting all job results, and generating a task result.
Description
Technical field
The present invention relates to task scheduling and tactful technical field, relate in particular to a kind of trustship type, unattended distributed task dispatching system and method relates to Patent classificating number G06 and calculates; Calculate; Counting G06F electricity Digital data processing G06F9/00 presetting apparatus, for example, the program that controller G06F9/06 application deposits in, namely using the storage inside for the treatment of facility comes reception program and keeps the G06F9/46 multiprogramming device G06F9/50 resource of program to distribute, for example, CPU (central processing unit).
Background technology
Computer applied algorithm is actually by computing machine and carries out a series of work, such as copying a file, start a process, closing a window etc.Along with computer applied algorithm becomes increasingly complex, its needed calculated amount is also more and more surprising, finishes if so complicated program is transferred to a computing machine, needs to consume a large amount of time, loses more than gain.Relatively current solution is, use distributed task scheduling system, the work that the subtask that a complexity and huge computer applied algorithm are resolved into a plurality of suitable sizes can be finished by a computing machine/workstation in other words within the suitable time.Then, dispatching system uses many workstation collaborative works to solve a huge computer applied algorithm workstation of these work allocations to the some in the network, and efficient has had very large change.
But existing distributed task scheduling distribution/dispatching system is more, lays particular emphasis on the work of how more effectively dividing and shares out the work.And when the workstation processing is transferred in the work of distributing, dispatching system then not too is concerned about problems such as the treatment situation of each workstation and treatment effeciencies, dispatching system can not or seldom can be carried out management and running to the work of just processing at each workstation, we know the algorithm careful precision how of dividing no matter in advance and sharing out the work, the uneven situation because of work allocation also can occur, and then cause the low of system-computed efficient.
Summary of the invention
The present invention is directed to the proposition of above problem, and the distributed task dispatching method of a kind of trustship type of development has following steps:
S100. a certain decomposable Task-decomposing is become a plurality of work that workstation is finished of transferring to;
S200. a plurality of workstations of described a plurality of work all being given the computing of participation task in advance at random according to quantity are processed;
S300. carry out by artificial neural network the calculated capacity of pending work being estimated in the process of work disposal at described workstation, the work that calculated capacity is large, priority allocation is to the strong workstation of computing power;
S400. after arbitrary workstation in the workstation of described participation computing is finished a described job, result treatment is carried out in this work;
S500. after described a plurality of workstations are finished whole work, gather whole working results, generate task result.
Described step S300 specifically comprises step:
S310. add up the hardware parameter of the workstation of all participation tasks processing, estimate the ability to work of described each workstation;
S320. for completed work, set its calculated capacity=work consuming time * finish the computing power of the workstation of this work;
The parameter of the work of S330. finishing at first in the selecting system and calculated capacity as the sample of artificial neural network, obtain initial neural network relation function;
S340. with the parameter of the work finished successively in the system and calculated capacity as the described artificial neural network of sample substitution, constantly improve the relation function of artificial neural network;
S350. in the artificial neural network that the parameter substitution of work to be allocated is current, estimate the calculated capacity of all work to be allocated; The strong workstation of computing power in the workstation that the participation task processes is distributed in the work that calculated capacity is large.
Among the described step S310, set in a plurality of workstations of participation task processing, the computing power of the workstation that computing power is the strongest is 100, and the computing power of all the other each workstations is the number of 0-100.
Described step S400 specifically comprises the steps:
S410. as a result verification: arbitrary workstation is finished the computing of a job in all working station that participates in computing, and the result of this work is comprised at least: whether operation result exists the verification that whether meets the demands with the operation result designated value;
S420. data are prepared: if the result of work, carries out the rename destination file by verification, mobile destination file is to specified path and/or extract data to the operation of the file of specified format;
S430. record the path that the described destination file of preparing through data is deposited;
S440. repeating step S410-430 until computing is finished in last work of this workstation, generates a destination file that comprises the handled all working of this workstation.
Described step S500 specifically comprises:
S510. after a workstation is finished the whole work that distribute, receive the operation result file of this workstation or the store path of destination file;
S520. repeating step S510 until the processing that shares out the work is separately finished at all working station of the work of participation, gathers the destination file at all working station, generates the destination file of task or the address of task result, finishes the computing of described distributed task scheduling.
A kind of distributed task dispatching system of trustship type comprises at least one server and a plurality of workstation:
After described server reception client becomes a plurality of work with the task of client upload, with Task-decomposing, again work allocation is adjusted the distribution of work in real time to a plurality of workstations and in the process that workstation is processed; Workstation receives the work by server-assignment, processes, and after handling the result is back to server, the processing of finishing the work;
Described server has: the whole processing module of task allocation schedule module and task;
Described workstation has: work disposal module and working result pretreatment module;
During work:
Task allocation schedule module: receive the task that the client uploads, Task-decomposing is become a plurality of work that can be finished separately by any one workstation in described a plurality of workstations processing; A plurality of work that decomposition obtains are all given each workstation in advance at random according to number;
After the processing of workstation startup to described work, task allocation schedule module is assessed the computing power of the workstation that the participation task is processed, for each workstation is set a numerical value that represents the ability of its calculating;
Described task scheduling modules is estimated the calculated capacity of work to be allocated according to completed work by artificial neural network;
The work that described task scheduling modules is large with calculated capacity, priority allocation is processed to the strong workstation of computing power;
Described work of being decomposed by task scheduling modules is stored in the server, when described workstation need to be processed work, need work to be processed be handed down to workstation by task scheduling modules;
Each workstation receives the work by described server-assignment, carries out calculation process by the processing module in this workstation, generates destination file; When any one work in a plurality of work that are assigned with that this workstation receives finish obtain destination file after, described working result pretreatment module this destination file is comprised at least whether destination file exists and destination file in the inspection that whether meets the demands of particular value;
Check complete after, the working result pretreatment module is carried out data and is prepared, these data are prepared to comprise at least: the conversion of the rename of destination file, movement and/or file layout;
After all work is finished in this workstation, the whole processing module of the task that the virtual route that described working result pretreatment module is deposited the destination file of all working uploads onto the server;
When whole work that a workstation is finished to distribute, the whole processing module of described task is stored the part virtual route of the described as a result literary composition of finishing the work that this workstation uploads or is downloaded destination file;
When relating to each workstation of processing a complete task and finish all working that it is assigned with, the whole processing module of described task gathers all destination files, generates download path and/or generates the task result file; Described download path and/or task result file are sent to the data transmission port of server, finish the computing of distributed task scheduling.
Described server also has task real-time informing module, and the progress msg that this notification module uses Email and note form that task is processed is at least notified to the user.
To the database of user transparent, this database comprises at least:
The task management tables of data of the current Processing tasks of register system and historic task;
At least record the work management tables of data of mission number corresponding to the startup of the corresponding workstation numbering of numbering, this work, running parameter, duty, work of each work that is become by task division and deadline and this work; And
Record described task real-time informing module and process the real-time informing management data list of progress msg to the task of user/supvr's transmission, task is processed in the engineering, described task real-time informing module is called the information of record in this real-time informing management data list, to user's transmission of appointment.
Also have: accept user instruction, assisting users is finished at least and is comprised: newly-built task, submission task, initiating task and the client of checking latest activity and having finished the work.
Have consistent with described server capability server.
In the work disposal process, the computing power that described task processing module is finished work on hand according to workstation is estimated the calculated capacity of each workstation residue work, according to the remaining displacement volume in different operating station, adjusts in real time the distribution of work.
Description of drawings
Technical scheme for clearer explanation embodiments of the invention or prior art, the below will do one to the accompanying drawing of required use in embodiment or the description of the Prior Art and introduce simply, apparently, accompanying drawing in the following describes only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.
The distributed task dispatching grid topological structure of Fig. 1 trustship type
Fig. 2 is general function structured flowchart of the present invention
Fig. 3 is overall operation flow process of the present invention
Fig. 4 is invention client station list structure figure
Fig. 5 is database structure figure of the present invention
Fig. 6 is task management data list structure figure in the database
Fig. 7 is work management data list structure figure in the database
Real-time informing management data list structure in Fig. 8 database
Fig. 9 is client task tabulation functions of mouse right key menu of the present invention
Embodiment
For the purpose, technical scheme and the advantage that make embodiments of the invention is clearer, below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is known complete description:
A kind of distributed task dispatching method of trustship type mainly comprises the steps:
S100. a certain decomposable distributed task scheduling is resolved into a plurality of work that workstation is finished of transferring to: the calculated amount of this work is huge especially usually, if give single computing machine or workstation carries out the time that computing may continue several months or several years.
Task described in the present invention is formed by a plurality of parameter combinations, and principal character is that can be divided into suitable granularity can be by the fritter of computing machine independent operating; If can not divided task then be not suitable for carrying out Distributed Calculation.Accordingly, a specific parameter combinations that is become by Task-decomposing is exactly work of the present invention.
The used application program of weather forecasting (being forecasting software): WFC (abbreviation of Weather Forecast), (each parameter needs particular value to this program, gets 2 such as wind-force, and wind direction gets 30 as input parameter take above-mentioned four parameters, temperature gets 20, humidity gets 45), output weather forecasting information, output format is as follows: clear to cloudy, 2 grades of component, 30 ° of wind directions, 20 ℃ of temperature, humidity 15%.
For convenience of description, here with wind-force as parameter X 1, wind direction is as parameter X 2, temperature is as parameter X 3, humidity is as parameter X 4; In the practical problems, the span of parameters is continuous (this is realistic physical phenomenon also), but calculate in order to use the computing machine science of carrying out, must carry out problem abstract, the value of parameter is dispersed, here we simultaneously for ease of explaining " task ", " work " these two nouns, we have done following setting according to the request for utilization of WFC software:
The value of parameters is:
X1(wind-force): 1,2,3,4(is totally 4 data points)
The X2(wind direction): 0,30,60,90(is totally 4 data points)
The X3(temperature): 15,20,25(is totally 3 data points)
X4(humidity): 40,45,50,55,60,65(is totally 6 data points)
Like this, four parameters (X1, X2, X3, X4) form 4*4*3*6=288 parameter combinations altogether, and are as shown in the table:
These 288 parameter combinations have just formed " task " among the present invention, and the just corresponding work of each parameter combinations, each work can both be carried out independently computing by the WFC application program, for example, first group of work Job1(corresponding to parameter combinations is X1=1, X2=0, X3=15, X4=40), the WFC application program is carried out computing according to the parameter of Job1, obtains operation result Result1 after the some time.
S200. a plurality of workstations of described a plurality of work all being given the computing of participation task according to quantity are processed: after obtaining described 288 work, described 288 work are all given each workstation according to the number mean random, namely satisfy " work " quantity that any 2 WA are assigned to poor≤1.The quantity of supposing the workstation of participation task computing has 10, and then according to the mean random apportion design, each workstation can be assigned to 28 or 29 work.Simultaneously, the method that adopts mean random to distribute has also guaranteed to avoid causing the situation that is distributed in a workstation in the large working set of calculated amount.
Further, in order to make things convenient for the optimization allotment of task, the work described in the present invention before issuing the workstation processing, all is to be stored in the described server; The distribution of described work can be described as just setting up a pending work sequence for a workstation, and determining has which by the work that this workstation is processed.Only have when a certain workstation and finish a upper job in the sequence, when preparation was processed next task, server just was issued to workstation with the task in the sequence, processes.So just make things convenient for system according to the situation of the real time execution of each workstation, adjust the work sequence of each workstation, and then reach the purpose that optimization shares out the work.
S310. add up the hardware parameter of the workstation of all participation tasks processing, estimate the ability to work of described each workstation: for work being carried out real-time optimization allotment, before the distributed component arithmetic system, at first in the system, or be described as participating in the hardware performance of the workstation of distributed task scheduling computing, assess.
Use in the present invention key hardware (CPU, internal memory, the buffer memory etc.) performance parameter of allocation of computer to weigh the computing power of this computing machine.During practical operation, can utilize operating system to carry or popular hardware assessment software is given a mark to each workstation.
In the present invention, for arbitrary workstation, its computing power is exactly an intrinsic known attribute.In order to make things convenient for computing, in all working station end computing machine of certain task of participation, computing power with that the strongest computing machine of digital 100 calibrated and calculated abilities, for other each workstation end computing machines, its corresponding computing power is (greater than 0 and less than or equal to 100, namely numerical value is larger represents that then the computing power of this computing machine is stronger) between 0 to 100.
S320: in described step S200, by equal first task in the sequence of the pending work of each workstation of obtaining of point-score at random in advance, when the work beginning, at first be issued to each predetermined workstation, process, the work of recording each work is consuming time.Consuming time with this work, be defined as the calculated capacity of this work in conjunction with the computing power of the workstation of finishing this work:
For each work in the same task, under identical computing environment, (adopt same computer to carry out computing), certain work complete needed time of computing shorter (each work is continuously uninterrupted computing), then calculated capacity corresponding to this work is less.
The calculated capacity of work can be subject to the impact of each combination parameter.Be combined into the parameters of a described job, the value of any parameter all can have influence on the calculated capacity of work, and simultaneously, different parameters is not identical to the influence degree of the calculated capacity of same work yet.
Any be operated in do not have computing complete before, we can not obtain the required definite time of this work of computing, namely can not obtain the calculated capacity of this task.But from whole task run management view, we can go to estimate the calculated capacity that is in the work for the treatment of the computing state according to those work of having finished in the task.
Moving the impact that a required time of work mainly is subjected to two aspect factors, is the computing power of the workstation end computing machine of this work of operation on the one hand, is the calculated capacity of this work itself on the other hand.
Therefore, treat the calculated capacity of the work of computing state in order to utilize complete work to estimate, must eliminate first the impact of the computing power of workstation end computing machine.Among the present invention, for the complete work of certain workstation operation, the computing power that * workstation end computing machine consuming time is made in recruitment characterizes the calculated capacity of this work.The computing power of making station end computing machine characterizes the calculated capacity of this work.Namely
Calculated capacity=work is consuming time * computing power
As seen, the calculated capacity with work consuming time of working is directly proportional, and is inversely proportional to the computing power of computing machine.
In order to estimate the calculated capacity of the pending work in each work sequence, the employing artificial neural network algorithm is realized the assessment to the calculated capacity that is in the work for the treatment of the computing state.
The parameter of the work of S330. finishing at first in the selecting system and calculated capacity, as the sample of artificial neural network:
Among the present invention, will be computing finish the combination parameter of (being the first job in the pending work sequence of described each workstation) part work as the input Xt of artificial neural network, calculated capacity (computing power of the * computing machine consuming time of working) corresponding to the part work that computing is finished is as the target output Yt of artificial neural network, the part work of finishing by computing obtains m group sample (Xt, Yt), t=1,2,3,,,,, m.Namely as follows:
Use the BP neural network algorithm above-mentioned relation analyzed, finally obtain the relation function of calculated capacity Y and the parameters that forms work, that is, Y=f (X1, X2, X3,,,, Xi,,,,, Xn).
In order to make described artificial neural network constantly obtain study, use in the present invention step S340. with the parameter of the work finished successively in the system (first job in the separately sequence finished of each workstation) and calculated capacity as the described artificial neural network of sample substitution, artificial neural network is learnt.
Same, continue the example of weather forecast, according to foregoing description, this task comprises 288 groups of parameter combinations altogether, every group of corresponding one " work " of parameter combinations, here we should finish 100 " work " at hypothesis, therefore we have also just formed 100 groups of samples, because the information of each sample (parameter combinations that sample is corresponding, computing is consuming time, the computing power of corresponding computing machine etc.) all known, so we can obtain following information easily:
Utilize these 100 samples and then obtain relation function Y.
Here we suppose the relation function Y=10X1+5X2+8X3+9X4+100 of this task.
Utilize this relation function, we bring the parameter information of (X1=40, X2=90, X3=25, X4=65) (supposing that this work is in the computing state for the treatment of) work into can obtain calculated capacity corresponding to this work in the relation function.
S350. other are in the work for the treatment of the computing state obtains correspondence equally according to this mode calculated capacity, therefore, have formed a calculated capacity tables of data for the work for the treatment of the computing state all in this task.By the work that calculated capacity is large of the task scheduling modules in the server, distribute to the strong workstation of computing power in the workstation that the participation task processes, the real-time allotment to work in the process of finishing the work.
S410. as a result verification: arbitrary workstation is finished the computing of a job in all working station that participates in computing, and the result of this work is comprised at least: whether operation result exists the verification that whether meets the demands with the operation result designated value;
S420. data are prepared: if the result of work, carries out the rename destination file by verification, mobile destination file is to specified path and/or extract data to the operation of the file of specified format;
S430. record the path that the described destination file of preparing through data is deposited;
S440. repeating step S410-430 until computing is finished in last work of this workstation, generates a destination file that comprises the handled all working of this workstation.
S510. after a workstation is finished the whole work that distribute, receive the operation result file of this workstation or the store path of destination file;
S520. repeating step S510 until the processing that shares out the work is separately finished at all working station of the work of participation, gathers the destination file at all working station, generates the destination file of task or the address of task result, finishes the computing of described distributed task scheduling.
Shown in Fig. 1-9:
The distributed task dispatching system of trustship type comprises client, server and workstation.
Client is the remote subscriber interface of server-side application, and being installed in arbitrarily can be by network connection to the computing machine of server.The client assisting users finish such as newly-built task, submission task, initiating task and check latest activities, the function such as finish the work.
The user can arrange the sending short messages in groups device property parameters under the task real-time informing module in the server by client, and Main Function is the port attribute that the sending short messages in groups device is set.
Use the SerialPort control under .NET Framework 4.0 platforms to realize serial communication among the present invention.According to the relevant information of sending short messages in groups device hardware device, its (sending short messages in groups device) serial ports attribute is set.The serial ports attribute mainly contains: port numbers, and baud rate, parity check bit, data bit length, Handshake Protocol, position of rest reads overtimely, writes the parameters such as overtime.
Among the present invention when this sending short messages in groups device serial ports setup of attribute, provide convenient, flexible human oriented design: when the arranging of uncertain serial ports property parameters, client is provided with default setting and comes the auxiliary parameter setting, the automatic search function is provided in the program, can realizes easily the correct setting of parameters.
When parameter after setting completed, can the property verified connection, if each property parameters of sending short messages in groups device all arranges correctly, then point out parameter to arrange correctly, finish the sending short messages in groups setting.If it is wrong that the sending short messages in groups device has at least a property parameters to arrange, then client can eject dialog box, and the prompting parameter arranges wrong, and reexamine and revise the parameter setting this moment, and the property verified connection is until parameter arranges correctly.
The computer name of the corresponding computing machine of all right display server end of client, Computer IP address, and udp broadcast group.
Client also can send scan instruction to server, is mainly used to scan online workstation, uses to cooperate submission, initiating task.
Server sends online workstation tabulation to client after scanning is finished, the workstation acquiescence in the tabulation all is selected state, can share out the work, if the user wants to cancel certain workstation, clicks cancellation and get final product in tabulation.
Same, in the workstation tabulation, the user can select to select flexibly to start, suspends and reclaim corresponding work.The effect that starts the work that suspends in the workstation choose is that the work that suspends in the workstation that will choose starts, and the work that after this this workstation the is corresponding calculating that just brings into operation successively is until move complete or with its time-out.The effect that suspends the work of formation in the workstation of choosing is the work stoppage of formation in the workstation that will choose; The effect of the work that suspends in the workstation that recovery is chosen is that the work that suspends in the workstation that will choose is reclaimed, and makes it be in holding state.
In the present embodiment, client is in order to realize the correlation function of above-mentioned control work, has workstation tabulation functions of mouse right key menu, this menu is comprised of 11 function menus, be respectively [newly-built task], [work that suspends in the task that startup is chosen], [work of formation in the time-out task of choosing], [work of standby in the submission task of choosing], [work that suspends in the task that recovery is chosen], [task is chosen in deletion], [removing completed task], [starting whole tasks], [suspending whole tasks], [attribute], 11 function menus such as [task management settings].
The effect of [newly-built task] is to create new task, can eject newly-built task dialogue frame when clicking [newly-built task], and the user can finish the creation task operation easily;
The Main Function of [work that suspends in the task that startup is chosen] is that the work that is in halted state in will choosing of the task starts, and the after this work of the formation calculating that just brings into operation successively in this task is until move complete or with its time-out;
The effect of [work of formation in the time-out task of choosing] is the work stoppage that is in quene state in will choosing of the task;
The effect of [work of standby in the submission task of choosing] is that each workstation is submitted in the work of standby in will choosing of the task, makes it be in halted state;
The effect of [work that suspends in the task that recovery is chosen] is to be in the work of halted state in will choosing of the task to reclaim, and makes it be in holding state;
The effect of [task that deletion is chosen] is the task deletion that will choose, and the Task tables of data of the task of being about to choose from database D B moves to the Old_Task tables of data;
The effect of [removing completed task] be with move all complete tasks all from database D B the Task tables of data move to the Old_Task tables of data;
The effect of [starting whole tasks] is that all tasks that will be in halted state all start operation calculating;
The effect of [suspending whole tasks] is all to be in task in the formation by quene state displacement halted state;
The effect of [attribute] is the relevant information of checking choosing of task, is comprised of 2 parts.First is ' routine ', and establishment, beginning, the deadline of workstation number that task names, task run state, task enable, work (the work number of operating work number, the work number of not submitting to, time-out, the work number in the formation, complete work number and the number of must working of operation), time cumulation, actual run time, speed-up ratio, formation prediction and task that task comprises arranged.。Second portion is ' parameter ', and 3 parts such as, parameter setting capable by task order, order preview form.
The effect of [task management setting] is the correlation parameter that the task real-time informing is set, and is comprised of 3 parts.First is ' notice arranges ' item, is comprised of notice form, announcement period.Wherein the notice form provides 2 kinds of selections, is respectively SMS notification (needing the support of sending short messages in groups device hardware), mail notification; Announcement period provides 2 kinds of selections, is respectively the time interval, percentage-proportion intervals.The time interval is namely carried out the real-time informing of a Task Progress every the selected time period, percentage-proportion intervals is namely carried out the real-time informing of Task Progress according to selected percentage-proportion intervals.The user can select according to user demand the corresponding selection of notice form, announcement period the inside.
Server-side application is installed on the server computer, opens rear at running background.The main functional modules of server-side application has: central processing module, task scheduling modules, as a result post-processing module.The function of modules is:
Central processing module: the Main Function of this module is by receiving the instruction from client, database DB is carried out corresponding operating, realizing final functions such as creation task, submission, startup, deletion.
Task scheduling modules: the Main Function of this module is (for same work according to the computing power of each workstation end, it is shorter to finish the required time of this work, then the computing power of this workstation end computing machine is stronger), the distribution of working in the real-time optimization task, reach best operational efficiency, namely finish the time optimization of a required by task.
The core methed of realizing task scheduling has following 2 parts, and first takes the predistribution of working in the task, and second portion is to take the Optimized Operation of working in the task.
Work predistribution:
Among the present invention, " work " in task starts before the operation, at first takes " work " in the task randomly orderedly, concentrates on same the probability on the workstation to avoid height " work " consuming time; Then take the equal point-score of going ahead of the rest, " work " that a task comprises all given each workstation end WA according to number, namely satisfy " work " quantity that any 2 WA are assigned to poor≤1.(work predistribution was finished in ' newly-built task ' time, finished before the startup operation of working in task.）
The optimization scheduling:
After " work " in task starts operation, task scheduling modules will according to certain algorithm, dynamically be adjusted work allocation according to the computing power of workstation end computing machine, so that the required minimal time of finishing the work in real time.For each workstation end computing machine, it is shorter to finish the required time of same work, and then the computing power of this workstation end computing machine is stronger.
Key hardware (CPU, internal memory, buffer memory etc.) performance parameter with allocation of computer among the present invention is weighed the computing power of this computing machine.Therefore, in the present invention, for arbitrary computing machine, its computing power is a build-in attribute.In all working station end computing machine of certain task of participation, computing power with that the strongest computing machine of digital 100 calibrated and calculated abilities, computing power for other each workstation end computing machines, its corresponding computing power is (greater than 0 and less than or equal to 100, namely numerical value is larger represents that then the computing power of this computing machine is stronger) between 0 to 100.
In same task, the calculated capacity of a job is subject to the impact of each running parameter, and the present invention weighs the calculated capacity of work with the combination of parameter.Be combined into the parameters of work, the value of any parameter all can have influence on the calculated capacity of work, simultaneously, different parameters on the calculated capacity of same work to affect intensity not identical yet.
Any be operated in do not have computing complete before, we can not obtain the required definite time of this work of computing, namely can not obtain the calculated capacity of this task.But from whole task run management view, we can go to predict the calculated capacity that is in the work for the treatment of the computing state according to those work of having finished in the task.Moving the impact that any required time of work mainly is subjected to two aspect factors, is the computing power of the workstation end computing machine of this work of operation on the one hand, is the calculated capacity of this work itself on the other hand.Therefore, treat the calculated capacity of the work of computing state in order to utilize complete work to predict, must eliminate first the impact of the computing power of workstation end computing machine.Among the present invention, for the complete work of certain workstation operation, the computing power that * workstation end computing machine consuming time is made in recruitment characterizes the calculated capacity of this work.Namely
A mistake! Do not find Reference source.Calculated capacity=work is consuming time * computing power
As seen, the calculated capacity with work consuming time of working is directly proportional, and is inversely proportional to the computing power of computing machine.
The employing artificial neural network algorithm is realized the assessment to the calculated capacity that is in the work for the treatment of the computing state among the present invention.(essence of artificial neural network has embodied a kind of funtcional relationship between the network input and output.By choosing different model structures and activation function, can form various artificial neural network, obtain the relational expression between the different input and output, and reach different purposes of design, finish different functions.）
Post-processing module as a result: the Main Function of this module is that the destination file that " work " that each workstation end is finished produces is processed.
Result treatment is divided into two stages, and the pre-service of workstation end and server end are processed eventually.
The pre-service of workstation end:
1) trigger condition 1: any one " work " operation is complete;
2) pre-service: carry out successively the following step;
A. as a result verification: whether the check result file exists, whether the value of assigned address meets the demands etc. in the destination file; Then the completion status of " work " is set to " finishing " and enters next step by verification, otherwise just the completion status of " work " is set to " failure " and jumps out pre-service;
B. data are prepared: rename destination file, mobile destination file, extract data to the file of specified format (supporting plain text, MS-EXCEL, MS-ACCESS) or what is not done;
3) trigger condition 2: on the end of work at present station last " work " finish pre-service;
4) upload preparation: the virtual route that the pretreated destination file of transmission process is deposited is to server end.Server end is processed eventually:
1) trigger condition 1: any one workstation end is finished and is uploaded preparation;
2) data are downloaded: download the destination file of workstation end or the virtual route of event memory file only;
3) trigger condition 2: the destination file on last workstation end is downloaded the virtual route of complete or destination file and is stored complete;
4) process eventually: if download destination file, then gather and finally process, final destination file is distributed on the data transmission interface of server end; If the storing virtual path then maps to it on data transmission interface of server end; (data transmission interface of server end is supported the multiple transport protocols such as http, ftp)
5) notice: use the various ways notice user tasks such as Email, note to finish.
The user can directly download final destination file by client, also can therefrom extract download link, uses third party's instrument to download.
Database D B is installed on computing machine corresponding to server end, the information that is used for storing all creation tasks.Be illustrated in figure 5 as the structural drawing of database D B, this database is made of 6 tables of data, is respectively Task tables of data, Old_Task tables of data, Job tables of data, Old_Job tables of data, Inform tables of data, Old_Inform tables of data.
The Task tables of data is used for storing the information of institute's creation task, and the Old_Task tables of data is used for the task from the deletion of Task tables of data is backed up.The Job tables of data is used for storing the information of the Job that institute's creation task comprises, and the Old_Job tables of data is used for the task from the deletion of Job tables of data is backed up.Among the present invention, [job_status] field in the Job tables of data among the database D B is used for indicating the running status of job, and following table is depicted as Job state value table.The Inform tables of data is used for the real-time follow-up parameters information of store tasks, and the Old_Inform tables of data is used for the task from the deletion of Inform tables of data is backed up.
The above; only be the better embodiment of the present invention; but protection scope of the present invention is not limited to this; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses; be equal to replacement or change according to technical scheme of the present invention and inventive concept thereof, all should be encompassed within protection scope of the present invention.
Claims (10)
1. the distributed task dispatching method of a trustship type has following steps:
S100. a certain decomposable Task-decomposing is become a plurality of work that workstation is finished of transferring to;
S200. a plurality of workstations of described a plurality of work all being given the computing of participation task in advance at random according to quantity are processed;
S300. carry out by artificial neural network the calculated capacity of pending work being estimated in the process of work disposal at described workstation, the work that calculated capacity is large, priority allocation is to the strong workstation of computing power;
S400. after arbitrary workstation in the workstation of described participation computing is finished a described job, result treatment is carried out in this work;
S500. after described a plurality of workstations are finished whole work, gather whole working results, generate task result.
2. a kind of method for scheduling task according to claim 1, be further characterized in that: described step S300 specifically comprises step:
S310. add up the hardware parameter of the workstation of all participation tasks processing, estimate the ability to work of described each workstation;
S320. for completed work, set its calculated capacity=work consuming time * finish the computing power of the workstation of this work;
The parameter of the work of S330. finishing at first in the selecting system and calculated capacity as the sample of artificial neural network, obtain initial neural network relation function;
S340. with the parameter of the work finished successively in the system and calculated capacity as the described artificial neural network of sample substitution, constantly improve the relation function of artificial neural network;
S350. in the artificial neural network that the parameter substitution of work to be allocated is current, estimate the calculated capacity of all work to be allocated; The strong workstation of computing power in the workstation that the participation task processes is distributed in the work that calculated capacity is large.
3. a kind of distributed task dispatching method according to claim 2, be further characterized in that: among the described step S310, set in a plurality of workstations of participation task processing, the computing power of the workstation that computing power is the strongest is 100, and the computing power of all the other each workstations is the number of 0-100.
4. a kind of method for scheduling task according to claim 1, be further characterized in that: described step S400 specifically comprises the steps:
S410. as a result verification: arbitrary workstation is finished the computing of a job in all working station that participates in computing, and the result of this work is comprised at least: whether operation result exists the verification that whether meets the demands with the operation result designated value;
S420. data are prepared: if the result of work, carries out the rename destination file by verification, mobile destination file is to specified path and/or extract data to the operation of the file of specified format;
S430. record the path that the described destination file of preparing through data is deposited;
S440. repeating step S410-430 until computing is finished in last work of this workstation, generates a destination file that comprises the handled all working of this workstation.
5. a kind of method for scheduling task according to claim 3, be further characterized in that: described step S500 specifically comprises:
S510. after a workstation is finished the whole work that distribute, receive the operation result file of this workstation or the store path of destination file;
S520. repeating step S510 until the processing that shares out the work is separately finished at all working station of the work of participation, gathers the destination file at all working station, generates the destination file of task or the address of task result, finishes the computing of described distributed task scheduling.
6. the distributed task dispatching system of a trustship type comprises at least one server and a plurality of workstation:
After described server reception client becomes a plurality of work with the task of client upload, with Task-decomposing, again work allocation is adjusted the distribution of work in real time to a plurality of workstations and in the process that workstation is processed; Workstation receives the work by server-assignment, processes, and after handling the result is back to server, the processing of finishing the work; It is characterized in that
Described server has: the whole processing module of task allocation schedule module and task;
Described workstation has: work disposal module and working result pretreatment module;
During work:
Task allocation schedule module: receive the task that the client uploads, Task-decomposing is become a plurality of work that can be finished separately by any one workstation in described a plurality of workstations processing; A plurality of work that decomposition obtains are all given each workstation in advance at random according to number;
After the processing of workstation startup to described work, task allocation schedule module is assessed the computing power of the workstation that the participation task is processed, for each workstation is set a numerical value that represents the ability of its calculating;
Described task scheduling modules is estimated the calculated capacity of work to be allocated according to completed work by artificial neural network;
The work that described task scheduling modules is large with calculated capacity, priority allocation is processed to the strong workstation of computing power;
Described work of being decomposed by task scheduling modules is stored in the server, when described workstation need to be processed work, need work to be processed be handed down to workstation by task scheduling modules;
Each workstation receives the work by described server-assignment, carries out calculation process by the processing module in this workstation, generates destination file; When any one work in a plurality of work that are assigned with that this workstation receives finish obtain destination file after, described working result pretreatment module this destination file is comprised at least whether destination file exists and destination file in the inspection that whether meets the demands of particular value;
Check complete after, the working result pretreatment module is carried out data and is prepared, these data are prepared to comprise at least: the conversion of the rename of destination file, movement and/or file layout;
After all work is finished in this workstation, the whole processing module of the task that the virtual route that described working result pretreatment module is deposited the destination file of all working uploads onto the server;
When whole work that a workstation is finished to distribute, the whole processing module of described task is stored the part virtual route of the described as a result literary composition of finishing the work that this workstation uploads or is downloaded destination file;
When relating to each workstation of processing a complete task and finish all working that it is assigned with, the whole processing module of described task gathers all destination files, generates download path and/or generates the task result file; Described download path and/or task result file are sent to the data transmission port of server, finish the computing of distributed task scheduling.
7. the distributed task dispatching system of a kind of trustship type according to claim 6, be further characterized in that: described server also has task real-time informing module, and the progress msg that this notification module uses Email and note form that task is processed is at least notified to the user.
8. the distributed task dispatching system of a kind of trustship type according to claim 7, be further characterized in that described server also has: to the database of user transparent, this database comprises at least:
The task management tables of data of the current Processing tasks of register system and historic task;
At least record the work management tables of data of mission number corresponding to the startup of the corresponding workstation numbering of numbering, this work, running parameter, duty, work of each work that is become by task division and deadline and this work; And
Record described task real-time informing module and process the real-time informing management data list of progress msg to the task of user/supvr's transmission, task is processed in the engineering, described task real-time informing module is called the information of record in this real-time informing management data list, to user's transmission of appointment.
9. the distributed task dispatching system of a kind of trustship type according to claim 6, be further characterized in that also and have: accept user instruction, assisting users is finished at least and is comprised: newly-built task, submission task, initiating task and the client of checking latest activity and having finished the work.
10. the distributed task dispatching system of a kind of trustship type according to claim 6, its feature also with have consistent with server with described server capability.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210517389.8A CN103049330B (en) | 2012-12-05 | 2012-12-05 | A kind of trustship type distributed task dispatching method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210517389.8A CN103049330B (en) | 2012-12-05 | 2012-12-05 | A kind of trustship type distributed task dispatching method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049330A true CN103049330A (en) | 2013-04-17 |
| CN103049330B CN103049330B (en) | 2015-08-12 |
Family
ID=48061980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210517389.8A Active CN103049330B (en) | 2012-12-05 | 2012-12-05 | A kind of trustship type distributed task dispatching method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103049330B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104181873A (en) * | 2013-05-28 | 2014-12-03 | 深圳市菲恩测控科技有限公司 | Method, device and system for achieving digital product detection |
| CN104281449A (en) * | 2014-09-16 | 2015-01-14 | 上海卫星工程研究所 | Managed spacecraft task management system and management method |
| CN105612536A (en) * | 2013-10-09 | 2016-05-25 | 高通股份有限公司 | Method and apparatus to control and monitor neural model execution remotely |
| CN106020966A (en) * | 2015-03-31 | 2016-10-12 | 史迪奇菲克斯公司 | Systems and methods for intelligently distributing tasks received from clients among a plurality of worker resources |
| CN107742183A (en) * | 2017-10-10 | 2018-02-27 | 海尔集团公司 | task scheduling system, method and storage medium |
| CN108710530A (en) * | 2018-02-24 | 2018-10-26 | 深圳市艾龙电子有限公司 | Task distribution formula processing method, device, network-termination device and storage medium |
| CN108897608A (en) * | 2018-05-31 | 2018-11-27 | 中国科学院软件研究所 | A kind of intelligent universal task scheduling system that data-driven is expansible |
| CN109343939A (en) * | 2018-07-31 | 2019-02-15 | 国家电网有限公司 | A kind of distributed type assemblies and parallel computation method for scheduling task |
| CN109976809A (en) * | 2017-12-28 | 2019-07-05 | 北京中科寒武纪科技有限公司 | Dispatching method and relevant apparatus |
| CN110968065A (en) * | 2019-12-18 | 2020-04-07 | 北京云杉世界信息技术有限公司 | Fresh food processing control method and device |
| CN111191796A (en) * | 2019-11-25 | 2020-05-22 | 上海联影智能医疗科技有限公司 | System and method for enhancing distributed medical networks |
| CN112016830A (en) * | 2020-08-27 | 2020-12-01 | 广东电网有限责任公司 | Patent file evaluation task allocation method and device |
| WO2022151071A1 (en) * | 2021-01-13 | 2022-07-21 | Oppo广东移动通信有限公司 | Node determination method and apparatus of distributed task, device, and medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11366457B1 (en) * | 2018-11-16 | 2022-06-21 | On-Time.Ai, Inc. | Controling operation of machine tools using artificial intelligence |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521044A (en) * | 2011-12-30 | 2012-06-27 | 北京拓明科技有限公司 | Distributed task scheduling method and system based on messaging middleware |
-
2012
- 2012-12-05 CN CN201210517389.8A patent/CN103049330B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521044A (en) * | 2011-12-30 | 2012-06-27 | 北京拓明科技有限公司 | Distributed task scheduling method and system based on messaging middleware |
Non-Patent Citations (2)
| Title |
|---|
| 于学军，程博: "基于BP神经网络的工作量估算模型", 《计算机科学》, vol. 39, no. 10, 31 October 2012 (2012-10-31), pages 98 - 2 * |
| 杨文有: "基于MPI的动态负载均衡任务分配方法设计实现", 《淮南师范学院学报》, vol. 12, no. 3, 31 December 2010 (2010-12-31) * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104181873A (en) * | 2013-05-28 | 2014-12-03 | 深圳市菲恩测控科技有限公司 | Method, device and system for achieving digital product detection |
| CN105612536A (en) * | 2013-10-09 | 2016-05-25 | 高通股份有限公司 | Method and apparatus to control and monitor neural model execution remotely |
| CN104281449A (en) * | 2014-09-16 | 2015-01-14 | 上海卫星工程研究所 | Managed spacecraft task management system and management method |
| CN106020966A (en) * | 2015-03-31 | 2016-10-12 | 史迪奇菲克斯公司 | Systems and methods for intelligently distributing tasks received from clients among a plurality of worker resources |
| CN107742183A (en) * | 2017-10-10 | 2018-02-27 | 海尔集团公司 | task scheduling system, method and storage medium |
| CN109976809A (en) * | 2017-12-28 | 2019-07-05 | 北京中科寒武纪科技有限公司 | Dispatching method and relevant apparatus |
| CN108710530A (en) * | 2018-02-24 | 2018-10-26 | 深圳市艾龙电子有限公司 | Task distribution formula processing method, device, network-termination device and storage medium |
| CN108897608A (en) * | 2018-05-31 | 2018-11-27 | 中国科学院软件研究所 | A kind of intelligent universal task scheduling system that data-driven is expansible |
| CN108897608B (en) * | 2018-05-31 | 2021-09-07 | 中国科学院软件研究所 | Data-driven extensible intelligent general task scheduling system |
| CN109343939A (en) * | 2018-07-31 | 2019-02-15 | 国家电网有限公司 | A kind of distributed type assemblies and parallel computation method for scheduling task |
| CN109343939B (en) * | 2018-07-31 | 2022-01-07 | 国家电网有限公司 | Distributed cluster and parallel computing task scheduling method |
| CN111191796A (en) * | 2019-11-25 | 2020-05-22 | 上海联影智能医疗科技有限公司 | System and method for enhancing distributed medical networks |
| CN111191796B (en) * | 2019-11-25 | 2024-03-22 | 上海联影智能医疗科技有限公司 | System and method for enhancing a distributed medical network |
| CN110968065A (en) * | 2019-12-18 | 2020-04-07 | 北京云杉世界信息技术有限公司 | Fresh food processing control method and device |
| CN112016830A (en) * | 2020-08-27 | 2020-12-01 | 广东电网有限责任公司 | Patent file evaluation task allocation method and device |
| WO2022151071A1 (en) * | 2021-01-13 | 2022-07-21 | Oppo广东移动通信有限公司 | Node determination method and apparatus of distributed task, device, and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103049330B (en) | 2015-08-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103049330B (en) | A kind of trustship type distributed task dispatching method and system | |
| CN102063336B (en) | Distributed computing multiple application function asynchronous concurrent scheduling method | |
| CN102254246B (en) | Workflow managing method and system | |
| CN103309946B (en) | Multimedia file processing method, Apparatus and system | |
| CN103617062B (en) | The render farm Dynamic Deployment System of a kind of flexibility and method | |
| CN104407912B (en) | A kind of virtual machine configuration method and device | |
| CN104657220A (en) | Model and method for scheduling for mixed cloud based on deadline and cost constraints | |
| CN102567312A (en) | Machine translation method based on distributive parallel computation framework | |
| CN101685480A (en) | Parallel computing method for security and stability analysis of large power grid and computing platform | |
| CN107992392A (en) | A kind of automatic monitoring repair system and method for cloud rendering system | |
| CN103838627A (en) | Workflow dispatching method based on workflow throughput maximization | |
| CN104731657A (en) | Resource scheduling method and system | |
| Kaliaev | Multiagent approach for building distributed adaptive computing system | |
| CN105808347A (en) | Cluster server deployment calculation method and device | |
| CN108595255B (en) | Workflow task scheduling method based on shortest path algorithm in geographically distributed cloud | |
| CN109118063A (en) | Task based on unit time distribution develops cloud platform | |
| Wang et al. | Task scheduling for MapReduce in heterogeneous networks | |
| CN104796467A (en) | Method for calculating limitation range of QoS (quality of service) optimal combined services with QoS correlations | |
| CN102156659A (en) | Scheduling method and system for job task of file | |
| CN104156505A (en) | Hadoop cluster job scheduling method and device on basis of user behavior analysis | |
| CN116954873B (en) | Heterogeneous computing system, and method, device, equipment and medium for selecting power nodes of heterogeneous computing system | |
| CN109242240A (en) | Task based on unit time distribution and timeliness control develops cloud platform | |
| CN104735134B (en) | A kind of method and apparatus serviced for providing calculating | |
| CN101072226A (en) | Grid computing method and system | |
| CN117331668A (en) | Job scheduling method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |