Summary of the invention
The characteristics of not considering high performance software operation characteristic and isomeric group that exist at cluster job scheduling method in the prior art cause the efficient reduction of high performance software, the problem that the isomeric group service efficiency reduces, the object of the present invention is to provide a kind of group operation self-adapting dispatching method and system based on application characteristic, with in addressing the above problem one of at least.
For achieving the above object, according to an aspect of the present invention, a kind of isomeric group operation self-adapting dispatching method based on application characteristic is provided, may further comprise the steps: operation dispatching server is collected the hardware information of each node of isomeric group, and each node is divided into the groups of nodes with different characteristic according to the hardware information of each node; Whether store the operation characteristic of the operation of current submission in the job scheduling middleware Query Database; If have, operation dispatching server goes the groups of nodes that the operation of current submission is submitted to individual features to operation according to operation characteristic; If do not have, operation dispatching server is submitted to the arbitrary node group with the operation of current submission and goes operation, the job scheduling middleware write down current submission operation service data and according to the Operational Data Analysis operation characteristic, the job scheduling middleware stores the application type and the operation characteristic of the operation of current submission in the database into.
According to the present invention, after the groups of nodes that operation dispatching server is submitted to the operation of current submission according to operation characteristic individual features is gone operation, the job scheduling middleware write down current submission operation service data and the history data of the operation of service data and current submission added up after reanalyse operation characteristic, the operation characteristic of the operation of the current submission of more storing in the new database then.
According to the present invention, service data comprises the node number of internal memory use amount, process number, use, each node memory, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and the CPU holding time of use.
According to the present invention, operation characteristic comprises memory-intensive type and CPU intensity.
According to the present invention, the memory-intensive type be meant internal memory use amount and use each node memory sum ratio greater than 90% and the ratio of internal memory use amount and process number greater than 1G; The CPU intensity is meant that the ratio of CPU holding time and working time is greater than 90%.
According to the present invention, the groups of nodes with different characteristic comprises groups of nodes and the many groups of nodes of CPU that internal memory is big; If the operation characteristic of the operation of current submission is the memory-intensive type, operation dispatching server just is submitted to the operation of current submission the big groups of nodes of internal memory and goes operation; If the operation characteristic of the operation of current submission is the CPU intensity, operation dispatching server just is submitted to the operation of current submission the many groups of nodes of CPU and goes operation.
Correspondingly, the invention provides a kind of isomeric group operation self-adapting dispatching system based on application characteristic, comprise: grouping module, be used to collect the hardware information of each node of isomeric group, and each node be divided into groups of nodes with different characteristic according to the hardware information of each node; Enquiry module is used for the operation characteristic whether Query Database stores the operation of current submission; Submit module to, be used for according to the groups of nodes that operation characteristic is submitted to individual features with the operation of the current submission operation of getting on; Logging modle is used to write down the service data of the operation of current submission; Analysis module is used to analyze service data and draws operation characteristic, and the operation characteristic of the operation of current submission in the new database more.
According to the present invention, service data comprises the node number of internal memory use amount, process number, use, each node memory, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and the CPU holding time of use.
According to the present invention, operation characteristic comprises memory-intensive type and CPU intensity, the memory-intensive type be meant internal memory use amount and use each node memory sum ratio greater than 90% and the ratio of internal memory use amount and process number greater than 1G, the CPU intensity is meant that the ratio of CPU holding time and working time is greater than 90%.
According to the present invention, analysis module is to reanalyse operation characteristic after the history data with the operation of the service data of the operation of current submission and current submission adds up.
By above-mentioned at least one technical scheme of the present invention, be divided into the groups of nodes with different characteristic by each node with isomeric group according to hardware information, dispatch server goes the groups of nodes that the operation of current submission is submitted to individual features to operation according to the operation characteristic of the operation of current submission.The characteristics of operation and isomeric group self have been taken into full account like this, move by the groups of nodes that is fit to this operation of operation, improved the efficient of job run on the one hand, for the high performance software that comprises many operations, improve the efficient of high performance software, also improved the service efficiency of isomeric group simultaneously.And the each run data of job scheduling middleware record operation, and after this operation each run finishes, the service data of this operation and the history data of this operation are added up, reanalyse the operation characteristic of this operation, thereby constantly proofread and correct, improve the accuracy of operation and operation groups of nodes coupling.
Embodiment
As shown in Figure 1, isomeric group operation self-adapting dispatching method based on application characteristic of the present invention may further comprise the steps: S110, operation dispatching server is collected the hardware information of each node of isomeric group, and each node is divided into the groups of nodes with different characteristic according to the hardware information of each node; Whether S120 stores the operation characteristic of the operation of current submission in the job scheduling middleware Query Database; S130, if having, operation dispatching server goes the groups of nodes that the operation of current submission is submitted to individual features to operation according to operation characteristic; If do not have, operation dispatching server is submitted to the arbitrary node group with the operation of current submission and goes operation, the job scheduling middleware write down current submission operation service data and according to the Operational Data Analysis operation characteristic, the application type and the operation characteristic of the operation of current submission stored in the database.
In step S110, the hardware information of each node of the isomeric group of collection comprises the information of the internal memory of the information (as dominant frequency, the model of CPU) of the CPU of each node and each node at least.Operation dispatching server is divided into groups each node according to the hardware information of each node, the node that hardware information is identical or characteristics are identical is classified as one group, for example that CPU is many nodes are divided into one group, the node that internal memory is big is divided into one group, thereby each node of isomeric group is divided into the groups of nodes with different hardware feature.
In step S120, the inquiry of job scheduling middleware is specifically designed to the database of storage operation application type and operation characteristic, see the application type and the operation characteristic of the operation that whether stores current submission, generally speaking, the job run characteristics that application type is identical are also identical.
In step S130, if store the operation characteristic of the operation of current submission in the database, the job scheduling middleware is this operation characteristic notice operation dispatching server, and operation dispatching server goes the groups of nodes that the operation of current submission is submitted to individual features to operation according to this operation characteristic; If do not store the operation characteristic of the operation of current submission in the database, operation dispatching server will be submitted to groups of nodes with the operation of current submission according to the scheduling strategy of acquiescence and go operation, the job scheduling middleware write down current submission operation service data and according to the Operational Data Analysis operation characteristic, application type and the operation characteristic with the operation of current submission stores in the database then.Wherein, service data comprises the node number of internal memory use amount, process number, use, each node memory, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and the CPU holding time of use.Each operation is after having moved, and the job scheduling middleware all can be noted service data.After adding up the history data of the operation of the service data of the operation of current submission and current submission, the job scheduling middleware reanalyses the operation characteristic of the operation of current submission, then the operation characteristic of this operation of more storing in the new database.That is to say, the every operation of this operation once, service data of job scheduling middleware record, and this service data and this operation all previous service data in the past added up reanalyses the operation characteristic of this operation then.Only can prevent the deviation that may occur according to the operation characteristic that service data draws like this, this is a process of constantly proofreading and correct.After this operation repeatedly moves, the job scheduling middleware will be more accurate to the analysis of the operation characteristic of this operation.Operation characteristic comprises memory-intensive type and CPU intensity.Wherein, the memory-intensive type be meant internal memory use amount and use each node memory sum ratio greater than 90% and the ratio of internal memory use amount and process number greater than 1G; The CPU intensity is meant that the ratio of CPU holding time and working time is greater than 90%.The internal memory use amount is meant the internal memory sum that all processes of the type operation are used.
Have a plurality of operations in the high performance software operational process and need be submitted to the node operation of getting on, move the operational efficiency that can improve high performance software by the groups of nodes of selecting optimum operation.In actual applications, high performance software also can be submitted to server end by web interface (socket).As shown in Figure 2, isomeric group comprises server end and groups of nodes, the management node of isomeric group is as the operation dispatching server of isomeric group, it also is the server end shown in Fig. 2, web server of server end operation, client can be connected to the web server by the web browser, carry out alternately with the web server, each groups of nodes that links to each other with server end all has certain feature, on client, move high performance software, the operation that client produces is dispatched by server end, and server end is according to the feature of operation, and the groups of nodes that submits the job to individual features is gone operation.
Correspondingly, the invention provides a kind of isomeric group operation self-adapting dispatching system based on application characteristic.As shown in Figure 3, the isomeric group operation self-adapting dispatching system based on application characteristic of present embodiment, comprise: grouping module 210, be used to collect the hardware information of each node of isomeric group, and each node be divided into groups of nodes with different characteristic according to the hardware information of each node; Enquiry module 220 is used for the operation characteristic whether Query Database stores the operation of current submission; Submit module 230 to, be used for according to the groups of nodes that operation characteristic is submitted to individual features with the operation of the current submission operation of getting on; Logging modle 240 is used to write down the service data of the operation of current submission; Analysis module 250 is used to analyze service data and draws operation characteristic, and the operation characteristic of the operation of current submission in the new database more.
Grouping module 210 is at first collected the hardware information of each node of isomeric group, comprises the information of the internal memory of the information (as dominant frequency, the model of CPU) of the CPU of each node and each node; Hardware information according to each node divides into groups each node then, the node that hardware information is identical or characteristics are identical is classified as one group, for example that CPU is many nodes are divided into one group, the node that internal memory is big is divided into one group, thereby makes each node of isomeric group form the groups of nodes with different characteristic.
Enquiry module 220, inquiry is specifically designed to the database of storage operation application type and operation characteristic, sees the application type and the operation characteristic of the operation that whether stores current submission.
Submit module 230 to, be used for the operation of getting on of groups of nodes that the operation characteristic of the operation of the current submission that inquires according to enquiry module is submitted to individual features with the operation of current submission,
Logging modle 240 writes down the service data of each operation, if therefore the operation of current submission moves follow-up submission the to once more, logging modle 240 can be noted twice service data of this operation.
Analysis module 250, the service data that is used for analytic record module 240 records, each operation submits to operation to finish, logging modle 240 is noted after the service data, analysis module 250 all can be analyzed the operation characteristic of this operation, when analyzing, moved repeatedly operation for submission, analysis module 250 is that all previous service data before up-to-date service data and this operation is analyzed after adding up, and the operation characteristic of this operation that latest analysis is drawn replaces the operation characteristic of the original storage of this operation in the database at last.Like this, the operation characteristic of this operation of storing in the database is dynamic change, and the result is tending towards more accurately.Therefore, even there is not the operation characteristic of the operation of current submission in enquiry module 220 Query Databases, certain groups of nodes that the submitted module of this operation 230 acquiescences are submitted to isomeric group operation of getting on, logging modle 240 is noted after the service data of this operation, analysis module 250 just can be analyzed the operation characteristic that draws this operation, and the application type and the operation characteristic of this operation stored in the database, when the operation of current submission needs to submit to after current operation once more, enquiry module 220 just can inquire the operation characteristic of this operation, if operation is repeatedly submitted in this operation to, logging modle 240 is noted repeatedly service data, and analysis module 250 is analyzed the operation characteristic that draws in view of the above will be tending towards accurate.
Wherein, the service data of logging modle 240 records comprises the node number of internal memory use amount, process number, use, each node memory, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and the CPU holding time of use.The operation characteristic of storing in the database comprises memory-intensive type and CPU intensity.The memory-intensive type be meant internal memory use amount and use each node memory sum ratio greater than 90% and the ratio of internal memory use amount and process number greater than 1G; The CPU intensity is meant that the ratio of CPU holding time and working time is greater than 90%.The internal memory use amount is meant the internal memory sum that all processes of the type operation are used.
By above-mentioned at least one technical scheme of the present invention, be divided into the groups of nodes with different characteristic by each node with isomeric group according to hardware information, dispatch server goes the groups of nodes that the operation of current submission is submitted to individual features to operation according to the operation characteristic of the operation of current submission.The characteristics of operation and isomeric group self have been taken into full account like this, move by the groups of nodes that is fit to this operation of operation, improved the efficient of job run on the one hand, for the high performance software that comprises many operations, improve the efficient of high performance software, also improved the service efficiency of isomeric group simultaneously.And the each run data of job scheduling middleware record operation, and after this operation each run finishes, the service data of this operation and the history data of this operation are added up, reanalyse the operation characteristic of this operation, thereby constantly proofread and correct, improve the accuracy of operation and operation groups of nodes coupling.
The above is the preferred embodiments of the present invention only, is not limited to the present invention, and for a person skilled in the art, the present invention can have various changes and variation.Within the spirit and principles in the present invention all, any modification of being done, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.