Summary of the invention
The feature of high performance software operation characteristic and isomeric group of not considering existed for cluster job scheduling method in prior art causes the problem that the efficiency of high performance software reduces, isomeric group service efficiency reduces, the object of the present invention is to provide a kind of group operation self-adapting dispatching method based on application characteristic and system, with in solving the problem one of at least.
For achieving the above object, according to an aspect of the present invention, provide a kind of isomeric group operation self-adapting dispatching method based on application characteristic, comprise the following steps: operation dispatching server collects the hardware information of each node of isomeric group, and each node is divided into the node group with different characteristic according to the hardware information of each node; The operation characteristic of the operation when submit whether is stored in job scheduling middleware Query Database; If had, the Hand up homework when submit goes to run to the node group of individual features according to operation characteristic by operation dispatching server; If do not had, Hand up homework when submit goes to run to arbitrary node group by operation dispatching server, the service data of the operation of submit worked as in job scheduling middleware record and according to Operational Data Analysis operation characteristic, the application type of the operation when submit and operation characteristic are stored in database by job scheduling middleware.
According to the present invention, after the Hand up homework when submit goes to run to the node group of individual features according to operation characteristic by operation dispatching server, job scheduling middleware record when the operation of submit service data and service data is reanalysed operation characteristic with after the history data of the operation of submit is cumulative, the operation characteristic of working as the operation of submit then more stored in new database.
According to the present invention, service data comprises internal memory use amount, process number, the nodes of use, each node memory of use, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and CPU holding time.
According to the present invention, operation characteristic comprises memory-intensive and CPU is intensive.
According to the present invention, memory-intensive refers to that the ratio of each node memory sum of internal memory use amount and use is greater than 90% and the ratio of internal memory use amount and process number is greater than 1G; CPU intensity refers to that the ratio of CPU holding time and working time is greater than 90%.
According to the present invention, the node group with different characteristic comprises the large node group of internal memory and the many node group of CPU; If when the operation characteristic of the operation of submit is memory-intensive, the Hand up homework when submit just goes to run to the node group that internal memory is large by operation dispatching server; If when the operation characteristic of the operation of submit is that CPU is intensive, the Hand up homework when submit just goes to run to the node group that CPU is many by operation dispatching server.
Correspondingly, the invention provides a kind of isomeric group operation self-adapting dispatching system based on application characteristic, comprise: grouping module, for collecting the hardware information of each node of isomeric group, and carry out each node to be divided into the node group with different characteristic according to the hardware information of each node; Whether enquiry module, for storing the operation characteristic of the operation when submit in Query Database; Submit module to, for being gone to run on the node group of individual features by the Hand up homework when submit according to operation characteristic; Logging modle, for recording the service data of the operation when submit; Analysis module, draws operation characteristic for analyzing service data, and more in new database when the operation characteristic of the operation of submit.
According to the present invention, service data comprises internal memory use amount, process number, the nodes of use, each node memory of use, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and CPU holding time.
According to the present invention, operation characteristic comprises memory-intensive and CPU is intensive, memory-intensive refers to that the ratio of each node memory sum of internal memory use amount and use is greater than 90% and the ratio of internal memory use amount and process number is greater than 1G, and CPU intensity refers to that the ratio of CPU holding time and working time is greater than 90%.
According to the present invention, analysis module is that the service data of the operation when submit is reanalysed operation characteristic with after the history data of the operation of submit is cumulative.
By the present invention's at least one technical scheme above-mentioned, by each node of isomeric group to be divided into the node group with different characteristic according to hardware information, the Hand up homework when submit goes to run to the node group of individual features according to the operation characteristic of the operation when submit by dispatch server.Take into full account the feature of operation and isomeric group self like this, run by the node group of applicable this operation of operation, improve the efficiency of job run on the one hand, for the high performance software comprising many operations, improve the efficiency of high performance software, also improve the service efficiency of isomeric group simultaneously.And each run data of job scheduling middleware record operation, and after this operation each run is complete, the service data of this operation and the history data of this operation are added up, reanalyse the operation characteristic of this operation, thus constantly correct, improve the accuracy of operation and operation node group coupling.
Embodiment
As shown in Figure 1, isomeric group operation self-adapting dispatching method based on application characteristic of the present invention comprises the following steps: S110, operation dispatching server collects the hardware information of each node of isomeric group, and each node is divided into the node group with different characteristic according to the hardware information of each node; Whether S120, store the operation characteristic of the operation when submit in job scheduling middleware Query Database; S130, if had, the Hand up homework when submit goes to run to the node group of individual features according to operation characteristic by operation dispatching server; If do not had, Hand up homework when submit goes to run to arbitrary node group by operation dispatching server, job scheduling middleware record is worked as the service data of the operation of submit and according to Operational Data Analysis operation characteristic, the application type of the operation when submit and operation characteristic is stored in database.
In step s 110, the hardware information of each node of the isomeric group of collection at least comprises the information of the information (dominant frequency, model as CPU) of the CPU of each node and the internal memory of each node.Each node divides into groups according to the hardware information of each node by operation dispatching server, the node that hardware information is identical or feature is identical is classified as one group, such as nodes many for CPU is divided into one group, node large for internal memory is divided into one group, thus each node of isomeric group is divided into the node group with different hardware feature.
In the step s 120, the inquiry of job scheduling middleware is specifically designed to the database of storage operation application type and operation characteristic, see the application type and operation characteristic that whether store when the operation of submit, generally, the job run feature that application type is identical is also identical.
In step s 130, which, if store the operation characteristic of the operation when submit in database, job scheduling middleware is by this operation characteristic notice operation dispatching server, and the Hand up homework when submit goes to run to the node group of individual features according to this operation characteristic by operation dispatching server; If there is no the operation characteristic of the operation stored when submit in database, Hand up homework when submit goes to run to node group by operation dispatching server by the scheduling strategy according to acquiescence, job scheduling middleware record is worked as the service data of the operation of submit and according to Operational Data Analysis operation characteristic, then the application type of the operation when submit and operation characteristic is stored in database.Wherein, service data comprises internal memory use amount, process number, the nodes of use, each node memory of use, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and CPU holding time.Every subjob is after having run, and job scheduling middleware all can record service data.Job scheduling middleware by the service data of the operation when submit and the operation characteristic reanalysing the operation when submit after the history data of the operation of submit adds up, the operation characteristic of this operation then more stored in new database.That is, this operation often runs once, job scheduling middleware record service data, and this service data and this operation all previous service data is in the past added up, and then reanalyses the operation characteristic of this operation.Can prevent the deviation that the operation characteristic only drawn according to service data may occur like this, this is a process constantly corrected.After this operation repeatedly runs, the analysis of job scheduling middleware to the operation characteristic of this operation will be more accurate.Operation characteristic comprises memory-intensive and CPU is intensive.Wherein, memory-intensive refers to that the ratio of each node memory sum of internal memory use amount and use is greater than 90% and the ratio of internal memory use amount and process number is greater than 1G; CPU intensity refers to that the ratio of CPU holding time and working time is greater than 90%.Internal memory use amount refers to the internal memory sum that all processes of the type operation are used.
Having multiple operation in high performance software operational process to need to be submitted to node and to get on operation, running the operational efficiency that can improve high performance software by selecting the node group of most suitable operation.In actual applications, high performance software also can pass through web interface (socket) and is submitted to server end.As shown in Figure 2, isomeric group comprises server end and node group, the management node of isomeric group is as the operation dispatching server of isomeric group, also be the server end shown in Fig. 2, server end runs a web server, client can be connected to web server by web browser, carry out alternately with web server, the each node group be connected with server end has certain feature, run high performance software on the client, the operation that client produces is dispatched by server end, server end is according to the feature of operation, the node group submitting the job to individual features goes to run.
Correspondingly, the invention provides a kind of isomeric group operation self-adapting dispatching system based on application characteristic.As shown in Figure 3, the isomeric group operation self-adapting dispatching system based on application characteristic of the present embodiment, comprise: grouping module 210, for collecting the hardware information of each node of isomeric group, and carry out each node to be divided into the node group with different characteristic according to the hardware information of each node; Whether enquiry module 220, for storing the operation characteristic of the operation when submit in Query Database; Submit module 230 to, for being gone to run on the node group of individual features by the Hand up homework when submit according to operation characteristic; Logging modle 240, for recording the service data of the operation when submit; Analysis module 250, draws operation characteristic for analyzing service data, and more in new database when the operation characteristic of the operation of submit.
Grouping module 210, first collects the hardware information of each node of isomeric group, comprises the information of the information (dominant frequency, model as CPU) of the CPU of each node and the internal memory of each node; Then according to the hardware information of each node, each node is divided into groups, the node that hardware information is identical or feature is identical is classified as one group, such as nodes many for CPU is divided into one group, node large for internal memory is divided into one group, thus makes each node of isomeric group form the node group with different characteristic.
Enquiry module 220, inquiry is specifically designed to the database of storage operation application type and operation characteristic, sees the application type and operation characteristic that whether store when the operation of submit.
Submit module 230 to, the Hand up homework when submit goes to run by the operation characteristic for the operation when submit inquired according to enquiry module on the node group of individual features,
Logging modle 240, records the service data of every subjob, if therefore when the operation of submit submits operation to again follow-up, logging modle 240 can record the service data of twice of this operation.
Analysis module 250, for the service data that analytic record module 240 records, each Hand up homework runs complete, after logging modle 240 records service data, analysis module 250 all can carry out the operation characteristic analyzing this operation, when analysis, for submitting the operation run repeatedly to, analysis module 250 analyzes after cumulative to up-to-date service data and the service data all previous before of this operation, and the operation characteristic of this operation finally latest analysis drawn replaces the operation characteristic that in database, this operation stored originally.Like this, the operation characteristic of this operation stored in database is dynamic change, and result is tending towards more accurately.Therefore, even if not when the operation characteristic of the operation of submit in enquiry module 220 Query Database, certain node group that this operation submitted module 230 acquiescence is submitted to isomeric group gets on operation, after logging modle 240 records the service data of this operation, analysis module 250 just can analyze the operation characteristic drawing this operation, and the application type of this operation and operation characteristic are stored in database, when the operation of submit again needs to submit to after current operation, enquiry module 220 just can inquire the operation characteristic of this operation, if this operation repeatedly submits operation to, logging modle 240 records repeatedly service data, analysis module 250 analyzes the operation characteristic drawn accordingly will be tending towards accurate.
Wherein, the service data that logging modle 240 records comprises internal memory use amount, process number, the nodes of use, each node memory of use, memory usage threshold value, one process EMS memory occupation amount threshold value, working time and CPU holding time.The operation characteristic stored in database comprises memory-intensive and CPU is intensive.Memory-intensive refers to that the ratio of each node memory sum of internal memory use amount and use is greater than 90% and the ratio of internal memory use amount and process number is greater than 1G; CPU intensity refers to that the ratio of CPU holding time and working time is greater than 90%.Internal memory use amount refers to the internal memory sum that all processes of the type operation are used.
By the present invention's at least one technical scheme above-mentioned, by each node of isomeric group to be divided into the node group with different characteristic according to hardware information, the Hand up homework when submit goes to run to the node group of individual features according to the operation characteristic of the operation when submit by dispatch server.Take into full account the feature of operation and isomeric group self like this, run by the node group of applicable this operation of operation, improve the efficiency of job run on the one hand, for the high performance software comprising many operations, improve the efficiency of high performance software, also improve the service efficiency of isomeric group simultaneously.And each run data of job scheduling middleware record operation, and after this operation each run is complete, the service data of this operation and the history data of this operation are added up, reanalyse the operation characteristic of this operation, thus constantly correct, improve the accuracy of operation and operation node group coupling.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.