The Web service method for packing and the system that are used for weather forecast computing system WRF
Technical field
The present invention relates to field of computer technology, more specifically, relate to a kind of Web service method for packing and Web service package system for mesoscale numerical value weather forecast computing system WRF.
Background technology
Meso-scale meteorology is to develop rapidly important branch in the modern Meteorological Science, and the atmosphere Mesoscale Motion that it is studied is related to discovery and prevention that Regional Gravity And is wanted diastrous weather.One side wherein is by meso-scale model, and the Study of Meso Scale Weather process is carried out deep modeling effort and forecast experiments.Along with developing rapidly of computer technology in recent years, meso-scale model reaches its maturity.Mesoscale numerical value weather forecast pattern has many, and for example WRF computing system (Mesoscale Model5) is comparatively widely pattern of present domestic and international application.
Mesoscale numerical value weather forecast pattern WRF is the mesoscale model of new generation that grows up in previous generation MM5 pattern, is the abbreviation of Weather Research Forecasting model.The WRF modular system is by common mesoscale Forecast Mode of new generation and the assimilation system that participates in developing research of the scientist of many American Studies departments and university.The WRF modular system will become the instrument that improves from the cloud yardstick to different scale significant weather feature forecast precisions such as synoptic scales, and emphasis is considered 1~10 kilometer horizontal grid.This pattern possesses high scalability, is pattern the most frequently used in the current Meteorology Forecast System.
The calculated amount of carrying out marine numerical simulation with WRF is generally larger, therefore need to carry out parallel computation at HPCC.In the mode of operation that has now, the user need to use long-range character terminal to be connected to the HPCC system usually, and carries out sequence of operations and just can finish this numerical simulation under command line environment, generally comprises following several flow process:
Prepare the executable file of WRF and the data file that needs thereof.These files usually need to be placed on the overall situation of cluster and share in the storage, and the instruments such as user's normal operation FTP, SFTP are uploaded these files to cluster, and in most of the cases, the user also needs with the text edit tool these files to be made amendment.
If on the calculating group system job scheduler is arranged, then the user need to use text editor to write a script file, wherein fills in the computational resource quantity that needs application, and the order of operation WRF program, then uses job scheduling order submit job.If there is not job scheduler, then the user need to be with the process number of operation WRF program and the process initiation device that passes to MPI in the mode that the information exchanges such as which node operation are crossed command line parameter, such as mpirun, mpiexec etc.Process initiation device startup WRF program by MPI is carried out parallel computation.
After the program operation is finished, use the instruments such as FTP, SFTP to download destination file and check result of calculation to the subscriber's local computing machine.
There is following defective in the method for existing operation WRF calculation procedure:
(1) use difficulty large.Want to finish smoothly the calculation task of WRF program, the user often needs to be familiar with very much computer literacy and the instruments of a lot of meter specialties, such as (SuSE) Linux OS basic operation, Shell script, text editing commonly used and character handling implement, the use of MPI environment, use of job scheduling software etc.
(2) execution efficient is low.Want to submit to smoothly the calculation task of a WRF program, need to carry out a plurality of flow processs, need to carry out complicated loaded down with trivial details operation in command line environment, thereby cause the submission of parallel computation task to waste time and energy inefficiency.
(3) error probability is large.For most of domestic consumer, carry out complicated loaded down with trivial details operation in command line environment, the probability that the human operational error occurs is very large.Particularly for the unfamiliar user of operating process, all easily encounter problems at the links of carrying out.
(4) security risk is high.Need to the capable terminal access authority of domestic consumer's open command, for the keeper of HPCC, there be the risk of security in traditional method of operating to the capable terminal of domestic consumer's open command.User's maloperation may affect the Systems balanth operation, and the hacker also might avail oneself of the opportunity to get in.
For the problem in the correlation technique, effective solution is proposed not yet at present.
Summary of the invention
For the problem in the correlation technique, the present invention proposes a kind of Web method for packing and Web package system of the user's of reduction centering yardstick numerical value weather forecast computing system WRF submit job.
Technical scheme of the present invention is achieved in that
On the one hand, provide a kind of Web service method for packing for mesoscale numerical value weather forecast computing system WRF, having comprised:
Receive the information relevant with task by the Web service interface;
Extract useful information and generate the job script file by script generator according to the received information relevant with task;
The job script file that generates is submitted to the calculating group system;
Described calculating group system is called described mesoscale numerical value weather forecast computing system WRF according to described job script file and is carried out corresponding operation.
In optional embodiment, before receiving the step of the information relevant with task by the Web service interface, described Web method for packing also comprises: obtain corresponding application configuration file according to the application request that receives; And, according to the corresponding Web service interface of ruuning situation generation of described application configuration file and described mesoscale numerical value weather forecast computing system WRF.
In optional embodiment, the described information relevant with task comprises one of following or combination in any: the core cpu number of the catalogue of path, described mesoscale numerical value weather forecast computing system WRF place, required input file, fill order, required computing node, each computing node and memory size and the computing time of executing the task.
In optional embodiment, extract useful information by script generator and generate before the job script according to the received information relevant with task, described Web method for packing also comprises: whether legally detect the described information relevant with task, detect when being, the received information relevant with task of executive basis is by script generator extraction useful information and generate the step of job script; Detect when no, generate illegal information and be presented on the described Web service interface.
In optional embodiment, the job script that generates is submitted to the meteorological forecast system that calculates of described mesoscale numerical value to carry out operation, comprise: in the job script file submit job scheduling queue that generates, dispatch to carry out operation by job scheduling system.
In optional embodiment, before receiving the information relevant with task by the Web service application interface, detect the queuing message of described calculating group system, comprising: queue name, can use operation number and available computing node number in total check figure, the formation.
In optional embodiment, before receiving the information relevant with task by the Web service interface, detect user's Disk Quotas and charging amount.
A kind of Web service package system for mesoscale numerical value weather forecast computing system WRF is provided on the other hand, has comprised:
The Web application module is used for receiving the information relevant with task by the Web service interface;
Script generation module is used for according to the received information extraction useful information relevant with task and generates the job script file;
Module is submitted in operation to, is used for the job script file that generates is submitted to the calculating group system; So that described calculating group system is called described mesoscale numerical value weather forecast computing system WRF according to described job script file and is carried out corresponding operation.
In optional embodiment, described Web application module comprises: the Command Line Parsing module is used for obtaining corresponding application configuration file according to the application request that receives; And Web service interface generation module is used for generating corresponding Web service interface according to the ruuning situation of described application configuration file and described mesoscale numerical value weather forecast computing system WRF.
In optional embodiment, described Web service package system also comprises detection module, whether legal for detection of the described information relevant with task, detect when being, described script generation module is according to the received information extraction useful information relevant with task and generate the job script file; Detect when no, generate illegal information and be presented on the described Web service interface.
Description of drawings
Fig. 1 illustrates the process flow diagram that is used for according to an embodiment of the invention the Web service method for packing of mesoscale numerical value weather forecast computing system WRF.
Fig. 2 illustrates the schematic block diagram that is used for according to another embodiment of the present invention the Web service package system of mesoscale numerical value weather forecast computing system WRF.
Embodiment
Below in conjunction with accompanying drawing the embodiment of the invention is further described.
Fig. 1 is the process flow diagram that is used for according to an embodiment of the invention the Web service method for packing of mesoscale numerical value weather forecast computing system WRF.As shown in Figure 1, this Web service method for packing comprises:
S101 receives the information relevant with task by the Web service interface.
Alternatively, before step S101, this Web service method for packing can also comprise the application configuration file that obtains correspondence according to the application request that receives; Ruuning situation according to described application configuration file and described mesoscale numerical value weather forecast computing system WRF generates corresponding Web service interface.
In embodiments of the present invention, each application for mesoscale numerical value weather forecast computing system WRF all can have a corresponding application configuration file.Comprise in this application configuration file: the corresponding every initial value of the page, Command Line Parsing can be configured parsing according to the ruuning situation of calculating group system and the actual conditions of application, generates the corresponding page, the namely corresponding Web service interface of using.
Alternatively, the information relevant with task can comprise: in catalogue and the fill order of path, described mesoscale numerical value weather forecast computing system place, required input file.
Alternatively, before step S101, can also receive the computational resource information relevant with the operation that will carry out by described Web service interface simultaneously or afterwards, described computational resource information comprises: core cpu number, memory size and computing time.
Alternatively, the user can also by the Web service interface, select whether to use Virtual network computer (Virtual Network Computing, VNC) function or CheckPoint function (being again computing function of breakpoint).Alternatively, the user can also be by the node in the Web service interface submission calculating group system, single node core number, network speed agreement, fill order, application required input file etc.
S102 is extracted useful information and is generated the job script file by script generator according to the received information relevant with task.
S103 submits to the calculating group system with the job script file that generates, and carries out corresponding operation so that described calculating group system is called described mesoscale numerical value weather forecast computing system WRF according to described job script file.
In optional embodiment, described Web method for packing also comprised before step S102: whether legally detect the described information relevant with task, detect when being execution in step S103; Detect when no, generate illegal information and be presented on the described Web service interface.
In optional embodiment, calculate group system and comprise job scheduling system, step S103 comprises: the job script file that generates is submitted in the job scheduling formation of calculating group system, dispatched in order to call described mesoscale numerical value weather forecast computing system WRF by job scheduling system and carry out corresponding operation.
In optional embodiment, before step S101, the Web method for packing also comprises: detect the queuing message of described calculating group system, comprising: queue name, can use operation number and available computing node number in total check figure, the formation.
In optional embodiment, before receiving the information relevant with task by the Web service interface, detect user's Disk Quotas and charging amount.
Fig. 2 is the schematic block diagram according to the Web service package system that is used for mesoscale numerical value weather forecast computing system WRF of the embodiment of the invention.As schematically shown in Figure 2, this Web service package system comprises: Web application module 10 is used for receiving the information relevant with task by the Web service interface; Script generation module 20 is used for according to the received information extraction useful information relevant with task and generates the job script file; And module 30 is submitted in operation to, is used for the job script file that generates is submitted to the calculating group system; So that described calculating group system is called described mesoscale numerical value weather forecast computing system WRF according to described job script file and is carried out corresponding operation.
In optional embodiment, the described information relevant with task comprises one of following or combination in any: the core cpu number of the catalogue of path, described mesoscale numerical value weather forecast computing system WRF place, required input file, fill order, required computing node, each computing node and memory size and the computing time of executing the task.
In optional embodiment, described Web application module comprises: the Command Line Parsing module is used for obtaining corresponding application configuration file according to the application request that receives; And Web service interface generation module is used for generating corresponding Web service interface according to the ruuning situation of described application configuration file and described mesoscale numerical value weather forecast computing system WRF.
In optional embodiment, described Web service package system also comprises detection module, whether legal for detection of the described information relevant with task, detect when being, described script generation module is according to the received information extraction useful information relevant with task and generate the job script file; Detect when no, generate illegal information and be presented on the described Web service interface.
In optional embodiment, operation submission module is submitted to the calculating group system with the job script file that generates and is comprised: the job script file that generates is submitted in the job scheduling formation of calculating group system, carried out job scheduling by job scheduling system and carry out corresponding operation in order to call described mesoscale numerical value weather forecast computing system WRF.
In optional embodiment, described detection module also was used for before the Web service application interface receives the information relevant with task, detect the queuing message of described calculating group system, comprising: queue name, can use operation number and available computing node number in total check figure, the formation.
In optional embodiment, described Web service package system also comprises the amount computing module, be used for before receiving the information relevant with task by the Web service interface, detect user's Disk Quotas and charging amount, so that the Web service package system determines whether to provide the Web packing service for this user's operation according to user's Disk Quotas and charging amount.Further, the Disk Quotas according to the user can also be the corresponding resource of this user assignment for calculating group system.
The present invention encapsulates by Web, submit to flow process to be hidden in bottom the operation of mesoscale numerical value weather forecast computing system WRF, offer the Web page of a simple, intuitive of user, the user only needs to carry out on the Web page shirtsleeve operation just can realize a whole set of flow processs such as data file preparation, computational resource application, operation submission, result check.By the Web encapsulation, can greatly reduce user's use threshold, improve service efficiency, reduce error probability.
Its beneficial effect can be embodied in the following aspects:
(1) reduces the use difficulty.The user no longer needs to be grasped the calling of (SuSE) Linux OS basic operation, Shell script, text editing commonly used and character handling implement, MPI environment, job scheduling software etc. and its specialty background irrelevant knowledge and instrument, just can easily finish the work such as preparation, execution of mesoscale numerical value weather forecast computing system WRF calculation task, greatly reduce the use threshold.
(2) increase work efficiency.The order line operation Hide All of very complicated is carried out at the bottom automatic batch at bottom, can save a large amount of manual operation time, improves service efficiency.
(3) success ratio of raising job invocation.In the embodiment of the invention, can carry out various validity checkings to parameter and data that the user submits at the Web interface.Such as, check whether the computational resource of user's application surpasses its authority, and whether the parameter of user's input is legal, whether user's input file form is correct etc., if the mistake of judging can provide corresponding prompting at the Web interface.Can greatly improve the success ratio that parallel task is submitted to by these validity checkings.
(4) can realize like a cork sophisticated functions.Some complexity or Premium Features during mesoscale numerical value weather forecast computing system WRF operation such as the selection of computational grid, start cpu process binding etc., if need to specify complicated parameter by command line mode, complex operation uses difficulty larger.After carrying out the Web encapsulation, the user only need to just can realize these functions by some Option Boxs on the interface.
(5) safety in utilization of raising HPCC system.After the operation of the job invocation of domestic consumer was encapsulated in Web, what the user carried out all was enclosed operation, has avoided to a great extent affect system stability or safe user's illegal operation.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.