The Web service method for packing and the system that are used for marine numerical simulation ROMS
Technical field
The present invention relates to field of computer technology, more specifically, relate to a kind of Web service method for packing for marine numerical simulation computing system ROMS and Web service package system.
Background technology
Numerical simulation is the important means of ocean research, is bringing into play more and more important effect in scientific research of seas.And the partial differential equations of ocean numerical model by numerical solution control seawater movement realizes the numerical simulation to seawater movement.In the numerical model of numerous oceans, ROMS (Regional Ocean Modeling System) is one of them zone and ocean circulation pattern, and as one of main ocean businessization pattern, ROMS is used in a large number.ROMS adopts Fortran90/95 to write, and supports to share and distributed two kinds of computer body system structures, has that computation-intensive, IO read-write amount are large, the network bandwidth requirement high.The ROMS pattern has preferably parallel-expansion, and problem scale hour can accelerate to 100 nuclears nearly, can reach 200-600 nuclear when problem scale is larger.Simultaneously, ROMS also can form coupled mode with other maritime meteorology pattern, the drive pattern of perhaps using as other maritime meteorology.
The calculated amount of carrying out marine numerical simulation with ROMS is generally larger, therefore need to carry out parallel computation at HPCC.In the mode of operation that has now, the user need to use long-range character terminal to be connected to HPCC usually, and carries out sequence of operations and just can finish this numerical simulation under command line environment, generally comprises following several flow process:
The calculated amount of carrying out marine numerical simulation with ROMS is generally larger, therefore need to carry out parallel computation at HPCC.In the mode of operation that has now, the user need to use long-range character terminal to be connected to the HPCC system usually, and carries out sequence of operations and just can finish this numerical simulation under command line environment, generally comprises following several flow process:
Prepare the executable file of ROMS and the data file that needs thereof.These files usually need to be placed on the overall situation of cluster and share in the storage, and the instruments such as user's normal operation FTP, SFTP are uploaded these files to cluster, and in most of the cases, the user also needs with the text edit tool these files to be made amendment.
If on the calculating group system job scheduler is arranged, then the user need to use text editor to write a script file, wherein fills in the computational resource quantity that needs application, and the order of operation ROMS program, then uses job scheduling order submit job.If there is not job scheduler, then the user need to be with the process number of operation ROMS program and the process initiation device that passes to MPI in the mode that the information exchanges such as which node operation are crossed command line parameter, such as mpirun, mpiexec etc.Process initiation device booting ROM S program by MPI is carried out parallel computation.
After the program operation is finished, use the instruments such as FTP, SFTP to download destination file and check result of calculation to the subscriber's local computing machine.
There is following defective in the method for existing operation ROMS calculation procedure:
(1) use difficulty large.Want to finish smoothly the calculation task of ROMS program, the user often needs to be familiar with very much computer literacy and the instruments of a lot of meter specialties, such as (SuSE) Linux OS basic operation, Shell script, text editing commonly used and character handling implement, the use of MPI environment, use of job scheduling software etc.
(2) execution efficient is low.Want to submit to smoothly the calculation task of a ROMS program, need to carry out a plurality of flow processs, need to carry out complicated loaded down with trivial details operation in command line environment, thereby cause the submission of parallel computation task to waste time and energy inefficiency.
(3) error probability is large.For most of domestic consumer, carry out complicated loaded down with trivial details operation in command line environment, the probability that the human operational error occurs is very large.Particularly for the unfamiliar user of operating process, all easily encounter problems at the links of carrying out.
(4) security risk is high.Need to the capable terminal access authority of domestic consumer's open command, for the keeper of HPCC, there be the risk of security in traditional method of operating to the capable terminal of domestic consumer's open command.User's maloperation may affect the Systems balanth operation, and the hacker also might avail oneself of the opportunity to get in.
For the problem in the correlation technique, effective solution is proposed not yet at present.
Summary of the invention
For the problem in the correlation technique, the present invention proposes a kind of user of reduction to Web method for packing and the Web package system of marine numerical simulation computing system ROMS submit job.
Technical scheme of the present invention is achieved in that
On the one hand, provide a kind of Web service method for packing for marine numerical simulation computing system ROMS, having comprised:
Receive the information relevant with task by the Web service interface;
Extract useful information and generate the job script file by script generator according to the received information relevant with task;
The job script file that generates is submitted to the calculating group system;
Described calculating group system is called described marine numerical simulation computing system ROMS according to described job script file and is carried out corresponding operation.
In optional embodiment, before receiving the step of the information relevant with task by the Web service interface, described Web method for packing also comprises: obtain corresponding application configuration file according to the application request that receives; And, according to the corresponding Web service interface of ruuning situation generation of described application configuration file and described marine numerical simulation computing system ROMS.
In optional embodiment, the described information relevant with task comprises one of following or combination in any: the core cpu number of the catalogue of path, described marine numerical simulation computing system ROMS place, required input file, fill order, required computing node, each computing node and memory size and the computing time of executing the task.
In optional embodiment, extract useful information by script generator and generate before the job script according to the received information relevant with task, described Web method for packing also comprises: whether legally detect the described information relevant with task, detect when being, the received information relevant with task of executive basis is by script generator extraction useful information and generate the step of job script; Detect when no, generate illegal information and be presented on the described Web service interface.
In optional embodiment, the job script that generates is submitted to described marine numerical simulation computing system ROMS system to carry out operation, comprise: in the job script file submit job scheduling queue that generates, dispatch to carry out operation by job scheduling system.
In optional embodiment, before receiving the information relevant with task by the Web service application interface, detect the queuing message of described calculating group system, comprising: queue name, can use operation number and available computing node number in total check figure, the formation.
In optional embodiment, before receiving the information relevant with task by the Web service interface, detect user's Disk Quotas and charging amount.
A kind of Web service package system for marine numerical simulation computing system ROMS is provided on the other hand, has comprised:
The Web application module is used for receiving the information relevant with task by the Web service interface;
Script generation module is used for according to the received information extraction useful information relevant with task and generates the job script file;
Module is submitted in operation to, is used for the job script file that generates is submitted to the calculating group system; So that described calculating group system is called described marine numerical simulation computing system ROMS according to described job script file and is carried out corresponding operation.
In optional embodiment, described Web application module comprises: the Command Line Parsing module is used for obtaining corresponding application configuration file according to the application request that receives; And Web service interface generation module is used for generating corresponding Web service interface according to the ruuning situation of described application configuration file and described marine numerical simulation computing system ROMS.
In optional embodiment, described Web service package system also comprises detection module, whether legal for detection of the described information relevant with task, detect when being, described script generation module is according to the received information extraction useful information relevant with task and generate the job script file; Detect when no, generate illegal information and be presented on the described Web service interface.
Description of drawings
Fig. 1 illustrates the process flow diagram that is used for according to an embodiment of the invention the Web service method for packing of marine numerical simulation computing system ROMS.
Fig. 2 illustrates the schematic block diagram that is used for according to another embodiment of the present invention the Web service package system of marine numerical simulation computing system ROMS.
Embodiment
Below in conjunction with accompanying drawing the embodiment of the invention is further described.
Fig. 1 is the process flow diagram that is used for according to an embodiment of the invention the Web service method for packing of marine numerical simulation computing system ROMS.As shown in Figure 1, this Web service method for packing comprises:
S101 receives the information relevant with task by the Web service interface.
Alternatively, before step S101, this Web service method for packing can also comprise the application configuration file that obtains correspondence according to the application request that receives; Ruuning situation according to described application configuration file and described marine numerical simulation computing system ROMS generates corresponding Web service interface.
In embodiments of the present invention, each application for marine numerical simulation computing system ROMS all can have a corresponding application configuration file.Comprise in this application configuration file: the corresponding every initial value of the page, Command Line Parsing can be configured parsing according to the ruuning situation of calculating group system and the actual conditions of application, generates the corresponding page, the namely corresponding Web service interface of using.
Alternatively, the information relevant with task can comprise: in catalogue and the fill order of path, described marine numerical simulation computing system ROMS place, required input file.
Alternatively, before step S101, can also receive the computational resource information relevant with the operation that will carry out by described Web service interface simultaneously or afterwards, described computational resource information comprises: core cpu number, memory size and computing time.
Alternatively, the user can also by the Web service interface, select whether to use Virtual network computer (Virtual Network Computing, VNC) function or CheckPoint function (being again computing function of breakpoint).Alternatively, the user can also be by the node in the Web service interface submission calculating group system, single node core number, network speed agreement, fill order, application required input file etc.
S102 is extracted useful information and is generated the job script file by script generator according to the received information relevant with task.
S103 submits to the calculating group system with the job script file that generates, and carries out corresponding operation so that described calculating group system is called described marine numerical simulation computing system ROMS according to described job script file.
In optional embodiment, described Web method for packing also comprised before step S102: whether legally detect the described information relevant with task, detect when being execution in step S103; Detect when no, generate illegal information and be presented on the described Web service interface.
In optional embodiment, calculate group system and comprise job scheduling system, step S103 comprises: the job script file that generates is submitted in the job scheduling formation of calculating group system, dispatched in order to call described marine numerical simulation computing system ROMS by job scheduling system and carry out corresponding operation.
In optional embodiment, before step S101, the Web method for packing also comprises: detect the queuing message of described calculating group system, comprising: queue name, can use operation number and available computing node number in total check figure, the formation.
In optional embodiment, before receiving the information relevant with task by the Web service interface, detect user's Disk Quotas and charging amount.
Fig. 2 is the schematic block diagram according to the Web service package system that is used for marine numerical simulation computing system ROMS of the embodiment of the invention.As schematically shown in Figure 2, this Web service package system comprises: Web application module 10 is used for receiving the information relevant with task by the Web service interface; Script generation module 20 is used for according to the received information extraction useful information relevant with task and generates the job script file; And module 30 is submitted in operation to, is used for the job script file that generates is submitted to the calculating group system; So that described calculating group system is called described marine numerical simulation computing system ROMS according to described job script file and is carried out corresponding operation.
In optional embodiment, the described information relevant with task comprises one of following or combination in any: the core cpu number of the catalogue of path, described marine numerical simulation computing system ROMS place, required input file, fill order, required computing node, each computing node and memory size and the computing time of executing the task.
In optional embodiment, described Web application module comprises: the Command Line Parsing module is used for obtaining corresponding application configuration file according to the application request that receives; And Web service interface generation module is used for generating corresponding Web service interface according to the ruuning situation of described application configuration file and described marine numerical simulation computing system ROMS.
In optional embodiment, described Web service package system also comprises detection module, whether legal for detection of the described information relevant with task, detect when being, described script generation module is according to the received information extraction useful information relevant with task and generate the job script file; Detect when no, generate illegal information and be presented on the described Web service interface.
In optional embodiment, operation submission module is submitted to the calculating group system with the job script file that generates and is comprised: the job script file that generates is submitted in the job scheduling formation of calculating group system, carried out job scheduling by job scheduling system and carry out corresponding operation in order to call described marine numerical simulation computing system ROMS.
In optional embodiment, described detection module also was used for before the Web service application interface receives the information relevant with task, detect the queuing message of described calculating group system, comprising: queue name, can use operation number and available computing node number in total check figure, the formation.
In optional embodiment, described Web service package system also comprises the amount computing module, be used for before receiving the information relevant with task by the Web service interface, detect user's Disk Quotas and charging amount, so that the Web service package system determines whether to provide the Web packing service for this user's operation according to user's Disk Quotas and charging amount.Further, the Disk Quotas according to the user can also be the corresponding resource of this user assignment for calculating group system.
The present invention encapsulates by Web, submit to flow process to be hidden in bottom the operation of marine numerical simulation computing system ROMS, offer the Web page of a simple, intuitive of user, the user only needs to carry out on the Web page shirtsleeve operation just can realize a whole set of flow processs such as data file preparation, computational resource application, operation submission, result check.By the Web encapsulation, can greatly reduce user's use threshold, improve service efficiency, reduce error probability.
Its beneficial effect can be embodied in the following aspects:
(1) reduces the use difficulty.The user no longer needs to be grasped the calling of (SuSE) Linux OS basic operation, Shell script, text editing commonly used and character handling implement, MPI environment, job scheduling software etc. and its specialty background irrelevant knowledge and instrument, just can easily finish the work such as preparation, execution of marine numerical simulation computing system ROMS calculation task, greatly reduce the use threshold.
(2) increase work efficiency.The order line operation Hide All of very complicated is carried out at the bottom automatic batch at bottom, can save a large amount of manual operation time, improves service efficiency.
(3) success ratio of raising job invocation.In the embodiment of the invention, can carry out various validity checkings to parameter and data that the user submits at the Web interface.Such as, check whether the computational resource of user's application surpasses its authority, and whether the parameter of user's input is legal, whether user's input file form is correct etc., if the mistake of judging can provide corresponding prompting at the Web interface.Can greatly improve the success ratio that parallel task is submitted to by these validity checkings.
(4) can realize like a cork sophisticated functions.Some complexity or the Premium Features of marine numerical simulation computing system ROMS operation such as the selection of computational grid, start cpu process binding etc., if need to specify complicated parameter by command line mode, complex operation uses difficulty larger.After carrying out the Web encapsulation, the user only need to just can realize these functions by some Option Boxs on the interface.
(5) safety in utilization of raising HPCC system.After the operation of the job invocation of domestic consumer was encapsulated in Web, what the user carried out all was enclosed operation, has avoided to a great extent affect system stability or safe user's illegal operation.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.