WO2012018248A1

WO2012018248A1 - System and method for job management

Info

Publication number: WO2012018248A1
Application number: PCT/MY2011/000048
Authority: WO
Inventors: Nazarudin Wijee; Fadzli Mohd Raihan; Mohd Sidek Salleh
Original assignee: Mimos Berhad
Priority date: 2010-08-04
Filing date: 2011-05-18
Publication date: 2012-02-09
Also published as: MY179572A

Abstract

A job management system (100) comprising a grid portal (45) serving as user interface for generic job submission and monitoring, a job scheduler interface (15) serving as an interface for job submission module (25) and job monitoring modules (55) to communicate with job scheduler (10), a job database application programming interface (API) (40) serving as an interface for Grid Portal (45) to communicate with job database (35), and a job management service (20) serving as access point for the grid portal (45) to job scheduler (10) wherein the job management system (100) includes job database (35), job database interface (30), job submission module (25) and job monitoring module (55).

Description

SYSTEM AND METHOD FOR JOB MANAGEMENT

TECHNICAL FIELD The present invention relates to a job management and more particularly to a system and method for job management system in grid computing connecting grid portal with a plurality of job scheduler.

BACKGROUND ART

Grid computing or grid portal is a mechanism for managing heterogenous a set of computing elements, operating systems, policy decisions and environments. A long term vision of enterprise grid computing community may be non-dedicated and interoperability of various disparate systems, wherein the various disparate systems being part of similar organization or various organizations. Grid computing may be looked upon by various experts as technology whereby the technology may potentially change the way people administrate and interact.

Traditionally, grid portal architecture involves web browser, grid portal, and job scheduler. The 3 tiers system represents physical and administrative boundaries between the grid user, the Grid Portal and the scheduler. The grid portal is a web based user interface to access grid resources via web browser over a network. Current practice, the grid portal is developed using browser-supported language such as HTML, JSP, Java, JavaScript and etc. The Grid Portal services include the following: i) job submission wherein executing step-by-step job submission to the grid resources ii) job monitoring wherein online monitoring of submitted jobs with running, waiting or queuing status. Users have the option to view output file online, as well as terminate and restart running jobs.

iii) job reports wherein online view of completed, terminated or restarted jobs, enabling users to view and download output files and restart terminated jobs.

Grid portal services have a lot of advantages wherein it includes automotive repetitive tasks of scheduling multi-step processing, increase job success rate through a guided step-by-step job submission, hides the complexity, by reducing the leaning curve of using the grid applications through the command line interface and provide online monitoring of job status anytime and anywhere, by enabling fast decision making in order to terminate problematic jobs and restart with new elements. Job scheduler is a resource management system comprises of at least one resource management server and a plurality of resource agent components or devices. The resource agent devices are suitably configured to receive the jobs or tasks scheduled by the resource management server and subsequently execute the assigned jobs. One of the many significant tasks of in relation to resource management involving a plurality of computing resources is to provide effectual scheduling of predefined tasks among these computing resources over a network. The scheduling is principally configured to perform job scheduling for resources on a larger scale, or as mentioned, between multiple computing resources. This particular task therefore entails additional vital tasks such as monitoring the plurality of computer resources and serves as the primary entry point for any jobs or tasks submission.

The current portal architecture and implementation is tightly coupled with job scheduler. The portal communicates with the job scheduler by using application programming interfaces (API) provided by the job scheduler. However, there are lots of drawbacks from this method as the grid portal is tightly coupled with the job scheduler and must be in-line with the scheduler API evolution. In order for a grid portal to support multiple job schedulers, the grid portal has to take care of supporting multiple API call to the job scheduler. This normally causes problems of maintaining the grid portal if the job scheduler is evolved. There is also the unavailability of generic or standard job management system which mapped the API call to job scheduler. Patent number US20050262506 and US6438553 have disclosed system and method related to job management service. However, comparing the present invention and the prior arts, these publications have not disclosed any of job management system that intermediately connects the grid portal and the job schedulers which is more feasible and reliable, particularly in managing plurality of job schedulers for job submission and monitoring process.

Therefore, there exists a need for a job management system that may be able to handle a plurality of job schedulers at the same time without interrupting the whole process of job submission and monitoring. SUMMARY OF THE INVENTION

The present invention relates to a job management system and more particularly to a job management system in grid computing connecting grid portal and job scheduler.

In a preferred embodiment of the present invention, a job management system comprising a grid portal serving as user interface for generic job submission and monitoring, a job database application programming interface (API) serving as interface for grid portal to access job database, a job scheduler interface serving as an interface for job submission module and job monitoring modules to communicate with job scheduler and a job management service serving as access point for the grid portal to job scheduler wherein the job management includes job database, job database interface, service interval timer module, job submission module and job monitoring module.

In another preferred embodiment of the present invention, said job database stores job information in the job management system.

In another preferred embodiment of the present invention, said job database application programming interface (API) includes grid portal to communicate with job database.

In another preferred embodiment of the present invention, said job database interface includes job submission and job monitoring modules to communicate with job database. In another preferred embodiment of the present invention, said service interval timer module send signals to job submission module for job submission process and job monitoring module for job monitoring process.

In another preferred embodiment of the present invention, said job submission module executes job submission process. In another preferred embodiment of the present invention, said job monitoring module executes job monitoring process.

In another preferred embodiment of the present invention, a method of job management system comprising submitting jobs via job submission module and monitoring jobs via job monitoring module.

In another preferred embodiment of the present invention, said job submission process comprising querying generic job database for new record, mapping the record with correct job schedulers, creating job submission commands as job submission script, executing the job submission script, retrieving actual job identifier and updating generic job database record with actual job identifier.

In another preferred embodiment of the present invention, job monitoring process comprising querying generic job database for an active job, mapping the active job with correct job schedulers API, executing job scheduler API to trace job information and status and updating generic job database record with actual job information and status.

The present invention consists of features and a combination of parts hereinafter fully described and illustrated in the accompanying drawings, it is being understood that various changes in the details may be made without departing from the scope of the invention or sacrificing any of the advantages of the present invention.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

To further clarify various aspects of some embodiments of the present invention, a more particular description of the invention will be rendered by references to specific embodiments thereof, which are illustrated, in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail through the accompanying drawings in which:

FIG.1 illustrates architecture of job management system.

FIG.2 illustrates job submission process. FIG. 3 illustrates job monitoring process.

FIG.4 illustrates generic job database information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Now referring to FIG. 1 , The job management system (100) consists of a grid portal (45) to serve as a user interface for generic job submission (25) and monitoring (55) process, a job database application programming interface (API) (40) serving as an interface for Grid Portal (45) to communicate with job database (35), a job scheduler interface (15) serves as an interface for job submission (25) and job monitoring (55) modules.

The job scheduler interface (15) communicates with a job scheduler (10) and a job management service (20) to serve as an access point for the grid portal (45) to job scheduler (10). The job management service (20) includes a job database (35) for information storage, job database interface (30) for job submission (25) and a job monitoring (55) module to communicate with said job database (35). The job submission module (25) configured for job submission process (200) while job monitoring module (55) is configured for job monitoring process (300).

The job management service (20) is configured to provide a service interval timer (60) to send signal request to job submission (25) and job monitoring (60) modules and the job submission (25) and job monitoring (55) module translates it to standard communication protocol that can be read and understood by the job scheduler (10). The job management service (20) also receives acknowledgement and information from the job scheduler (10) and presented it in a form that can be retrieved by the Grid Portal (45). This architecture offers loosely coupled system between the Grid Portal (45) and the job scheduler (10) to support multiple job schedulers (10). Job Submission Module (25) received signal from service internal timer (60) to submit jobs which includes query generic job database (35) for new record for job submission request, map records of the job submission request with correct job schedulers API (15), create job submission commands to be used as job submission script, thereafter execute job submission script and retrieve actual job identifier and update generic job database record with actual job identifier.

Job Monitoring Module (25) received signal from service internal timer (60) to monitor jobs which includes query generic job database (35) for existing record for job monitoring request, map records of the job monitoring request with correct job schedulers API (15), create job monitoring commands to be used as job monitoring script, thereafter execute job monitoring script, retrieve the monitoring or trace job information and update generic job database record with monitoring or trace job information.

The job management system (100) is configured to provide generic job database (35) to be deployed in any Relational Database Management System which is accessible by the grid portal (45), A job database application programming interface (API) (40) serving as an interface for Grid Portal (45) to query and update the generic job database (35) record. The generic job database (35) may contain information as shown in FIG. 4. The job management service (20) also configured to provide job database interface (30) for job submission (25) and job monitoring (55) modules to query and update the generic job database (35) record. The job database interface (30) is used to update existing job record and to query existing job records.

Referring to FIG. 2, the process (200) commences (1 10) with service interval timer (60) sending a signal to the job submission process (200). Thereafter, the job submission process (200) constructs a query (1 15) to the job database (135) for new jobs. The job submission process (200) will be terminated if there is no availability (125) of new jobs (120) or otherwise, if there is availability (130) of a new job (120), the process proceed to the next step. The job submission process (200) then maps (140) the new job with available scheduler. In the next process, the job submission (200) creates a job submission commands (145) based on the job database (135) and mapped the scheduler (140). Job submission commands will be submitted to the scheduler (150) by said job submission process (200). Job submission (200) then updates (155) the job database with actual job identifier, which was retrieved from said scheduler. The process ends (160) once the task has been completed.

Referring to FIG. 3, a generic job monitoring process (300) commences (210) with service interval timer (60) sending a signal to said job monitoring process (300). Thereafter, the job monitoring process (300) constructs a query (215) to the job database (220) for a list of active jobs. The job monitoring process (300) will be terminated if there is no availability (225) of new jobs (230) or otherwise, if there is availability (235) of a new job (230), the process continues with the next step. The job monitoring process will then map the active jobs with available scheduler (240) and create a monitoring or trace job commands (245). The monitoring and trace job commands (245) will then be sent to the scheduler (140). In the next process, the job monitoring process (300) retrieves the job monitoring or traces job information (245) from the scheduler (140) and update (250) the monitoring or trace job into job database. The process then ends (255) once the task has been executed.

In the foregoing specification, specific embodiments of the present invention have been described. However, one of ordinary skill in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present invention as set forth in the various embodiments discussed above and the claims that follow. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present invention. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements as described herein.

Claims

1) A job management system (100) comprising:

a grid portal (45) serving as user interface for generic job submission and monitoring; a job scheduler interface (15) serving as an interface for job submission module (25) and job monitoring modules (55) to communicate with job scheduler (10); and a job management service (20) serving as access point for the grid portal (45) to job scheduler (10), wherein the job management system (100) includes job database (35), job database interface (API) (30), job submission module (25) and job monitoring module (55).

2) The system in claim 1 wherein said job database (35) stores job information in the job management system.

3) The system in claim 1 wherein said job database interface (30) includes job submission and job monitoring modules to communicate with job database.

4) The system in claim 1 wherein said job database application programming interface (API) (40) serving as an interface for Grid Portal (45) to communicate with job database (35).

5) The system in claim 1 wherein said job submission module (25) executes job submission process.

6) The system in claim 1 wherein said job monitoring module (55) executes job monitoring process. 7) A method of job management system comprising submitting jobs via job submission module (25) and monitoring jobs via job monitoring module (55).

8) The method in claim 7 wherein said job submission process comprising:

querying generic job database for new record;

mapping the record with correct job schedulers;

creating job submission commands as job submission script;

executing the job submission script;

retrieving actual job identifier; and

updating generic job database record with actual job identifier.

9) The method in claim 7 wherein job monitoring process comprising:

querying generic job database for an active job;

mapping the active job with correct job schedulers API;

executing job scheduler API to trace job information and status; and

updating generic job database record with actual job information and status.