TWI758736B - Computer program product and apparatus for scheduling and executing jobs employed in parallel-processing environment - Google Patents

Abstract

The disclosure discloses a computer program product for scheduling and executing jobs employed in a parallel-processing environment being loaded and executed by a processing unit of an application server to include program code to: obtain a job from a job item list, in which a status of the job is wait and a scheduled start time is earlier than the current time; provide a job identity and an optimistic lock version of the job to a database server; request the database server to update a record of a job table to increase an optimistic lock version of the record by one, modify an owner of the record with an identity of the application server, and modify a status of the record with running, where the record before being updated needs to include the job identity and the optimistic lock version of the job; and enqueue the job into a queue to execute when receiving an update success message from the database server.

Description

Computer program products and devices for job scheduling and execution in parallel processing environments

The present invention relates to a parallel processing technology, in particular to a computer program product and device for work scheduling and execution in a parallel processing environment.

A server cluster consists of a group of servers that work loosely or tightly together. Each server in a server cluster is configured to perform the same tasks, controlled and scheduled by software. Therefore, there is a need for a computer program product and apparatus for work scheduling and execution in a parallel processing environment for optimizing the work load of the entire system.

In view of this, how to alleviate or eliminate the above-mentioned deficiencies in related fields is a problem to be solved.

This specification relates to a computer program product for job scheduling and execution in a parallel processing environment, including code that can be loaded and executed by a processing unit of an application server: a job is obtained from a list of work items, wherein the job's The status is waiting, and the scheduled start time of the job is earlier than the current time; provide the job identification code and optimistic lock version number of the job to the database server; request the database server to update the record of the job data table, which is used to convert the record Add one to the optimistic lock version number, change the record's executor to the ID of the application server, and change the record's status to running, where the record must contain the job's job ID and the job's optimistic lock version before updating. number; and when a successful update message is received from the database server, the job is queued for execution.

The present specification further relates to an apparatus for scheduling and executing tasks in a parallel processing environment, including a memory and a processing unit. Memory allocation space for queuing and storing work item lists. The processing unit is coupled to the memory, and is used to obtain work from the work item list, wherein the state of the work is waiting, and the scheduled start time of the work is earlier than the current time; the work identification code and the optimistic lock version number of the work are provided to the data. the database server; requesting the database server to update the work data table record for incrementing the optimistic lock version number of this record by one, changing the executor of this record to the identifier of the application server executed by the processing unit, and Change the status of this record to running, where the record must contain the job ID of the job and the optimistic lock version number of the job before updating; and enqueue the job when the update success message is received from the database server to the queue to execute.

Other advantages of the present invention will be explained in more detail in conjunction with the following description and drawings.

The following description is a preferred implementation manner to complete the invention, and its purpose is to describe the basic spirit of the invention, but it is not intended to limit the invention. Reference must be made to the scope of the following claims for the actual inventive content.

It must be understood that the words "comprising" and "including" used in this specification are used to indicate the existence of specific technical features, values, method steps, operation processes, elements and/or components, but do not exclude the possibility of adding More technical features, values, method steps, job processes, elements, components, or any combination of the above.

The use of words such as "first", "second", "third", etc. in the claims is used to modify the elements in the claims, and is not used to indicate that there is a priority order, a preceding relationship between them, or an element Prior to another element, or chronological order in which method steps are performed, is only used to distinguish elements with the same name.

In order to allow multiple application servers to process work in parallel, a database server can be used to manage all the work that needs to be performed, and then the application servers can request and execute the work. FIG. 1 is a schematic diagram of multiple application servers requesting a database server to provide executable work, according to some embodiments. In these embodiments, application servers 112, 114, and 116 in server cluster 110 start a timer 113, 115, and 117, respectively, to periodically make requests to database server 130 over network 100 for Get a list of tasks that need to be performed, the details of the tasks, and then execute accordingly. However, since application servers 112, 114, and 116 make requests to database server 130 independently, it is necessary to ensure that each job is performed by one application server rather than two or more Application server execution. In addition, if the application servers 112, 114 and 116 make requests to the database server 130 too frequently, the database server 130 will also be too busy and cause a system bottleneck.

In order to solve the above disadvantages, an embodiment of the present invention provides an application server to coordinate job scheduling and execution in a parallel processing environment by accessing records in the Jobs Table. FIG. 2 is an architecture diagram of a network system according to an embodiment of the present invention. The network system 20 includes a Server Cluster 210 consisting of a plurality of Application Servers 212, 214 and 216, each of which can be configured to execute a variety of It can cover but not limited to digital bank management, online banking, mobile customer service, enterprise internal process management, big data storage, big data data integration, big data data retrieval and other fields. In addition, each application server can also be configured to provide File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP), Hypertext Transfer Protocol Secure (Hypertext Transfer Protocol Secure, HTTPS Request), email-managed services, or any combination of the above. The network system 10 sets up a database server (Database Server) 230 to store a work data table for coordinating work execution among the application servers 212, 214 and 216, ensuring that each work is performed by an application server to execute. The application servers 212, 214 and 216 and the database server 230 can communicate with each other through the network 200, and the network 200 can be the Internet, a wired Local Area Network (LAN), Wireless LAN, Peer-to-Peer (P2P Network), or any combination of the above. It should be noted here that the application servers 212, 214 and 216 and the database server 230 may also refer to software servers, which are executed on a computer with a multi-core processor. In the network environment of server cluster, it can also be applied to a single computer with parallel processing capability.

FIG. 3 is a system architecture diagram of a computing device according to an embodiment of the present invention. This system architecture may be implemented in any of application servers 212 , 214 and 216 , database server 230 , including at least processing unit 310 . The processing unit 310 may be implemented in a variety of ways, such as in dedicated hardware circuits or general-purpose hardware (eg, a single processor, a multiprocessor with parallel processing capabilities, a graphics processor, or other processor capable of computing), and in When the code or software is executed, the functions described later are provided. The system architecture further includes a memory 350 and a storage unit 340. The memory 350 stores data required during the execution of the code, such as variables, data tables, etc., and the storage unit 340 stores various electronic files, such as , web pages, documents, audio files, video files, databases, etc. The system architecture further includes a communication interface 360 for the processing unit 310 to communicate with other electronic devices. The communication interface 360 may be a wireless telecommunications module (Wireless Telecommunications Module), a local area network (LAN) communication module or a wireless local area network communication module (WLAN). The Wireless Telecommunications Module may include any combination of modems (Modem) supporting 2G, 3G, 4G or above technology generations. The input device 330 may include a keyboard, a mouse, a touch panel, and the like. The user can press hard keys on the keyboard to input characters, control the mouse by operating the mouse, or control the running application by making gestures on the touch panel. The gestures may include single-click, double-click, single-finger drag, multi-finger drag, etc., but are not limited thereto. The system architecture may optionally include a display unit 320, and the display unit 320 may include a display panel (eg, a thin film liquid crystal display panel, an organic light-emitting diode panel, or other panels with display capability) for displaying input characters, Numbers, symbols, moving traces of dragging the mouse, drawn patterns or pictures provided by the application are provided to the user for viewing.

In order to coordinate a plurality of application servers to execute incoming scheduled jobs, the storage device 340 of the database server 230 stores a relational database, including a job data table "Jobs". A job data table can store multiple records, each record containing information about an incoming scheduled job. A job that is scheduled is a job that cannot be executed immediately, but cannot be executed until a specified time has elapsed. A work data sheet can contain sample fields as shown in Table 1: Table 1 field name describe id working code fromId Re-executed working code jobName job title startDateTime Schedule start time status Status (Wait/Running/Done/Error) step long-term work status progress owner Executor parameters parameter version Optimistic lock version number Among them, the field "id" is the primary key of the job record, and the fields "jobName", "startDateTime" and "paramters" respectively store the name of the scheduled job, the start execution time and the parameters of the job content. The parameter can include a Group ID to distinguish whether multiple scheduled jobs are substantially the same. If the parameters of multiple jobs contain the same group ID, it means that these jobs are substantially the same. For example, if a user enters or imports billing data on the user interface, and then clicks the "Monthly Billing" button several times after specifying a schedule completion time, multiple entries will be added to the memory list data table that include: A record of the same group ID. When creating a new schedule job, the initial state of the field "status" of the newly added record is "Wait", the initial value of the field "owner" is null, and the initial value of the field "step" is "0", and the initial value of the field "version" is "1". The field "version" is used to control which application server executes the scheduling job. The following paragraphs describe in detail how to use this field to determine the details of the execution application server. When an application server executes work, the application server modifies the field "status" of the corresponding record to "Running", and during the execution process, the content of the field "step" can be continuously updated to Reflects the current execution progress. When a job is executed successfully, the application server modifies the field "status" of the corresponding record to "Done". When a job fails, the application server modifies the field "status" of the corresponding record to "Error". When a job needs to be re-executed for some reason, the application server will modify the field "status" of the corresponding record to "Error", and add a record in the memory list data table, in the field "id" is assigned to the new job code, while the fields "jobName", "startDateTime" and "paramters" store the value of the original record, and the field "fromId" stores the job code of the original record, indicating that this is a re-execution of a specific job Work.

The application server 212, 214 or 216 may periodically request the database server 230 to provide the content of the i closest jobs waiting to be performed so that the jobs can be performed when the scheduled time is reached in the future. 4 is a flowchart of a method for updating a work item list according to an embodiment of the present invention. The processing unit 310 of the application server 212, 214 or 216 (hereinafter referred to as the processing unit 310 for brevity) loads and executes a specific software phantom. Implemented in groups. The specific instructions are as follows:

Step S410: Receive a work item change instruction. The operator can issue a work item change command through the Man-Machine Interface (MMI) to start the application server.

Step S420 : requesting the database server 230 to provide i closest jobs to be executed, where i can be set to any integer from 4 to 6. The processing unit 310 may issue the following Structured Query Language (SQL) command to the database server 230 for requesting: select Top i * from Jobs where status='Wait' order by startDateTime

Step S430 : Store the work item list obtained from the database server 230 . The processing unit 310 can directly store the Result Set obtained from the database server 230 as a work item list, and store it in the memory 350 of the application server.

Step S440: Start the timer to count a preset time. For example, the processing unit 310 may set the preset time as any time from 30 minutes to 1 hour, so as to request the database server 230 to update the work item list after a preset time interval.

5 is a flowchart of a method for extracting and executing work according to an embodiment of the present invention. The processing unit 310 (hereinafter referred to as the processing unit 310 for brevity) of the application server 212, 214 or 216 loads and executes a specific software phantom. It is implemented during group time, and a loop (including the operations of steps S510 to S560 ) is repeatedly executed, for continuously fetching executable work from the work item list, and storing it in a queue for execution. Details are as follows:

Step S510: Determine whether there is executable work in the work item list. If yes, go to step S530; otherwise, go to step S520. The processing unit 310 may search whether there is a work in the work item list, the status of which is "Wait" and the scheduled start time "startDateTime" is earlier than the current time. If so, the job is executable.

Step S520: Wait for 1 second. In this step, the check in step S510 can be performed every at least one second.

Step S530 : requesting the database server 230 to update the record corresponding to the executable work in the work data table, so as to obtain the execution right. The processing unit 310 can provide the work identifier and the optimistic lock version number of the executable work to the database server 230, and request the database server 230 to update the record corresponding to the executable work in the work data table, so that the record of the record can be updated. Add one to the optimistic lock version number, change the executor of this record to the identifier of this server, and change the state of this record to executing, where this record needs to contain the job identifier of the executable work and the available work before updating. Optimistic lock version number of the execution work. Specifically, the processing unit 310 can issue the following SQL commands to the database server 230 for updating: Update Jobs set owner=myName, version=this.version+1, status='Run' where id=this.id & version=this.version In this SQL command, there are two conditions for a successful update, one is that the work code "id" must match, and the other is that the optimistic lock version number "version" must match before the update is performed. If there is no record meeting the update condition in the work data table, the database server 230 will reply the update failure message to the processing unit 310 . For example, when the record corresponding to the executable job in the job data table is deleted (that is, the job is removed from the schedule), the database server 230 cannot find any record that meets the update condition. Or, when the optimistic lock version number in the record corresponding to the executable work in the work data table is updated by other application servers (that is, other application servers have obtained the execution right of the executable work first), Database server 230 will not find any records that meet this update criteria. Once the update condition is satisfied, the database server 230 will update the executor "owner" in the record corresponding to the executable work to the identifier "myName" of the application server, and add the optimistic lock version number "version" to 1, and update the status "status" to "Run". It should be noted that after adding 1 to the optimistic lock version number "version" of the corresponding record, other application servers will fail if they try to update the same record with the above SQL command, thus ensuring that this work can only be performed by An application server implementation. If the update is successful, the processing unit 310 will receive the update success message from the database server 230 .

Step S540 : Determine whether the update is successful according to the message returned by the database server 230 . If yes, go to step S560; otherwise, go to step S550.

Step S550: Delete the executable work from the work item list.

Step S560: List the executable work in the queue. FIG. 6 is a schematic diagram of a queue configuration according to an embodiment of the present invention. For example, the memory 350 (hereinafter referred to as the memory 350 for brevity) of the application server (such as the application server 212, 214 or 216) may allocate space for the queue 600 to be used according to the scheduled start time Stores executable work in a list of work items in chronological order. The queue 600 includes a collection (Collection) formed by multiple items (Entry), and each item (Entry) can store the content of one job. The basic principle of the operation of the queue 600 is to add work (which can be called enqueuing) from the end position (such as the position indicated by the index T), and remove the work from the starting position (such as the position indicated by the index H) (which can be called the position). to dequeue). That is to say, the first job added to queue 600 will also be the first to be removed, which conforms to the First-In First-Out (FIFO) principle. For example, at the time point of 16:00:00, queue 600 has stored 5 jobs, which are specified to be at 15:59:05, 15:59:05, 15:59:05, and 15:59:10 respectively. and 15:59:15 start execution. Among them, the parameter fields of the first three jobs "X" contain the same group ID, representing substantially the same job. Jobs "X", "Y" and "Z" represent different jobs.

When the processing unit 310 of the application server 212, 214 or 216 (hereinafter referred to as the processing unit 310 for brevity) loads and executes a specific software module, work is sequentially extracted from the top of the queue 600 and executed. Before each job is executed, the processing unit 310 will first determine whether there is any other job in the queue 600 that is the same as the fetch job. If so, discard the job and not execute it. Specifically, the processing unit 310 can determine whether there is any job in the queue 600 , and the parameter of which includes the same group identification code as the fetched job. If so, it means that the same job exists in the main queue 600. FIG. 7 is a schematic diagram of work in the execution queue 600 according to an embodiment of the present invention. At the time point of 16:00:01, since the queue 600 contains three substantially identical jobs "X", the processing unit 310 directly takes out the first two jobs "X" from the queue 600 and discards them without executing them. Execute the third job "X". If not, perform the fetched work.

The scheduled work usually needs to be executed for a period of time, which may be as long as several hours. During the execution, the processing unit 310 can continuously request the database server 230 to update the content of the field “step” in the record corresponding to the work to reflect the current execution progress. When the execution is completed, the processing unit 310 may request the database server 230 to update the field "status" in the record corresponding to the job to "Done". When the execution fails, the processing unit 310 may request the database server 230 to update the field "status" in the record corresponding to the job to be "Error".

All or part of the steps in the method of the present invention can be implemented by a computer program, such as a computer's operating system, a specific hardware driver in the computer, or a software application. In addition, it can also be implemented in other types of programs as shown above. Those skilled in the art can write the methods of the embodiments of the present invention into computer programs, which are not described for brevity. The computer program implemented by the method according to the embodiment of the present invention can be stored in a suitable computer-readable data carrier, such as DVD, CD-ROM, USB disk, hard disk, or can be stored in a computer that can be accessed through a network (for example, the Internet route, or other suitable vehicle) to access the web server.

Although the above-described elements are included in FIG. 3 , it is not excluded that more other additional elements can be used to achieve better technical effects without departing from the spirit of the invention. In addition, although the steps in FIG. 4 to FIG. 5 are performed in the specified order, those skilled in the art can modify the order of these steps on the premise of achieving the same effect without violating the spirit of the invention. Therefore, this The invention is not limited to using only the sequence described above. In addition, those skilled in the art can also integrate several steps into one step, or in addition to these steps, perform more steps sequentially or in parallel, and the present invention is not limited thereby.

Although the present invention is described using the above embodiments, it should be noted that these descriptions are not intended to limit the present invention. On the contrary, this invention covers modifications and similar arrangements obvious to those skilled in the art. Therefore, the scope of the appended claims is to be construed in the broadest manner so as to encompass all obvious modifications and similar arrangements.

100: Internet 110: Server Cluster 112, 114, 116: Application Servers 113, 115, 117: Timer 130:Database server 20: Network System 200: Internet 210: Server Cluster 212, 214, 216: Application Servers 230:Database Server 310: Processing Unit 320: Display unit 330: Input Device 340: Storage Device 350: Memory 360: Communication Interface S410~S440: Method steps S510~S560: method steps 600: Queue X,Y,Z: work

FIG. 1 is a schematic diagram of multiple application servers requesting a database server to provide executable work, according to some embodiments. FIG. 2 is an architecture diagram of a network system according to an embodiment of the present invention. FIG. 3 is a system architecture diagram of a computing device according to an embodiment of the present invention. FIG. 4 is a flowchart of a method for updating a work item list according to an embodiment of the present invention. FIG. 5 is a flowchart of a method for extracting and executing work according to an embodiment of the present invention. FIG. 6 is a schematic diagram of a queue configuration according to an embodiment of the present invention. FIG. 7 is a schematic diagram of work in an execution queue according to an embodiment of the present invention.

S510~S560: method steps

Claims (10)

A computer program product, applied to a database server to coordinate and manage the work scheduling and execution of a parallel processing environment of multiple application servers, including programs that can be loaded and executed by a processing unit of an application server Code: Get a job from a work item list, where the work item list is periodically obtained by the application server from the database server, a status of the job is waiting, and a scheduled start time of the job earlier than the current time; providing a job identification code and an optimistic lock version number of the above-mentioned job to the database server; requesting the above-mentioned database server to update a record of a job data table for an optimistic lock of the above record Add one to the version number, change an executor of the above record to the identifier of the above application server, and change a state of the above record to running, wherein the above record must contain a job identifier of the above job before updating and the optimistic locking version number of the above-mentioned work; and when receiving the update success message from the above-mentioned database server, the above-mentioned work is listed in a queue for execution. The computer program product of claim 1, comprising program code: when receiving the update failure message from the database server, deleting the above-mentioned work from the above-mentioned work item list. The computer program product of claim 1, comprising code: extracting the above-mentioned job from the above-mentioned queue; determining whether there is any other job identical to the above-mentioned job in the above-mentioned queue; if there is any other job identical to the above-mentioned job in the above-mentioned queue for any other work, the said work shall be discarded and not performed; and If no other job identical to the above job exists in the above queue, the above job is executed. The computer program product according to claim 3, comprising code: during the execution of the above-mentioned work, continuously requesting the above-mentioned database server to update the content of a progress in the above-mentioned record of the above-mentioned work data table, which is used to reflect the current execution schedule. The computer program product according to claim 3, comprising program code: when the execution of the above-mentioned work is completed, the above-mentioned status in the above-mentioned record of the above-mentioned work data table is requested to be updated by the above-mentioned database server to be completed; and when the above-mentioned work execution fails , requesting the database server to update the above record in the above work data table with the above status as error. A device for work scheduling and execution applied in a parallel processing environment connected to a database server, comprising: a memory for allocating space to a queue and storing a work item list, wherein the work item list is The device periodically obtains from the database server; and a processing unit, coupled to the memory, for obtaining a job from the work item list, wherein a state of the job is waiting, and a row of the job The start time of the process is earlier than the current time; provide a job identification code and an optimistic lock version number of the above-mentioned job to the above-mentioned database server; An optimistic lock version number is incremented by one, an executor of the above-mentioned record is changed to the identifier of an application server executed by the above-mentioned processing unit, and a state of the above-mentioned record is changed to executing, wherein the above-mentioned record must be updated before updating. A job identifier including the above job and the above optimistic lock version number of the above job; and when the above database server is downloaded from the above When receiving a message that the update is successful, the above tasks are listed in the above queue for execution. The apparatus for job scheduling and execution in a parallel processing environment according to claim 6, wherein when the processing unit receives a message of update failure from the database server, the job is deleted from the work item list . The apparatus for job scheduling and execution in a parallel processing environment according to claim 6, wherein the processing unit extracts the job from the queue; determines whether there is any other job identical to the job in the queue; If any other job identical to the above job exists in the above queue, the above job is discarded and not executed; and if there is no other job identical to the above job in the above queue, the above job is executed. The apparatus for scheduling and executing jobs in a parallel processing environment according to claim 8, wherein the processing unit continuously requests the database server to update the records in the job data table during the execution of the job. A progress content, used to reflect the current execution progress. The apparatus for scheduling and executing jobs in a parallel processing environment according to claim 8, wherein when the job is completed, the processing unit requests the database server to update the above records in the job data table The status is completed; and when the execution of the job fails, the processing unit requests the database server to update the record in the job data table and the status is error.

Images