CN113485815A - Job batch processing system, method, device, storage medium and electronic equipment - Google Patents

Job batch processing system, method, device, storage medium and electronic equipment Download PDF

Info

Publication number
CN113485815A
CN113485815A CN202110850102.2A CN202110850102A CN113485815A CN 113485815 A CN113485815 A CN 113485815A CN 202110850102 A CN202110850102 A CN 202110850102A CN 113485815 A CN113485815 A CN 113485815A
Authority
CN
China
Prior art keywords
processed
job
batch
queue
jobs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110850102.2A
Other languages
Chinese (zh)
Inventor
帅红波
谢炜琪
黄显超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bank of China Ltd
Original Assignee
Bank of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bank of China Ltd filed Critical Bank of China Ltd
Priority to CN202110850102.2A priority Critical patent/CN113485815A/en
Publication of CN113485815A publication Critical patent/CN113485815A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a job batch processing system, a method, a device, a storage medium and an electronic device, wherein the job batch processing system comprises: a batch scheduling system and a batch application system; the batch application system comprises a plurality of application ends for executing the jobs; the batch scheduling system is used for establishing a job queue to be processed, and the job queue to be processed comprises a plurality of jobs to be processed; the batch application system is used for determining a target application end capable of executing the operation; and the target application end is used for grabbing the job to be processed from the job queue to be processed. The job batch processing system automatically captures the jobs to be processed through the client, can greatly reduce the workload of the batch scheduling system, enables the batch scheduling system to only maintain the job queue to be processed, and is beneficial to improving the concurrency of the jobs to be processed.

Description

Job batch processing system, method, device, storage medium and electronic equipment
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a system, a method, an apparatus, a storage medium, and an electronic device for processing a job batch.
Background
In the interaction process of the existing batch scheduling system and the batch application system, the batch scheduling system actively pushes jobs to a batch application end, and the batch application end runs the jobs after receiving job execution requests.
For job scheduling in the push mode, the batch scheduling system needs to acquire information of the batch application system according to the routing information, select an application end to be executed from the batch application system, then send the job to the application end for processing, and wait for the return of the job execution result.
That is, all the scheduling operations of all the jobs are performed in the batch scheduling system, and when the number of concurrent scheduling operations is too large, all the job scheduling processes are processed in the scheduling stage, which is not favorable for improving the concurrency.
Disclosure of Invention
In view of the above, to solve the above problems, the present invention provides a system, a method, a device, a storage medium and an electronic device for processing job batches, wherein the technical scheme is as follows:
a job batch processing system, the job batch processing system comprising: a batch scheduling system and a batch application system; the batch application system comprises a plurality of application ends for executing the jobs;
the batch scheduling system is used for establishing a job queue to be processed, and the job queue to be processed comprises a plurality of jobs to be processed;
the batch application system is used for determining a target application end capable of executing the operation;
and the target application end is used for grabbing the job to be processed from the job queue to be processed.
Preferably, in the job batch processing system, the batch application system is further configured to feed back a processing result of the job to be processed to the batch scheduling system.
Preferably, in the job batch processing system, the batch scheduling system is further configured to determine whether a subsequent job of the processed job to be processed is placed in the job queue to be processed;
and if so, putting the subsequent operation of the processed operation to be processed into the operation queue to be processed.
A job batch processing method is applied to the batch application system; the job batch processing method comprises the following steps:
determining a target application end capable of executing the operation;
and grabbing the jobs to be processed from a job queue to be processed, wherein the job queue to be processed is established by the batch scheduling system, and the job queue to be processed comprises a plurality of jobs to be processed.
Preferably, in the above job batch processing method, the method further includes:
and feeding back the processing result of the job to be processed to the batch scheduling system.
A work batch processing apparatus, comprising:
the determining module is used for determining a target application end capable of executing the operation;
and the grabbing module is used for grabbing the jobs to be processed from a job queue to be processed, the job queue to be processed is established by the batch scheduling system, and the job queue to be processed comprises a plurality of jobs to be processed.
A job batch processing method is applied to the batch scheduling system; the job batch processing method comprises the following steps:
establishing a job queue to be processed, wherein the job queue to be processed comprises a plurality of jobs to be processed;
the to-be-processed job can be grabbed from the to-be-processed job queue by a target application end; the target application end is an application end capable of executing the operation in the batch application system.
Preferably, in the above job batch processing method, the job batch processing method further includes:
judging whether the subsequent operation of the processed operation to be processed is put into the operation queue to be processed;
and if so, putting the subsequent operation of the processed operation to be processed into the operation queue to be processed.
A computer-readable storage medium having computer-executable instructions stored therein for performing any of the above described methods of job batching.
An electronic device, the electronic device comprising: at least one processor, and at least one memory, bus connected with the processor;
the processor and the memory complete mutual communication through the bus;
the processor is configured to call the program instructions in the memory to execute any one of the above-mentioned job batch processing methods.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a job batch processing system, comprising: a batch scheduling system and a batch application system; the batch application system comprises a plurality of application ends for executing the jobs; the batch scheduling system is used for establishing a job queue to be processed, and the job queue to be processed comprises a plurality of jobs to be processed; the batch application system is used for determining a target application end capable of executing the operation; and the target application end is used for grabbing the job to be processed from the job queue to be processed.
The job batch processing system automatically captures the jobs to be processed through the client, can greatly reduce the workload of the batch scheduling system, enables the batch scheduling system to only maintain the job queue to be processed, and is beneficial to improving the concurrency of the jobs to be processed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic diagram of a job batch processing system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for batch processing jobs according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating another method for batch processing jobs according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an apparatus for batch processing of jobs according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of another job batch processing apparatus according to an embodiment of the present invention;
FIG. 6 is a flowchart illustrating a method for batch processing jobs according to another embodiment of the present invention;
FIG. 7 is a flowchart illustrating a method for batch processing jobs according to another embodiment of the present invention;
fig. 8 is a schematic diagram of a hardware architecture of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a job batch processing system according to an embodiment of the present invention.
The job batch processing system includes: a batch scheduling system 11 and a batch application system 12; the batch application system 12 includes a plurality of application terminals for executing jobs;
the batch scheduling system 11 is configured to establish a job queue to be processed, where the job queue to be processed includes a plurality of jobs to be processed;
the batch application system 12 is configured to determine a target application end capable of executing a job;
and the target application end is used for grabbing the job to be processed from the job queue to be processed.
In this embodiment, the batch scheduling system 11 and the batch application system 12 perform data interaction with each other.
Specifically, the job batch processing system receives an instruction for starting the execution of a job flow from the outside, analyzes the job flow, and places the job to be processed into a job queue to be processed.
It should be noted that all the jobs to be processed in the job queue to be processed are jobs without any preceding work or with completed preceding work.
The target application end refers to an application end which does not have a job operation, or an application end which has redundant resources and can receive the job operation again.
That is, the job batch processing system uses the idle resources of the application end, and when the application end has a demand for executing the job, the application end actively accesses the batch scheduling system to capture the job to be processed from the job queue to be processed, and processes the job.
Therefore, the workload of the batch scheduling system can be greatly reduced, the batch scheduling system only needs to maintain the queue of the jobs to be processed, and the concurrency of the jobs to be processed is favorably improved.
Optionally, in another embodiment of the present invention, the batch application system 12 is further configured to feed back a processing result of the job to be processed to the batch scheduling system.
In this embodiment, after the batch application system 12 is abnormally offline, if there is a job that does not feed back the processing result, the job result is actively queried to the application end at regular time or manually.
Optionally, in another embodiment of the present invention, the batch scheduling system 11 is further configured to determine whether a job subsequent to the job to be processed that has been processed is placed in the job queue to be processed;
and if so, putting the subsequent operation of the processed operation to be processed into the operation queue to be processed.
In this embodiment, based on the job to be processed that has been completed, a determination is made as to the job following it to update the queue of jobs to be processed.
Optionally, based on all the above embodiments of the present invention, in another embodiment of the present invention, a job batch processing method is further provided, referring to fig. 2, and fig. 2 is a schematic flow diagram of the job batch processing method according to the embodiment of the present invention.
The method is applied to the batch application system; the job batch processing method comprises the following steps:
s101: determining a target application end capable of executing the operation;
s102: and grabbing the jobs to be processed from a job queue to be processed, wherein the job queue to be processed is established by the batch scheduling system, and the job queue to be processed comprises a plurality of jobs to be processed.
In this embodiment, the target application refers to an application that has no job running, or an application that has redundant resources to receive the job running again.
That is, the job batch processing system uses the idle resources of the application end, and when the application end has a demand for executing the job, the application end actively accesses the batch scheduling system to capture the job to be processed from the job queue to be processed, and processes the job.
Therefore, the workload of the batch scheduling system can be greatly reduced, the batch scheduling system only needs to maintain the queue of the jobs to be processed, and the concurrency of the jobs to be processed is favorably improved.
Optionally, in another embodiment of the present invention, referring to fig. 3, fig. 3 is a schematic flowchart of another job batch processing method according to the embodiment of the present invention.
The method further comprises the following steps:
s103: and feeding back the processing result of the job to be processed to the batch scheduling system.
In this embodiment, after the batch application system is abnormally offline, if there is a job that does not feed back the processing result, the job result is actively queried to the application terminal at regular time or manually.
Optionally, based on the above embodiment of the present invention, in another embodiment of the present invention, a job batch processing apparatus is further provided, referring to fig. 4, and fig. 4 is a schematic structural diagram of a job batch processing apparatus according to an embodiment of the present invention.
The job batch processing apparatus includes:
a determining module 13, configured to determine a target application end that can execute a job;
and the grabbing module 14 is configured to grab the job to be processed from a job queue to be processed, where the job queue to be processed is established by the batch scheduling system, and the job queue to be processed includes a plurality of jobs to be processed.
Optionally, referring to fig. 5, fig. 5 is a schematic structural diagram of another job batch processing apparatus according to an embodiment of the present invention.
The job batch processing apparatus further includes:
and the feedback module 15 is configured to feed back the processing result of the job to be processed to the batch scheduling system.
In the embodiment, by using the idle resources of the application end, when the application end has a demand for executing the job, the application end actively accesses the batch scheduling system to capture the job to be processed from the job queue to be processed, and processes the job.
Therefore, the workload of the batch scheduling system can be greatly reduced, the batch scheduling system only needs to maintain the queue of the jobs to be processed, and the concurrency of the jobs to be processed is favorably improved.
Optionally, based on the above embodiment of the present invention, in another embodiment of the present invention, a job batch processing method is further provided, referring to fig. 6, and fig. 6 is a schematic flow diagram of another job batch processing method provided in the embodiment of the present invention.
The method is applied to the batch scheduling system; the job batch processing method comprises the following steps:
s201: establishing a job queue to be processed, wherein the job queue to be processed comprises a plurality of jobs to be processed;
the to-be-processed job can be grabbed from the to-be-processed job queue by a target application end; the target application end is an application end capable of executing the operation in the batch application system.
Optionally, referring to fig. 7, fig. 7 is a schematic flowchart of another job batch processing method according to an embodiment of the present invention.
The job batch processing method further comprises the following steps:
s202: judging whether the subsequent operation of the processed operation to be processed is put into the operation queue to be processed;
s203: and if so, putting the subsequent operation of the processed operation to be processed into the operation queue to be processed.
In this embodiment, the job batch processing system receives an instruction for starting execution of a job flow from the outside, parses the job flow, and places a job to be processed in a job queue to be processed.
It should be noted that all the jobs to be processed in the job queue to be processed are jobs without any preceding work or with completed preceding work.
Further, based on the job to be processed that has been completed, a determination is made as to which job it is to follow in order to update the queue of jobs to be processed.
Optionally, in another embodiment of the present invention, a computer-readable storage medium is further provided, where computer-executable instructions are stored in the computer-readable storage medium, and the computer-executable instructions are used to execute the job batch processing method described in the foregoing embodiment.
Optionally, in another embodiment of the present invention, an electronic device is further provided, and referring to fig. 8, fig. 8 is a schematic diagram of a hardware architecture of the electronic device according to the embodiment of the present invention.
The electronic device includes: at least one processor 16, and at least one memory 17, a bus 18, connected to the processor 16;
wherein, the processor 16 and the memory 17 complete the communication with each other through the bus 18;
the processor 16 is configured to call the program instructions in the memory 17 to execute the job batch processing method according to the above embodiment.
The present invention provides a system, a method, an apparatus, a storage medium, and an electronic device for processing job batches, which are described in detail above, and the present invention is described in the following by applying specific examples to explain the principle and the implementation of the present invention, and the description of the above embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include or include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A work batch processing system, comprising: a batch scheduling system and a batch application system; the batch application system comprises a plurality of application ends for executing the jobs;
the batch scheduling system is used for establishing a job queue to be processed, and the job queue to be processed comprises a plurality of jobs to be processed;
the batch application system is used for determining a target application end capable of executing the operation;
and the target application end is used for grabbing the job to be processed from the job queue to be processed.
2. The job batch processing system of claim 1, wherein the batch application system is further configured to feed back a processing result of the job to be processed to the batch scheduling system.
3. The batch processing system of jobs according to claim 2, wherein the batch scheduling system is further configured to determine whether a job succeeding a job to be processed that has been processed is placed in the queue of jobs to be processed;
and if so, putting the subsequent operation of the processed operation to be processed into the operation queue to be processed.
4. A job batch processing method is characterized in that the method is applied to the batch application system; the job batch processing method comprises the following steps:
determining a target application end capable of executing the operation;
and grabbing the jobs to be processed from a job queue to be processed, wherein the job queue to be processed is established by the batch scheduling system, and the job queue to be processed comprises a plurality of jobs to be processed.
5. The job batch processing method of claim 4, further comprising:
and feeding back the processing result of the job to be processed to the batch scheduling system.
6. An apparatus for processing a work batch, comprising:
the determining module is used for determining a target application end capable of executing the operation;
and the grabbing module is used for grabbing the jobs to be processed from a job queue to be processed, the job queue to be processed is established by the batch scheduling system, and the job queue to be processed comprises a plurality of jobs to be processed.
7. A job batch processing method is characterized in that the method is applied to the batch scheduling system; the job batch processing method comprises the following steps:
establishing a job queue to be processed, wherein the job queue to be processed comprises a plurality of jobs to be processed;
the to-be-processed job can be grabbed from the to-be-processed job queue by a target application end; the target application end is an application end capable of executing the operation in the batch application system.
8. The job batch processing method according to claim 7, further comprising:
judging whether the subsequent operation of the processed operation to be processed is put into the operation queue to be processed;
and if so, putting the subsequent operation of the processed operation to be processed into the operation queue to be processed.
9. A computer-readable storage medium having computer-executable instructions stored thereon for performing the method of job batch processing of any one of claims 4-5 and/or the method of job batch processing of any one of claims 7-8.
10. An electronic device, characterized in that the electronic device comprises: at least one processor, and at least one memory, bus connected with the processor;
the processor and the memory complete mutual communication through the bus;
the processor is configured to call program instructions in the memory to execute the job batch processing method of any one of claims 4 to 5, and/or the job batch processing method of any one of claims 7 to 8.
CN202110850102.2A 2021-07-27 2021-07-27 Job batch processing system, method, device, storage medium and electronic equipment Pending CN113485815A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110850102.2A CN113485815A (en) 2021-07-27 2021-07-27 Job batch processing system, method, device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110850102.2A CN113485815A (en) 2021-07-27 2021-07-27 Job batch processing system, method, device, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN113485815A true CN113485815A (en) 2021-10-08

Family

ID=77944039

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110850102.2A Pending CN113485815A (en) 2021-07-27 2021-07-27 Job batch processing system, method, device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN113485815A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102012840A (en) * 2010-12-23 2011-04-13 中国农业银行股份有限公司 Batch data scheduling method and system
CN106095585A (en) * 2016-06-22 2016-11-09 中国建设银行股份有限公司 Task requests processing method, device and enterprise information system
CN111338770A (en) * 2020-02-12 2020-06-26 咪咕文化科技有限公司 Task scheduling method, server and computer readable storage medium
CN111427694A (en) * 2020-03-26 2020-07-17 北京金山云网络技术有限公司 Task execution method, device, system and server
CN113032119A (en) * 2021-03-23 2021-06-25 北京三快在线科技有限公司 Task scheduling method and device, storage medium and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102012840A (en) * 2010-12-23 2011-04-13 中国农业银行股份有限公司 Batch data scheduling method and system
CN106095585A (en) * 2016-06-22 2016-11-09 中国建设银行股份有限公司 Task requests processing method, device and enterprise information system
CN111338770A (en) * 2020-02-12 2020-06-26 咪咕文化科技有限公司 Task scheduling method, server and computer readable storage medium
CN111427694A (en) * 2020-03-26 2020-07-17 北京金山云网络技术有限公司 Task execution method, device, system and server
CN113032119A (en) * 2021-03-23 2021-06-25 北京三快在线科技有限公司 Task scheduling method and device, storage medium and electronic equipment

Similar Documents

Publication Publication Date Title
CN107370667B (en) Multithreading parallel processing method and device, readable medium and storage controller
CN109343941B (en) Task processing method and device, electronic equipment and computer readable storage medium
CA3158476A1 (en) Method for processing response timeout, and systems for server and client to process response timeout
CN110413384B (en) Delay task processing method and device, storage medium and electronic equipment
CN106407021A (en) Queue-based business request processing method
CN105516086B (en) Method for processing business and device
CN111026541B (en) Rendering resource scheduling method, device, equipment and storage medium
CN110806928A (en) Job submitting method and system
CN108762905A (en) A kind for the treatment of method and apparatus of multitask event
CN110659131A (en) Task processing method, electronic device, computer device, and storage medium
CN112817745A (en) Task processing method and device
CN115086298A (en) File transmission method and device
CN112286688A (en) Memory management and use method, device, equipment and medium
CN107819855B (en) Message distribution method and device
CN111143210A (en) Test task scheduling method and system
CN110852603A (en) High-throughput wind control data processing method, device, equipment and storage medium
CN113485815A (en) Job batch processing system, method, device, storage medium and electronic equipment
CN113778581A (en) Page loading method, electronic equipment and storage medium
CN107247784A (en) The control method and task manager of a kind of distributed transaction
CN112486638A (en) Method, apparatus, device and storage medium for executing processing task
CN110912958A (en) HTTP connection processing method, device, equipment and medium
CN114153573A (en) Multitasking method based on single thread pool and related device
CN112783613B (en) Method and device for scheduling units
CN110008010B (en) System calling method, device, equipment and readable storage medium
CN111737036A (en) Service processing method and device based on robot process automation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination