JPH0895810A - Batch job execution system - Google Patents

Abstract

PURPOSE: To process plural job steps simultaneously in parallel by using a buffer for data transmission on a main storage for data transmission between job steps constituting a batch job instead of a temporary file. CONSTITUTION: Instead of the common temporary file that a job step 7-1 uses for output and a job step 7-2 uses for input respectively, the FIFO buffer 5-1 for data transmission is prepared on the main storage. A step parallel execution management part 4 allocates different processes 6-1 and 6-2 to the job steps 7-1 and 7-2 and places them into parallel operation. A transmitted data management part 1 queues output data in the buffer 5-1 for data transmission in order when the process 6-1 makes a request to write the temporary file, and takes the data queued in the buffer 5-1 for data transmission out in order and sends them to the process 6-1 when the process 6-1 makes a request to read the temporary file.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a batch job execution method for a computer, and more particularly, to a plurality of job steps for transmitting data between job steps via a temporary file, instead of the temporary file The present invention relates to a batch job execution method in which a transmission buffer is used to operate concurrently in parallel.

[0002]

2. Description of the Related Art Conventionally, a batch job composed of a plurality of job steps is executed by sequentially executing the job steps constituting the batch job one by one in the order of description.

Further, in executing a batch job composed of a plurality of job steps, when data generated in a preceding job step is processed in a succeeding job step, a temporary file is used to store the data between the job steps. Communicating. As the temporary file, in addition to the file on the magnetic disk medium, a file on the electronic disk medium may be used to improve the input / output speed.

[0004]

By the way, as one of the methods for increasing the job execution speed, there is a multi-tasking method adopted in a real-time job for processing an online transaction. That is, this is a method of processing the respective transactions in parallel by separate processes. However, this multi-tasking method is rarely used for batch jobs. One of the reasons is that the batch job originally had the purpose of processing having a strong sequentiality.

However, for example, when there is a calculation process of P and a calculation process of Q, and these calculations can be performed independently of each other, one job step for performing the calculation process of P and the calculation process of Q is performed by one process. The calculation result can be obtained earlier by performing parallel processing in two processes, that is, a process for performing the calculation process P and a process for performing the calculation process Q, rather than executing the process. In other words, the multitasking method may be effective in batch jobs as well.

Similarly, when there is a job step P for performing the calculation processing and a job step Q for performing the calculation processing, the execution of the job steps is not executed sequentially, but both job steps are executed. It is also possible to allocate different processes and execute them in parallel.

However, in any of the above cases, there is no problem because calculation processing in each job step can be performed independently of each other, but a batch job in which data generated in a preceding job step is processed in a succeeding job step In the case of (1), since the conventional method uses a temporary file for data transmission, the job steps are not substantially processed in parallel.
In other words, in the temporary file, because all the data is written out due to the structure of the file and the data cannot be read until the file is completed, the preceding job step writes all the data in the temporary file. This is because the subsequent job step will start to read the job step after writing the job step, and the job steps will have to be executed sequentially.

Therefore, an object of the present invention is to provide a buffer for data transmission for storing individual data in a first-in first-out format on the main memory instead of a temporary file, so that a preceding job step and a succeeding job step are asynchronously separated from each other. The batch job execution method enables multiple job steps to be operated in parallel substantially at the same time by enabling data transmission in each data unit, and the batch job execution time can be shortened. To provide.

[0009]

In order to achieve the above-mentioned object, the present invention processes a plurality of data and sequentially outputs processed data of each data to a temporary file.
And a second job step for sequentially inputting and processing individual processed data from the temporary file.
Step parallel execution management section for allocating separate processes to the job step and the second job step to operate in parallel, and for data transmission of the first-in first-out format provided on the main memory instead of the temporary file. A buffer and a process assigned to the first job step sequentially queue output data in the data transmission buffer when a write request is made to the temporary file, and the output data is assigned to the second job step. And a transmission data management unit that sequentially takes out the data queued in the data transmission buffer and transmits the data to the process assigned to the second job step when there is a read request for the temporary file from another process. I have it.

Further, in order to be able to restore the contents of the data transmission buffer on the main memory when a failure occurs, the history of the contents of the data transmission buffer is saved in a permanent file in parallel with the operation of the process. The transmission data saving unit is provided. After the job execution is stopped due to a cause such as a failure, when the job is re-executed from the beginning of the job step that was halfway completed, the transmission data recovery unit Reads the history of the contents of the data transmission buffer corresponding to the temporary file used for output by the normally ended job step from the permanent file and stores all the data output by the normally ended job step The data transmission buffer is restored to the main memory.

[0011]

In the batch job execution method of the present invention, a buffer for data transmission of a first-in first-out format is provided on the main memory instead of the temporary file, and the step parallel execution management unit processes a plurality of data. A first job step that sequentially outputs processed data for each individual data and a second job step that sequentially processes each processed data output from the first job step are separately provided. Processes are allocated and operated in parallel, and the transmission data management unit sequentially queues output data in the data transmission buffer when there is a write request for the temporary file from the process allocated to the first job step. And a read request for the temporary file is issued from the process assigned to the second job step. Wherein the data transmission buffer retrieves the queued data sequentially transmitted to the processes assigned to the second job step when the.

[0012]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a block diagram of an embodiment of a batch job execution system according to the present invention.
Is a transmission data saving unit, 3 is a transmission data recovery unit, 4 is a step parallel execution management unit including the job management unit 11 and the process management unit 12, 5-1 to 5-3 are data transmission buffers on the main memory, and 6 -1 to 6-3 are processes, 7 is a batch job, 7-1 to 7-3 are job steps constituting the batch job 7, 8 is a permanent file on an external storage medium such as a magnetic disk medium, and 9 is data management. 10 is a restart control unit, and 13 is a non-volatile control table.
Such components are mounted on a computer equipped with a multi-process processing function, such as a multiprocessor computer, capable of simultaneously executing a plurality of processes in parallel.

FIG. 2 shows job step 7 of batch job 7.
It is explanatory drawing of the process example performed in -1 to 7-3. As shown in the figure, the job step 7-1 includes a plurality of data A1.
~ AN are sequentially processed to generate processed data B1 to BN and processed data C1 to CN for each individual data,
The former data B1 to BN are stored in the temporary file X, and the latter data C1 to CN are stored in the temporary file Y. The job step 7-2 is the data B1 from the temporary file X.
To BN are sequentially input and processed, and processed data D1 to DN are generated for each individual data and stored in the temporary file Z. In job step 7-3, processed data C1 to CN and D1 to DN are sequentially input and processed from the temporary file Y and the temporary file Z. In this embodiment, the three job steps 7-1 to 7-3 for exchanging data via the temporary files X, Y, Z are operated in parallel.

FIG. 3 shows a JCL description example of each job step 7-1 to 7-3. Since the job step 7-1 uses the temporary files X and Y, the JCL description has an assignment statement $ ASSSIGN regarding the temporary files X and Y, and the job step 7-2 uses the temporary files X and Z. The JCL description has an assignment statement $ ASSSIGN regarding temporary files X and Z. Further, since the job step 7-3 uses the temporary files Y and Z, the JCL description has an assignment statement $ ASSSIGN regarding the temporary files Y and Z. Further, job step 7-1
In the JCL description of the job step 7-1 and the job step 7-2 immediately after that, there is a statement $ PARALLEL that instructs to operate in parallel, and in the JCL description of the job step 7-2, Job step 7-
There is a statement $ PARALLEL to instruct that 2 and the job step 7-3 immediately after that be operated in parallel. Note that the statement $ PA is included in the JCL description of job step 7-3.
There is no RALLEL.

FIG. 4 is a flow chart showing an example of the processing of job step 7-1. As shown in the figure, first, the temporary files X and Y are opened (S1), and then S
Data A1 to AN shown in FIG. 2 from a YSIN file or the like
Are input one by one (S2), and predetermined data processing is performed (S4). Then, the data of the processing result is written to the temporary file X (S5), the data input in step S2 is subjected to another data processing different from step S4 (S6), and the resulting data is written to the temporary file Y. (S7). When the above is executed up to the last data AN (YES in S3), the temporary files X and Y are closed (S8), and the process is ended.

FIG. 5 is a flow chart showing an example of the processing of job step 7-2. First, the temporary file X,
Z is opened (S11), then the data B1 to BN are input one by one from the temporary file X (S12), and predetermined data processing is performed (S14). Next, the resulting data is written to the temporary file Z (S15). When the above is executed up to the last data BN of the temporary file X (Y in S13)
ES), close the temporary files X and Z (S16),
The process ends.

FIG. 6 is a flow chart showing an example of the processing of job step 7-3. First, the temporary file Y,
Z is opened (S21), and then data C1 to CN and data D1 to DN are input one by one from the temporary files Y and Z (S21).
22, S24), and predetermined data processing is performed (S25).
Then, the resulting data is output to, for example, a SYSOUT file (S26). When the above is executed up to the last data CN of the temporary file Y (YES in S23), the temporary files Y and Z are closed (S27), and the process ends.

FIG. 7 is a flowchart showing a processing example of the job management means 11 of the step parallel execution management unit 4. As shown in the figure, the job management unit 11 is configured to execute the batch job 7
The job step description (JCL description) is sequentially input from the beginning (S31), the input job step description is analyzed, and preprocessing is executed (S33). In this pretreatment,
Processing such as securing resources necessary for the job step is performed. At this time, in this embodiment, when the assignment statement $ ASSSIGN of the temporary file appears, the temporary file is not actually reserved, but a data transmission buffer is reserved on the main memory instead. Further, information indicating which data transmission buffer is assigned to which partial file is created and set in the control table 13. Next, assign a process to execute the job step,
The process management means 12 is instructed to execute the process (S34). Next, in the job step description, $
By checking whether or not the PARALLEL statement exists, it is checked whether or not parallel execution of the succeeding job step is instructed (S35), and if instructed, step S3 is performed.
Returning to step 1, the same processing as above is repeated for the next job step description.

On the other hand, when the job step description for which parallel execution is not instructed has been processed (NO in S35), the completion of the process corresponding to the job step that started the execution is waited for (S36), and either When the process ends, post-processing of job steps corresponding to the ended process is performed (S37). In this post-processing, the resources used by the job step are released. However, the release of the data transmission buffer, which is a resource shared with other job steps, is not released until the end of the job. Next, it is judged whether or not there is a running process among the processes corresponding to other job steps (S3).
8) If there is any, go to step S36 to wait for the end. If not remaining, step S31
And process the subsequent job step description.

The data transmission buffer 5 shown in FIG.
1 to 5-3 are reserved in the main memory instead of the temporary file by the above-mentioned job managing means 11, the data transmission buffer 5-1 is the temporary file X, and the data transmission buffer 5-2 is the temporary. The data transfer buffer 5-3 corresponds to the file Y, and the temporary file Z corresponds to the data transfer buffer 5-3. Such a relationship is held in the control table 13 as shown in FIG. Note that the control table 13 has a column for registering a status, and in this column, the transmission data management unit 1 sets a status such as which temporary file is being used by which process.

FIG. 9 shows data transmission buffers 5-1 to 5-
3 is a block diagram showing a configuration example of No. 3, and each of the data transmission buffers 5-1 to 5-3 has queues 511, 521 and
Has 31. When writing data to the data transmission buffer 5-1, a pointer indicating the address on the main memory of the written data is queued in the queue 511, and when reading data, the pointer is dequeued from the queue 511, and the pointer is dequeued. The data pointed to by is transmitted to the read request source. The other data transmission buffers 5-2 and 5-3 also have the same configuration.

10 to 13 are flowcharts showing an example of processing of the transmission data management unit 1. As shown in FIG. 10, when there is an open request for a temporary file from the process being executed, the transmission data management unit 1 sends the control table 1
The fact that the request source is in use is registered in the status field corresponding to the open-requested temporary file in 3 (S41). Further, as shown in FIG. 11, when there is a close request for a temporary file from the process being executed, the status column corresponding to the temporary file requested to be closed in the control table 13 indicates that the request source has finished using the file. Describe (S51).

Further, as shown in FIG. 12, when the process being executed issues a data output request for a temporary file, the transmission data management unit 1 refers to the control table 13 and transmits the data corresponding to the temporary file requested to be output. The data buffer is recognized (S61), and the data requested to be output is stored as one object in the data transmission buffer (S62). Further, the transmission data saving unit 2 of FIG. 1 is notified of history data including the data transfer buffer name of the storage destination and the stored data (S63).

As shown in FIG. 13, when the process being executed issues a data input request for a temporary file, the transmission data management unit 1 refers to the control table 13 and transmits the data corresponding to the temporary file requested. The data buffer is recognized (S71), one data is taken out from the data transmission buffer and transmitted to the request source (S7).
3). At this time, when no data is stored in the data transmission buffer (NO in S72), based on the description in the status column corresponding to the data transmission buffer in the control table 13, the process other than the input request source process is executed. It is checked whether the process (that is, the process of the data output source) has finished using the data transmission buffer (S7).
4) If the use is not completed, the process waits for a predetermined time (S75), and the process S72 is retried again. On the other hand,
If the other processes have finished using the data, it means that all the data has been transmitted, so the end of data is notified to the input request source (S76).

FIG. 14 is a flowchart showing a processing example of the transmission data saving unit 2. As shown in the figure, when the transmission data saving unit 2 is notified of the history data by the transmission data management unit 1, the transmission data saving unit 2 writes the history data in the permanent file 8 (S81). Therefore, as shown in FIG. 15, the permanent file 8 sequentially stores history data including a data transmission buffer name and data.

FIG. 16 is a flowchart showing a processing example of the restart control unit 10. As shown in the figure, the restart control unit 10 recognizes a temporary file for which a job step that has been terminated midway and that uses it for output is recognized as a normal file, and responds to it. A buffer for data transmission to be secured is secured in the main memory (S9
1) The transmission data restoration unit 3 is requested to restore the data stored in the data transmission buffer (S92). Then, the step parallel execution management unit 4 is instructed to re-execute the job from the beginning of the job step that has ended halfway (S93).

FIG. 17 is a flow chart showing a processing example of the transmission data restoration unit 3. As shown in the figure, the transmission data restoration unit 3 extracts all history data including the data transmission buffer name to be restored from the permanent file 8 (S10).
1) The data in the extracted history data is stored in the data transmission buffer to be restored (S102).

The operation of this embodiment having the above-mentioned structure will be described below.

When the job description as shown in FIG. 3 is given and the execution of the job is instructed, the job management means 11 of the step parallel execution management unit 4 starts the analysis in order from the first sentence of the job description. When the top job step description is input (S31 in FIG. 7), job step execution preprocessing is performed (S33). If the first job step is the job step 7-1, the assignment statement of the temporary files X and Y exists in the description, so that the data transmission buffer 5-1 is mainly used instead of the temporary file X. The data transmission buffer 5-2, which is secured in the memory and replaces the temporary file Y, is secured in the main memory. Further, it is registered in the control table 13 that the temporary file X corresponds to the data transmission buffer 5-1 and the temporary file Y corresponds to the data transmission buffer 5-2, as shown in FIG.
When the pre-processing for executing the job step is completed, one process is assigned to the job step 7-1,
The process management means 12 is instructed to execute the process (S34). If the process 6-1 is assigned to the job step 7-1, the job step 7-1 will be executed as the process 6-1 under the control of the process management means 12.

The job managing means 11 next checks whether or not the $ PARALLEL statement is present in the description of the job step 7-1 (S35). In the case of FIG. 3, since it exists,
The job management means 11 inputs the job step description of the next job step 7-2 (S31), analyzes it and performs preprocessing (S33). Since the temporary files X and Z are declared here, a data transmission buffer to replace them is secured in the main memory. However, since the temporary file X has already been secured as the data transmission buffer 5-1, the data transmission buffer 5-used instead of the temporary file Z is used.
Only 3 is reserved in the main memory. Further, the fact that the temporary file Z corresponds to the data transmission buffer 5-3 is set in the control table 13 as shown in FIG. Then, one process is assigned to the job step 7-2 and executed under the control of the process management means 12 (S34). Therefore, if process 6-2 is assigned to job step 7-2, job step 7-2 is assigned to process 6-
It will be executed as 2.

Next, since the $ PARALLEL statement is present in the description of the job step 7-2, the job management means 11 inputs the job step description of the next job step 7-3 (S31) and analyzes it. Pre-processing (S3
3). Although the temporary files Y and Z are declared here, the data transmission buffers 5-2 and 5 corresponding to them are given.
Since -3 has already been reserved, no new data transmission buffer is reserved. Then, in job step 7-3, 1
One process, that is, the process 6-3 is assigned and executed under the control of the process management means 12 (S34).

Next, the job management means 11 checks whether or not the $ PARALLEL statement exists in the description of the job step 7-3 (S35), but since it does not exist in the example of FIG. 3, the processes 6-1 to 6-6 are executed. -3 waits for the end of execution (S36).

As a result, the job step 7-1 for performing the processing as shown in FIG. 4, the job step 7-2 for performing the processing as shown in FIG. 5, and the job step for performing the processing as shown in FIG. Three job steps 7-3 and 7-3 will be simultaneously executed in parallel.

Now, in the process of FIG. 4 of the process 6-1 which is the execution unit of the job step 7-1, when there is an open request for the temporary files X and Y (S1), it is the OS for file processing in general. The data is transmitted to the data management unit 9, which is a functional unit. The data management unit 9 checks whether or not the file name of the open request target is registered in the control table 13, and if registered, passes control to the transmission data management unit 1. Files that are not registered are processed in the same manner as in the past. Such division is
It is also performed at the time of a close request and a data input / output request. Therefore, when an open request is issued to the temporary files X and Y, control is passed to the transmission data management unit 1, and the transmission data management unit 1 executes the processing shown in FIG. In the status column corresponding to X and Y, it is set that the process 6-1 is in use.

The processing of the process 6-1 proceeds, and the data B1 which is the processed result of the leading input data A1 shown in FIG.
When a request to output the data to the temporary file X is issued, the control is transferred to the transmission data management unit 1, and the transmission data management unit 1
As shown in step S6, the data transmission buffer 5-1 corresponding to the temporary file X is recognized from the control table 13 (S6).
1) The output requested data B1 is queued in the data transmission buffer 5-1 (S62). That is, as shown in FIG. 9, while writing the data B1 in the main memory,
The pointer indicating the address is queued in the queue 511. Then, the history data including the data transmission buffer name of the storage destination and the stored data B1 is notified to the transmission data saving unit 2. The transmission data saving unit 2 stores the notified history data in the permanent file 8 as shown in FIG. 15 (S81 in FIG. 14).

When the process 6-1 issues a request to output the data C1 obtained by processing the data A1 to the temporary file Y, the transmission data management section 1 transmits the data corresponding to the temporary file Y from the control table 13. The data buffer 5-2 is recognized, the data C1 is written in the main memory as shown in FIG.
Queue to 1. Then, history data including the data transmission buffer name of the storage destination and the stored data C1 is notified to the transmission data saving unit 2, and the transmission data saving unit 2 stores the history data in the permanent file 8.

On the other hand, when there is an open request for the temporary files X and Z in the process 6-2 operating in parallel with the process 6-1 (S11 in FIG. 5), the transmission data management section 1 causes the transmission data management unit 1 In the status column corresponding to the temporary files X and Z, the fact that the process 6-2 is in use is set. In addition, the process 6-1 proceeds,
When there is a data input request from the temporary file X (S1
2) As shown in FIG. 13, the transmission data management unit 1 refers to the control table 13 to recognize the data transmission buffer 5-1 corresponding to the temporary file X (S71), and is queued there. One of the existing data is taken out and transmitted to the process 6-2 (S73). That is, queue 5 in FIG.
One pointer is dequeued from 11, and the data B1 designated by it is transmitted to the process 6-2. here,
If the process 6-2 requests input before the process 6-1 outputs the first data B1, there is no data in the data transmission buffer 5-1. Therefore, it is determined as NO in S72 of FIG. To be done. At this time, the status column corresponding to the data transmission buffer 5-1 of the control table 13 is checked to confirm that the process 6-1 other than the requesting process 6-2 is still in use, and the predetermined value is determined. After waiting for time, the input request from process 6-2 is reprocessed. As a result, data B from process 6-1
When 1 is output, it will be transmitted to process 6-2.

Further, when the process 6-2 issues a request to output the data D1 obtained by processing the data B1 to the temporary file Z, the transmission data management unit 1 causes the data transmission buffer 5-3 corresponding to the temporary file Z. The data D1 is stored in.
Also, history data including the stored data D1 is stored in the permanent file 8 by the transmission data saving unit 2.

Further, when there is a request to open the temporary files Y and Z from the process 6-3 operating in parallel with the processes 6-1 and 6-2, there is a data input request from the temporary files Y and Z. Also in the case, the same processing as described above is performed, and the data fetched from the data transmission buffers 5-2 and 5-3 is transmitted to the process 6-3.

Since the process 6-1 has finished processing up to the last data AN shown in FIG. 2, when a close request for the temporary files X and Y is issued, the transmission data management unit 1
Registers in the status column corresponding to the temporary files X and Y in the control table 13 that the process 6-1 has finished using it.

When the process 6-2 finishes processing the last data BN shown in FIG. 2 and requests the input of the next data from the temporary file X, the transmission data management unit 1 causes the data corresponding to the temporary file X. Data is to be fetched from the transmission buffer 5-1, but since the transmission up to the last data BN has been completed, there is no data to be fetched anymore.
For this reason, the transmission data management unit 1 proceeds from S73 to S73 in FIG.
In step 74, the status column corresponding to the data transmission buffer 5-1 in the control table 13 is checked. At this point, since the description indicating that the process 6-1 has finished using is registered, the transmission data management unit 1 empties the data transmission buffer 5-1 because all the data has been transmitted. Then, the process 6-2 of the request source is notified that the data is completed (S76). Accordingly, the process 6-2 determines that the processing up to the last data has been completed (S13 in FIG. 5), requests the closing of the temporary files X and Z, and ends the processing.

The operation after the process 6-3 finishes processing the last input data CN is also the same as that of the process 6-2 described above.

As described above, three processes 6-1 to 6-3, that is, three continuous job steps are performed while asynchronously transmitting data using the data transmission buffers 5-1 to 5-3. 7-1 to 7-3 are executed in parallel.

When the batch job 7 has been normally executed up to the last job step, it ends. However, if a failure occurs during the execution of any job step and it becomes impossible to continue the process, the job execution ends abnormally. In such a case, it is necessary to re-execute the job from the beginning of the job step that has ended halfway. In this embodiment, since the data transmission buffer on the main memory is used instead of the temporary file, the contents of the data transmission buffer on the main memory are lost when restarting after a failure. However, since the transmission data history is stored in the permanent file 8 by the transmission data saving unit 2, it can be restored by the transmission data restoring unit 3. Hereinafter, this point will be described.

As a part of the restart control, the restart control section 10 checks the information of the abnormally ended job, recognizes the job step that has ended halfway, and the temporary file used for input by this job step that has ended halfway. That is, the control table 13 of FIG. 8 is searched for the data transmission buffer corresponding to the temporary file for which the job step using it for output has been completed normally. And in FIG.
As shown in FIG. 5, the data transmission buffer is secured in the main memory (S91), the data transmission buffer name is notified to the transmission data restoration unit 3, and restoration is requested (S92).
Further, a new control table 13 is created by leaving only the portion related to the secured data transmission buffer in the control table 13 and deleting the rest. Transmission data recovery unit 3
As shown in FIG. 17, all the history data including the notified data transmission buffer name is extracted from the permanent file 8 (S101), and the data in the extracted history data is requested to be restored in the main memory. The data is stored in the upper data transmission buffer (S102). After that, the restart control unit 10
Instructs the step parallel execution management unit 4 to re-execute the job from the beginning of the job step that was terminated midway (S
93).

Therefore, for example, job steps 7-1 to 7
-3 is being executed in parallel, only job step 7-1 ends normally, and job steps 7-2, 7
-3 is a temporary file that is used for input in the job steps 7-2 and 7-3 that have ended halfway and the job step that uses it for output has ended normally Is the temporary file X and the temporary file Y, the data transmission buffers 5-1 and 5-2 corresponding to them are secured in the main memory, and the data stored in the permanent file 8 by the transmission data restoration unit 3. All the history data of the transmission buffer 5-1 is extracted, the data therein is stored in the data transmission buffer 5-1, and all of the data transmission buffer 5-1 stored in the permanent file 8 is stored. The history data is extracted and the data therein is stored in the data transmission buffer 5-2.
Then, the job is re-executed from job step 7-2.

The embodiments of the present invention have been described above.
The present invention is not limited to the above embodiments, and various other additions and changes can be made. For example, although the instruction to execute the subsequent job steps in parallel is inserted as one job step statement ($ PARALLEL) in the immediately preceding job step description, it may be inserted between the job step descriptions. Also, the leading $ in the following job step description
A parameter for instructing simultaneous execution with the immediately preceding job step may be inserted in the STEP sentence.

[0049]

According to the batch job execution system of the present invention described above, the following effects can be obtained.

For data transmission between job steps, since a data transmission buffer on the main memory that can queue and dequeue individual data independently is used, the preceding job as in the case of using a temporary file is used. Each time the preceding job step outputs individual processed data, the processed job data is output after the step outputs all the data, and the subsequent job step does not have to start reading the data. Subsequent job steps can be read, allowing multiple job steps to operate substantially concurrently in parallel. As a result, the execution time of the entire batch job can be shortened.

When a data transmission buffer on the main memory is used instead of a temporary file, data loss at the occurrence of a failure becomes a problem. However, with the configuration provided with the transmission data saving unit, data is transferred in parallel with the process operation. Since the history of the contents of the transmission buffer is saved in a permanent file, data loss can be prevented. Further, in the configuration including the transmission data recovery unit, the contents of the data transmission buffer corresponding to the temporary file used for input by the job step that is terminated halfway can be restored in the main memory, and it is necessary to re-execute the job. The time can be shortened.

[Brief description of drawings]

FIG. 1 is a block diagram of one embodiment of the present invention.

FIG. 2 is an explanatory diagram of a processing example of job steps 7-1 to 7-3.

FIG. 3 is a diagram showing a JCL description example of job steps 7-1 to 7-3.

FIG. 4 is a flowchart showing an example of processing of job step 7-1.

FIG. 5 is a flowchart showing an example of processing of job step 7-2.

FIG. 6 is a flowchart showing an example of processing of job step 7-3.

FIG. 7 is a flowchart illustrating a processing example of a job management unit of a step parallel execution management unit.

FIG. 8 is a diagram showing an example of contents of a control table.

FIG. 9 is a block diagram showing a configuration example of data transmission buffers 5-1 to 5-3.

FIG. 10 is a flowchart showing a processing example when a file open request is made by the transmission data management unit.

FIG. 11 is a flowchart showing a processing example when a transmission data management unit requests a file close.

FIG. 12 is a flowchart illustrating a processing example of a transmission data management unit when a data output request is made.

FIG. 13 is a flowchart illustrating a processing example of a transmission data management unit when a data input request is made.

FIG. 14 is a flowchart illustrating a processing example of a transmission data saving unit.

FIG. 15 is a diagram showing an example of the contents of a permanent file.

FIG. 16 is a flowchart illustrating a processing example of a restart control unit.

FIG. 17 is a flowchart illustrating a processing example of a transmission data restoration unit.

[Explanation of symbols]

 1 ... Transmission data management unit 2 ... Transmission data saving unit 3 ... Transmission data restoration unit 4 ... Step parallel execution management unit 5-1 to 5-3 ... Data transmission buffer 6-1 to 6-3 ... Process 7 ... Batch job 7-1 to 7-3 ... Job step 8 ... Permanent file 9 ... Data management unit 10 ... Restart control unit 11 ... Job management unit 12 ... Process management unit 13 ... Control table

Claims (3)

[Claims] 1. A first method for processing a plurality of data and sequentially outputting processed data of each data to a temporary file.
And a second job step for sequentially inputting and processing individual processed data from the temporary file, the first job step and the second job step A step parallel execution management unit that allocates separate processes to and to operate in parallel, a first-in first-out data transmission buffer provided on the main memory instead of the temporary file, and the first job step When there is a write request for the temporary file from the allocated process, output data is sequentially queued in the data transmission buffer, and a read request for the temporary file is issued from the process allocated to the second job step. Queuing in the data transmission buffer when there is Batch job execution method, characterized in that it comprises a transmission data management unit for transmitting the processes assigned to the second job step data by sequentially taken out. 2. A transmission data saving unit for saving the history of the contents of the data transmission buffer to a permanent file in parallel with the operation of the process.
The described batch job execution method. 3. When a job is re-executed from the beginning of a job step that is terminated after execution of the job is stopped due to a failure or the like, the job step that is terminated is the temporary file used for input. Data transfer that stores all the data output by the normally completed job step by reading the history of the contents of the data transmission buffer corresponding to the temporary file used for output by the normally completed job step from the permanent file 3. The batch job execution method according to claim 2, further comprising a transmission data restoration unit that restores the buffer for main storage in the main memory.