JPH09269901A - Data transfer method between plural jobs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To transfer inter-job data where a subsequent job which receives all generation data of a preceding job and a subsequent job which receives only a part of data coexist. SOLUTION: An open processing execution part 3 stores information on the respective jobs of a processing requesting source in a table 9 as a common information table and an individual information table for the respective jobs, identifies whether the job is that for receiving all data or that receiving only a part of data in the case of the job on a side for receiving data, and records an identification result in the table 9. A job synchronism control part 5 refers to the table 9, writes data into a buffer area 7 with the processing request of the preceding job. It unconditionally transfers data in the buffer area 7 to the subsequent job if the job is that for receiving all data and transfers only data satisfying the condition to the subsequent job among data in the buffer area 7 at the time of the processing request of the subsequent job.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of inheriting data between jobs, and more particularly, in a computer system used in a bank or the like, inheriting data from a job executing online processing to a job executing batch processing. Regarding processing method.

[0002]

BACKGROUND OF THE INVENTION The present invention relates to a process for taking over data between jobs. A job is a unit of processing in a computer system and is composed of one or more programs. When a user requests a job to be executed, each program is executed sequentially or in parallel.
For the sake of simplicity, hereinafter, in the data transfer process, the job that generates data will be referred to as a “preceding job”, and the job that receives the data will be referred to as a “successor job”. When there are a plurality of preceding jobs, all or some of them may be collectively referred to as a “preceding job group”. Similarly, the plurality of subsequent jobs are collectively referred to as a "successor job group".

In a computer system used in a bank or the like, when it is necessary to perform a predetermined task, the task is often executed not by a single program but by a combination of a plurality of programs. At this time,
One job is completed by linking multiple jobs. This processing mode can minimize the influence of an abnormality when it occurs, and is highly maintainable. Also,
Even when changing the work contents accompanying a new service, it is possible to add jobs to execute new work between existing jobs instead of recreating the entire existing program. . However, the operation mode of the system in which a plurality of jobs are combined involves data transfer processing between jobs because it is necessary to cooperate with different jobs.

When it is necessary to perform data transfer processing between jobs, most commonly, the preceding job stores all the data in the external storage device in the form of a file, and after the file generation is completed, the succeeding job is executed. The method is to start reading files sequentially. On the other hand, a so-called "pipeline method" described in Japanese Patent Laid-Open No. 3-46033 is used when it is desired to execute the data transfer processing at a higher speed. The present technology permits the preceding job and the succeeding job to access the files to be handed over at the same time, and guarantees the passing of jobs between records (components of the file). In other words, when the preceding job generates one record and then the succeeding job issues a read request for this record, this record is immediately transferred to the succeeding job, and the following records are also passed between jobs in the same way. To go. As a result, it is possible to prevent the start of processing of the succeeding job group from being waited until all the data is generated, and it is possible to execute all the job groups that perform the data transfer processing in parallel, thus reducing the processing time required for the transfer processing. be able to.

Further, in the pipeline method, data is transferred between jobs via a buffer area prepared on the main storage device or an auxiliary storage device corresponding to the storage device, not on the external storage device. By using the main storage device and the auxiliary storage device that can transfer data at a higher speed than the external storage device, the data transfer processing time can be further shortened. However, the main storage device has a smaller data storage capacity than the external storage device, and it is generally impossible to store a huge amount of data that can be handed over between jobs in a computer system such as a bank in the buffer area. is there. Therefore, in this pipeline method, data that has been transferred is sequentially deleted from the buffer area, and that area is reused for storing new data. Therefore, this pipeline method is applied when the data writing order of the preceding job and the data reading order of the succeeding job are the same.

[0006] The pipeline method was originally applied to the data transfer processing from one preceding job to one succeeding job. However, thereafter, in order to further reduce the processing time, a method of increasing the number of jobs to be applied has been proposed. As a result, when there are multiple successor jobs requesting the data generated by the preceding job, the processing to pass the data to all of the succeeding jobs at the same time and the processing speed of the succeeding jobs are faster than the preceding job. If the job is slow and slow, prepare a copy job that operates exactly the same as the succeeding job and divide the data into the succeeding job and the copy job to pass the data to shorten the transfer time. Is.

[0007]

The operation of financial computer systems such as banks is divided into online processing during the day and batch processing during the night. The online processing is to process user requests such as cash withdrawal and money transfer, while the batch processing is mainly to summarize the processing contents of the daytime online processing. Batch processing is often started after waiting for the end of online processing, but at present, there is an increasing demand for non-stop operation (24-hour operation) of online processing operations of banks.
In such an operation, the execution history data must be sequentially passed to the job that executes the batch processing during the execution of the online processing. At this time, in order to prevent a delay in online processing, it is desirable to use the above-mentioned pipeline method for data transfer between online processing and batch processing.

Further, in order to execute batch processing at high speed,
A method may be used in which the execution history data of online processing is divided into several groups, and batch processing (aggregation processing) is partially executed for each of the groups.
In the operation mode of such batch processing, when data is sequentially transferred using the pipeline method from online processing to batch processing as described above, if data distribution to each batch processing can be realized at the time of transfer, more processing will be performed. The time can be shortened.

On the other hand, financial systems such as banks require particularly high reliability. Therefore, in some cases, the online processing execution history data
You may need to save. However, as shown above, considering the case where data is transferred using a pipeline method from online processing to batch processing, and when data is divided and transferred to multiple batch processing jobs, each batch processing It is necessary to transfer all the data to the external storage device at a time separately from the data division transfer to the job. However, such processing cannot be realized by the conventional pipeline method.

Therefore, when it is necessary to divide the data into a plurality of batch processes and transfer the data and to store the data to be transferred in a batch, the data transfer between jobs using a high-speed pipeline method is applied. I couldn't. In this case, the data to be delivered must be temporarily stored in the external storage device, and after all data has been generated, the data must be distributed to each batch process. Alternatively, you can use a pipeline method to pass data to each batch process, and then save the data individually in each batch process. In this case, save the online execution history data after dividing it. And then
Reliability is severely compromised. Further, depending on the contents of the division, it may not be possible to restore the original execution history data by saving the delivery data for each batch process.

Further, the use of online processing has been increasing year by year, and in order to cope with this, a computer system in which a plurality of computers for online processing are prepared and the load is distributed has appeared. In such a computer system, it is necessary to combine the execution history data of each online computer into one before passing the execution history data of the online processing to the batch processing side. If the operation mode is such that the batch processing starts after the completion of the online processing, it is possible to collect the execution history data of each online processing at the start of the execution of the batch processing. However, when directly transferring data from online processing to batch processing using the pipeline method, after receiving data from multiple jobs (online processing side), these data are set according to a certain standard. In comparison with the pipeline method, complicated processing such as dividing the data into multiple jobs (batch processing side) according to another standard and passing it at the same time, and saving all the data to the external storage device at once Required. The conventional pipeline method does not have a function for realizing such processing.

As described above, particularly in the non-stop operation (24-hour operation) of bank online processing that may occur in the future, a pipeline method is widely used for the processing of transferring data from a plurality of online processing jobs to a plurality of batch processing jobs. To be applicable is an issue.

[0013]

In order to solve the above problems, the present invention generates data when one or more jobs need to receive one or more data generated by one or more jobs. Jobs that receive data, a function that executes all jobs that receive data in parallel, and a job that receives data from jobs that generate data via a buffer area prepared on the main storage device or an auxiliary storage device that is similar to that To classify the data generated by all jobs on the data generating side into one or more groups in the computer system that has the function of sequentially passing the generated data even if the data generation is not completed. The classification standard is set in advance, all jobs on the data receiving side are classified into Jobs that receive only data that belong to one of the classified groups are defined, job control statements for each job are defined according to the classification, and the data generated by each job on the data generating side is stored in the buffer. A job that stores data in a region and receives data from a job that receives the data, is a job that receives all the data, or a job that receives only data that belongs to any one of the groups. If it is determined that the job that receives the data is a job that receives all data as a result of the determination, all data that has been generated via the buffer area for the job is As a result of the above-mentioned determination, the job receiving the data receives only the data belonging to any group. If it is determined that the job that has been so passed against the job only data of the group receiving the said job via the buffer area.

Further, in the definition of the job control statement of each job, whether the job receiving data is a job receiving all data or a job receiving only data belonging to any group. Is specified in the job control statement of the job, and when the job that receives the data is specified as a job that receives only data belonging to any group, the job receives it. The type of group is also described in the job control statement of the job.

Further, in the job control statement of the data generating job, the number of jobs generating data, the number of jobs receiving all data, and the number of jobs receiving only data belonging to a specific group are specified. The number of jobs that have been accessed for the files to be inherited that can be seen from the job that generates data (in the buffer area) is defined as the job that generates data, the job that receives all data, and the data that belongs to a specific group. Are recorded separately for each job that receives, and each time a new job accesses the file to be inherited,
It is determined whether the job is a job that generates data, a job that receives all data, or a job that receives only data belonging to a specific group, and in the determination, the newly accessed job is If it is determined that the job is a data generating job, the number of jobs that generate the data described in the job control statement of the data generating job and the recorded accessed data are compared. When the number of jobs to be generated is compared and the number of jobs to generate the accessed data is smaller than the number of jobs to generate the data described in the job control statement, access of the new job is permitted and the accessed data is When 1 is added to the number of jobs to be generated and when they are equal, access to the new job is denied, the same job is abnormally terminated, When it is determined that the job that newly accessed the file to be connected is a job that receives all data, the number of jobs that receive all the data described in the job control statement of the job that generates the data and , Comparing the number of jobs receiving all the recorded accessed data, and accessing the new job when the number of jobs receiving all accessed data is smaller than the number of jobs receiving all the data described in the job control statement. Is permitted, and 1 is added to the number of jobs that generate all accessed data. When they are equal, the access of the new job is denied, the same job is abnormally terminated, and in the determination, for the file to be taken over. If it is determined that the newly accessed job is a job that receives only data belonging to a specific group, Compare the number of jobs that receive only the data belonging to the specific group described in the job control statement of the job that generates the data and the number of jobs that receive only the data that belongs to the recorded and accessed specific group, When the number of jobs that receive only the data that belongs to the accessed specific group is smaller than the number of jobs that receive only the data that belongs to the specific group described in the job control statement, the access of the new job is permitted and the accessed specific 1 is added to the number of jobs that generate only data belonging to the group, and when they are equal, access to the new job is denied and the job is abnormally terminated.

Further, when one or more jobs need to receive one or more data generated by one or more jobs, a function for executing all the jobs for generating data and the jobs for receiving data in parallel. , And through the buffer area prepared on the main memory device or its equivalent auxiliary memory device, to the job that receives the data from the job that generates the data, even if all the data generation is not completed In a computer system having a function of executing processing, all jobs that receive data are classified into jobs that receive all data and jobs that receive only part of the data, and the job control statements for each job are classified according to the classification. Defined in the job control statement of the job that receives only some data. Stores data generated by each job on the side that generates data in the buffer area,
When a data reception request is made by the job receiving the data, it is determined whether the job is a job that receives all the data or a job that receives only the part of the data. As a result of the determination, when it is determined that the job that receives the data is a job that receives all the data, all the generated data is passed to the job via the buffer area,
As a result of the determination, when it is determined that the job receiving the data is a job receiving only the partial data, only the data satisfying the condition of the received data is sent to the job via the buffer area. I will try to deliver it.

Further, in the definition of the job control statement of each job, it is specified whether the job receiving the data is a job receiving all the data or a job receiving only a part of the data. , In the job control statement of the job, and when the job receiving the data is designated as a job that receives only a part of the data, the condition of the data received by the job is
It is described in the job control statement of the job.

Further, in the job control statement of the job for generating data, the number of jobs for generating data, the number of jobs for receiving all data, and the number of jobs for receiving only some data are described. , The number of jobs that have been accessed for the files to be taken over that can be seen from the job that generates data (in the buffer area), the job that generates data, the job that receives all data, and the job that receives only some data Data is recorded separately, and each time a new job accesses the file to be inherited, the job is a job that generates data, a job that receives all data, or data that belongs to a specific group. It is determined whether the job is a job that receives only, and in the determination, the newly accessed job generates data. If it is determined that the job is a job of the data generating side, the number of jobs that generate the data described in the job control statement of the job that generates the data and the job that generates the recorded accessed data. The number of jobs that generate the accessed data is smaller than the number of jobs that generate the data described in the job control statement, the new job is permitted to access and the accessed data is generated. When the number is equal to 1, and when they are equal, access to the new job is denied, the job is abnormally terminated, and in the determination, the job newly accessed to the takeover target file is a job that receives all data. If it is determined that the number of jobs that receive all the data described in the job control statement of the job that generates the data, Compare the number of jobs that receive all the accessed data, and when the number of jobs that receive all the accessed data is less than the number of jobs that receive all the data described in the job control statement, permit access to the new job, When 1 is added to the number of jobs that generate all the accessed data, the access of the new job is denied, the same job is abnormally terminated, and the transfer target file is newly accessed in the determination. If it is determined that the job is a job that receives only a part of the data, the number of jobs that receives only the data that belongs to the specific group described in the job control statement of the job that generates the data, and the record Compare the number of jobs that receive only some accessed data and receive only some accessed data If the number of jobs to be received is less than the number of jobs that receive only part of the data described in the job control statement, access to the new job is permitted, and 1 is added to the number of jobs that receive only part of the accessed data. , And when they are equal, the access of the new job is denied and the same job is abnormally terminated.

Further, in a computer system in which one or more computers can cooperate with each other via a control computer and control the processing as one computer as a whole, they are executed in different computers. One or more data generated by one or more jobs, through the control computer, a process of delivering to one or more jobs executed in a computer different from the computer that executes those jobs, Any of the above-described data transfer methods between a plurality of jobs is applied to the process of transferring a job via the control computer.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a diagram illustrating the principle of the present invention.
In FIG. 1, reference numeral 1 denotes a pipeline execution control unit, which includes a process reception unit 2, an open process execution unit 3, a close process execution unit 4, a job synchronization control unit 5, a data input / output control unit 6, a buffer area 7, and a pipeline. The information storage area 8 is included. The relationship and operation of each will be described in detail later. Reference numeral 9 denotes a pipeline information storage table, which is a storage area for storing various types of information necessary for normally executing the data transfer processing, and is created in the pipeline information storage area 8 described above. The detailed configuration of the table 9 will be described later with reference to FIG. Reference numeral 10 denotes one or more preceding jobs, which is a group of jobs on the side that creates takeover data among jobs that execute data takeover processing between jobs using the present invention. Reference numeral 11 denotes one or more succeeding jobs, which is a group of jobs on the side that receives the inherited data among the jobs that execute the data inheritance process between jobs using the present invention.

The processing accepting unit 2 accepts processing requests from the preceding job 10 and the succeeding job 11. There are four types of processing requests from each job to the pipeline execution control unit 1, namely, file open processing request, file close processing request, data write (write, get, etc.) processing request, and data read (read, put, etc.) processing request. is there. Upon receiving the processing request, the processing analysis unit 2 analyzes which of the above four types of contents the request is, and if it is a file open processing request, calls the open processing execution unit 3 and similarly performs the file close processing. If it is a request, the close process execution unit 4 is called, and if it is a data write process request and a data read process request, the job synchronization control unit 5 is called.

The open process execution unit 3 executes preparation for taking over, such as newly creating the pipeline information storage table 9 and the buffer area 7 used for taking over data. The processing contents of the open processing execution unit 3 will be described later in detail with reference to FIGS. 6 to 10.

The close processing execution unit 4 executes post processing associated with the completion of data transfer. In particular, the most important processing content is to erase the table 9 and the buffer area 7 used for the data transfer processing when all the jobs that have executed the data transfer processing issue the file close processing requests. However, the processing of the close processing execution unit 4 can be realized by using a known technique, and detailed description thereof will be omitted.

The job synchronization control unit 5 determines whether the processing request is a data writing processing request or a data reading processing request, and executes processing corresponding to each. In general, each job that inherits data is executed independently of each other, and therefore the timing of issuing each data processing request varies. Even in such a situation, in order to transfer the data normally, processing such as temporarily setting the job execution to a waiting state (waiting state) and canceling the waiting state is necessary. Will be needed. The job synchronization control unit 5 executes such a synchronization process between jobs. Details will be described later with reference to FIGS. 11 and 12.

The data input / output control unit 6 is called when the job synchronization control unit 5 determines that data transfer is possible in response to a data writing process request or a data reading process request from each job. When the requested content is a data write processing request, the called data input / output control unit 6 transfers the data from the data storage area (not shown) in the preceding job 10 to the buffer area 7 in the pipeline execution control unit 1. . Conversely, when the request content is a data read processing request, the data is transferred from the buffer area 7 to the data storage area (not shown) in the succeeding job 11. As described above, the buffer area 7 is an area for temporarily storing the data transferred between jobs.

FIG. 2 shows the system configuration of the first embodiment. In this embodiment, a total of six jobs in the computer 201 use the pipeline execution control unit 1 to perform data transfer processing. Specifically, the first preceding job 202 and the second preceding job 203 independently generate takeover data. The pipeline execution control unit 1 receives the inherited data from these two jobs in the order of write requests, and passes it to the succeeding job group. Of the four succeeding job groups, the first succeeding job 204 and the second succeeding job 204
The succeeding job 205 and the third succeeding job 206 extract the necessary data from the inherited data and receive them. Specific extraction criteria in that case will be described later with reference to FIG. 4th remaining
The subsequent job 207 receives all the data to be inherited. As described above, in the first embodiment, all jobs having a data transfer relationship are executed in parallel, and the data strings generated by the plurality of preceding jobs are divided and passed to some of the succeeding jobs. The process of passing all data to the succeeding job is executed simultaneously and in units of one record.

FIG. 3 shows the structure of a record transferred between jobs in the first embodiment. The present embodiment is premised on handling execution history information that occurs in banking operations. Therefore, this transfer record also includes typical execution history information of banking operations. In the transfer record of FIG. 3, the processing start time 301 is a field for storing the time when the processing request from the user is generated. The account number 302 is a field that stores the account number of the processing request target. The branch name 303 is a field that stores the name of the branch where the processing target account exists. The area name 304 is a field that stores the name of the area to which the branch office belongs. Here, it is assumed that the Bank manages all branches in several regions.
Batch processing is often divided into such regions and aggregated. In such a case, each record is classified based on the area name 304. Next, the usage name 305 is a field for storing the name of the user corresponding to the account number 302. Processing type 306
Is a field for storing the type of the target processing. Here, the type of processing is the type of online service prepared by the bank, and generally includes types such as “cash withdrawal” and “cash transfer”. Processing type 30
In 6, a character string or a code corresponding to each predetermined process is stored. Next, processing amount 307
Is a field for storing the amount of money to be processed online. The account balance 308 is a field for storing the changed balance of the account indicated by the account number 302 as a result of the processing. In this embodiment, each field is defined to have a length of 8 bytes. However, the length is not particularly limited, and the optimum length may be determined according to the contents of each system. Also, regarding the configuration of the table, new fields may be added or unnecessary fields may be deleted as necessary.

FIG. 4 shows job control statements for each job in the first embodiment. The job control statement is a character string describing various information necessary for executing a job, such as specifying a program to be executed and a file to be accessed when executing the job in the computer system. In FIG. 4, a job control statement 401 is a job control statement defining the first preceding job 202 in the present embodiment shown in FIG.
Similarly, in the following, 402 is the second preceding job 203, 403.
Are job control statements defining first succeeding jobs 204, 404, second succeeding jobs 205, 405, third succeeding jobs 206, 406, and fourth succeeding job 207, respectively. The job control statement itself is a known technique, and detailed description thereof will be omitted. Here, only the part related to the present invention will be mainly described.

In the job control statement, the character string "JOB"
The line including "" means the beginning of each job control statement, and the character string (OUTJOB1, OUTJOB2, etc. in FIG. 4) described before the same character string "JOB" is called the job name. The line containing the character string "EXEC" is a place to specify the program to be executed in each job, and the character string (STEP1 in FIG. 4) described before the character string "EXEC" is used as the job step name. Call. The program executed at each job step is the character string "PROGRA
It is specified by describing the program name after “M =”. In the job control statement, the character string "FD"
A line including a line and a group of lines continued by the line and a comma are a part defining a file accessed by the job defined in this job control statement, and are called a file definition statement. In the file definition statement, the character string "FD"
The character string written immediately before is a file identifier. Inside the program, the file to be accessed (the file that can be seen from the preceding job (in the buffer area))
Is specified by a temporary file name instead of the actual file name. The temporary file name is the file identifier. By adopting such a method, the operation of the system is made flexible. Furthermore, in the file definition statement, the file name, “DE
The file storage medium type (magnetic disk device, magnetic tape device, etc.) is designated by "VICE". In this embodiment, when data is taken over by using the pipeline method, "DEVICE = PIPELINE" is designated. At this time, an inter-job data transfer relationship is established with a job for which the same file name is specified.

In this embodiment, the file identifier is set to "OU" when a data write request is made from the program.
The file identifier is defined to start with "T" and start with "IN" when a data read request is made. That is, in FIG. 4, it can be recognized that the jobs “OUTJOB1” and “OUTJOB2” request writing of data to the file “FILE.PIPE.X”, and thus become a preceding job in the data transfer processing. . Conversely, "INJOB1" and "INJOB2"
The jobs “INJOB3” and “INJOB4” request reading of data from the same file, and become a succeeding job in the data transfer processing.

Further, in this embodiment, the number of jobs to be taken over is defined by the parameter "PIPENO" in the file definition statement for the preceding job. This parameter has a format in which the first, second, and third parameters are enclosed in parentheses. In order from the first parameter, "the number of preceding jobs", "the number of subsequent jobs received by dividing the data", and "all data" Specify the number of subsequent jobs that receive In the job control statements 401 and 402, “PIPENO = (2,
3, 1) ”is specified, in this embodiment, the number of preceding jobs is 2, the number of succeeding jobs received by dividing the data is 3, and the number of succeeding jobs receiving all the data is 1. It can be recognized as a configuration.

Further, in the present embodiment, "PIPESLCT" (PI
The abbreviation system parameter of PESELECT defines whether the succeeding job is a job that receives divided data or a job that receives all data. This parameter has a format in which four parameters, the first, second, third, and fourth parameters are enclosed in parentheses, and "YES" is set in the first parameter.
When it is specified, it means that it is a succeeding job that receives the divided data, and when "NO" is specified in the same parameter, it means that it is a succeeding job that receives all the data. The second parameter to the fourth parameter specify the condition of the data received by the job only when the data is divided and received. Therefore, when "NO" is designated in the first parameter, the designations of the second and subsequent parameters are omitted or the designations are ignored.

In the present embodiment, as a partial data receiving instruction, it is determined whether a specific portion (field) of the inherited record is equal to a specific value (key) designated separately. However, you can specify that you want to receive data only when both are the same. Specifically, the second parameter specifies from which position (offset) of the inherited record should be the field for condition determination, and the third parameter specifies the length of the field.
In both cases, the unit is bytes. The fourth parameter specifies the key to be compared with the field. For example, in the job control statement 403, it is 2
8 bytes starting from the 4th byte is the character string "XXXXXXX
It specifies that the job receives data only when it is equal to “XXX”. Similarly, the job control statement 404,
In 405, it is specified to receive data only when the field at the same position is equal to the character strings “YYYYYYYY” and “ZZZZZZZZ”, respectively.

When the offset is designated, the beginning of the record is set to 0.
Is calculated as That is, the area having a length of 8 bytes from the 24th byte from the beginning is, as shown in FIG.
04 ”is instructed. Here, in this embodiment, all the branches of the bank are “XXXXXXXXX” and “YYYYYY”.
It is assumed that the area is divided into three areas of "YYY" and "ZZZZZZZZ". Therefore, all the inherited data must be stored in only one of the first succeeding job 204, the second succeeding job 205, and the third succeeding job 206 defined by 403, 404, and 405 in FIG. Transferred. That is, when the data passed to the three succeeding jobs are recombined, they all match. As described above, in this embodiment, the data strings generated by the first preceding job 202 and the second preceding job 203 are merged in time series, and the first succeeding job 204, the second succeeding job 205, and the second succeeding job 205 It can be seen that processing is performed such that all the data is delivered to the fourth succeeding job 207 at the same time as being divided into three and delivered to the third succeeding job 206.

Next, FIG. 5 shows the configuration of the pipeline information storage table 9 for storing the information used for normally performing the data takeover process. The table 9 is created one by one for each data takeover process (for each file viewed from the preceding job),
Common information table 501 and individual information table 5 respectively
02. The common information table 501 is composed of a plurality of fields storing common information commonly referred to by each job during the data transfer process, and only one exists in one pipeline information storage table 9.
On the other hand, the individual information table 502 is made up of a plurality of fields storing information unique to each job for which data transfer processing is performed, and exists in one pipeline information storage table 9 for the number of jobs for which processing requests have been made. .

Of the components of the common information table 501, the file name field 503 stores the file name of the data transfer target file. The buffer address field 504 stores the start address of the buffer used as a temporary data storage area during the data transfer process. In the buffer block number field 505, the number of blocks in the secured buffer area (the number of records that can be stored) is stored. Number of preceding jobs field 50
In 6, the number of preceding jobs designated by the PIPENO first parameter of the job control statement is stored. Similarly, the first succeeding job number field 507 stores the number of all the data transfer target succeeding jobs designated by the PIPENO third parameter, and the second succeeding job number field 508 stores the PIP
The number of succeeding jobs for divided data transfer designated by the ENO second parameter is stored. The opened preceding job number counter 509 stores the number of preceding jobs for which an open command has been issued to the file to be taken over at an arbitrary time. Similarly, the opened first succeeding job counter 510 stores the number of all data transfer succeeding jobs for which an open command has been issued at an arbitrary time, and the opened second succeeding job counter 511 stores It stores the number of succeeding jobs for divided data transfer for which an open command has been issued at an arbitrary time. Shared output counter 512
Stores the cumulative value of the number of transferred data from all the preceding jobs, and similarly stores the cumulative value of the transferred data of the input job in the shared input counter 513.

Further, the next table pointer 514 stores the head address of another pipeline information storage table 9 created immediately after the pipeline information storage table 9 is created. When each component of the pipeline execution control unit 1 needs to extract a predetermined pipeline information storage table, all the tables can be searched by sequentially tracing the chain by the pointer 514. It should be noted that 0 is stored in the next table pointer 514 of the pipeline information storage table 9 created most recently. The succeeding job table pointer 515 and the preceding job table pointer 516 store the start address of a predetermined individual information table 502, respectively. Here, the individual information table 502 corresponding to each job for which a processing request is made is created in one pipeline information storage table 9. Of these, in the individual information table 502 corresponding to the preceding job, the head address of the table corresponding to the job that issued the earliest open command is stored in the preceding job table pointer 516. Similarly, in the succeeding job table pointer 515, the head address of the table corresponding to the job that issued the earliest open command in the individual information table 502 corresponding to the succeeding job is stored.

Next, each field constituting the individual information table 502 will be described. Same table 502
Among them, the job name field 517 stores the job name defined in the job control statement and is used as an identifier in table search or the like. Open flag 518
In the table, information on whether the job corresponding to the table 502 has issued an open command to the takeover applicable file is stored. Similarly, the close flag 519 stores information as to whether or not the job corresponding to the table 502 has already issued the close command. Flags are set when the open command is issued and when the close command is issued. The wait flag 520 stores information indicating whether or not the job corresponding to the table is in the execution waiting state. The wait flag 520 is set to ON immediately before each job issues a wait instruction during the data transfer processing, and is set to OFF when the wait is released.

In the pipeline type field 521,
At the time of data transfer processing, the position of the job corresponding to the table 502 is stored. For the pipeline type, any one of three types of “preceding job”, “all data transfer succeeding job”, and “divided data transfer succeeding job” is selected. The meaning of each type is as described above.
The individual counter 522 stores a cumulative value of the number of pieces of data that have been processed in each job. Transfer condition field 52
In 3, the reception condition at the time of divided transfer designated by the second parameter after “PIPESLCT” of the job control statement is stored. This transfer condition field is used only for jobs requesting divided data transfer. In the table for the job requesting the transfer, the same field 5
23 exists. The next table pointer 524 in the individual information table 502 stores the head address of another individual information table 502 created immediately after the creation of the individual information table 502. However, the individual information table 502 corresponding to the preceding job and the individual information table 502 corresponding to the succeeding job are distinguished from each other, and the individual information pointed to by either the preceding job table pointer 516 or the succeeding job table pointer 515, respectively. Allocate to the table 502 group. Both the preceding job and the succeeding job
Individual information table 50 that issued the most recent open command
0 is stored in the next table pointer 524 of 2. The above is a description of the configuration of the pipeline information storage table 9. The usage of each field and the timing of content update will be referred to in the following description.

As described above, the process accepting unit 2 that accepts the process request from each job is either the open process executing unit 3, the close process executing unit 4, or the job synchronization control unit 5 according to the content of the process request. Call one. Among these, the processing content of the close processing execution unit 4 is described in
This is based on the known technology described in Japanese Patent Publication No. -46033. The processing contents of the remaining open processing execution unit 3 and job synchronization control unit 5 will be described below in order.

FIG. 6 is a processing flow chart of the open processing execution unit 3. As described above, the open process executing unit 3 is called by the process accepting unit 2 when the process request sent from the job to the pipeline executing unit 1 is the open process request. The called open process execution unit 3 first has a pipeline information storage table 9 having the same file name field 503 as the file requested to be processed in the pipeline information storage area 8.
It is determined whether or not exists (step 601). This determination is continued until the next table pointer 514 of each pipeline information storage table 9 is sequentially searched, a predetermined table is found, or the end of the chain is reached, and the search of all tables is completed.

If it is determined in step 601 that there is no table satisfying the conditions, the common information table 501 is first created from the pipeline information storage table 9 corresponding to the processing request target file. Details of creating the table will be described later with reference to FIG. Subsequent to the creation of the table, the start address of the newly created table is stored in the next table pointer 514 of the final table 9 connected by the respective next table pointers 514 in the existing pipeline information storage table 9 (step 602). ) Do.

Whether the common information table 501 is newly created or not, the access type to the file for which the processing request is made is subsequently determined (step 604).
Here, the access type refers to the difference between a request to write data to a file and a request to read data. Such designation is usually given in the program or in a job control statement. In the present embodiment, as described above, in the job control statement, if the file identifier of the target file starts with "OUT", the data writing is "IN".
If it starts with, it is defined that data reading is requested. Therefore, the determination in step 604 can be realized by referring to the file identifier in the job control statement. If it is determined in step 604 that a data write request has been made to the target file, the preceding job open process is continuously executed (step 605), and if a data read request has been made, a subsequent job open process is performed. Is executed (step 606). Details of each processing will be described with reference to FIGS. 7 and 8.

First, FIG. 7 shows a process flowchart of the preceding job open process (step 605) in the open process execution unit 3. First, the opened preceding job number counter 509 and the preceding job number field 506 in the pipeline information storage table 9 corresponding to the processing target file that has already been searched or newly created in the previous step are compared (step 701). If the former has reached the latter, it means that a predetermined number of preceding jobs are already being processed, and therefore the job requested this time is abnormally terminated (step 702).
If it is determined in step 701 that the opened preceding job number counter 509 is smaller than the preceding job number field 506, the processing request from the current job is accepted and the following processing is continued. First, the individual information table 502 corresponding to the same job is newly created in the pipeline information storage table 9 (step 703). Detailed processing contents of the new creation of the table will be described later with reference to FIG.

After creating the new individual information table 502, a table chain for the table is created. At this time, it is judged whether or not the value of the opened preceding job number counter 509 is 0 (step 704), and if the counter 509 is not 0, the final table 5 of the table chain for the individual information table 502 for the existing preceding job.
02, the start address of the newly created individual information table 502 is stored in the next table pointer 524 (step 70).
5) Do it. When the value of the opened preceding job number counter 509 is 0 in the determination of step 705, the existing individual information table 502 for the preceding job does not exist, and the newly created table becomes the head of the table chain for the preceding job. That is, the leading address of the newly created individual information table 502 is stored in the preceding job table pointer 516 in the common information table 501 (step 7).
06) After creating the table chain, the counter 509 for the number of opened preceding jobs in the common information table 501
Is incremented by 1 (step 707) to complete the preceding job open process.

Next, a processing flowchart of the succeeding job open processing (step 606) will be shown. Also in the succeeding job opening process, the basic processing flow is based on the preceding job opening process described above. However, for the succeeding job, it is necessary to distinguish between the job requesting the transfer of all data and the job requesting the transfer of divided data. First, of the job control statements of the processing request source job, the PIP in the file definition statement of the processing target file
Determine the contents of the ESLCT first parameter (step 80
1) Yes. When the content of the parameter is "NO", it indicates that the job requests transfer of all data. Therefore, subsequently, the opened first succeeding job number counter 510 and the first succeeding job number field 507 are compared (step 802). Hereinafter, similar to the processing for the preceding job, if it is determined that a predetermined number of all data request succeeding jobs have already been opened, the job for which the processing is requested this time is abnormally terminated (step 803). After the individual information table 502 corresponding to the job is created (step 804), 1 is added to the opened first succeeding job counter (step 805). Here, the contents of the individual information table creation (step 804) will be described with reference to FIG.
On the other hand, in the determination of step 801, PIPESLC
When the content of the T first parameter is “YES”, it means that the job requests the transfer of the divided data. In this case, the second succeeding job number field 508 for managing the succeeding job of the divided data transfer request and the opened second succeeding job number counter 511 are used, and according to the processing of steps 802 to 805, Step 8
The processing from 06 to step 808 is performed.

In this way, after processing corresponding to either the all data transfer request succeeding job or the divided data transfer request succeeding job, a table chain is created.
Regarding the table chain, regardless of the difference between all data transfer and divided data transfer, one table chain is managed as a succeeding job group. Therefore, it is determined whether or not the value obtained by adding the number of opened first subsequent job counters 510 and the number of opened second subsequent job counters 511 is 0 (step 809). The head address of the newly created individual information table 502 is stored in the next table pointer 524 in the final table of the subsequent job correspondence table chain of (step 810), and if the same value is 0, the succeeding job in the common information table 501 The head address of the newly created individual information table 502 is stored in the table pointer 515 (step 811). The above is the processing content of the subsequent job open processing.

FIG. 9 is a detailed process flowchart of the common information table creating process (step 602) of the processes of the open process executing section 3. In the same process, first, a new pipeline information storage table 9 corresponding to the file to be processed is newly created (step 901), and then,
A common information table 501 is newly created inside the table 9 (step 902). Further, a buffer area 7 corresponding to the same file is newly created (step 903). Hereinafter, necessary information will be stored in the newly created table. First, the file name of the file to be processed specified in the job control statement is stored in the file name field 503 (step 904). The start address of the buffer area 7 newly created in step 903 is stored in the buffer address field 504 (step 90
5) Do it. In the buffer block count field 505,
The number of blocks in the buffer area newly created in step 903, that is, the number of data that can be stored is stored (step 9
06) In many cases, the system determines in advance how many blocks should be secured for one buffer area 7. However, if there is not enough storage capacity to secure the predetermined number of blocks at the time of allocating the buffer area, the maximum number of blocks will be allocated within the usable range at that time. Is normal. It is also possible to specify the number of blocks included in one buffer area with a job control statement or the like. In this way, the number of blocks for each buffer is not necessarily a fixed value, so it is necessary to store that number.

Next, the type of access from the processing request job to the file to be processed is determined (step 907).
If the access type is write processing, the same job will be a preceding job in pipeline processing, and various job numbers should be specified in the job control statement. Therefore, when the access type is write processing, the value of the first parameter is set in the preceding job number field 506 and the value of the second parameter is set in the second subsequent job number field 508 from the PIPENO parameter in the job control statement. The value of the third parameter is stored in the first succeeding job number field 507 (step 908). If the PIPENO parameter is omitted, it is assumed that all data is transferred from one preceding job to one succeeding job, and in step 908, the preceding job number field 50
6, 1 is stored in the first subsequent job number field 507, and 0 is stored in the second subsequent job number field 508. In the fields other than the fields in which the data are stored up to the previous step, 0 or information equivalent to "empty" is stored (step 909). When all the initial values are 0 when a new table is created, it is not necessary to intentionally perform step 909. Finally, the head address of the newly created table is stored in the next table pointer 514 in the last table in the table chain of the existing pipeline information storage table 9 (step 910).

FIG. 10 shows the individual information table creation (steps 703, 804, among the processing of the open processing execution part).
807) is a detailed processing flowchart. In the same process, first, a new individual information table 502 corresponding to the process request source job is newly created in the pipeline information storage table 9 corresponding to the process target file (step 100).
1) Yes. Hereinafter, necessary information is stored in the newly created table. First, the job name field 517
The processing request source job stores the job name defined in the job control statement (step 1002). "1" is stored in the open flag 518 (step 100
3) Yes. Next, the access type to the processing request file is determined (step 1004), and if the access type is a writing process, "preceding job" is stored in the pipeline type field 521 (step 1005). When the access type is read processing, the content of the PIPESLCT first parameter in the job control statement is further determined (step 1006). If the content of the parameter is "NO", "all data transfer succeeding job" is stored in the pipeline type field 521 (step 1007). If the content of the parameter is "YES", the pipeline type field 521 is stored. After storing "divided data transfer succeeding job" in step 1008, the contents of the PIPESLCT second parameter and thereafter are stored in the transfer condition field 523 (step 1009). Fields other than the information stored up to the previous step store information indicating 0 or its equivalent "empty" (step 101).
0). This completes the individual information table creation process.

Next, the processing contents of the job synchronization control section 5 will be described. When the processing request content from each job is a data writing request or a data reading request, the job synchronization control unit 5 is called by the processing receiving unit 2 that has received the request. The called job synchronization control unit 5 divides the request content into a data write request and a data read request, and executes processing corresponding to each. Each processing content will be described with reference to FIGS. 11 and 12.

FIG. 11 is a processing flowchart of the job synchronization control section 5 when a write request is issued. First,
A table 9 having the same file name field 503 as the file name of the processing request target file is selected from all the pipeline information storage tables 9 existing in the pipeline information storage area 8 (step 1101). Furthermore,
A table 502 having the same job name field 517 as that of the processing request source job is selected from the table 9 (step 1102).

Subsequently, the data is transferred to the buffer area 7. The content of the shared output counter 512 determines to which block number belonging to the buffer area 7 the data should be transferred. Further, after processing the request from each job, the counter 512 is updated. However, in order to normally execute processing for asynchronous data write requests from a plurality of jobs, while a request from one job is being processed, another output is executed until the shared output counter 512 is updated. It is necessary to prohibit write requests from jobs. In order to realize this, in the present invention, after the table selection up to step 1102, the shared output counter is first locked. Here, locking the counter is one of the methods for managing exclusive control of resources. The resource locked by one job is prohibited from being updated by another job until the same job releases the lock.

Using the same locking method, step 1103
At, it is determined whether the shared output counter 512 is locked by another job. If it is locked, the determination in step 1103 is repeated until the lock is released. If it is not locked by another job, the lock of the shared output counter 512 is secured (step 1104). After securing the lock, you need not be aware of the existence of other jobs until the lock is released. After securing the lock, it is first determined whether or not there is a free area in the buffer area 7 (step 1105). If there is no free space,
Job execution must wait until free space is created. Therefore, the wait flag in the individual information table 502 selected in step 1102 is set to "1".
(Step 1106), the lock of the shared output counter is released (step 1107), and then the execution state of the job being processed is set to wait (step 1108).
I do. After that, the wait is released by another job,
After the execution is restarted, the wait flag is set to "0" (step 1109), the process returns to step 1103, and the process is continued from the lock request of the shared output counter 512. If there is a free area in the buffer 7 in step 1105, the shared output counter 51 continues.
Data is transferred from the data storage area of the processing request source job to the block address in the buffer 7 determined by 2 (step 1110). Shared output counter 51 after transfer
1 is added to 2 (step 1111), and the counter 51
The lock of No. 2 is released (step 1112).

By the processing up to the previous step, one data has been stored in the buffer 7. Therefore, if there is a succeeding job that is waiting after storing new data, it is necessary to cancel the wait state of the same job. Therefore, first, the individual information table 5 chained from the succeeding job table pointer 515 of the pipeline information storage table 9 selected in step 1101.
In 02, it is determined whether or not there is a wait flag 520 set to "1" (step 1113). If there is a table satisfying the condition, all the wait states of the jobs corresponding to the corresponding table 502 are canceled (step 1114). Here, the corresponding job is identified by referring to the job name field 517 in the table. In this embodiment, the wait state of the succeeding job is canceled when one new data is stored in the buffer, but the frequency of wait may increase in the same process. In order to prevent this, there is also a method of executing the wait release determination process including step 1113 and step 1114 for each writing of a plurality of data (for example, 5 or 10) instead of for each new data storage. Conceivable.

Next, the processing contents when the job synchronization control unit 5 receives a data read request will be shown. Figure 12
6 is a processing flowchart of the job synchronization control unit 5 when a data read request is made. Similar to the data write request, in step 1201 and step 1202, the processing request source job, the pipeline information storage table 9 corresponding to the processing target file, and the individual information table 5
Select 02. After that, it is determined whether or not uninput data is stored in the buffer (step 1203). Here, it should be noted that the lock processing of the counter is not performed unlike the data write request. That is, in response to the data read request, the individual counter 522 in the individual information table 502 corresponding to each job.
(The counter value of this counter is the number of times the data in the buffer is viewed, and is not affected by the subsequent determination result). The counter 522 is for each job 1
Since they exist independently, no lock processing is required. The determination as to whether or not there is uninput data can be realized by determining whether or not the value of the counter 522 is smaller than the value of the shared output counter 512.

When it is determined by the above-described processing whether or not the uninput data exists in the buffer for each job, if the uninput data does not exist, the process execution is waited until the same data is generated. To do. That is, the wait flag 520 of the selected individual information table 502 is set to "1" (step 1204), and then the execution of the processing request source job is set to the wait state (step 120).
5) Do it. After that, the wait state is canceled by another job, the process is restarted, the wait flag 520 is set to "0", and the determination as to whether or not there is uninput data in the buffer is repeated (step 1203). .

In the same judgment, if there is uninput data, the contents of the pipeline type field 521 in the individual information table 502 are continuously judged (step 1
207) If the content of the field 521 is “divided data transfer subsequent job”, then the individual counter 52
It is determined whether or not the data in the buffer designated by 2 satisfies the transfer condition specified in the transfer condition field 523 (step 1208). If the determination is satisfied, then the same data is transferred to the data storage area of the processing request source job (step 1209). Step 120
7, the pipeline type field 521
If the content of is a "all data transfer succeeding job", the predetermined data is unconditionally transferred to the data storage area of the processing request source job (step 1209). In either case, subsequently, 1 is added to the individual counter 522 (step 1210).
I do.

Further, it is determined whether or not the data processed before the counter addition is erased from the buffer 7. Here, the data may be deleted from the buffer only when the processing of all succeeding jobs 11 is completed. Whether or not the processing of all the succeeding jobs 11 is completed, that is, the value of the individual counter 522 is equal to or more than the value after the addition in step 1210 for all the individual information tables 502 chained from the succeeding job table pointer 515. It is determined whether or not (step 1211). If the same condition is satisfied, the data processed immediately before, that is, the data corresponding to [individual counter-1] is erased from the buffer 7 (step 1212). Further, at this time, if there is a preceding job 10 in the wait state, it is necessary to cancel the wait state. That is, of all the tables chained from the preceding job table pointer 516 of the same common information table 501, it is determined whether or not there is a table with a wait flag 520 of "1" (step 1213), and a table satisfying the condition is satisfied. If exists, the wait states of the jobs corresponding to those tables are released (step 1214).

In the condition judgment of step 1211, the individual information table 5 having the individual counter 522 having a value smaller than the individual counter 522 of the succeeding job 11 currently being processed.
If 02 exists, the data cannot be erased from the buffer at this stage, so the process is terminated without performing the wait release determination of the preceding job.

As in the case of canceling the wait state of the succeeding job 11, in canceling the wait condition of the preceding job,
A method may be considered in which the frequency of waiting for a job is reduced by performing a wait release determination each time a plurality of pieces of data are deleted, instead of executing the deletion each time one piece of data is deleted from the buffer.

According to the processing described above, in the first embodiment, the data transferred independently from the two preceding jobs 202 and 203 are received in the order of generation, and the received data is transferred from the first succeeding job 204 to the third succeeding job 204. For the three succeeding jobs 206, the only data that satisfies the condition specified by the job control statement is selected and passed, and at the same time, all the data is received for the fourth succeeding job 207. Implement the process of passing.

Next, the second embodiment will be described. The system configuration of the second embodiment is shown in FIG. In the second embodiment, similarly to the first embodiment, the data generated from a plurality of preceding jobs is divided and handed over to some succeeding jobs, and all the data is passed to another succeeding job. The process of handing over is realized by executing all jobs in parallel. The difference from the first embodiment is that each job is executed on a different computer. Hereinafter, this point will be mainly described in detail.

In the system shown in FIG. 13, a plurality of computers are controlled by tangential lines such as optical fibers.
The computer system is connected to No. 01, and executes as a single computer as a whole while coordinating exclusive control and data sharing through the control computer 1301. In addition, this embodiment is assumed to be operated by a bank. Therefore, each computer is divided into an online processing computer group 1302 that exclusively executes online processing and a batch processing computer group 1303 that exclusively executes batch processing. The computers belonging to the online processing computer group 1302 are online processing computers 1304, and the computers belonging to the batch processing computer group 1303 are batch processing computers 13.
Call 05. However, such a difference in nickname is due to a difference in program configuration executed on each computer, and there is often no difference in hardware configuration between computers.

The online processing request from the user 1306 is sent to the computer 1 belonging to the online processing computer group 1302.
It is distributed to any one of 304 and processed on the same computer. Which computer to allocate to is arbitrary. You may allocate according to the order determined beforehand. In general, when each online processing request is made, it is often assigned to a computer with the lowest load such as a CPU. In each online processing computer 1304, the same online processing program 1308 is executed, and each processing request is processed independently. However,
The database 1307 storing user account information and the like is connected to all online processing computers so that the processing can be executed without change regardless of which computer is distributed.

On the batch processing computer group 1303 side, various tabulation processing is executed based on the processing history information (generally called “journal”) output from the online processing computer group 1302. If the processing request from the user 1306 is unspecifiedly distributed to each online processing computer 1304, it is meaningless to execute the batch processing for each online computer. Therefore, it is necessary to collect the journals of all the online processing computers 1308 into one and then transfer the data to each batch processing program as needed. Therefore, all the journals of each online processing program 1308 are designated to be transferred to the inter-computer pipeline execution control unit 1309 using the present invention.

Inter-computer pipeline execution control unit 1309
The configuration of is similar to that of the pipeline execution control unit 1 of FIG.
However, while the pipeline execution control unit 1 is executed in a single computer, the inter-computer pipeline execution control unit 1309 needs to be aware that the transfer partner jobs are executed on different computers. There is. For example, in the pipeline execution control unit 1, a place where data is simply transferred from the job to the buffer area 7 is
The inter-computer pipeline execution control unit 1309 involves data transfer between computers. If the control computer cannot directly execute jobs such as setting the wait status of a job or canceling the wait status, it can be executed once for the operating system on a specific computer. Must be accompanied by a communication requesting. However, in a system that uses such a plurality of computers to perform processing as if it were a single computer, the known technology is mainly used for the job synchronization control part and the data input / output control part by the buffer area for data transfer. By using, it is possible to realize the processing realized on a single computer. That is, there is no difference in the configurations of the pipeline execution control unit 1 and the inter-computer pipeline execution control unit 1309 if the processing is replaced only for such a portion.

In the second embodiment shown in FIG. 13, of the batch processing computers 1305, the first batch processing computer 13
The 05th to n-th batch processing computers 1305 execute different first batch processing programs 1310, second batch processing programs 1311, ..., And nth batch processing programs 1312. In this embodiment, all the journal data is divided into several chunks, and batch aggregation processing is performed in parallel for each chunk to shorten the batch processing time. Therefore, it is assumed that each of the batch processing computers 1305 respectively executes the partial batch processing thus divided. At this time, the standard of division is generally, for example, as shown in the first embodiment, that all branches are divided into several areas and batch aggregation is performed in each area. Therefore, also in this embodiment, it is assumed that the same transfer record as that in the first embodiment shown in FIG. 3 is assumed, and that each batch processing program is distributed based on the area name 304 of the record. To do.

By the way, a computer system of a bank or the like is required to have high reliability, and therefore journal data is usually stored for a certain period of time. In such storage, it is more effective from the viewpoint of reliability to manage all the data collectively, not the data divided into each batch processing computer. At this time, if the present invention is utilized, divided transfer of data and batch transfer of all data can be executed simultaneously. That is, in FIG. 13, the backup program 1313 is executed on one computer in the batch processing computer group 1303 named 0th batch processing computer 1305 for convenience, and the program 1313 is executed by the inter-computer pipeline execution control unit 1309. By requesting the transfer of all data, it is possible to save all the transferred data at once. Here, the backup program 13
13 is a program that sequentially outputs the input data to another storage medium. In this embodiment, a saved journal file 1314 which receives data from the inter-computer pipeline execution control unit 1309 and is prepared in a magnetic tape device or the like.
Data is saved by outputting the data to.

As described above, in the second embodiment, the journals generated from the plurality of online processing programs 1308 are collected into one, and are transferred to the plurality of batch processing programs according to a predetermined condition while being combined into one. It realizes the saving of all the collected data. Further, all the related programs are executed at the same time, and the generated data are sequentially transferred to the batch processing program, so that the processing time can be shortened.

The linking means of each program is designated by the job control statement as in the first embodiment. Each job control statement may have the same format as that of the first embodiment shown in FIG. 4, except that each job is executed on a different computer. However, for each program name and file name, the specified online processing program name,
It is necessary to change the batch processing program name, backup program name, and journal specification file name.

For comparison, FIGS. 14 to 16 show processes using the prior art without using the present invention. First, Figure 1
4 shows a state in which online processing using the conventional technique is being executed. In the conventional bank system, after the daytime online processing is completed, the batch processing for totalizing the same processing is started. That is, the data from the online processing and the batch processing as shown in FIG. 13 are not sequentially transferred, and the journals from each online processing program are stored in the external storage device for each computer. It FIG. 14 shows an example of processing by two online processing computers. The online processing program 1308 on each online computer 1304 has a first journal file 1401 and a second journal 1402, respectively.
Has been generated. Note that the processing request from the user 1306 is distributed to one of the online computers 1304 and processed, which is the same in the conventional technique.

After the online processing is completed, batch processing is started. However, according to the prior art, each online computer 1
Since the journals in 304 are individually generated, it is necessary to combine the two journal files into one for the purpose of saving the reliability as described above. In FIG. 15, such multiple journal files 1
The state where 401 and 1402 are put together in one saved journal file 1314 by using the merge program 1501 is shown.

The preparation for starting the batch processing is completed as described above, and the batch processing shown in FIG. 16 is started. Even when the conventional technique is used, the batch process is divided into several parts based on a predetermined standard, and the respective processes are executed in parallel to reduce the processing time. Therefore, the inter-computer pipeline execution control unit 1602 is also used to transfer data to each computer. However, the conventional inter-computer pipeline execution control unit 1602 does not have a function of judging the received data by itself and distributing it to each succeeding job. Therefore, before transferring to each computer, one of the computers 130
5, it is necessary to execute the distribution program 1601 which judges the condition of the input data and distributes it to each output file. Each distribution data generated by the program 1601 is sent to each batch processing computer 1305 via the conventional inter-computer pipeline execution control unit 1602.
Batch processing programs 1310 and 1311, which are being executed in
By transferring to 1312, the intended processing is realized. In this example, the number of batch processing programs is three.

Subsequently, the execution order of each program is divided into the case where the conventional technique is used and the case where the technique of the present invention is used, and they are shown in FIGS. 17 and 18, respectively. In both figures, the vertical axis is the time axis, and the temporal flow of processing is schematically shown. Also, programs having a data transfer relationship are connected by a tangent line, and a distinction is made as to whether each transfer uses an external storage device or a pipeline method. In the form using the conventional technique of FIG. 17, the online processing program 1308 and the batch processing program 1
310, 1311, 1312 are merge programs 1
It can be seen that the execution of 501 is sequentially executed. On the other hand, in the embodiment of the present invention shown in FIG. 18, the online processing program and the batch processing program can be executed simultaneously, and thus the processing time can be shortened. Furthermore, there is no need to prepare an unnecessary external storage device when linking data, and data distribution to each batch process and batch saving of all data can be performed at the same time.
Not only the processing time but also the labor of the operator can be reduced.

Next, the third embodiment will be described. The difference from the first and second embodiments lies in the receiving condition when specifying to receive the data in a divided manner in the succeeding job. In the embodiments described so far, FIG.
As shown in, the contents of the area name 304 in the transfer data are “XXXXXXXXX” and “YYYYYYYY”, respectively.
It was divided into three subsequent jobs by any of "ZZZZZZZZ". Further, as mentioned in the description of the first embodiment, in the banking system handled in these embodiments, all branches are divided into three areas, which are indicated by the above three area names. Therefore, each data is always transferred to any one of these three subsequent jobs, and is never transferred to two or more subsequent jobs. Therefore, the data obtained by dividing and transferring the data divided into these three succeeding jobs has no excess or deficiency as compared with the case where all the data are collectively transferred. By using this feature, you can save the journal by saving each data string transferred to each succeeding job even if only the divided transfer is performed without simultaneously performing the divided transfer of data and the batch transfer of all data. Can meet the demand. However, it should be noted that in this case, the reliability is markedly lower than that of the form involving batch transfer of all data.

On the other hand, FIG. 19 shows the job control statement for each job of the third embodiment. The number of executed processes is the same as in the first embodiment: the number of preceding jobs = 2, the number of data division transfer request subsequent jobs = 3, the total data transfer request subsequent jobs = 1
It is. In FIG. 19, 1901 and 1902 are job control statements for preceding jobs, and 1903, 1904, and 19.
Reference numeral 05 is a job control statement for the job following the data division transfer request, and reference numeral 1906 is a job control statement for the job following the all data transfer request. The biggest difference from Fig. 4 is 19
The contents of the PIPESLCT parameter from 03 to 1905, that is, the transfer conditions for division transfer.
Here, it is assumed that the transfer record has the structure shown in FIG. 3 as in the first embodiment. At that time, in the job control statement 1903, “the area name 304 is“ XXXXXXXXX ”
X only transfer records equal to "", job control statement 190
4 indicates that only the records whose processing type 306 is equal to “CCCCCCCC” are transferred, and the job control statement 1905 states that “only the records whose processing amount 307 is larger than 1000000 are transferred”. The processing type is a field that stores the type of online processing, such as "cash withdrawal".
It is assumed that “C” is a code of any processing type.

In the first and second embodiments, the reason for dividing and transferring the data to a plurality of batch processes is that all records are divided into a plurality of regional units and, so to speak, the aggregation process is executed in parallel for each subset. By doing so, the processing time was shortened. On the other hand, the reason why the data is divided and transferred in the third embodiment is that when different batch processes for executing different aggregation processes are required because different data are required for each batch process program. To distribute the data. At this time, with respect to the transfer conditions of the above three jobs, there are transfer records that satisfy all of the conditions, and there are transfer records that do not satisfy any of the conditions. Therefore, in the present embodiment, it is not possible to achieve the purpose of journal storage only by individually storing the data strings divided and transferred for each job. In this way, for each succeeding job requesting divided transfer, a transfer condition is set so that the set of data transferred to each of the succeeding jobs may be duplicated or missing from the original journal data. In this case, simultaneous execution of data division transfer and batch transfer of all data using the present invention is more effective.

Next, the fourth embodiment will be described. FIG. 20 shows a system configuration diagram of the fourth embodiment.
Compared to the second embodiment shown in FIG. 13, in FIG.
The difference is that there are a plurality of jobs requesting the batch transfer of all data, that is, a fourth batch processing program 2001, a fifth batch processing program 2002, and a sixth batch processing program 2003. In the systems up to the third embodiment, the purpose of collectively transferring all the data separately from the divided data transfer was to store the journal data exclusively. Therefore, while it is expected that the preceding job on the data generation side or the succeeding job that requests divided data transfer will change its number arbitrarily depending on the situation, the subsequent job that requests batch transfer of all data One was always sufficient.

On the other hand, in the fourth embodiment, among the batch processes distributed to the respective batch process computers 1305, some batch process computers require batch data only as input. Batch processing on another batch processing computer requires input of all data. Even if the batch processing program distributed to each batch processing computer has such a property, it is possible to simultaneously transfer the data generated by the online processing side to all batch computers by using the technique of the present invention. is there.

Next, the fifth embodiment will be described. FIG. 21 shows a system configuration diagram of the fifth embodiment.
Also in the fifth embodiment, as in the previous embodiments, the selection data is transferred and all the data is transferred at the same time. Here, the first batch processing program 1310 and the second batch processing program 13 that receive the selection data
The data selection criterion 11 is equal to the job control statements 1903 and 1904 of the third embodiment shown in FIG. 19, respectively. At the same time, the backup program 1313 that receives all the data and saves the journal data in a batch is executed by the third batch processing computer 130.
It is being executed in 5.

The difference from the above-described embodiments is that the first batch processing computer 1305 and the second batch processing computer 1305 in which the first batch processing program 1310 and the second batch processing program 1311 are executed. In this case, the backup program 1313 is further executed. At this time, each backup program 1313 requests the inter-computer pipeline execution control unit 1301 to transfer the selected data. Furthermore, each backup program 1
The data selection criterion of 313 is the same batch processing computer 1
Similarly to the data selection standard of the batch processing program executed in 305, the job control statement 19 shown in FIG.
03 and 1904, respectively. That is, the backup program 1313 in the first batch processing computer 1305 is the first batch processing program 1
Requesting to receive the same data as 310,
Similarly, the backup program 1313 in the second batch processing computer 1305 requests to receive the same data as the second batch processing program 1311. In the fourth embodiment, an example in which all data transfers are multiplexed is shown, but in the fifth embodiment, so to speak, an example in which selected data transfers are multiplexed (duplexed) is shown. By implementing such processing, it becomes possible not only to save all the journal data all at once, but also to satisfy the request to partially save only the data satisfying a specific condition. In this embodiment, the first partially saved journal file 21 extracted under different conditions
01, the second partial storage journal file 2102 is generated.

[0083]

According to the present invention, when it is necessary to transfer data between jobs, there is one or more jobs that generate data, receive data from all these jobs in the order of generation, and receive the same data. The pipeline method is used even when there are one or more jobs requesting to receive all at once, and at the same time, there are one or more jobs requesting to receive only necessary data according to a predetermined condition. Inter-job data transfer processing can be applied, and processing time can be shortened.

[Brief description of drawings]

FIG. 1 is a diagram showing the principle of the present invention.

FIG. 2 is a diagram showing a system configuration of a first embodiment.

FIG. 3 is a diagram showing a configuration example of a transfer record.

FIG. 4 illustrates a job control statement according to the first embodiment.

FIG. 5 is a diagram showing a configuration of a pipeline information storage table.

FIG. 6 is a diagram showing a processing flowchart of an open processing execution unit.

FIG. 7 is a diagram illustrating a processing flowchart of a preceding job open processing.

FIG. 8 is a diagram showing a processing flowchart of a subsequent job opening processing.

FIG. 9 is a diagram showing a processing flowchart of a common information table creation processing.

FIG. 10 is a diagram showing a processing flowchart of individual information table creation processing.

FIG. 11 is a diagram illustrating a processing flowchart of a job synchronization control unit when a data write request is made.

FIG. 12 is a diagram illustrating a processing flowchart of a job synchronization control unit when a data read request is issued.

FIG. 13 is a diagram showing a system configuration of a second embodiment.

FIG. 14 is a diagram showing a state during execution of online processing using a conventional technique.

FIG. 15 is a diagram for explaining creation of a saved journal file from a plurality of journal files according to a conventional technique.

FIG. 16 is a diagram for explaining batch processing using a conventional technique.

FIG. 17 is a diagram for explaining the flow of processing when a conventional technique is used.

FIG. 18 is a diagram for explaining the flow of processing when the present invention is used.

FIG. 19 is a diagram showing a job control statement in the third embodiment.

FIG. 20 is a diagram showing a system configuration of a fourth embodiment.

FIG. 21 is a diagram showing a system configuration of a fifth embodiment.

[Explanation of symbols]

 1 Pipeline Execution Control Unit 2 Process Reception Unit 3 Open Process Execution Unit 4 Close Process Execution Unit 5 Job Synchronization Control Unit 6 Data Input / Output Control Unit 7 Buffer Area 8 Pipeline Information Storage Area 9 Pipeline Information Storage Table 10 Predecessor Job 11 Successor job

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Hirofumi Nagasuka 1099, Ozenji, Aso-ku, Kawasaki-shi, Kanagawa Incorporated company Hitachi, Ltd. Systems Development Laboratory

Claims (7)

[Claims] 1. A function of executing, in parallel, all the jobs that generate data and the jobs that receive data when one or more jobs need to receive one or more data generated by one or more jobs. And, even if all the data generation is not completed, it is sequentially passed to the job that receives the data from the job that generates the data through the buffer area prepared on the main storage device or an auxiliary storage device that is similar to it. In a computer system that has the function of executing processing, the classification criteria for classifying the data generated by all jobs on the data generating side into one or more groups are set in advance, and all the data receiving side Only jobs that receive all data and data that belong to any of the groups classified by the above classification criteria Jobs to be received are classified, job control statements for each job are defined according to the classification, the data generated by each job that generates data is stored in the buffer area, and the job that receives the data When a request for data reception is made, it is determined whether the job is a job that receives all the data or a job that receives only data belonging to any one of the groups, and as a result of the determination, When it is determined that the job of the data receiving side is a job that receives all the data, all the generated data is passed to the job via the buffer area, and as a result of the determination, the side that receives the data If it is determined that the job is a job that receives only the data that belongs to any of the groups, Multiple jobs between the data migration method is characterized by passing against the job only groups of data through the buffer area to take. 2. The method of transferring data between a plurality of jobs according to claim 1, wherein in the definition of the job control statement of each job, the job that receives the data is a job that receives all data, or In the job control statement of the job, specify whether it is a job that receives only data that belongs to another group, and the job that receives the data is a job that receives only data that belongs to any group A method of inheriting data between a plurality of jobs, characterized in that, if specified, the type of group received by the job is also described in the job control statement of the job. 3. The data transfer method between multiple jobs according to claim 1, wherein the number of jobs generating data, the number of jobs receiving all data, in a job control statement of a job generating data,
Describe the specification of the number of jobs that receive only the data that belongs to a specific group, and specify the number of jobs that have been accessed for the files to be inherited as seen from the job that generates the data (in the buffer area). The job is created, the job that receives all the data, and the job that receives only the data that belongs to a specific group are recorded separately, and each time the new job accesses the file to be inherited, the job that creates the data Job, a job that receives all data, or a job that receives only data that belongs to a specific group, and the newly accessed job is the job that generates the data. If it is determined that the job is in the job control statement of the job that generates the data, The number of jobs that generate the described data is compared with the number of jobs that generate the recorded accessed data, and the number of jobs that generate the accessed data generates the data described in the job control statement. When the number of jobs is smaller than the number of jobs to be performed, the access of the new job is permitted, 1 is added to the number of jobs that generate the accessed data, and when they are equal, the access of the new job is denied, and the job is abnormally terminated. When it is determined in the above determination that the job newly accessing the transfer target file is a job that receives all data, all the data described in the job control statement of the job that generates the data is Compare the number of jobs to receive with the number of jobs to receive all the recorded accessed data, and receive all accessed data When the number of jobs is smaller than the number of jobs that receive all the data described in the job control statement, the access of the new job is permitted, and 1 is added to the number of jobs that generate all the accessed data. When a new job is rejected, the same job is abnormally terminated, and it is determined in the above determination that the newly accessed job is a job that receives only data belonging to a specific group. Compares the number of jobs that receive only the data belonging to the specific group described in the job control statement of the job that generates the data with the number of jobs that receive only the data that belongs to the recorded and accessed specific group. Then
When the number of jobs that receive only the data that belongs to the accessed specific group is less than the number of jobs that receive only the data that belongs to the specific group described in the job control statement, the access of the new job is permitted and the accessed specific 1 is added to the number of jobs that generate only data belonging to the group, and when they are equal, the access of the new job is denied and the same job is abnormally terminated. 4. A function that, when one or more jobs need to receive one or more data generated by one or more jobs, executes all the jobs that generate the data and the jobs that receive the data in parallel. , And through the buffer area prepared on the main memory device or its equivalent auxiliary memory device, to the job that receives the data from the job that generates the data, even if all the data generation is not completed In a computer system having a function of executing processing, all jobs that receive data are classified into jobs that receive all data and jobs that receive only some data, and the job control statements for each job are classified according to the classification. Is defined, and the condition of the received data is specified in the job control statement of the job that receives only some data. The data generated by each job of the data generating side is stored in the buffer area, and when the job of the data receiving side requests the data reception, the job is the job of receiving all the data. Yes, or if it is a job that receives only the part of the data, and if the result of the determination is that the job that receives the data is a job that receives all the data, On the other hand, if all the generated data is passed through the buffer area, and as a result of the determination, the job that receives the data is determined to be a job that receives only the partial data, Data between a plurality of jobs, characterized in that only data satisfying the conditions is transferred to the job via the buffer area. Can splicing method. 5. The method of transferring data between a plurality of jobs according to claim 4, wherein in the definition of the job control statement of each job, the job that receives the data is a job that receives all data, or When the job control statement of the job describes that the job is a job that receives only partial data, and the job that receives the data is specified as a job that receives only a part of the data In the method, the data transfer method between a plurality of jobs is characterized in that the condition of the data received by the job is described in the job control statement of the job. 6. The multi-job data transfer method according to claim 4, wherein the number of jobs generating data, the number of jobs receiving all data, in a job control statement of a job generating data,
Describe the specification of the number of jobs that receive only part of the data, and generate the data for the number of jobs that have already been accessed for the files to be inherited as seen from the job that generates the data (in the buffer area). This job is a job that generates data each time a job, a job that receives all data, and a job that receives only some data is recorded separately and a new job accesses the file to be inherited. It is determined whether it is a job that receives all data or a job that receives only data that belongs to a specific group. In the determination, it is determined that the newly accessed job is the job that generates data. If it is, the job that generates the data described in the job control statement of the job that generates the data The number of jobs for generating the recorded accessed data, and when the number of jobs for generating the accessed data is smaller than the number of jobs for generating the data described in the job control statement, the new The access of the job is permitted, 1 is added to the number of jobs that generate the accessed data, and when they are equal, the access of the new job is denied and the same job is abnormally terminated. On the other hand, when it is determined that the newly accessed job is a job that receives all the data, the number of jobs that receive all the data described in the job control statement of the job that generates the data and the recorded The number of jobs that receive all accessed data is compared, and the number of jobs that receive all accessed data is described in the job control statement. When it is smaller than the number of jobs that receive all data, the access of the new job is permitted, 1 is added to the number of jobs that generate all the accessed data, and when they are equal, the access of the new job is denied and the same job is If it is determined that the job that newly accessed the takeover target file is a job that receives only a part of the data, the job control statement of the job that generates the data is terminated. Compare the number of jobs that receive only the data that belongs to the specific group described in, and the number of jobs that receive only the recorded part of the accessed data, and find the number of jobs that receive only the part of the accessed data. When the number of jobs that receive only part of the data described in the job control statement is less than the number of jobs, access of the new job is permitted. It adds 1 to the number of jobs to receive only the accessed some data, when equal, a plurality of jobs among the data migration method is characterized in that deny access of the new job, abnormally terminated the job. 7. A computer system in which one or more computers cooperate with each other via a control computer and are capable of controlling processing as if they were one computer as a whole, are executed in different computers. One or more data generated by one or more jobs, through the control computer, a process of delivering to one or more jobs executed in a computer different from the computer that executes those jobs, A data transfer method between a plurality of jobs, characterized in that the data transfer method between a plurality of jobs according to any one of claims 1 to 6 is applied to a process of transferring a job through the control computer.