JP2008234142A - Batch processing restoration method and system, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently carry out data restoration for reprocessing when processing abnormal finish occurs in a batch processing system, of updating a database and a sequential file, with interlocking by one process (program). <P>SOLUTION: The sequential file 60 at a commit time point of the database 50 is automatically saved into a backup file 70, in a business program, so as to secure a content of the sequential file 60 matched with the database 50 after committed, and the automatic data recovery is carried out for the sequential file 60, from the backup file 70 by a recovery processing function of a processing control tool of the batch processing, together with restoration processing by roll-back processing of the database 60 by a DBMS (Data Base Management System) function, in the processing abnormal finish. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a data restoration technique for reprocessing when an abnormal end of processing occurs when updating a database and a sequential file linked in one process (program) in update batch processing using a computer system. In particular, the present invention relates to a technique suitable for easily and accurately realizing preparation for reprocessing in the case of abnormal termination in a process in which a database commit (update confirmation) is performed a plurality of times for mass update. Is.

  In batch processing, a process of sequentially reading records stored in an input file, processing the read records, and sequentially storing them in an update file (database) is performed for each job.

  Such batch processing is performed according to the update system batch processing program. If a failure occurs during the update process of the file record, all the state before the update is temporarily set to prevent double update of the file record. The file record is returned, and the update process is started again from the beginning of the file record.

  Therefore, there are problems that the recovery time becomes long and various troubles occur when the database (update file) is returned to the state before the update.

  As a conventional technique for solving such a problem, for example, there is a technique described in Patent Document 1. This technique eliminates the update file recovery process when a failure occurs in the update batch processing program, so that the batch process can be resumed immediately.

  Specifically, in a batch processing system that reads a record sequentially from an input file, processes the read record, and sequentially stores it in an update file for each job, the number of records processed for each job is calculated. A processing record recording means for recording; a means for searching for the number of records corresponding to the job recorded in the processing record recording means in response to a start instruction for a predetermined job; and When a numerical value other than “0” is indicated, means for starting processing from the next record of the numerical value in the input file is provided.

  In another conventional technique, in the process of reading records sequentially from the input file and updating the database, the following measures are taken when the process ends abnormally.

  In the first conventional technique, in the process of sequentially reading records from the input file and updating the database, if the process ends abnormally, the state before the commit (update confirmation) is performed by the database rollback function. Automatically restore the database to its pre-update state. In this technique, at the time of reprocessing, after the cause of abnormal termination is removed, database update processing is executed again from the first record of the input file.

  In the second prior art, in the process of updating a large amount of data to the database, commit (update confirmation) is performed at the stage when the process for the appropriate number of records is completed so that the area for rollback of the database does not become enormous. Let it be implemented. In this case, when the update process ends abnormally, the database is automatically recovered until before the last commit (update confirmation). With this technology, at the time of reprocessing, after removing the cause of abnormal termination, the records of the input file up to the time of the last commit (update confirmation) are skipped, and the database update processing is executed from the records after the last commit (update confirmation) Let This prevents double updates.

  In the third conventional technique, in the process of sequentially reading records from the input file and updating the sequential file that is the output file, if the process ends abnormally, the file is recovered based on the backup of the output file obtained before the process. Or, when reprocessing is performed, double update is prevented from occurring by skipping the input records for the number of updated records.

  However, if the system and the sequential file are updated in one process (program) at the same time, and the system that performs the process of committing the database (update confirmation) multiple times for mass update, the process ends abnormally. The database after the rollback and the sequential file after the update are not synchronized because the processed time points are different. Such a case cannot be dealt with by the conventional technique including the technique of Patent Document 1 described above.

  For example, after reading records sequentially from the input file with a business program and performing a check process for record items, the record that was OK in the check process (database update determination) is updated to the database, and the record that is NG In the process of outputting to the sequential file as error information, when a large amount of data is input, when the appropriate number of records are processed so that the database rollback area is not too large and insufficient, the commit is performed. .

  In such a case, in order to perform reprocessing after abnormal termination, a backup of the database and sequential file before execution of the process is acquired, and at the time of abnormal termination, the database and sequential file are processed from the backup before execution. After recovering, it is necessary to perform processing from the first record again.

  As a result, the same processing as the first time must be repeated up to the number of cases in the middle of reprocessing, resulting in wasted time and labor.

Japanese Patent Laid-Open No. 06-012267

  The problem to be solved is that, in the conventional technique, in a system that performs processing of updating a sequential file with a single program and updating a sequential file, the database is Is automatically recovered to the point of the last commit with the rollback function, but the sequential file becomes the abnormal end point and the data in the database and the sequential file become inconsistent.

  The object of the present invention is to solve the problems of the prior art, and to deal with a processing error (such as re-execution) by automatically saving a sequential file linked with the commit time of the database and automatically recovering from the processing error. It is possible to carry out simply and accurately.

  In order to achieve the above object, in the present invention, the sequential file at the time of committing the database is automatically saved in the backup file in the business program, and the contents of the sequential file consistent with the database after committing are secured. In addition, when processing ends abnormally, automatic recovery of data from backup files is performed for sequential files as well as recovery processing by database rollback processing using the DBMS function, using the recovery processing function of the processing control tool for batch processing. . In this way, by combining sequential file automatic backup at the time of commit and sequential file recovery performed by the recovery function of the batch processing process control tool, which is a conventional technology, when committing multiple times in a single program Databases and sequential files can be kept in a consistent state.

  According to the present invention, in a batch processing system that performs processing for updating a database multiple times with a single business program and updating a sequential file, for example, even when the processing of a business program ends abnormally. The sequential file can be automatically recovered to the content at the time of the most recent commit completion that is consistent with the rolled back database, and even if a processing abnormal termination occurs when updating a large amount of data Even when processing abnormal termination occurs, re-execution upon abnormal termination can be performed easily and accurately without being aware of the consistency between the commit of the database and the update of the sequential file.

  The best mode for carrying out the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of a batch processing system according to the present invention, FIG. 2 is a flowchart showing an example of processing operation according to the present invention of the batch processing system in FIG. 1, and FIG. FIG. 4 is an explanatory diagram showing an example of database recovery processing and sequential file recovery processing when the update processing is abnormally terminated by the batch processing system. FIG. 4 is a diagram of updating the database and sequential file until the batch processing system in FIG. FIG. 5 is an explanatory diagram showing an example of processing operation, FIG. 5 is an explanatory diagram showing an example of database recovery processing operation at the time of abnormal termination of update processing by the batch processing system in FIG. 1, and FIG. It is explanatory drawing which shows the recovery processing operation example of the sequential file at the time of completion | finish.

  The batch processing system in FIG. 1 updates the database 50 and the sequential file (described as “output file” in the figure) 60 in a single process (program) by programmed computer processing and terminates abnormally. Data recovery for reprocessing at the time of occurrence, consisting of a CPU (Central Processing Unit), main memory, display device, input device, external storage device, etc., CD-ROM etc. via an optical disk drive device etc. After installing the program or data recorded in the storage medium in the external storage device, the program is read from the external storage device into the main memory and processed by the CPU, thereby executing the functions of each processing unit. Commit (update confirmation) of database 50 multiple times for update Preparation for reprocessing when the process ends abnormally is easily and accurately performed.

  In FIG. 1, the batch processing system executes a process of sequentially reading records from an input file 30 stored in a storage device and updating a database 50 and a sequential file 60 stored in the storage device by a business program. As a function realized by a computer using a program, a normal execution processing unit 10 and an abnormal end processing unit 20 are provided.

  The normal execution processing unit 10 includes an input file reading unit 11, a database update unit 12 including a file output unit 12a and a database update unit 12b, a database update confirmation unit 13 including a commit unit 13a and an output file backup unit 13b, and a commit number information record. Unit 14, reads the input file 30, writes / updates to the database 50, commits the database 50 (confirmed update), records the commit number information at the time of committing the database 50, and saves the sequential file 60. (Backup process).

  That is, the input file reading unit 11 sequentially reads records corresponding to the job from the input file 30 in response to the program operation command and passes them to the database update unit 12.

  The database update unit 12 checks the data item with the data check processing function for the record passed from the input file reading unit 11, and if the data item is correct, the database update unit 12b performs data editing / processing. The database 50 is written / updated later.

  When the data check by the data check processing function detects that there is an error in the data item, the database update unit 12 outputs the record as error information to the sequential file 60 by the file output unit 12a.

  At this time, if the commit counter value designated from the commit interval parameter 40, which is an external parameter, matches the record count value recorded by the business program, the database update confirming unit 13 is activated.

  The database update confirmation unit 13 commits (update confirmation) to the database 50 by the commit unit 13a, and records the record contents of the sequential file 60 by the output file backup unit 13b as “output file backup” in the figure. Description) Duplicated in 70 and saved. As a result, the sequential file 60 at the time of commit (definite update) of the database 50 is synchronized.

  The commit number information recording unit 14 performs commit (update confirmation) on the database 50 and saves the record contents of the sequential file 60 to the backup file 70 by the commit unit 13a and the output file backup unit 13b of the database update confirmation unit 13. The processing information is recorded in the storage device.

  Thus, in the job processing system of this example, the record contents of the sequential file 60 at the time of committing the database 50 are automatically saved in the backup file 70, so that consistency with the database 50 after the rollback can be obtained. The backup file 70 is secured.

  When the process ends abnormally, the recovery process unit 21 in the abnormal end process unit 20 performs the process of recovering the record contents of the sequential file 60 from the secured backup file 70 together with the recovery process of the database 50. Synchronize.

  The recovery processing unit 21 of this example includes a rollback unit 21a that performs rollback processing, which is an existing function of DBMS (Data Base Managing System), and recovery processing as a recovery function of a processing control tool for batch processing that is a conventional technology The output file return unit 21b is configured to perform rollback processing on the database 50 by the rollback unit 21a, and recovery processing is performed on the sequential file 60 using the output file backup 70 by the output file return unit 21b.

  As a result, when the business program that commits the update of the database 50 multiple times ends abnormally, the sequential file 60 that is consistent with the rolled back database 50 has the same contents as the sequential file 70 at the time of the most recent commit completion. Can be recovered automatically.

  In this way, in the batch processing system of this example, by combining the sequential file automatic backup at the time of committing and the recovery function of the processing control tool for batch processing, the database 50 and sequential when committing multiple times with a single program are performed. The file 60 is in a consistent state.

  Hereinafter, the details of the update process of the database 50 and the sequential file 60 by the normal execution processing unit 10 will be described with reference to FIG.

  As described above, here, the records of the input file 30 are sequentially read by the business program, the data items are checked by the data check process, and if the data items are correct, the database 50 is updated after the data is edited and processed, If there is an error in the data item, a record is output as error information to the sequential file 60. At this time, the commit counter value (commit interval parameter 40) specified by the external parameter and the record count recorded by the business program are processed. When the values match, an example is used in which the commit (definite update) of the database 50 and the sequential file 60 are saved, and the sequential file 60 at the time of commit (definite update) of the database 50 is synchronized.

  In executing the process, first, a commit counter value that is a basis for the commit execution determination of the database 50 is read (step S201).

  Next, one record is read from the input file 30 (step S202), and 1 is set to the record counter value for counting the number of record inputs of the input file 30 that is the commit determination value (step S203).

  Next, the processing part (steps S204 to S214) that is repeated until the record reading of the input file 30 is completed (EOF) will be described.

  First, an item check is performed on the input record to determine whether to update the database 50 (step S205).

  As a result of the determination, if the record is an update target record in the database 50, the data item is edited / processed, temporarily written in the database 50 (step S206), and the determination result is a non-update target record in the database 50. If there is, the input record is output to the sequential file 60 (output file) (step S207).

  Next, the commit counter value and the record counter value are compared (step S208). If the counter values are the same, commit (update confirmation) to the database 50 is performed (step S209) and all the sequential files 60 (output files) The record is output to a sequential file for backup (backup file 70) (step S210).

  Then, the record counter value is set to “0” (step S211), the next one record is read from the input file 30 (step S212), and 1 is added to the record counter value (step S213). Return to S204).

  In the comparison in step S208, if the counter values are different, one record is read from the input file 30 (step S212), 1 is added to the record counter value (step S213), and the beginning of the repeated portion (step S204). Return to.

  If the input file 30 is the last record, commit (update confirmation) is executed (step S215) after the process of the repeated portion (steps S204 to 214) is completed, and the process is terminated. At the time of commit execution in steps S209 and S215, the commit number information recording unit 14 records information such as the number of input records at that time in the storage device.

  Next, an execution example of business processing for automatically recovering the sequential file 60 at the time of abnormal termination of update processing by the abnormal termination processing unit 20 will be described with reference to FIGS.

  In the explanation of this example, the records of the input file 30 are sequentially read by the business program, the record reading of the input file 30 is 450,000th, the exception data (data that does not cause an error in the data check process), the shortage of the database 50 area, etc. As an example, the automatic recovery function of the sequential file 60 synchronized with the recovered database 50 when the process ends abnormally as described above.

  As shown in FIG. 4, every time the number of records read from the input file 30 reaches 100,000, the commit (update confirmation) in the database 50 and the backup file 70 (evacuation file) for the sequential file 60 (output file) are stored. When the save process is executed and the record reading of the input file 30 is 450,000th and the process ends abnormally (step 301), the database 30 is processed by the rollback unit 21a of the recovery processing unit 21 as shown in FIG. The DBMS rollback function automatically recovers to the end of the 400,000th processing, which is the final commit time (steps 302 to 303).

  Regarding the sequence file 60, the backup file 70 at the end of the 400,000th processing, which is the final commit time of the database 50, is automatically acquired, and the output file return unit 21b of the recovery processing unit 21, that is, the process control of batch processing As shown in FIG. 6, the sequential file 60 is automatically recovered from the backup file 70 by the tool recovery function (steps 305 to 307).

  As a result, the database 50 and the sequential file 60 are properly synchronized. When reprocessing is performed after the cause of the abnormal termination is removed, the number of input files up to the time of the final commit (update confirmation) is read. It is possible to eliminate useless input processing by the conventional technique of skipping.

  As described above with reference to FIGS. 1 to 6, in this example, in the business program, the sequential file 60 at the time of committing the database 50 is automatically saved in the backup file 70, and the committed database 50 and The contents of the sequential file 60 with consistency are secured, and when the process ends abnormally, the recovery process by the rollback process of the database 50 by the DBMS function is performed, and the process control tool of the batch process is also applied to the sequential file 60. Automatic data recovery from the backup file 70 is performed by the recovery processing function.

  In this way, a single program is obtained by combining automatic backup of the sequential file 60 to the backup file 70 at the time of committing and sequential file recovery performed by the recovery function of the processing control tool for batch processing, which is a conventional technique. Thus, the database 50 and the sequential file 60 in the case of committing a plurality of times can be brought into a consistent state.

  Thus, in a batch processing system that commits the update of the database 50 a plurality of times with a single business program and updates the sequential file 60, for example, even when the processing of the business program ends abnormally, The sequential file 60 can be automatically recovered to the content at the time of the latest commit completion that is consistent with the rolled back database 50.

  As described above, even when processing abnormal termination occurs when updating a large amount of data by automatically saving the sequential file 60 linked with the commit time of the database 50 and recovering when processing is abnormal, the database Re-execution at the time of abnormal termination can be performed easily and accurately without being aware of the consistency between 50 commits and sequential file 60 update.

  That is, when a large amount of data is updated, the sequential file 60 can also be automatically recovered in accordance with the recovery point by the rollback of the database 50 when the processing ends abnormally. It can be done well.

  In addition, this invention is not limited to the example demonstrated using FIGS. 1-6, In the range which does not deviate from the summary, various changes are possible. For example, in this example, a batch processing system and a database system are provided in one computer device, but the batch processing system and the database system are connected to each other via a network. It is good also as a structure provided in the apparatus.

  The computer configuration of this example may be a computer configuration without a keyboard or optical disk drive. In this example, an optical disk is used as a recording medium. However, an FD (Flexible Disk) or the like may be used as a recording medium. As for the program installation, the program may be downloaded and installed via a network via a communication device.

It is a block diagram which shows the structural example of the batch processing system which concerns on this invention. It is a flowchart which shows the processing operation example which concerns on this invention of the batch processing system in FIG. It is explanatory drawing which shows the example of recovery processing of the database at the time of the update process abnormal end by the batch processing system in FIG. 1, and recovery processing of a sequential file. It is explanatory drawing which shows the update process operation example of the database and sequential file until update process abnormal end by the batch processing system in FIG. It is explanatory drawing which shows the recovery processing operation example of the database at the time of the update process abnormal end by the batch processing system in FIG. It is explanatory drawing which shows the recovery processing operation example of the sequential file at the time of the update process abnormal end by the batch processing system in FIG.

Explanation of symbols

  10: Normal execution processing unit, 11: Input file reading unit, 12: Database update unit, 12a: File output unit, 12b: Database update unit, 13: Database update confirmation unit, 13a: Commit unit, 13b: Output file backup unit , 14: commit number information storage unit, 20: abnormal end processing unit, 21: recovery processing unit, 21a: rollback unit, 21b: output file return unit, 30: input file, 40: commit interval parameter, 50: database, 60: Sequential file (“output file”), 70: Backup file (“output file backup”).

Claims (4)

A method of performing recovery processing from abnormal termination of data update processing of a database and a sequential file in a batch processing system by a programmed computer,
A first step of storing data of the sequential file at the time of committing the database in a backup device;
And a second step of recovering data from the backup device for the sequential file together with the recovery processing of the database. The batch processing recovery method according to claim 1,
The second step is
Perform the database recovery process using the rollback function of the DBMS,
A batch processing recovery method characterized in that data recovery from the backup device for the sequential file is performed using a recovery function of a batch processing control tool. A system for performing recovery processing from abnormal termination of data update processing of a database and a sequential file in a batch processing device by a programmed computer,
A batch processing system comprising means for executing processing at each step in the batch processing recovery method according to claim 1.   The program for making a computer perform the process in each step in the batch process recovery method in any one of Claim 1 or Claim 2.

Images