JP2004054816A - Database management method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent a system from stopping due to the shortage of a log recording region due to logging that is generated during data loading. <P>SOLUTION: The amount of log that is generated during the execution of a data load program 24 is estimated from an input data file 42 for recording added data, and the processing of the data load program 24 is completed when there is not any sufficient free space after comparing with the free space of a log file 32. The free space of the log file 32 is monitored during the addition of data, and transaction for adding data is settled when there are no sufficient regions for recording a log, thus suspending processing for loading data. After the supervisor of a database management apparatus secures the free space of the log file 32, the data load program 24 is executed again, and data addition processing is resumed. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technology of a database management device that manages data accessed from a plurality of transactions and records update information as a log.
[0002]
[Prior art]
The database management device is a computerized system that records and holds data. In the database management device, a logical processing unit including a series of data search and update is called a transaction.
If a transaction ends normally, updates processed by the transaction are considered permanent until updated by another transaction, and even if a failure occurs after the normal end of the transaction, the database management device can update the data. Guarantee recovery.
If a failure occurs or the transaction is canceled before the transaction ends, it is guaranteed that the data updated so far returns to the data before the update (rollback).
In order to guarantee the recovery of such a transaction, the database management apparatus records an update history generated in a transaction called a log.
Normally, the log includes data before update and data after update. When the transaction is rolled back, the database is returned to the state before update by the data before update in the log (UNDO processing), and the data is recovered for recovery after a failure occurs. Recovers the database to the updated data using the updated data in the log (REDO process).
These prior arts are described in J. Gray, A.J. Reuter et al., "Transaction Processing: Concepts and Techniques", Morgan Kaufman Publishers, Inc. It is described in.
[0003]
[Problems to be solved by the invention]
In the above-described related art, since a log generated by a transaction to be recovered is necessary for recovering a transaction, if the log recording area is insufficient, recovery of the transaction cannot be guaranteed.
At this time, the database management device cannot continue the transaction processing, suspends the transaction being processed, and becomes unable to accept a new transaction, or an area for recording a log necessary for suspending the transaction. If not, the system must be shut down.
For this reason, the administrator of the database management apparatus needs to estimate the amount of logs generated by transactions occurring during operation and secure an area for recording a sufficient amount of logs.
However, when performing data loading for the purpose of registering a large amount of data, the log generated by updating the database increases in proportion to the update amount. Insufficient, and the database management device will be stopped.
When the log estimate is insufficient, for example, the amount of data added daily increases in the process of operating the database management device, the amount of data to be registered becomes large, and the estimated value of the initial log amount is insufficient, In some cases, the amount of data to be added increases and the amount of data to be registered becomes large.
An object of the present invention is to prevent the system from being stopped due to a shortage of a log recording area during data loading, particularly during data loading of a large amount of data.
[0004]
[Means for Solving the Problems]
In order to achieve the above object, the present invention performs a data load process of adding data to a database by inputting an input data file recording data to be added, according to a received transaction. Before starting, estimate the amount of log generated by the additional data in the input data file, and determine whether the free space in the log file is smaller than the estimated log amount. End the loading process.
Further, during the process of adding data from the input data file to the database, the free space in the log file is monitored, and when the free space becomes smaller than a predetermined amount, the transaction is committed and the data loading process is terminated. .
Further, when the transaction is committed, the position information of the next data to be added in the input data file is recorded in the update information file, and a message of interruption of the data addition processing is displayed on the terminal device which issued the transaction.
[0005]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings.
FIG. 2 shows a configuration of a database management apparatus 10 suitable for registering a large amount of data according to the present invention.
The database management device 10 includes an external storage device 11 and is connected to a terminal 13 via a network 12. The external storage device 11 has a database 31 for recording data managed by the database management device 10 and a log as a data update history. Is stored in the log file 32, which is an area for recording.
The database management device 10
An inquiry request issued from a program executed in the terminal 13 or an inquiry request issued from the data load program 24 is received and analyzed to determine an access route to the database (that is, a required data in the database 31). Tables are stored in several parts, so determine the order in which these tables are combined or accessed using what kind of index structure) An inquiry control unit 21 for instructing the access control unit 22;
A database access control unit 22 for controlling a data buffer 26 for the purpose of inputting / outputting to / from the database 31 and reducing the number of input / output operations;
A dictionary management unit 23 for managing definition information of data stored in the database,
A log management unit 25 that receives a log record generated when the database 31 is updated and manages input / output to / from the log buffer 27 and the log file 32;
It comprises a data load program 24 that reads a data load input data file 41 in which the contents of data to be added are recorded, and controls data addition processing.
The data load input data file 41 is a file of data to be stored in a database, and the data of the file is input from a terminal via a network or by a database management device, for example.
[0006]
Next, the operation at the time of data loading processing suitable for registering a large amount of data according to the present invention will be described with reference to the processing flow of FIG.
The operator instructs the terminal 13 to execute the data load program 24. At this time, a table in the database 31 to which data is to be added and a data load input data file 41 in which data to be added and accessible by the database management apparatus 10 are recorded are specified.
The data load program 24 estimates the amount of log generated by execution of the instructed data load based on the amount of data to be added in the data load input data file (step 201), and obtains the free space of the log file 32 (step 201). 202).
Next, the estimated value of the log amount obtained is calculated as a percentage of the free space of the log file registered in the database management apparatus in advance (for example, the percentage may be registered in the dictionary management unit in advance, or a system not shown or It is determined whether or not the value is smaller than the value obtained by multiplying by (registered in the definition file) (step 203).
If it is determined to be false in step 203, a warning message is displayed on the terminal 13 (step 216) because there is a possibility that the free space of the log file may become insufficient due to the execution of the data load, and the process ends.
In particular, when registering a large amount of data, the free space of the log file is likely to be insufficient, and the operator can be immediately notified of the shortage.
FIG. 3 shows an example of a warning message displayed on the terminal 13.
When it is determined in step 203 that the estimated value of the log amount does not exceed a certain ratio of the free space of the log file, one data (one record) of the data to be added consisting of a plurality of data from the data load input data file 41 is determined. Is read (step 204), an inquiry for adding data to the database is generated (step 205), and execution of the inquiry is requested to the inquiry control unit 21 (step 206).
At this time, the message of the inquiry request includes the identifier of the request source of the inquiry, and at the time of the inquiry request in step 206, a value indicating the data load program is set.
[0007]
The inquiry control unit 21 receives the inquiry (Step 207), analyzes the inquiry (Step 208), and in the case of the inquiry requested in Step 206, obtains that the data is added as a result of the analysis, and obtains the database access control. It instructs the section 22 to add data (step 209).
[0008]
The database access control unit 22 updates the data in the data buffer 26 with the data stored in the part where the data is added in the database 31 (step 210), and edits the data addition log (step 211).
Here, the log includes a log type for identifying the contents of the log such as addition, update, and deletion. If the identifier of the inquiry request source included in the inquiry message is that of the data load program, the log type is data load. It is set that data is added by the program.
After editing the log, the log management unit 25 is requested to output an update log (step 212).
[0009]
The log management unit 25 writes the log record to the log buffer (Step 213).
The log records recorded in the log buffer are written to the log file 32 when the log buffer becomes full or when any transaction commits or rolls back.
Here, the control of the data buffer 26 and the log buffer 27 conforms to a generally well-known write-ahead logging (WAL) protocol, and the contents of the log buffer 27 are stored in a log file before the updated contents of the data buffer 26 are written to the database 31. 32 is controlled.
The log management unit 25 determines the type of the log for which the output request has been received, and if the log is a log generated by adding data by the data load program 24, checks the free space of the log file 32. If the size of the log written to the log buffer is not larger than a value obtained by multiplying the capacity by a certain percentage registered in the database management apparatus in advance (step 214), "normal termination" is returned as a response to the inquiry.
[0010]
The data load program 24 that has received the response to the inquiry determines whether or not additional data remains in the data load input data file 41 (step 215), and if additional data remains, repeats the processing from step 204 onward. If no additional data remains, the data loading process ends.
[0011]
If the size of the log written to the log buffer is larger than the value obtained by multiplying the free space of the log file 32 by a certain percentage registered in the database management device in advance in step 214, the log is written at the time when a write request to the log file 32 is generated. Since the data load program 24 may not be recorded, the transaction generated by the execution of the data load program 24 is committed (step 218), and "auto commit" is returned to the data load program as a response to the inquiry.
[0012]
The data load program 24 receives "automatic commit" as a response to the inquiry, records the position of the additional data to be added next in the data load input data file 41 in the update information file 42 (step 219), and adds the data. A message indicating that the processing has been interrupted is displayed on the terminal 13 (step 220), and the processing ends. FIG. 4 shows an example of a data load interruption message displayed on the terminal 13.
[0013]
If a message is displayed on the terminal in step 220, the operator can know that the data loading process has been terminated due to lack of free space in the log file 32.
The operator specifies the data load input data file 41 and the update information file 42 after instructing the data load program 24 to execute the data load program 24 from the terminal 13 after performing the measure for increasing the free space of the log file 32.
In particular, in the case of registering a large amount of data, the free space of the log file is likely to be insufficient, and according to the present embodiment, the operator can quickly know the lack of the free space of the log file, and It is possible to appropriately cope with the shortage.
The data load program obtains the position of the next additional data in the data load input data file 41 recorded in the update information file 42 in step 219, and continues the data load processing according to the processing flow of FIG.
[0014]
Further, in another embodiment of the present invention, after displaying a warning message on the terminal 13 in step 216, the processing from step 204 onward is continued.
The operator who has instructed the execution of the data load program 24 can recognize from the message displayed on the terminal 13 immediately after instructing the execution that the free space of the log file 32 is insufficient, and immediately the free space of the log file 32 is obtained. By performing the process of increasing the capacity, it is possible to avoid a shortage of free space in the log file 32 during data loading.
Further, even if the operator does not take any measures to increase the log recording area, if the free space of the log file 32 during the data loading is reduced in step 214, the data loading process is terminated. It does not stop due to lack of free space.
[0015]
【The invention's effect】
According to the database management apparatus of the present invention, it is possible to warn the operator before starting the data addition processing of the data loading processing in which the log recording area is considered to be insufficient during the data loading processing. Stopping can be prevented beforehand, and is particularly effective when loading a large amount of data.
Further, when the free space of the area for recording the log becomes small due to the large amount of data added by the data loading, the data addition process is automatically interrupted, and the system can be prevented from being stopped.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic flow of a data loading process.
FIG. 2 is a diagram showing a configuration of a database management device for implementing the present invention.
FIG. 3 is a diagram showing a display example of a warning message on a terminal.
FIG. 4 is a diagram showing a display example of a data load interruption message on a terminal.
[Explanation of symbols]
Reference Signs List 10 database management device 11 external storage device 12 network 13 terminal 21 inquiry control unit 22 database access control unit 23 dictionary management unit 24 data load program 25 log management unit 26 data buffer 27 log buffer 31 database 32 log file 41 data load input data file 42 Update information file

Claims (6)

A database management method for performing a data load process of adding data to a database with an input data file recording data to be added as an input according to a received transaction,
Before starting the data addition processing, the amount of log generated due to the additional data in the input data file is estimated, and it is determined whether the free space in the log file is smaller than the estimated log amount. A database management method, wherein a data loading process is completed before starting. A database management method for performing a data load process of adding data to a database with an input data file recording data to be added as an input according to a received transaction,
During the process of adding data from the input data file to the database, the free space in the log file is monitored, and when the free space becomes smaller than a predetermined amount, the transaction is committed and the data loading process is terminated. Characteristic database management method. The database management method according to claim 2,
When the transaction is committed, the location information of the next data to be added in the input data file is recorded in the update information file, and a message of the interruption of the data addition process is displayed on the terminal device that issued the transaction. Characteristic database management method. A database management device that performs a data load process of adding data to a database by inputting an input data file recording data to be added according to a received transaction,
Means for estimating the amount of log generated by the additional data in the input data file before starting the data addition processing;
Means for determining whether the free space in the log file is smaller than the estimated log amount,
A database management device comprising means for terminating a data load process before starting a data addition process when a result of the determination is small. A database management device that performs a data load process of adding data to a database by inputting an input data file recording data to be added according to a received transaction,
Means for monitoring the free space in the log file during the process of adding data from the input data file to the database;
A database management device comprising means for committing the transaction and ending the data loading process when the free space becomes smaller than a predetermined amount. The database management device according to claim 5,
Means for recording the position information of the next data to be added in the input data file in the update information file when the transaction is committed, and displaying a data addition interruption message on the terminal device that issued the transaction. A database management device having means.

Images