JP2002157156A - Database management method and system, processing program therefor and recording medium stored with the program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that a database operation stop time or a batch operation time cannot be secured on account of 24-hour global use of a database system, e.g. typically under Web environment. SOLUTION: This database management method is as follows. Inputted data are stored in a first database. When set conditions are satisfied, data not registered in a second database is obtained from the first database on the basis of management information managing data registered in the second database. The obtained data is registered into the second database, and the management information of the obtained data is updated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a database management technique having a plurality of databases, and more particularly to a database management technique for storing information of one database in another database.

[0002]

2. Description of the Related Art As a high-availability solution, a database (DB) management system is duplicated and one system is a CPU.
In order to prevent the entire database system from stopping due to a system failure such as power down or power down, HA (Hi
gh Availability) system, and the like. For example, in the literature (Jim Gray and Andreas Reu
ter, Tranzaction Processing: Concepts and Techniqu
es, Morgan Kaufmann Publishers, 1993) discloses non-stop operation of hot standby as a measure against DB failure by HA system configuration. In addition, in order to guarantee high reliability, the disk device storing the database has a RAID (Redundant Array of Inexpensive Disk).
s) Often used together. This RAID configuration:
Literature (David A. Patterson et. Al., A Case for Redund
ant Arrays of Inexpensive Disks (RAID), ACM SIGMOD
1988.). Further, there is a replica DB as a function of accessing a database between distributed systems from the same viewpoint. This replica DB
The technology is described in the literature (Date, CJ, An Introduction to Datab
ase Systems: Volume 1 Fifth Edition, Addison-Wesle
y Publisher, 1990).

[0003]

However, the database management system is applied to a Web environment that is simultaneously accessed by tens of thousands of users, or a DWH system that manages a large-scale DB consisting of tens of TBs to which analysis data is added daily. Under the present circumstances, it is essential to cope with the DB amount, the number of users, the load amount, etc., which greatly increase and fluctuate with the operation of these database systems. The DB design (data division method, memory amount, processor / server number,
Disk allocation) can be used,
When the database system is initially operated, the database system must be operated with the minimum necessary system configuration in consideration of the enormous system cost that is expected to increase. Therefore, when the DB amount, the number of users, and the load amount increase or decrease, the initial DB design is reviewed and rebuilt. Heretofore, based on such a DB design policy, the operation of the database system has been stopped, that is, the DB service has been stopped, and the data division method, the amount of memory, the number of processors and servers, and the amount of disks have been changed as necessary. Was.

[0004] Various causes can be considered for the suspension of the DB service. For example, in a Web environment, the number of services increasing year by year, and in response to access requests from clients such as Web browsers, it is necessary to add a server of a database management system in accordance with the processing capacity required to process these requests. Will be needed. In addition, by adding services to the client, the number of application programs increases, and the viewpoint of the database becomes diversified. As a result, the number of corresponding tables increases, and as a result, the amount of the DB increases year by year. It is necessary to consider the imbalance of the DB amount and the balancing of the DB access load by adding disks for storing these DBs and changing the schema of the data division method. Further, in order to prevent a decrease in search efficiency due to disorder in the DB structure due to operations such as a large amount of DB deletion, it is necessary to periodically reorganize the DB.

[0005] These DB operation functions are performed by stopping the operation of the database system at night or batch operation at the weekend. However, at present, as the database system has been used globally for 24 hours globally, as represented by the Web environment, there are situations in which the DB operation stop time and the batch operation time cannot be secured. Therefore, D
There is an increasing demand for realizing these operations in parallel without stopping service B.

[0006] Such a database operation function is performed by suspending operation of the database system at night or batch operation at the weekend. However, when the database system is used globally for 24 hours on a global scale, as represented by the Web environment, a situation arises in which the database operation stop time and the batch operation time cannot be secured.

An object of the present invention is to provide a database management method and system for storing data in a database in parallel with a database service.

[0008]

The above object is achieved by the following means. A database management method in a database management system having a first database and a second database includes the following steps. (1) a first step of storing input data in the first database; (2) when a set condition is satisfied, based on management information for managing data registered in the second database. A second method of obtaining data not registered in the second database from the first database, registering the obtained data in the second database, and updating the management information for the obtained data. Step As described above, according to the set conditions, data not registered in the second database is obtained from the first database based on management information for managing data registered in the second database. Since the stored data can be registered in the second database, the data storage in the second database is performed by the first database service. In parallel it is possible to process with.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION Various factors can be considered for stopping a DB service. For example, in a Web environment, the number of services increasing year by year, and in response to access requests from clients such as Web browsers, it is necessary to add a server of a database management system in accordance with the processing capacity required to process these requests. Will be needed. In addition, by adding services to the client, the number of application programs increases, and the viewpoint of the database becomes diversified. As a result, the number of corresponding tables increases, and as a result, the amount of the DB increases year by year. It is necessary to consider the imbalance of the DB amount and the balancing of the DB access load by adding disks for storing these DBs and changing the schema of the data division method. Further, in order to prevent a decrease in search efficiency due to disorder in the DB structure due to operations such as a large amount of DB deletion, it is necessary to periodically reorganize the DB.

[0010] These DB operation functions are performed by stopping the operation of the database system at night or batch operation at the weekend. However, at present, as the database system has been used globally for 24 hours globally, as represented by the Web environment, there are situations in which the DB operation stop time and the batch operation time cannot be secured. Therefore, D
There is an increasing demand for realizing these operations in parallel without stopping service B. Do not stop this DB service,
In order to operate a database, it is necessary to solve the following problems. (I) Connectivity and expandability Since the server that performs database processing and the storage that manages the database can operate independently and independently, data management can be performed, so that the connection configuration and addition of each server and storage can be performed while the server or storage is operating. Changes are easier. (Ii) Data sharing Since storage can be shared among a plurality of servers, data movement and conversion can be performed independently of a flat form. (Iii) High availability Data that should not be destroyed can be multiplexed or copied to a remote location.

Satisfying these requirements is a SAN (Storage Area Network), which has attracted attention as next-generation IT.
Solution. The goals of the SAN solution are the following four points. (A) Significant improvement of storage input / output performance (Performance) (b) Flexible storage environment setting and expansion independent of server environment (Scalability) (c) Centralized operation of storage (Storage Management Efficiency)
(d) Disaster countermeasures by dramatically increasing the connection distance (Dat
a Protection) In particular, considering the scalability of the database system, it is promising to make use of the above feature (b).

Here, a specific database system will be discussed. In the major retail sales performance management system, the sales history information of each retail store operating in the area
It is integrated by an OS terminal or the like. This information is
It is associated with inventory management, order management, product delivery planning, and so on. It is also conceivable that the result of the analysis of the hot-selling products is immediately reflected in the in-store product arrangement. For this purpose, the record including the sales history code and the like is reflected in the RDB system in chronological order in the DB, and the D record is reflected.
Build B. Also, an OLAP system capable of instantaneous OLAP analysis using multiple analysis axes is constructed using the history DB accumulated here as a source. This OLAP system needs to reflect the history DB constructed by the RDB system. In the replica DB of the prior art, the extraction source R
The parts changed by the update to the DB are extracted, and these are reflected in the reflection destination RDB immediately or with a certain time delay. In the replica DB, since both the extraction source and the reflection destination are RDBs, basically, a copy of the same data (including a partial join or classification into a plurality of tables is included) is created. Here, there are two possible reflection methods from the RDB system to the OLAP system. (a) Initial creation Data loading of all the history DBs built in the RDB system and reflecting it in the OLAP system (b) Additional update Typical addition and update of the history DB in the OLAP system D for sales history DB
The B specifications and the access characteristics are as follows. Estimating from the sales results of major retailers, it is estimated that there are 7.7M records / day, and by registering records of 200B to 1KB length for each result, about 40GB to 200GB
/ Month history DB scale. About 0.4TB to 2T per year
B's DB will be added. This history DB constitutes a star schema, and is often used in a task of creating a statistical summary by an SQL aggregation query.

In the business of dynamically analyzing the sales history DB on a more multidimensional axis, an OLAP system is used.
It is necessary to construct 24-month history data, and to analyze the sales data for each sales district or each product by month-to-month ratio, month, period, and the like. Further, there is a feature that 80% or more accesses are concentrated in the history data of about three months, and there is an increasing need to further improve the freshness of the data. As discussed above, it goes without saying that the services of the RDB system and the OLAP system must be provided without interruption.

[0014] OL for non-stop DB service in time series, sales history code, etc.
In order to construct (initial creation, additional update) the AP analysis work without stopping the RDB work, the following problems must be solved. (1) Since the database area to be additionally updated in the RDB system and the database area to be initially created and additionally loaded in the OLAP system overlap, the RDB system and the OLAP
Since access conflicts in the system, the system performance of the RDB system decreases. (2) In the OLAP analysis work, 80% of recent data access is concentrated, and it is also subject to additional load to the OLAP system, so the load is concentrated on a local database area.

To solve these problems, the following solution is considered.

To solve the problem caused by the conflict between the database areas to be accessed, consider the following solution. (a) Separating the database storage area The database area accessed by the RDB system is mainly for data addition and reference, and it is only readout for additional loading and reflection on the OLAP server.
By using the multiplexing function of the disk device or the disk double writing provided by the OS, it is possible to cope with having a plurality of database areas physically. This database area is premised on that writing is synchronized. The contents of the primary Vol (with data addition and reference) and the secondary Vol (DB that has consistency at the designated time) are different, and the access load is divided by being physically isolated. (b) Use of update history information system log of database management system The database management system holds an update history information system log for recovery from a DB failure. That is, since this system log contains all update information to the database system, it is possible to extract update information from a certain point in time. In order to additionally load and reflect the data on the OLAP server, data on the corresponding history DB table extracted from the system log can be created. (c) Method of localizing the reflection part to OLAP server The method of dividing the database by key value range, hash division, etc. is performed. However, since recent data is subdivided and 80% access is concentrated. In order to enhance the parallelization effect, a multi-stage hierarchical division is adopted. That is, OLA
Since the data to be additionally loaded to the P server has a characteristic of gathering recently in time series, data that is close in time is localized as a divided section using the above-described division method. This minimizes the load on the RDB system and enables reading for additional loading.

When data is reflected from the RDB system to the OLAP system, it is required that OLAP analysis can be performed in a consistent state as a certain database. That is, it is necessary to show data that has been matched as a database at a certain point in time from the RDB system to the OLAP system. To solve this problem, the following means are applied. (a) Divided management of the database of the RDB system in chronological order Since the database is divided and managed in chronological order, the data access range to be additionally loaded can be localized. The RDB system side can determine which time series range should be additionally loaded this time by knowing up to which time series range was previously reflected in the OLAP system side. (b) Data reflection based on system log information In the RDB system, the update reflection information to the database is maintained and managed as a system log. Data to be additionally loaded into the OLAP system may be obtained by extracting an update history from a certain point in the corresponding history DB table from the system log information.

By combining with the above-mentioned SAN solution, the server for performing the DB processing and the storage for storing the DB are connected via a high-speed network such as an optical fiber. Dynamic server addition and disk addition can be easily operated by simply changing information. Furthermore, since each storage uses a RAID disk, normal operations are stopped by securing the reliability by multiplexing and allocating areas necessary for work in DB operation as dynamic devices and using functions such as release. DB reorganization operation can be performed without doing so. Specifically, the scale of the DB system is expanded by adding a server and a disk, that is, a scalable system can be constructed.

The above problem is solved by the following means. (1) In a system that manages DB division, a new first DB
A storage area and a second DB storage area are allocated, the copy of the first storage area is reflected on the second storage area, and when the copy instruction is released, writing to the first storage area is performed, and When a copy instruction is issued, data written in the first storage area is reflected in the second storage area while the copy instruction is released. (2) In (1), the copy instruction is released. During this time, the system log in which the transaction has been completed up to the time specified in the second storage area is reflected in the second storage area,
To read the storage area. (3) In (1), as a DB division management method, a key range division, a hash division, a round robin division, and a division unit that combines them in a pluralistic manner are used, and the latest data is subdivided in a time series to obtain data. To split. (4) In (1), the first DB storage area and the second D
The database area to which the B storage area is assigned is a database area for storing more recent data in chronological order from the time point of the previous reflection. (5) In a system that manages DB division, a first DB storage area is allocated, and pertinent data from the system log, which is a writing history of the first storage area, since the last time the data was reflected is read.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. An OLAP analysis job for a history DB mainly for additional writing is constructed (initial creation, additional update) without stopping the RDB job using a time series, a sales history code, or the like. DB
The division management has been applied as a method of localizing a DB failure and subdividing a backup unit. However, the division management is applied as a method of localizing a change in a DB amount due to additional writing and a portion reflected on an OLAP server.

FIG. 1 shows an overall configuration for reflecting data from the RDB system to the OLAP system. The RDB system includes a database management system 20 including application programs 10 and 11 and a data extraction unit 200. The OLAP system includes an OLAP server system 60 including application programs 11 and 12 and a data reflecting unit 600. The databases of the RDB system and the OLAP system are connected via a SAN. The database of the RDB system is divided into areas.
As shown by, 300 of area 1, 301 of area 2, 302 of area 3, 303 of area 4, and 30 of area 51
4, 305 of area 52 and 306 of area 53, respectively. 304 of area 51, 30 of area 52
5, 306 of the area 53 is stored and managed as a primary Vol. However, when the disk has a dual write function, 307 of the area 51, 308 of the area 52, and 3 of the area 53
In 09, the secondary Vol is stored and managed. The database of the OLAP system is divided into areas, and is stored and managed in 700 in area 1, 701 in area 2, and 702 in area 3.

When data is updated in the RDB system, the data is written to the database by the database management system 20. In this example, 3 of area 51 which is a positive Vol
04, 305 in area 52, and 306 in area 53. Writing to the primary Vol is writing to the secondary Vol in the background. OL from RDB system
The data reflection on the AP system is performed via the data extraction unit 200 of the database management system 20 and the data reflection unit 600 of the OLAP server system 60. OL
The AP server system 60 stores the data reflected from the RDB system in the area 700, the area 2 701, and the area 3 702 based on the division schema.

FIG. 2 is a configuration diagram of the database management system 20. There are application programs 10 and 11 created by the user and a database management system 20 that manages the entire database system such as query processing and resource management. The database management system 20 includes a system control 21, a logical processing unit 22, a physical processing unit 23, and a database 30, a dictionary 50, and a system log 40 for permanently or temporarily storing data to be accessed. . Also,
The database management system 20 is connected to another system via a network or the like.

In a SAN-structured system, all or part of the database 30, dictionary 50, and system log 40 are stored in a disk device via the SAN 80. In this case, in the dictionary 50, the schema division method of dividing the database 30 into each area, the mapping between the area and the primary and secondary Vols, and the like are set. By updating this dictionary 50 information with a table definition statement, a table change statement, etc., a storage part of a schema-divided database in a disk device connected via a SAN to a server system such as a database management system, an OLAP server system, etc. And the mapping with the storage side such as primary and secondary Vols associated with these areas are set and updated.

The system control 21 performs input of an inquiry from a user, return of an inquiry result, maintenance of a database by receiving a data load / data reorganization / reconstruction command, coordination control with another system, and the like. The logic processing unit 22 includes a query analysis 220 that performs syntax analysis and semantic analysis of the query, and an optimization process 22 that generates an appropriate processing procedure.
1 and a code generator 222 for generating a code corresponding to the processing procedure. A code interpretation and execution 223 for interpreting and executing a code based on the generated code.
Is provided. Furthermore, a table definition command, a table change command, an area-Vol mapping command, a primary / secondary Vo
It has a command analyzer 225 for performing command analysis such as a control command. The physical processing unit 23 includes a data access control 230 that implements condition determination, editing, and record addition of accessed data, a database buffer control 231 that performs access control such as reading and writing of a database record, and a resource shared by the system. Exclusive control 232. Further, reading and writing of the database 30 are performed via the database buffer 24. The system log 40 accumulates update history information such as data insertion, update, and deletion in the database 30. The dictionary 50 includes storage information such as database schema information such as tables and indexes of the database 30, a data division method of the database 30, names of areas to be stored and managed, and page sizes. When a command such as a change in the data division method is executed, the dictionary 50 is changed.

Next, as one embodiment of dividing the sales history table in time series, a key range dividing method is used as a database dividing method. In addition to the key range dividing method, the database dividing method includes a hash dividing method, a round robin dividing method, and a method of multidimensionally combining these methods. These methods can be applied as the dividing method. FIG. 3 manages the sales history table by dividing it in time series. The sales history table is divided and managed for each year, half year, and month. Specifically, in fiscal 1997, 300, 9
In fiscal 2008, area 2 was 301, in the first quarter of 1999 it was 302 in area 3, and in the second quarter of 1999, it was 30 in area 4.
For July, 1999 and July, the area 51 of 304, 1999, 8
Monthly data for area 52 305, September 1999 data for area 5
3 is stored at 306. In this case, in the dictionary 50, a key range section and an area name corresponding to the section are set.

For example, this database division is executed by the following table definition statement. CREATE TABLE Sales history table (purchase code CHAR (16), product code CHAR (16), region code CHAR (16), time code DATE, sales amount DEC (10)) IN ((area 1) time code> = '1997 -12 ', (area 2) time code>=' 1998-12 ', (area 3) time code>=' 1999-03 ', (area 4) time code>=' 1999-06 ', (area 51) Time code> = '1999-07', (area 52) Time code> = '1999-08', (area 53) Time code> = '1999-09'); Areas have not been allocated for the half year, but when the data has been reflected in September 1999, the areas 51 304 and 52 allocated to July, August, and September 1999, respectively. 305,
Area 306 of area 53 is merged and managed as area 5.

FIG. 4 shows the execution of a table change statement in the case of changing the database division method at the time when the data has been reflected in September 1999. 304 of area 51, area 5
The table change sentence in which the database managed in the second section 305 is 307 in area 5 including 306 in area 53 is as follows. ALTER TABLE Sales history table MERGE AREA (Area 51, Area 52, Area 53) INTO ((Area 5) Time code> = '1999-09'); Area 51, area 52, and area 53 corresponding to the section are renamed to area 5,
Is set.

FIG. 5 is a processing flow of the logic processing unit 22. Command analysis execution, query analysis, and query execution of a table definition statement, a table change statement, and the like are realized by the logical processing unit 22. It is confirmed whether the command input by the user is an analysis request, an execution request, or a command analysis execution (225). If it is an analysis request, query analysis (22
0), optimization processing (221), and code generation (222). If the request is an execution request, code interpretation is executed (223). If it is a command analysis execution request, command analysis is performed (224). If it is a table definition command (226), the dictionary 50 describes how to divide a table into a database, the correspondence to each area name, and the like.
(227). If it is a table definition command (22
8), dictionary 5 according to change of table database dividing method and addition, deletion, merging, division, etc. of each area
Register to 0. If the command is an area-Vol mapping command (230), primary and secondary Vol information corresponding to each area is registered in the dictionary 50. If the command is a primary / secondary Vol control command (232), a disconnection instruction and a reconnection instruction to the primary / secondary Vol are issued to the corresponding disk device (2).
33).

FIG. 6 shows the principle of disk double writing.
In the following explanation, in order to apply the disc double writing function,
It is disclosed here. Add a pair of primary and secondary Vol to the disk device and set 2
When performing double writing, the operation differs depending on whether or not the disk dual writing function exists.

If the disk dual write function does not exist ((a)), the primary Vo
A write request is issued to each disk device with respect to 30 of area 2 of 1 and 31 of area 2 of sub-Vol. That is, the same data is always held on the two disk devices by software.

If the disk dual writing function is present ((b)), the database buffer management
A write request is issued to 30 in area 2 of ol. In a disk device having a disk double writing function,
30 of primary Vol area 2 and 31 of secondary Vol area 2
Is specified, and the positive V
The writing of “ol” to area 2 30 is reflected as the writing of sub-vol 31 to area 2. That is, the same data is always held on two disk devices in terms of hardware as a function of the disk device.

Further, the primary Vol area 2 30 and the secondary Vol area 2 31 are defined by the following area-Vol mapping statement. CREATE AREA Area 2 AS PRIMARY Primary volume 2 SECONDARY Secondary volume 2; According to this area-Vol mapping statement, the dictionary 50 stores the area and the primary V corresponding to the area.
ol name “primary volume 2” and secondary Vol name “secondary volume 2” are set.

FIG. 7 shows the database buffer management 231.
It is a flow detail of. It is called when there is a read / write request for a database page in the data access process 230. First, it is checked whether the corresponding page exists in the database buffer (2310). If it exists,
The buffer position is notified and the process is terminated (2311). If not, a page to be flushed is determined instead to read the corresponding page into the database buffer (2312). When writing a page, it is checked whether the disk device has a double writing function (2313).
If the dual writing function exists, a write request is issued to the primary Vol of the disk device (2315), and the contents of the primary Vol of the disk device are reflected on the secondary Vol (2316). If the dual write function does not exist, a write request is issued to each of the primary and secondary Vol pairs of the disk device by means such as an OS (2314). Then, the requested page is read into the empty database buffer (2317), and the process ends. Next, an embodiment in which data is reflected from the RDB system to the OLAP system will be described.

First, there is a case where the disk is duplicated and a case where the system log base data is reflected. Further, when the disc is duplicated, the disc is divided depending on the presence or absence of the disc double writing function. Each case will be described below with reference to FIGS.

FIG. 8 shows a case where there is no disk double writing. This corresponds to (a) of FIG. 6 and realizes disk duplication by software. First, in the (i) designated time data creation phase, a database state that is consistent at the designated time is created in the secondary Vol. Writing from the database buffer 231 to the area 31 of the sub-Vol 2 is suppressed. Next, the data extraction unit 200 reflects the system log information in a time-series manner so that the database is completed in the transaction from the system log 40 to the designated time. Next, in the (ii) data extraction reflection phase, the data extraction unit 200 sets the sub-Vol.
Data is extracted from the
Area 1 according to the schema division method of the AP server system
, 700 in area 2, and 702 in area m. Further, (iii) as the primary and secondary Vol reconnection phases, the state written in the primary Vol area 2 30 during the execution of the above phases (i) and (ii) and the secondary Vol
In order to maintain consistency with the status of the area 31 in the area 2, the data extraction unit 200 reflects the system log information in a time-series manner so as to make the latest database from the system log 40 to the present time.

FIG. 9 shows a case where the disc has double writing. This corresponds to FIG. 6B, and is based on the assumption that the disk provided in the disk device is duplicated. First, (i)
In the designated time data creation phase, a database state that is consistent at the designated time is created in the secondary Vol. Positive Vo
At the same time as the writing of 30 from the area 2 of 1 to the 31 of the area 2 of the sub-Vol is suppressed, the information to be written to the 30 of the area 2 of the primary Vol is set to be contrary to the difference information 3000. Next, the data extraction unit 200 reflects the system log information in a time-series manner so that the database is completed in the transaction from the system log 40 to the designated time. Next, in the (ii) data extraction / reflection phase, the data extraction unit 200 extracts data from the area 31 of the sub-Vol 2 and sends the data to the OLA through the data reflection unit 600.
Data is registered in 700 in area 1, 701 in area 2, and 702 in area m according to the schema division method of the P server system. Further, as the (iii) primary / secondary Vol reconnection phase, during the execution of the phases (i) and (ii), the primary V
In order to maintain the consistency between the state written in the area 30 of the sub-vol 2 and the state of the sub-vol 31 in the area 2, the difference information 3
000 is reflected in 31 of the sub-volume 2 and becomes the latest database up to the present time.

FIG. 10 shows a case in which system log-based data is reflected. First, obtain the system log point that reflects the data up to the previous time. This system log
The point represents the designated time. Next, after the system log point, the system log up to the designated time is reflected. Basically, the system logs are stored in chronological order, and the data may be reflected in this order. Specifically, data is extracted from the system log 40, and the data is extracted from the system log 40 via the data reflecting unit 600 according to the schema division method of the OLAP server system.
Data is registered in 0, 701 in area 2 and 702 in area m.

A detailed flow of the above data extraction and data reflection is shown. FIG. 11 is a flow of the data extraction unit 200. It is confirmed whether or not the data is reflected on the system log base (2001). If the data is system log-based data reflection, a system log point to which data has been reflected up to the previous time is acquired and notified (2002). Next, after the system log point, the system log information up to the designated time is reflected (2003). If the data is not reflected on the system log base, it is checked whether or not the disk device has a dual writing function (2004).

If the disk device has a dual writing function, the data reflection synchronization relationship between the primary Vol and the secondary Vol of the disk device is cut off (2005), and at the same time, the contents written to the primary Vol are reflected in the difference information 3000. The disk device is set to the mode (2006). Next, based on the system log, system log information is reflected in order to bring the secondary Vol into a database state in which the transaction has been completed by the designated time (2007). Furthermore, Vice Vol
Is extracted and the data is reflected (2008). Finally, the state written in the primary Vol and the secondary V
In order to maintain consistency with the status of the disk device, the difference information 3000 is reflected on the secondary Vol (2009),
And the secondary Vol are reconnected (2010).

If the dual writing function does not exist in the disk device, a mode for writing only the primary Vol to the disk device by means such as the OS is set, and writing to the secondary Vol is suppressed (2011). Next, based on the system log,
The system log information is reflected in order to set “ol” to the database state in which the transaction is completed by the designated time (2012). Further, the contents of the sub-Vol are extracted and the data is reflected (2013). Finally, in the above steps, the positive V
In order to maintain the consistency between the state written in "ol" and the state of the secondary Vol, and to make the latest database up to the present time, the system log information is reflected in the secondary Vol (2014), and the mode is returned to the primary / secondary Vol write mode (2015). .

FIG. 12 is a flow chart of the data reflection unit 600. Receiving data from the data extraction unit 200 (6
000), perform initial creation and additional loading, and register data in each area according to the division schema (6001). Further, the management information 9000 is updated based on the content reflected in the OLAP system (6002).

The difference between the replica DB system and the RDB-OLAP cooperation system of the prior art shown in FIG. 13 will be disclosed. In the replica DB method, portions changed by updating the extraction source RDB 301 are extracted, and the extracted portions are reflected in the reflection destination RDB 302 immediately or with a certain time delay.
In the replica DB, since both the extraction source and the reflection destination are RDBs, basically, a copy of the same data (including a partial join or classification into a plurality of tables is included) is created.
Therefore, when accessing the data of the reflection destination RDB 302 by a query, there is no need to specifically consider the relevance to the extraction source RDB 301. On the other hand, the RDB-OLAP cooperation method extracts portions changed by updating to the RDB system, and extracts these portions immediately or with a certain time delay.
Reflected on the system. In the RDB-OLAP cooperation method, when an OLAP system is accessed by a query,
Data 301 and RDB-OL managed by the LAP system
It has a meaning in combination with management information 9000 (time interval until reflection, time, area division, customer division, etc.) managed by AP cooperation. The RDB system manages fact data, and the OLAP system derives trend analysis results and the like derived therefrom. Therefore, the prerequisites for the analysis work are derived from the management information 9000 described above. For example, in customer purchase analysis, it is necessary to perform evaluations on various analysis axes based on time series such as time intervals and times, sales of sold products by region, and classification of customers who bought products. Based on the time series, OLAP system data 70 to be analyzed in order to analyze on a scale such as year-on-year, quarterly, last week, and previous day
1 must be appropriately reflected from the RDB system in which fact information is stored and managed according to the above scale. That is, in the inquiry to the OLAP system, management information reflecting these measures and data 7 of the OLAP system are used.
Only with 01, meaningful results will be obtained.

FIG. 14 is a detailed flow of the OLAP inquiry processing section 900. First, an OLAP query is analyzed, and the OLA to be accessed as a result of the query is
The data 701 of the P system is determined (901). Next, the management information 9000 that sets the data reflection status in the OLAP system is read, and data necessary for the inquiry result is created (902). Finally, the data 701 of the OLAP system is accessed and returned as a result of the inquiry by merging with the management information 9000 (90
3).

FIG. 15 shows the configuration of the system log. The system log includes a log sequence number, a time stamp, a log type, and a log content. The log sequence number is assigned an identifier that is unique in the RDB system. The time stamp is set to record the time at which the event related to the system log has occurred. The log type is the start of the extraction reflection process (event_star
t), suspension of extraction reflection processing (event_suspend), restart of extraction reflection processing (event_restart), commit of data (eve)
nt_commit), data update (event_update), and data extraction / reflection (event_reflect). The log content records update history information to the database. Based on these system logs, it is possible to control the extraction and reflection processing from the RDB system to the OLAP system.

The present invention can be realized through a tightly-coupled / loosely-coupled multiprocessor system, a software system of a large-scale computer, or a tightly-coupled / loosely-coupled multiprocessor system in which a dedicated processor is prepared for each processing unit. It is also possible to realize through. Further, even a single processor system is applicable as long as processes are allocated for system operation. The present invention is also applicable to a configuration in which a plurality of processors share a plurality of disks with each other. Note that the processing of the above-described flowchart can be realized by executing a program with a general data processing device as shown in FIG. The program can be stored in a computer-readable storage medium such as a hard disk device or a floppy (registered trademark) disk, and can be accessed through a network.

[0047]

According to the present invention, data reflection on a database can be processed in parallel with a database service.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram showing an outline of a data extraction and storage process in a database system.

FIG. 2 is a schematic diagram showing a configuration of a database management system according to an embodiment of the present invention.

FIG. 3 is a configuration diagram when a schema division method at the time of table definition is applied;

FIG. 4 is a configuration diagram in a case where a schema dividing method at the time of changing a table is applied;

FIG. 5 is a configuration diagram of a detailed flow of a logical processing unit;

FIG. 6 is a configuration diagram of a disk double writing function.

FIG. 7 is a configuration diagram of a detailed flow of database buffer management.

FIG. 8 is a configuration diagram of a data extraction / storage method without a disk double writing function;

FIG. 9 is a configuration diagram of a data extraction / storage method in a case where a disk dual writing function is provided.

FIG. 10 is a configuration diagram of a system log-based data extraction and storage method.

FIG. 11 is a configuration diagram of a detailed flow of a data extraction unit.

FIG. 12 is a configuration diagram of a detailed flow of a data storage unit;

FIG. 13 is a conventional conceptual diagram of an RDB-OLAP cooperation system.

FIG. 14 is a configuration diagram of a detailed flow of an OLAP inquiry processing unit;

FIG. 15 is a configuration diagram of a system log.

[Explanation of symbols]

 10, 11, 12, 13 application program 20 database management system 30 database 40 system log 50 dictionary 60 OLAP server system 80 SAN

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) G06F 17/30 240 G06F 17/30 240A (72) Inventor Nobuyuki Yamashita 890 Kashimada, Saiwai-ku, Kawasaki-shi, Kanagawa Prefecture Hitachi, Ltd. F-term in business solution business (reference) 5B075 KK03 NR02 PR03 5B082 DE04 GA04 5B085 AC05 AC12 AC14

Claims (20)

[Claims] 1. A database management method in a database management system having a first database and a second database, wherein a first step of storing input data in the first database and a set condition are performed. When satisfied, data not registered in the second database is obtained from the first database based on management information for managing data registered in the second database, and the obtained data is stored in the second database. A second step of registering the management information in the database and updating the management information with respect to the obtained data. 2. The database management method according to claim 1, wherein the first step includes, when registering data in the first database, creating a system log indicating that the data has been registered; In the second step, when a set condition is satisfied, data not registered in the second database is obtained from a system log based on management information for managing data registered in the second database, A database management method, wherein the obtained data is registered in the second database, and the management information is updated for the obtained data. 3. The database management method according to claim 1, wherein said condition is a predetermined time. 4. The database management method according to claim 1, wherein said condition is a predetermined time. 5. The database management method according to claim 1, wherein the condition is an end of a process of registering the obtained data in the second database. 6. The database management method according to claim 1, wherein in the second step, a plurality of the obtained data are stored in a storage area, and the stored data is registered in the second database. A database management method comprising: 7. The database management method according to claim 1, wherein said condition is an end of a process of registering said plurality of obtained data in said second database. 8. A database management system in a database management system having a first database and a second database, wherein a first means for storing input data in the first database and a set condition are set. At this time, data not registered in the second database is obtained from the first database based on management information for managing data registered in the second database, and the obtained data is stored in the second database. And a second means for registering the management information with respect to the obtained data and updating the management information with respect to the obtained data. 9. The database management system according to claim 1, wherein the first means, when registering data in the first database, creates a system log indicating that the data has been registered, The second means obtains data not registered in the second database from a system log based on management information for managing data registered in the second database when a set condition is satisfied, A database management system, wherein the obtained data is registered in the second database, and the management information is updated for the obtained data. 10. The database management system according to claim 1, wherein said condition is a predetermined time. 11. The database management system according to claim 1, wherein said condition is a predetermined time. 12. The database management system according to claim 1, wherein said condition is completion of a process of registering said obtained data in said second database. 13. The database management system according to claim 1, wherein said second means stores a plurality of said obtained data in a storage area, and registers said stored data in said second database. A database management method comprising: 14. The database management system according to claim 1, wherein said condition is an end of processing for registering said plurality of obtained data in said second database. 15. A computer readable storage medium storing a database management program in a database management system having a first database and a second database, wherein a first data is stored in the first database. And, when a set condition is satisfied, obtaining data not registered in the second database from the first database based on management information for managing data registered in the second database, A computer-readable storage medium storing a database management program having a second step of registering the obtained data in the second database and updating the management information for the obtained data. . 16. A computer-readable storage medium storing a database management program in a database management system having a first database and a second database, wherein a first data is stored in the first database. And, when a set condition is satisfied, obtaining data not registered in the second database from the first database based on management information for managing data registered in the second database, A second step of registering the obtained data in the second database and updating the management information for the obtained data. 17. A first database storage area and a second database storage area are allocated, a copy of data stored in the first storage area is stored in a second storage area, and the copy instruction is canceled. When a request is input, data is written to the first storage area. When a recopy request is input, data written to the first storage area during the release of the copy instruction is written to the second storage area. A database management method characterized by storing. 18. A first database storage area and a second database storage area are allocated, and corresponding data after the last storage time from a system log, which is a writing history to the first storage area, is stored in a second storage area. A database management method characterized by storing data in a storage area. 19. A storage device comprising: first storage means; second storage means;
A copy of the data stored in the first storage means is stored in the second storage means, and when a request for cancellation of the copy instruction is input, the data is written to the first storage means and re-copying is performed. A controller for storing, in a second storage means, data written in the first storage means during the release of the copy instruction when a request is input. 20. The first storage means, the second storage means, and the system log, which is the writing history of the first storage means, and the corresponding data from the time when the data was stored up to the previous time are stored in the second storage means. A database management system comprising: a storage controller.