WO2012090281A1 - Data processing device - Google Patents

Abstract

A cooperation memory management unit (18) generates an individual data storage unit (21) for writing an updated record from a CDC (22), and an individual data storage unit (21) for writing update-reflected data generated by a conversion unit (12) from the updated record. The conversion unit (12) reads the updated record from the individual data storage unit (21) asynchronously with a receiving unit (11), generates update-reflected data, and writes the update-reflected data to the individual data storage unit (21). A CDW processing unit (13) reads the update-reflected data from the individual data storage unit (21) asynchronously with the receiving unit (11) and the conversion unit (12) and updates a cooperation destination table.

Description

Data processing device

The present invention relates to a database linkage technique for transferring data between a plurality of database tables.

For example, Patent Document 1 discloses database replication between a cooperation source database and a cooperation destination database having the same number, type, and size of columns constituting a table.

Japanese Patent Laid-Open No. 11-327988

Since database replication targeted by Patent Document 1 is data transfer between databases of the same specification, data linkage between a linkage source database and a linkage destination database having different numbers, types, and sizes of columns constituting a table is performed. There is a problem that it cannot be done.
When transferring data between multiple linkage source databases and multiple linkage destination databases, the number, type, and size of the columns that make up the table differ for each database, and the frequency of transactions and the update within each transaction. The number of records to be played varies from database to database.

The present invention has been made in view of these points, and has as its main object to efficiently perform database linkage between a plurality of linkage source databases and a plurality of linkage destination databases.

The data processing apparatus according to the present invention
A data processing apparatus that reflects the update in the linkage source table in the linkage destination table,
An updated record receiving unit for receiving an updated record updated in the cooperation source table;
A linkage destination table update unit for updating the linkage destination table in units of records;
Using the updated record, data that reflects the update in the cooperation source table, and an update reflection data generation unit that generates, as update reflection data, data that is at least part of the record in the cooperation destination table;
An updated record buffer generating unit for generating an updated record buffer for delivering an updated record from the updated record receiving unit to the update reflected data generating unit;
An update reflection data buffer generation unit that generates an update reflection data buffer for delivering update reflection data from the update reflection data generation unit to the cooperation destination table update unit;
The updated record receiver is
Write the received updated record to the updated record buffer,
The update reflected data generation unit
Asynchronously with the reception of the updated record by the updated record receiving unit, the updated record stored in the updated record buffer is read, and the update reflected data is generated using the read updated record. Write the update reflection data to the update reflection data buffer,
The cooperation destination table update unit
The update reflected data stored in the update reflected data buffer is read asynchronously with the reception of the updated record by the updated record receiving unit and the generation of the update reflected data by the update reflected data generating unit, and the read update reflected It is characterized in that the cooperation destination table is updated using data.

According to the present invention, an updated record buffer for writing updated records and an update reflected data buffer for writing updated reflected data are generated, and the updated reflected data generating unit is asynchronous with the updated record receiving unit. Reads updated records, generates update reflected data, writes update reflected data, and the linkage destination table update unit reads the update reflected data asynchronously with the updated record reception unit and the update reflected data generation unit. Since the table is updated, efficient pipeline processing is possible in database linkage between a plurality of linkage source databases and a plurality of linkage destination databases.

1 is a diagram showing a configuration example of a database cooperation system according to Embodiments 1 to 4. FIG. FIG. 3 is a diagram illustrating a configuration example of a database cooperation device according to the first embodiment. FIG. 6 is a diagram illustrating an example of cooperation table information according to the first embodiment. FIG. 4 is a diagram illustrating an example of cooperation management information according to the first embodiment. FIG. 6 is a diagram illustrating an example of cooperative connection information according to the first embodiment. The figure which shows the example of the update data arrival log which concerns on Embodiment 1. FIG. FIG. 3 is a diagram illustrating an example of a data configuration in a data processing control unit according to the first embodiment. FIG. 4 shows an example of update data information according to the first embodiment. FIG. 3 is a flowchart showing an operation example of the database linkage apparatus according to the first embodiment. FIG. 3 is a flowchart showing an operation example of the database linkage apparatus according to the first embodiment. The flowchart figure which shows the operation example of the database cooperation apparatus which concerns on Embodiment 2. FIG. The figure which shows the structural example of the database cooperation apparatus which concerns on Embodiment 3, and a data evacuation apparatus. The figure which shows the structural example of the database cooperation apparatus which concerns on Embodiment 4, and a data evacuation apparatus. FIG. 3 is a flowchart showing an operation example of the database linkage apparatus according to the first embodiment. FIG. 5 is a diagram illustrating an example of a hardware configuration of the database linkage apparatus according to the first to fourth embodiments.

Embodiment 1 FIG.
In the database (hereinafter referred to as DB) linkage technology, data changed in the linkage source database (linkage source table) is extracted, and the extracted data is converted into the format of the linkage destination database (linkage destination table). The converted data is reflected in the linked database.
Examples of data conversion include the following.
Conversion between uppercase and lowercase letters (abc → ABC): Conversion from lowercase letters to uppercase Conversion of numeric data units (1000 → 1): Conversion to K units Implementation of padding and change of the number of digits (123456 → 000123456): 6 digits From 9 to 9 digits

There are two main methods for linking databases.
The first method is a method of transferring data from one database serving as a cooperation source to one database serving as a cooperation destination, like an ETL (Extract Transform Load) tool.
The second method is a method of transferring data from a plurality of cooperation source databases to a plurality of cooperation destination databases in a Hub type configuration in which a plurality of cooperation source databases and a plurality of cooperation destination databases are connected.
In the first to fourth embodiments, a second method, that is, a database linkage method using a Hub type configuration will be described.

The hub-type database cooperation device receives a change data extracted in the cooperation source DB, a conversion process for matching the extracted data to the specification of the cooperation destination DB, and a CDW ( (Change Data Writer) processing.
When the conversion unit and the CDW processing unit that perform pipeline processing when performing database cooperation and the queue configuration between tables is uniform, in other words, when the conversion processing and the CDW processing are synchronized, the database cooperation The throughput of the device is reduced.
Embodiments 1 to 4 are mainly intended to solve this problem, and in order to improve the throughput of the database linkage apparatus, a configuration in which the conversion unit and the CDW processing unit operate asynchronously in pipeline processing. explain.
Hereinafter, a database linkage apparatus according to an embodiment will be described with reference to the drawings.

FIG. 1 shows a database linkage system according to the present embodiment, and shows a relationship among the database linkage apparatus 1, the linkage source system 2, and the linkage destination system 4.
The database linkage apparatus 1 is an example of a data processing apparatus.
An example of the internal configuration of the database linkage apparatus 1 will be described later.
The database cooperation device 1 is connected to a plurality of cooperation source systems 2 and a plurality of cooperation destination systems 4.
In FIG. 1, there are two cooperation source systems 2, that is, a cooperation source system A 2 a and a cooperation source system B 2 b.
Further, the cooperation destination system 4 includes the cooperation destination system X4x, the cooperation destination system Y4y, and the cooperation destination system Z4z.

Each cooperation source system 2 includes a cooperation source database 3 and a CDC (Change Data Capture) 22.
The cooperation source database 3 includes one or more cooperation source tables.
In the following, it is assumed that the collaboration source database A3a includes two collaboration source tables, and the respective collaboration source tables are referred to as a collaboration source table A-1 and a collaboration source table A-2.
Further, on the assumption that the collaboration source database B3b also includes two collaboration source tables, the respective collaboration source tables are referred to as a collaboration source table B-1 and a collaboration source table B-2.
Further, the CDC 22 monitors an update log of the cooperation source database 3, detects an update of the cooperation source database 3, and extracts a record updated in the cooperation source database 3.
The CDC 22 is realized by a known technique.
A record extracted by the CDC 22 is also referred to as an updated record.

Each cooperation destination system 4 includes a cooperation destination database 5.
The cooperation destination database 5 includes one or more cooperation destination tables.
In the following, it is assumed that the cooperation destination database X5x includes two cooperation destination tables, and the respective cooperation destination tables are referred to as a cooperation destination table X-1 and a cooperation source table X-2.
Further, on the assumption that the cooperation destination database Y5y also includes two cooperation destination tables, the respective cooperation destination tables are expressed as a cooperation destination table Y-1 and a cooperation source table Y-2.
Further, on the assumption that the cooperation destination database Z5z also includes two cooperation destination tables, the respective cooperation destination tables are referred to as a cooperation destination table Z-1 and a cooperation source table Z-2.

The database linkage apparatus 1 reflects the update in any linkage source database in any linkage destination table.
The database cooperation apparatus 1 can perform arbitrary database cooperation.
For example, the update of one linkage source table is performed such that the updated record of the linkage source table A-1 is reflected in each of the linkage destination table X-1, the linkage destination table Y-1, and the linkage destination table Z-1. The completed record may be reflected in two or more cooperation destination tables.
Further, a combination of a part of the updated record of the cooperation source table A-1, a part of the updated record of the cooperation source table B-1 and a part of the updated record of the cooperation source table B-2 A combination of updated records in a plurality of linkage source tables may be reflected in one linkage destination table as reflected in the table Z-2.

It should be noted that the schema configuration of the linkage source database 3 and the linkage destination database 5 may be different, more specifically, the schema, table, and column configuration configured in the database instance. The type, size, and stored data may be different from the column type, size, and stored data of the link destination database 5.

Further, the number of cooperation source systems 2, the number of cooperation destination systems 4, the number of cooperation source databases 3, the number of cooperation destination databases 5, the number of cooperation source tables, and the number of cooperation destination tables shown in the present embodiment are examples. Yes, the number of each can be set arbitrarily.

Next, FIG. 2 shows a configuration example of the database linkage apparatus 1 according to the present embodiment.
In the database linkage apparatus 1 according to the present embodiment, the configurations of the individual exclusive control unit 20 and the individual data storage unit 21 in the data storage unit 19 based on the data linkage definition by the reception unit 11, the conversion unit 12, and the CDW processing unit 13. And the configuration of the data storage unit 19 is managed by the cooperation management information 17.

In the data collaboration processing unit 23, the reception unit 11 receives the updated record detected by the CDC 22 in the collaboration source system 2 from the collaboration source table of the collaboration source database 3.
The receiving unit 11 is an example of an updated record receiving unit.
The receiving unit 11 is prepared for each cooperation source database 3.
In FIG. 2, only one receiving unit 11 is shown for the sake of simplification. However, in light of the configuration in FIG. 1, the database linkage apparatus 1 in FIG. 2 receives updated records from the linkage source database A 3 a. A receiving unit (hereinafter referred to as a receiving unit 11a) for receiving and a receiving unit (hereinafter referred to as a receiving unit 11b) for receiving updated records from the cooperation source database B3b are provided.

The conversion unit 12 converts the data extracted from the cooperation source database 3 into the data specification of the cooperation destination database 5 when the database definition specifications of the cooperation source database 3 and the cooperation destination database 5 are different.
For example, in the case of character string data, it has functions such as conversion from upper case to lower case and numerical operation such as digit alignment, and the conversion process is performed by a known technique.
The conversion unit 12 determines whether or not the input updated record needs to be converted, determines what conversion is necessary if conversion is necessary, converts the updated record if conversion is necessary, and after conversion Are updated in the individual data storage unit 21 described later.
On the other hand, if there is no need for conversion, an unconverted updated record is written in the individual data storage unit 21.
The data that the conversion unit 12 writes to the individual data storage unit 21 reflects the update in the cooperation source table, whether it is a post-conversion updated record or a non-conversion updated record, and records in the cooperation destination table This is data (update reflected data) that becomes at least a part of.
In other words, the conversion unit 12 generates the update reflected data by converting the updated record if conversion is necessary, and generates the update reflected data without converting the updated record if conversion is not necessary. Yes.
Thus, the conversion unit 12 functions as an example of the update reflected data generation unit.
The conversion unit 12 is prepared for each cooperation source table.
In FIG. 2, only one conversion unit 12 is shown for simplification of the drawing. However, in light of the configuration of FIG. 1, the database linkage device 1 of FIG. 2 corresponds to the linkage source table A-1. Corresponding to the conversion unit (hereinafter referred to as conversion unit 12a-1), the conversion unit corresponding to cooperation source table A-2 (hereinafter referred to as conversion unit 12a-2), and the cooperation source table B-1 A conversion unit (hereinafter referred to as conversion unit 12b-1) and a conversion unit (hereinafter referred to as conversion unit 12b-2) corresponding to cooperation source table B-2 are provided.

The CDW processing unit 13 has a function of reflecting the update reflection data generated by the conversion unit 12 in the cooperation destination database 5.
More specifically, the update reflected data is reflected in the cooperation destination database 5 in units of records by using an ODBC (Open DataBase Connectivity) or the like.
At this time, the CDW processing unit 13 collectively acquires a plurality of update reflection data from the conversion unit 12 and reflects them in the cooperation destination database 5.
That is, the CDW processing unit 13 updates two or more records in the cooperation destination table in one transaction using the acquired plurality of update reflection data.
The CDW processing unit 13 is an example of a cooperation destination table update unit.
The CDW processing unit 13 is prepared for each cooperation destination database 5.
In FIG. 2, only one CDW processing unit 13 is shown for the sake of simplification. However, in light of the configuration in FIG. 1, the database linkage apparatus 1 in FIG. 2 has a CDW corresponding to the linkage destination database X5x. A processing unit (hereinafter referred to as CDW processing unit 13x), a CDW processing unit (hereinafter referred to as CDW processing unit 13y) corresponding to cooperation destination database Y5y, and a CDW processing unit (hereinafter referred to as CDW processing unit 13y) corresponding to cooperation destination database Z5z. , CDW processing unit 13z).

In the data linkage interlocking buffer management unit 14, the linkage memory management unit 18 is a buffer area for transferring the updated record from the reception unit 11 to the conversion unit 12 on the main storage device (shared memory) of the database linkage device 1. The individual data storage unit 21 is generated, and the individual data storage unit 21 which is a buffer area for transferring update reflected data from the conversion unit 12 to the CDW processing unit 13 is generated.
The individual data storage unit 21 for transferring the updated record from the receiving unit 11 to the conversion unit 12 is an example of an updated record buffer, and an individual data for transferring update reflected data from the conversion unit 12 to the CDW processing unit 13 The data storage unit 21 is an example of an update reflected data buffer.
For this reason, the linkage memory management unit 18 is an example of an updated record buffer generation unit and an update reflection data buffer generation unit.

As described above, in the configuration of FIG. 2, the conversion unit 12 includes four conversion units 12a-1, 12a-2, 12b-1, and 12b-2. 18 generates four individual data storage units 21 corresponding to the respective conversion units.
In other words, the cooperation memory management unit 18 includes the individual data storage unit 21 for the reception unit 11a to transfer the updated record of the cooperation source table A-1 to the conversion unit 12a-1, and the reception unit 11a for the cooperation source table A-2. Individual data storage unit 21 for transferring the updated record of the data to the conversion unit 12a-2, and an individual data storage unit for the reception unit 11b to transfer the updated record of the cooperation source table B-1 to the conversion unit 12b-1. 21. The receiving unit 11b generates four individual data storage units 21 for transferring the updated records of the cooperation source table B-2 to the conversion unit 12b-2.
In the configuration of FIG. 2, there are three CDW processing units 13, that is, a CDW processing unit 13 x, a CDW processing unit 13 y, and a CDW processing unit 13 z. The individual data storage unit 21 is generated.
The individual data storage unit 21 of the CDW processing unit 13 is generated for each conversion unit 12 that is the source of update reflected data.
For example, when the CDW processing unit 13x receives update reflection data from each of the conversion unit 12a-1, the conversion unit 12a-2, and the conversion unit 12b-1, individual data storage in which only the conversion unit 12a-1 can write update reflection data The individual data storage unit 21 in which only the unit 21 and the conversion unit 12a-2 can write update reflected data, and the individual data storage unit 21 in which only the conversion unit 12b-1 can write update reflected data are generated.
Then, the individual exclusive control unit 20 provided for each individual data storage unit 21 performs exclusive control for writing updated records or update reflected data to each individual data storage unit 21.
The individual exclusive control unit 20 is an example of an exclusive control unit.
A logical address space in which one or more sets of the individual data storage unit 21 and the individual exclusive control unit 20 are stored is referred to as a data storage unit 19.
In the above-described example of the CDW processing unit 13x, a pair of the individual data storage unit 21 and the individual exclusive control unit 20 for the conversion unit 12a-1 and an individual data storage unit 21 and the individual exclusive control unit 20 for the conversion unit 12a-2. The logical address space in which the pair of the individual data storage unit 21 and the individual exclusive control unit 20 for the conversion unit 12b-1 is stored becomes the data storage unit 19 of the CDW processing unit 13x.

The data processing control unit 16 is a queue that stores messages for interprocess communication.
That is, each receiving unit 11 registers a message notifying the writing of the updated record in the data processing control unit 16 after writing the updated record in the individual data storage unit 21.
Each conversion unit 12 calls a message from the data processing control unit 16 and reads an updated record from the corresponding individual data storage unit 21.
Each conversion unit 12 writes the update reflected data in the individual data storage unit 21 and then registers a message informing the writing of the update reflected data in the data processing control unit 16.
Each CDW processing unit 13 calls a message from the data processing control unit 16 and reads the update reflected data from the corresponding individual data storage unit 21.

The update data arrival log 15 is information indicating the reception status of updated records by the receiving unit 11, and more specifically, information indicating the number of received records per unit time for each cooperation source table.
Details of the update data arrival log 15 will be described later.

The data linkage design unit 6 defines a linkage relationship between the linkage source database 3 and the linkage destination database 5.
The definition of the linkage relationship indicates the corresponding relationship between the schema, table, and column provided in the linkage source database 3 and the schema, table, and column provided in the linkage destination database 5.
The data linkage design unit 6 may be a tool equipped with a GUI (Graphical User Interface).
In addition, the data linkage design unit 6 may operate on hardware different from that of the database linkage device 1.
The data linkage design unit 6 outputs the database linkage definition as data linkage design information 7 and uses it as input data of the integrated linkage information 8.
Details of the data linkage design information 7 and the integrated linkage information 8 will be described later.

Here, an outline of the operation of the database linkage apparatus 1 according to the present embodiment will be described.

First, the cooperative memory management unit 18 calculates the size required for the individual data storage unit 21 for each conversion unit 12.
More specifically, the cooperation memory management unit 18 obtains, for each conversion unit 12, the number of received records per unit time from the target cooperation source table and the record length of the target cooperation source table. By integrating, the size required for the conversion unit 12 is calculated.
Then, the cooperative memory management unit 18 assigns the individual data storage unit 21 corresponding to the calculated size to the conversion unit 12.
Note that the number of received records per unit time from the linkage source table can be derived from the update data arrival log 15, and the record length of the linkage source table can be derived from the linkage source table information 9 or the like.

In addition, the cooperative memory management unit 18 calculates the capacity required for the individual data storage unit 21 for each CDW processing unit 13.
As described above, the linked memory management unit 18 generates the individual data storage unit 21 for each conversion unit 12 that is the update reflection data transfer source for each CDW processing unit 13. Calculate the required capacity in 12 units.
More specifically, the cooperative memory management unit 18 determines the data length of the update reflected data transferred from the conversion unit 12 by the CDW processing unit 13 for each conversion unit 12 from which the update reflected data is transferred, and the conversion unit. The number of received records per unit time from 12 linkage source tables and the number of records to which the CDW processing unit 13 reflects the update reflected data are integrated, and the individual data storage unit 21 for the capacity obtained by the integration. Secure.
Such integration processing is performed on all the transfer source conversion units 12 to generate individual data storage units 21 for each transfer source conversion unit 12 for one CDW processing unit 13.
The number of received records per unit time from the linkage source table can be derived from the update data arrival log 15, and the record length of the linkage destination table and the number of records to which the update reflected data is reflected in the linkage destination table Can be derived from the cooperation destination table information 10 or the like.
Further, the linked memory management unit 18 may be implemented as a shared library, and may be activated from processes that configure the receiving unit 11, the converting unit 12, and the CDW processing unit 13.

Next, the receiving unit 11 receives the updated record transmitted from the CDC 22 and writes the updated record in the data storage unit 19 assigned to the conversion unit 12 designated as the transfer destination of the received updated record. .
As described above, the receiving unit 11 exists for each cooperation source database 3, and the conversion unit 12 exists for each cooperation source table.
For example, the receiving unit 11a receives the updated record of the cooperation source table A-1 and the updated record of the cooperation source table A-1 of the cooperation source database 3.
When the updated record of the cooperation source table A-1 is received, the reception unit 11a writes the received updated record to the individual data storage unit 21 for the conversion unit 12a-1, and updates the cooperation source table A-2. When the received record is received, the receiving unit 11a writes the received updated record in the individual data storage unit 21 for the conversion unit 12a-2.

The conversion unit 12 reads the updated record from the assigned individual data storage unit 21, generates the update reflection data using the read updated record, and the CDW designated as the delivery destination of the generated update reflection data Write to the individual data storage unit 21 of the processing unit 13.
For example, when the transfer destination of the conversion unit 12a-1 is the CDW processing unit 13x, and the CDW processing unit 13x receives update reflected data from each of the conversion unit 12a-1 and the conversion unit 12b-1, the CDW processing unit 13x In contrast, an individual data storage unit 21 for the conversion unit 12a-1 and an individual data storage unit 21 for the conversion unit 12b-1 are generated.
Therefore, the conversion unit 12a-1 writes the update reflected data in the individual data storage unit 21 for the conversion unit 12a-1 of the CDW processing unit 13x.
Further, when the transfer destination of the conversion unit 12a-1 is the CDW processing unit 13x and the CDW processing unit 13y, the read updated record is converted in accordance with the cooperation destination database of the CDW processing unit 13x to perform the CDW processing. The update reflection data for the unit 13x is generated, and the update reflection data for the CDW processing unit 13y is generated by performing conversion according to the cooperation destination database of the CDW processing unit 13y.
Then, the update reflection data for the CDW processing unit 13x is written in the individual data storage unit 21 for the conversion unit 12a-1 of the CDW processing unit 13x, and the update reflection data for the CDW processing unit 13y is written in the conversion unit 12a of the CDW processing unit 13y. Write to the individual data storage unit 21 for -1.
Note that the conversion unit 12a-1 performs writing to the individual data storage unit 21 of the CDW processing unit 13x and writing to the individual data storage unit 21 of the CDW processing unit 13y in parallel.
In addition, reading of updated records, generation of update reflected data, and writing of update reflected data to the individual data storage unit 21 in each conversion unit 12 are asynchronous with reception of updated records by the receiving unit 11 and writing of updated records. To be done.

The CDW processing unit 13 reads the update reflection data from the individual data storage unit 21 and updates the cooperation destination table using the read update reflection data.
For example, when the CDW processing unit 13x receives update reflected data from each of the conversion unit 12a-1 and the conversion unit 12b-1, the individual data storage unit 21 for the conversion unit 12a-1 and the individual data for the conversion unit 12b-1 Update reflected data is read from each of the data storage units 21.
Then, for example, the CDW processing unit 13x updates the records in the cooperation destination table X-1 and the cooperation destination table X-2 using the read update reflection data.
In each CDW processing unit 13, the update reflected data is read and the cooperation destination table is updated using the updated reflected data. The receiving unit 11 receives the updated record, writes the updated record, and each converting unit 12 updates the updated record. Reading, generation of update reflection data, and writing of update reflection data to the individual data storage unit 21 are performed asynchronously.
For this reason, each CDW processing unit 13 can update two or more records in the cooperation destination table in one transaction.
In addition, each CDW processing unit 13 reads the update reflected data and updates the cooperation destination table in parallel with the reading of the update reflection data and the update of the cooperation destination table in the other CDW processing unit 13.

The above is the outline of the operation of the database linkage apparatus 1 according to the present embodiment.
Next, details of the data linkage design information 7, the linkage source table information 9, the linkage destination table information 10, the linkage management information 17, the linkage connection information 24, and the update data arrival log 15 will be described, and the database linkage according to the present embodiment Details of the operation of the apparatus 1 will be described.

The data linkage design information 7 may be described in a format such as XML (Extensible Markup Language), or may be a format that is externally taken into the database linkage device 1 in the form of a file.
The data linkage design information 7 includes linkage source table information 9, linkage destination table information 10, and linkage connection information 24.
2 shows a state in which the cooperation source table information 9, the cooperation destination table information 10, and the cooperation connection information 24 included in the data cooperation design information 7 are expanded in the memory of the database cooperation device 1.
When the data linkage design information 7 is a file described in the XML format, it is possible to read the contents of the database linkage design information 7 from the file each time the data linkage processing unit 23 performs the database linkage processing. Not right.
For this reason, in this embodiment, the data linkage design information 7 is received as an input and stored as the integrated linkage information 8 in the memory of the database linkage device 1.

The linkage source table information 9 and the linkage destination table information 10 are composed of table definition information 201 and column definition information 202 as shown in FIG.

In the table definition information 201, the CDC / CDW identifier is an identifier that identifies the CDC 22 or the CDW processing unit 13.
The table identifier is an identifier for uniquely identifying the cooperation source table or the cooperation destination table in the database cooperation device 1.
The table type is information indicating the type of the cooperation source table or the cooperation destination table, and the table name is information indicating the name of the cooperation source table or the cooperation destination table.
Further, the table definition information 201 includes information such as the number of columns of the linkage source table or linkage destination table, column information serving as a primary key, table information having a reference relationship, definition registration date, update date, and definition version. Is included.

In the column definition information 202, the column identifier is an identifier for uniquely identifying the column within the database linkage apparatus 1.
The table identifier is an identifier of the table to which the column belongs.
The data type indicates the data type of the column, and the numerical accuracy indicates the numerical accuracy (the number of digits, the number of significant digits less than a decimal) when the column data type is a numerical type.
The column definition information 202 includes information such as the data length of the data included in the column, the column name, and the parent column identifier in the case of a foreign key.

As shown in FIG. 4, the cooperation management information 17 includes a process management table 203, data storage unit information 204, individual exclusive control information 205, individual data storage information 206, external identifier information 207, and open handle information 208.

The process management table 203 includes a process ID, a process type, an external identifier, a state, a data processing control unit ID, a read control flag, a data storage unit ID, an initialization handle, and an open handle.
The process ID is an identifier of each process of the reception unit 11, the conversion unit 12, and the CDW processing unit 13.
The process type indicates whether the process specified by the process ID is a process of the reception unit 11, the conversion unit 12, or the CDW processing unit 13.
The data processing control unit ID is an identifier of the data processing control unit 16 and indicates whether it is a message queue from the reception unit 11 to the conversion unit 12 or a message queue from the conversion unit 12 to the CDW processing unit 13. Show.
The read control flag indicates whether data can be read from the individual data storage unit 21.

The data storage unit information 204 includes a data storage unit ID, individual data count, individual exclusive control information, connection management ID, cooperation target logical table ID, and individual data storage information.
The data storage unit ID is an identifier of the data storage unit 19.
The connection management ID is an identifier set for each combination of the cooperation source table and the cooperation destination database 5. In the above-described example, there are four linkage source tables (A-1, A-2, B-1, B-2) and three linkage destination tables (X, Y, Z). Connection management IDs are provided.
The cooperation target logical table ID indicates the identifier of the cooperation source table when the process identified by the process ID of the process management table 203 is the reception unit 11 and the conversion unit 12, and indicates the cooperation destination table when the process is the CDW processing unit 13. An identifier is shown.

The individual exclusive control information 205 includes an individual exclusive control ID, a buffer exclusive control flag, and an overflow control flag.
The individual exclusive control ID is an identifier of the individual exclusive control unit 20.

The individual data storage information 206 includes an individual data storage unit ID, a plurality of connection management ID duplications, and a total column length (Sl) to be linked.
The individual data storage unit ID is an identifier of the individual data storage unit 21.
The connection management ID duplication indicates how many records of the cooperation destination table are updated by the update reflected data read from the individual data storage unit 21 specified by the individual data storage unit ID.
For example, when the CDW processing unit 13x reflects one update reflected data from the conversion unit 12a-1 in the cooperation destination table X-1 and the cooperation destination table X-2 of the cooperation destination database 5x, the conversion unit 12a-1 In the individual data storage unit 21 that delivers the update reflected data to the CDW processing unit 13x, the connection management ID duplication number is 2.
The cooperation target column length total number (Sl) indicates the data length (record length) written in the individual data storage unit 21 specified by the individual data storage unit ID.
For example, in the case of the individual data storage unit 21 from the conversion unit 12 to the CDW processing unit 13, the data length (record length) of the update reflected data written in the individual data storage unit 21 by the writing source conversion unit 12 is indicated.

The external identifier information 207 indicates the CDCid when the process specified by the process ID of the process management table 203 is a receiving unit, the cooperation source logical table ID when the process is a conversion unit, and the cooperation destination database ID when the process is a CDW processing unit.

The open handle information 208 includes an open handle, input / output edges, exclusive control information (semaphore ID, etc.), and individual data storage information (address to memory, etc.).

Here, the input edge and the output edge will be described.
The input edge is a data input source, and the output edge is a data output destination.
For the receiving unit 11, the CDC is an input edge, and the individual data storage unit 21 for delivering the updated record to the conversion unit 12 is an output edge.
For the conversion unit 12, the individual data storage unit 21 for receiving updated records serves as an input edge, and the individual data storage unit 21 for delivering update reflected data to the CDW processing unit 13 serves as an output edge.
Further, for the CDW processing unit 13, the individual data storage unit 21 for receiving the update reflected data is an input edge, and the cooperation destination table is an output edge.
For example, when the CDW processing unit 13x receives update reflected data from each of the converting unit 12a-1 and the converting unit 12b-1, the individual data storage unit 21 for the converting unit 12a-1 and the converting unit 12b-1 The individual data storage unit 21 becomes an input edge.
When the CDW processing unit 13x updates the cooperation destination table X-1 and the cooperation destination table X-2, the cooperation destination table X-1 and the cooperation destination table X-2 are output edges, respectively.

Next, the cooperative connection information 24 includes connection input information 210, connection output information 211, and conversion unit connection information 212, as shown in FIG.

The connection input information 210 is information about the receiving unit 11 and includes a connection management ID, a connection management version, an input edge ID, an output edge number, a cooperation source logical table ID, and a conversion process identifier.
The conversion process identifier is the process ID of the conversion unit 12 to which the receiving unit 11 delivers the updated record.

The connection output information 211 is information about the CDW processing unit 13 and includes a connection management ID, a connection management version, an input edge ID, a CDWid, an output edge ID, runtime additional information (record length), and a cooperation destination logical table ID. .
The additional information (record length) at the time of execution is the data length (record length) of the update reflected data transferred from the conversion unit 12 that is the transfer source by the CDW processing unit 13.

The conversion unit connection information 212 is information about the conversion unit 12 and includes a process identifier, the number of output edges, output edge information, and an input edge ID. The output edge information 213 includes an output edge ID and additional information at the time of execution. It consists of a pair.

The update data arrival log 15 has a configuration as shown in FIG.
The update data arrival log 15 includes an ID for uniquely identifying a record, a linkage source logical table ID, a connection management ID, an average number of transactions per unit time, an average number of records per transaction (number of updated records), and a final update. Consists of days.

Next, details of the database linkage operation by the database linkage apparatus 1 according to the present embodiment will be described.

First, the conversion unit 12 and the CDW processing unit 13 call the cooperative memory management unit 18 as an initialization process when generating a process, and generate an individual data storage unit 21 for acquiring data from other processing units.
The processing flow for calculating the size of the individual data storage unit 21 based on FIG. 10 will be described below.

The CDW processing unit 13 notifies the associated memory management unit 18 of the CDWid that is an identifier of itself (CDW processing unit 13) (S201).
The linked memory management unit 18 extracts one or more connection output information 211 (FIG. 5) of the linked connection information 24 indicating the same CDWid as the notified CDWid, and classifies the extracted connection output information 211 for each connection management ID. (S202).
For example, when the CDW processing unit 13x updates the cooperation destination table of the cooperation destination database 5x using the update reflected data from each of the conversion unit 12a-1 and the conversion unit 12b-1, it cooperates with the cooperation source table A-1. Connection output information 211 in which a connection management ID corresponding to a combination with the destination database 5x is described, and a connection output in which a connection management ID corresponding to a combination of the cooperation source table B-1 and the cooperation destination database 5x is described. Information 211 exists.
As described above, when there are a plurality of types of connection management IDs described in the connection output information 211, the cooperative memory management unit 18 groups the connection output information 211 for each connection management ID in S202.

Next, for each group of the connection output information 211, the cooperative memory management unit 18 acquires the additional information at run time (record length) from the connection output information 211, and calculates the sum of the acquired additional information at run time (record length). Then, the sum of the additional information (record) at the time of execution is defined as the size (S1) of the record in the block of the individual data storage unit 21 (S203, S204).
Since the connection management ID is an identifier that identifies a combination of the cooperation source table and the cooperation destination database, the output edge ID may be different even if the connection output information 211 has the same connection management ID.
For example, when the CDW processing unit 13x updates the cooperation destination table X-1 and the cooperation destination table X-2 in the cooperation destination database 5x using the update reflection data from the conversion unit 12a-1, the connection management ID is Even if they are the same, there is connection output information 211 in which the output edge ID corresponding to the cooperation destination table X-1 is described, and connection output information 211 in which the output edge ID corresponding to the cooperation destination table X-2 is described. Will do.
In S203 and S204, the cooperation memory management unit 18 obtains the sum of the additional information (record length) at the time of execution for each group of the connection output information 211.

Next, using the connection management ID as a key, the cooperative memory management unit 18 obtains the average transaction count / unit time (Stn) and average record count / transaction (Stl) values from the update data arrival log 15 (FIG. 6). (S205).
Next, the cooperative memory management unit 18 acquires a buffer having a size corresponding to (Stn × connection management ID duplication number × Stl × Sl) as the individual data storage unit 21 (S206).
The plurality of connection management IDs used for the integration in S206 are grouped as having the same connection management ID as a result of the number of connection output information 211 having the same connection management ID, that is, the classification in S202. This is the number of connection output information 211.
Further, the cooperative memory management unit 18 stores the connector (for example, memory address) to the individual data storage unit 21 generated in S206 in the individual data storage unit ID column of the individual data storage information 206 (FIG. 4). The connection management ID duplication number used in the accumulation in S206 is stored in the connection management ID duplication number column.
The linked memory management unit 18 performs the processing from S203 onward for all connection management IDs belonging to the list of linked destination logical table IDs (S207).
After the processing from S203 is performed on all connection management IDs, the cooperative memory management unit 18 generates an initialization handle for the generated individual data storage unit 21, and uses the generated initialization handle as a CDW. Notify the processing unit 13.

As described above, the cooperative memory management unit 18 generates an individual data storage unit 21 for each connection management ID for each CDW processing unit 13.
The connection management ID is different for each cooperation source table, and the conversion unit 12 is provided for each cooperation source table.
For this reason, it can be said that the linked memory management unit 18 generates an individual data storage unit 21 for each conversion unit 12 that is the source of update reflected data for each CDW processing unit 13.
The size of the individual data storage unit 21 is the unit of the updated record from the linkage source table targeted by the conversion unit 12 and the data length (Sl) of the update reflected data transferred from the conversion unit 12 to the CDW processing unit 13. The size is obtained by integrating the number of receptions per time (Stn × Stl) and the number of records (multiple connection management IDs) that the CDW processing unit 13 reflects the update reflection data.

The operation of the initialization process of the conversion unit 12 will be described below.

The conversion unit 12 acquires the input edge ID from the conversion unit connection information 212 (FIG. 5) in which the ID of itself (the conversion unit 12) is described in the process identifier column, and notifies the cooperation memory management unit 18 of the input edge ID.
The cooperation memory management unit 18 extracts the connection input information 210 (FIG. 5) in which the same input edge ID as the notified input edge ID is described, and acquires the cooperation source logical table ID in the extracted connection input information 210. To do.
Next, the cooperation memory management unit 18 extracts the cooperation source table information 9 (table definition information: FIG. 3) in which the same cooperation source logical table ID as the acquired cooperation source logical table ID is included in the table identifier column. The column information in the extracted cooperation source table information 9 is acquired.
Next, the cooperative memory management unit 18 acquires the data length for each column identifier of the column definition information 202 (FIG. 3) associated with the acquired column information, and totals the acquired data length.
Next, the cooperation memory management unit 18 defines the total value as the total record target column length as the in-block record size (Sl) of the individual data storage unit.
Next, the linkage memory management unit 18 uses the linkage source table ID as a key to obtain the average transaction count / unit time (Stn) and average record count / transaction (Stl) values from the update data arrival log 15 (FIG. 6). get.
Next, the cooperative memory management unit 18 acquires a buffer having a size corresponding to Stn × St1 × S1 as the individual data storage unit 21.
Then, the cooperation memory management unit 18 stores the generated connector (for example, memory address) to the individual data storage unit 21 in the individual data storage unit ID column of the individual data storage information 206 (FIG. 4), and The original logical table ID is stored in the cooperation target logical table ID column of the data storage unit information 204 (FIG. 4).
Finally, the cooperative memory management unit 18 generates an initialization handle and notifies the conversion unit 12 of the generated initialization handle.

Next, an open process for establishing a connection with the data storage unit in which the reception unit 11 and the conversion unit 12 write processing data will be described.

The receiving unit 11 acquires an open handle for writing the updated record acquired from the CDC 22 to the individual data storage unit 21 of the conversion unit 12.
The receiving unit 11 notifies the cooperation memory management unit 18 of the initialization handle and the input edge ID.
The cooperation memory management unit 18 extracts the connection input information 210 (FIG. 5) of the cooperation connection information in which the same input edge ID as the notified input edge ID is indicated, and the conversion process identifier in the extracted connection input information 210 And obtain the cooperation source logical table ID.
Next, the cooperation memory management unit 18 extracts the process management table 203 (FIG. 4) of the cooperation management information 17 indicating the same process ID as the acquired conversion process identifier, and the data in the extracted process management table 203 Acquires the storage unit ID.
Next, from the data storage unit information 204 associated with the acquired data storage unit ID, the data storage unit information 204 indicating the same cooperation target logical table ID as the acquired cooperation source logical table ID is extracted. To do.
Next, the cooperative memory management unit 18 acquires the individual data storage unit ID stored in the individual data storage information in the extracted data storage unit information 204.
Next, the cooperative memory management unit 18 maps the shared memory using the acquired individual data storage unit ID as a key, and sets the individual exclusive control ID stored in the individual exclusive control information in the extracted data storage unit information 204 as a key. Create a semaphore for.
Further, the process management table 203 (FIG. 4) of the cooperation management information 17 indicating the same initialization handle as the initialization handle notified from the receiving unit 11 is extracted and associated with the extracted process management table 203. Stores the semaphore key in the individual exclusive control information column of the open handle information 208, stores the shared memory address (individual data storage unit address) in the individual data storage information column, and receives it in the input / output edge column The input edge ID notified from the unit 11 is stored, and the open handle is returned.

The conversion unit 12 acquires an open handle for writing the update reflected data into the individual data storage unit 21.
The conversion unit 12 notifies the cooperation memory management unit 18 of the initialization handle and the output edge ID.
The cooperative memory management unit 18 extracts the connection output information 211 (FIG. 5) of the cooperative connection information 24 in which the same output edge ID as the notified output edge ID is indicated, and connects to the CDWid from the extracted connection output information 211. Get the management ID.
Next, the cooperation memory management unit 18 extracts the process management table 203 (FIG. 4) of the cooperation management information 17 in which the same process ID as the acquired CDWid is shown, and is associated with the extracted process management table 203. The data storage unit information 204 indicating the same connection management ID as the acquired connection management ID is extracted from the stored data storage unit information 204 (FIG. 4).
Next, the cooperative memory management unit 18 uses the individual data storage unit ID in the individual data storage information 206 (FIG. 4) associated with the extracted data storage unit information 204 and the individual exclusive control information 205 (FIG. 4). The individual exclusive control ID is acquired.
Next, the cooperative memory management unit 18 maps the individual data storage unit 21 using the acquired individual data storage unit ID as a key.
More specifically, when implemented in C language, the operation is to map the shared memory by the map file.
Further, the cooperative memory management unit 18 generates a semaphore for exclusive control using the acquired individual exclusive control ID as a key.
Next, the linkage memory management unit 18 generates an open handle, extracts the process management table 203 (FIG. 4) indicating the same initialization handle as the initialization handle notified from the conversion unit 12, The generated open handle information is stored in the open handle information 208 associated with the extracted process management table 203, the semaphore key is stored in the individual exclusive control information column, and the shared memory is stored in the individual data storage information column. (The address of the individual data storage unit) is stored, the output edge ID notified from the conversion unit 12 is stored in the input / output edge column, and the open handle is returned to the conversion unit 12.

As described above, according to the present embodiment, the number of blocks and the block size in each individual data storage unit can be calculated from the database design information and the past update data arrival log.
As a result, the memory area secured by the data storage unit is optimized, the occurrence of memory overflow during execution of data linkage processing is suppressed, and the securing of unused memory that is not used is avoided.
Then, it is possible to maximize the number of conversion units 12 and CDW processing units 13 that can be operated on the server on which the database linkage apparatus 1 is mounted, and the throughput (number of processing records per unit time) of the database linkage apparatus 1 can be increased. Can be improved.
In addition, in order to generate the individual data storage unit 21 for each conversion unit 12, reading of updated records, generation of update reflection data, and writing of update reflection data to the individual data storage unit 21 in each conversion unit 12 are performed by the reception unit. 11 can be performed asynchronously with the reception of the updated record and the writing of the updated record.
Further, since the individual data storage unit 21 is generated for each CDW processing unit 13, the update reflected data is read out in each CDW processing unit 13 and the update of the cooperation destination table using the update reflected data is performed by the receiving unit 11. And writing of updated records, reading of updated records by each conversion unit 12, generation of update reflected data, and writing of update reflected data to the individual data storage unit 21 can be performed asynchronously.
For this reason, each CDW processing unit 13 can update two or more records in the cooperation destination table in one transaction.

Next, writing of the updated record to the individual data storage unit 21 by the receiving unit 11 will be described.

The receiving unit 11 extracts the process management table 203 (FIG. 4) of the cooperation management information 17 in which the open handle that matches the open handle acquired in the open process is extracted, and is associated with the extracted process management table 203. The semaphore of the individual exclusive control unit 20 is acquired from the individual exclusion control information of the open handle information 208, the right to write to the individual data storage unit 21 is acquired, and the updated record received from the CDC 22 is stored in the individual data storage unit 21. Write.
When the writing is completed, the semaphore of the individual exclusive control unit 20 is released, and a message for interprocess communication is written in the data processing control unit 16.
The data processing control unit 16 has a FIFO (First-In First-Out) structure having a list structure as shown in FIG. 7, and has a function of reflecting updated records in the cooperation destination database 5 in the order received from the CDC 22. .
The receiving unit 11 writes a message for inter-process communication in the data processing control unit 16 and then releases the semaphore of the read control flag.
Thereby, the data writing event is notified to the conversion unit 12, and the update order restriction of the cooperation source database 3 can be observed and the update data can be reflected in the cooperation destination database 5.

Next, writing of the update reflected data into the individual data storage unit 21 by the conversion unit 12 will be described with reference to FIG.
FIG. 9 shows processing of one transaction in the conversion unit 12.

The conversion unit 12 acquires the initialization handle from the process management table 203 in which the process ID of itself (the conversion unit 12) is described (FIG. 4), and stores the initialization handle in the individual data storage unit 21 linked to the initialization handle. A connection destination address is acquired (S100).
Based on the acquired connection destination address, the updated record is read from the individual data storage unit 21, and the read updated record is converted to generate the update reflected data.
Note that the conversion unit 12 acquires an open handle by referring to the link management information 17 using the output edge acquired from the link connection information 24 (FIG. 5) as a key when performing the conversion process.
Next, the conversion unit 12 acquires the semaphore of the individual exclusive control unit 20 managed using the open handle as a key (S101).
When all the blocks of the individual data storage unit 21 of the CDW processing unit 13 are in an unprocessed state, that is, when the buffer of the individual data storage unit 21 is filled with data that has not been dequeued by the CDW processing unit 13, the conversion unit 12 In this embodiment, the transaction arrival frequency per unit time is accumulated in the update data arrival log 15 and the statistical processing result is stored in the format shown in FIG.
Since the cooperative memory management unit 18 calculates the number of blocks of the individual data storage unit based on the information of FIG. 6, all the blocks of the individual data storage unit are unprocessed when the CDW processing unit 13 is operating normally. The occurrence of an event can be avoided.
At this time, when the conversion unit 12 writes update reflected data to a plurality of CDW processing units 13, it is necessary to acquire all the open handles corresponding to each output edge and the semaphores of the individual exclusive control unit 20 ( S102).
Next, the conversion unit 12 performs the writing process to the individual data storage unit 21 that has acquired the open handle and the semaphore in parallel (S103).
Next, the conversion unit 12 releases the semaphore of each individual exclusive control unit 20 corresponding to the output edge (S104).
Next, the conversion unit 12 confirms that all the semaphores of the individual exclusive control unit 20 have been released (S105).
Next, the conversion unit 12 acquires the semaphore of the data processing control unit 16 of the CDW processing unit 13 (S106).
Next, the conversion unit 12 performs an enqueue process to the data processing control unit 16, and stores a set of the output edge and update record information of the update data information 310 shown in FIG. 8 for each output edge (S107).
Next, the conversion unit 12 releases the semaphore of the data processing control unit 16 of the CDW processing unit 13 (S108).
When the enqueue of all the CDW processing units 13 to the data processing control unit 16 is completed, the processing of the conversion unit 12 for one transaction data is completed (S109).

Next, the data reflection process to the cooperation destination database by the CDW processing unit 13 will be described with reference to FIG.

The CDW processing unit 13 extracts the process management table 203 (FIG. 4) in which the same process ID as the process ID of itself (CDW processing unit 13) is shown, and the initialization stored in the extracted process management table 203 A handle is acquired (S401).
Next, the CDW processing unit 13 acquires the individual exclusive control information based on the data storage unit ID of the data storage unit information associated with the extracted process management table 203, and the individual data read by the CDW processing process The storage unit ID is acquired (S402).
Next, the write-waiting data stored in the data processing control unit 16 (FIFO structure) that controls data writing to the CDW processing unit 13 is dequeued.
The data processing control unit 16 stores a write request message corresponding to the update reflected data from the conversion unit 12 accumulated in the individual data storage unit 21.
At this time, a plurality of write request messages may be collectively dequeued from the data processing control unit 16 (S403).
The CDW processing unit 13 locks each individual data storage unit 21 based on the previously acquired individual exclusive control information and copies the stored update reflection data to the local memory (S404).
After obtaining all the write request messages from the data processing control unit 16 and copying all the update reflected data to the local memory (S405), the CDW processing unit 13 dequeues the write request message from the data processing control unit 16. Based on the update reflection data copied to the local memory in this order, an SQL statement to be issued to the DBMS (DataBase Management System) of the cooperation destination database 5 is generated (S406).
Next, the CDW processing unit 13 sequentially executes the SQL statements generated in S406 (S407).
At this time, the commit process is not performed for each SQL1 sentence.
After issuing SQL statements for all data acquired from the data processing control unit 16 and the individual data storage unit 21, the CDW processing unit 13 issues a commit process to the DBMS of the cooperation destination database 5.
As a result, the number of commit processes issued by the CDW processing unit 13 is reduced, and an effect of reducing the communication processing time with the cooperation destination database is obtained (S409).

The database linkage apparatus according to the present embodiment distributes update data to a plurality of linkage destination databases 5 for a plurality of linkage source databases 3.
And in the database cooperation apparatus which concerns on this Embodiment, the CDW process part which receives the input of conversion data from the some conversion part 12 which converts the specification of update data, and the some conversion part 12, and reflects in the cooperation destination database 5 13 is provided with a data linkage interlocking buffer management unit 14.
In the data linkage interlocking buffer management unit 14, for each CDW processing unit 13, an individual data storage unit 21 and an individual exclusive control unit 20 for each conversion unit 12 are configured, so that data writing from the conversion unit 12 is performed in parallel. Processing is possible and throughput is improved.

As described above, in the present embodiment,
A database linkage apparatus for transferring data between a plurality of databases has been described.

In the present embodiment,
Integrated linkage information with data linkage definition, data linkage processing unit, data linkage linked buffer management unit,
The data linkage processing unit includes at least a conversion unit and a CDW processing unit,
The data linkage interlocking buffer management unit includes a data processing control unit that controls data transfer between the conversion unit and the CDW processing unit for each CDW processing unit, and a data storage unit configured for each CDW processing unit,
The data storage unit includes an individual exclusive control unit and an individual data storage unit for each conversion unit that writes data to the CDW processing unit,
Database linkage device that asynchronously processes the acquisition and conversion of update records and the reflection to the linkage destination database table when the linkage source database table is updated, and reflects multiple update records to the linkage destination database in a batch Explained.

In the present embodiment,
The data linkage interlocking buffer management unit has linkage management information,
The linkage management information includes the correspondence relationship between the individual data storage unit configured by the process unit that performs the conversion process of the linkage source database for each CDW processing unit that performs the data reflection process, and the individual exclusive control information,
It has been described that the individual data storage unit is provided with a plurality of connection management IDs and the total column length to be linked.

In the present embodiment,
The integrated linkage information includes information that configures the linkage information to the linkage destination database in units of columns of the linkage source database as a combination of the linkage source database column identifier and the linkage destination database column identifier,
When the conversion unit of the data cooperation processing unit writes data to the CDW processing unit using the information, the data is collectively written in all the individual data storage units that are the data write destinations, and the data processing control is performed after the writing is completed. It has been explained that writing is notified to a plurality of the CDW processing units via the unit, and the plurality of CDW processing units are operated in parallel.

In the present embodiment,
An individual data storage unit for a specific CDW processing unit to read data is composed of a plurality of data buffers, and by providing individual exclusive control for each individual data storage unit, parallel writing from a plurality of conversion units is possible. Explained what to do.

In the present embodiment,
Accumulate the number of arrivals per unit time and the number of records per unit arrival of extracted data sent from the linkage source database as an update data arrival log,
The linked memory management unit includes statistical processing means for the log,
The block size in the individual data storage unit is calculated from the product of the total column length defined in the linkage information targeted by a specific CDW processing unit when the linkage information is updated and the number of records per unit arrival, and the number of data arrivals and blocks Calculate the data buffer size per linkage source table from the product of the number of linkage definitions for the same linkage source table and the size, and calculate the data buffer size for all linkage source tables for the specific CDW processing unit The calculation of the capacity of the individual data storage unit has been described.

Embodiment 2. FIG.
In this embodiment, an example of adding a new database definition between the cooperation source database 3 and the cooperation destination database 5 will be described with reference to the drawings.

For example, when a cooperation destination table is added in any of the cooperation destination databases 5, an individual for transferring update reflection data from the conversion unit 12 to the CDW processing unit 13 in correspondence with the added cooperation destination table. It is necessary to increase the capacity of the data storage unit 21 or generate a new individual data storage unit 21.
In the present embodiment, an example will be described in which the capacity of the individual data storage unit 21 is changed or a new individual data storage unit 21 is generated in accordance with a change in the situation.

A processing flow when the linkage definition is changed based on FIG. 11 will be described.
Here, the case where the cooperation destination table is added in the cooperation destination database 5, that is, the case where the number of output edges of the CDW processing unit 13 is increased will be described as an example.

The CDW processing unit 13 notifies the linked memory management unit 18 of the added output edge ID and its own CDWid.
At this time, it is assumed that the connection output information 211 (FIG. 5) including the added output edge ID has been generated.
The cooperative memory management unit 18 extracts the connection output information 211 indicating the notified output edge ID and CDWid, and acquires the connection management ID indicated in the extracted connection output information 211 (S301).
Next, the cooperation memory management unit 18 extracts the process management table 203 (FIG. 4) of the cooperation management information 17 indicating the same process ID as the notified CDWid, and associates it with the extracted process management table 203. Among the stored data storage unit information 204, the data storage unit information 204 showing the same connection management ID as the connection management ID acquired in S301 is searched (S302).
If the corresponding data storage unit information 204 cannot be extracted, that is, if the connection management ID acquired in S301 is not registered (YES in S303), a new individual data storage unit 21 and individual exclusive control unit 20 are generated. Since it is necessary, the cooperative memory management unit 18 acquires additional information (record length) at the time of execution from the connection output information 211 extracted in FIG. .
Next, the linked memory management unit 18 refers to the update data arrival log 15 and selects a record having a matching connection management ID acquired from the connection output information 211, and calculates the average number of transactions / unit time (Stn) and the average number of records. / The value of transaction (St1) is acquired (S304). At this time, when the conversion unit 12 to be a cooperation source is added, that is, when a table of the cooperation source database is newly added, the average number of transactions and the average number of records are not recorded in the update data arrival log There is. In this case, a predetermined value specified in advance is assigned as the average number of transactions and the average number of records, and is used as an index for calculating the size of the individual data storage unit 21.
Next, the cooperative memory management unit 18 acquires a buffer having a size corresponding to (Stn × number of duplicate connection management IDs) × St1 × S1 as an individual data storage unit, and stores the individual data in the individual data storage information 206 (FIG. 4). A connector for the buffer is added to the storage unit ID column, and a line for storing the connection management ID is added to the connection management ID column (S305).
Next, the CDW processing unit 13 establishes access to the individual data storage unit 21 (S306).

On the other hand, if there is a connection management ID in S303, the runtime additional information (record length) is acquired from the connection output information 211 acquired in S301, and the acquired runtime additional information (record length) is used for the search in S302. The result is added to the total column length (Sl) to be linked in the individual data storage information 206 associated with the data storage unit information 204 (FIG. 4) obtained (S307).
Further, the cooperative memory management unit 18 adds the connection management ID duplication number of the individual data storage information 206 (S308).
Next, the linked memory management unit 18 acquires the latest average transaction count / unit time (Stn) and average record count / transaction (Stl) values from the update data arrival log 15 using the connection management ID as a key, and Stn * St1 * S1 is calculated, and the incremental capacity of the individual data storage unit is calculated (S309).
Next, the cooperative memory management unit 18 searches the conversion unit connection information (FIG. 5) for the process identifier of the conversion unit 12 based on the output edge ID, and acquires the process identifier and the addition target CDW process. The execution of the access blocking process to the individual data storage unit 21 is notified to the unit 13 (S310).
After the access blocking is completed, the cooperative memory management unit 18 expands the capacity of the individual data storage unit 21 (S311).
Finally, the cooperative memory management unit 18 notifies the conversion unit 12 and the CDW processing unit 13 that have been blocked from accessing the individual data storage unit 21 that the reopening process has been performed (S312).

Next, the flow of the open process by the conversion unit 12 when the individual data storage unit 21 is newly generated during the processing of S304 to S306 will be described.

The conversion unit 12 notifies the association memory management unit 18 of the initialization handle and the output edge ID.
The cooperative memory management unit 18 acquires the CDWid and the connection management ID from the connection destination information 211 (FIG. 5) of the cooperative connection information using the output edge ID as a key.
Next, the cooperation memory management unit 18 extracts the process management table 203 (FIG. 4) of the cooperation management information 17 in which the same process ID as the acquired CDWid is shown, and is associated with the extracted process management table 203. Data storage unit information 204 having the same connection management ID as the acquired connection management ID is extracted.
Next, the cooperative memory management unit 18 acquires the individual data storage unit ID of the individual data storage information associated with the extracted data storage unit information 204 and the individual exclusive control ID of the individual exclusive control information.
Next, the cooperative memory management unit 18 maps the individual data storage unit 21 using the acquired individual data storage unit ID as a key.
More specifically, when implemented in C language, the operation is to map the shared memory by the map file.
Furthermore, the cooperative memory management unit 18 generates a semaphore for exclusive control using the individual exclusive control ID as a key.
Next, the linkage memory management unit 18 generates an open handle, extracts the process management table 203 (FIG. 4) indicating the same initialization handle as the initialization handle notified from the conversion unit 12, The generated open handle information is stored in the open handle information 208 associated with the extracted process management table 203, the semaphore key is stored in the individual exclusive control information column, and the shared memory is stored in the individual data storage information column. (The address of the individual data storage unit) is stored, the output edge ID notified from the conversion unit 12 is stored in the input / output edge column, and the open handle is returned to the conversion unit 12.

According to the present embodiment, when the cooperative connection information 24 is updated, only the processing of the conversion unit 12 and the CDW processing unit 13 related to the updated cooperative connection information 24 is interrupted, and the data storage unit 19 is opened. Processing will occur.
Conventionally, in order to perform inter-process communication by updating the cooperative connection information 24, the processing of all the conversion units and the CDW processing unit sharing the buffer has been interrupted.
However, according to the present embodiment, the generation target of the open processing is limited to only the conversion unit 12 corresponding to the table added as the cooperation source table and the CDW processing unit 13 operating corresponding to the database constituting the cooperation destination table. Thus, the throughput of the database linkage apparatus can be improved.

As described above, in this embodiment, when the cooperation destination table targeted by any of the CDW processing units 13 is changed, the CDW processing unit 13 other than the CDW processing unit 13 and the CDW processing unit 13 are updated. Generation of a new individual data storage unit 21 corresponding to the CDW processing unit 13 and processing of the CDW processing unit 13 are performed without affecting the operation of the conversion unit 12 other than the conversion unit 12 that is the transfer destination of the reflected data. The capacity of the existing individual data storage unit 21 can be changed.

As described above, in the present embodiment,
Linked memory that refers to the link destination table information when the integrated link information is updated, and calculates the data length written from the conversion unit based on the column configuration information of the new link destination table added to the link destination table information With a management department,
A database cooperation apparatus has been described in which the capacity of an individual data storage unit is calculated according to the data length of a record newly added as a cooperation destination database, and the individual data storage unit can be expanded.

Embodiment 3 FIG.
In the present embodiment, an example in which a secondary storage storage unit 26 for temporarily storing data in the individual data storage unit 21 is provided in addition to hardware for mounting the database linkage apparatus 1 will be described with reference to the drawings. To do.

FIG. 12 shows a system configuration example according to the present embodiment.
In FIG. 12, the data linkage design unit 6 and the data linkage design information 7 are not shown for reasons of drawing. However, as in FIG. Assume that the data linkage design information 7 is included.
In this embodiment, in addition to the configuration of the database cooperation device 1 shown in FIG. 1, a data saving device 27 including a secondary storage unit 26 that is a secondary storage device is provided.
The data saving device 27 and the database cooperation device 1 may be mounted on the same hardware device or different hardware devices.
Moreover, the structure which utilizes the data evacuation apparatus 27 with the some database cooperation apparatus 1 may be sufficient.

In the first and second embodiments, the individual data storage unit 21 is generated on the main storage device (shared memory).
When the cooperation destination database 5 is stopped for some reason, the CDW processing unit 13 cannot erase the update reflected data stored in the individual data storage unit 21 because the CDW processing unit 13 cannot perform the writing process to the cooperation destination database 5. Can not.
In this case, the memory in which the individual data storage unit 21 is mounted is depleted and the processing of the conversion unit 12 is in a waiting state, so that the influence is propagated to the reception unit 11 and reception of update data from the CDC 22 is delayed. End up.

Therefore, in the present embodiment, by providing the data saving device 27 including the secondary storage unit 26, the update reflected data to be stored in the individual data storage unit 21 is temporarily saved in the secondary storage unit 26. Thus, the processing unit 12 is prevented from waiting for processing, and the update data arrival delay from the CDC 22 is prevented from occurring.
That is, when the accumulated amount of the update reflected data in the individual data storage unit 21 exceeds a predetermined level and the overflow is close, the cooperative memory management unit 18 stores the individual data storage unit 21 for update reflected data in the secondary storage storage unit 26. Is generated.
The conversion unit 12 sends the update reflected data to the individual data storage unit 21 in the main storage device (shared memory) before the individual data storage unit 21 is generated in the secondary storage unit 26 by the cooperative memory management unit 18. After the individual data storage unit 21 is generated in the secondary storage unit 26 by the writing / cooperation memory management unit 18, the update reflected data is written in the individual data storage unit 21 in the secondary storage unit 26.
Similarly, when the accumulated amount of updated records in the individual data storage unit 21 exceeds a predetermined level and the overflow is near, the linked memory management unit 18 stores the individual data for the updated records in the secondary storage unit 26. The storage unit 21 is generated.
The receiving unit 11 stores the updated record in the individual data storage unit 21 in the main storage device (shared memory) before the individual data storage unit 21 is generated in the secondary storage unit 26 by the cooperative memory management unit 18. After the individual data storage unit 21 is generated in the secondary storage unit 26 by the writing / cooperation memory management unit 18, the updated record is written in the individual data storage unit 21 in the secondary storage unit 26.

As described above, in the present embodiment,
When there is input of linkage target data that exceeds the capacity of the individual data storage unit configured on the main storage device, an area for temporarily storing the linkage target data is secured in the secondary storage storage unit, and data processing control is performed. A database linkage apparatus whose section refers to a data area other than the main memory has been described.

Embodiment 4 FIG.
In the present embodiment, an example in which the secondary storage control unit 28 is provided in the data saving device 27 in addition to the configuration of the database cooperation device 1 illustrated in FIG. 12 will be described.

FIG. 13 shows a system configuration example according to the present embodiment.
In FIG. 13, the data linkage design unit 6 and the data linkage design information 7 are omitted for reasons of drawing. However, as in FIG. 2, the database linkage apparatus 1 includes the data linkage design unit 6. It is assumed that data linkage design information 7 is included.
In FIG. 13, a secondary storage control unit 28 is added to the data saving device 27 compared to the configuration of FIG. 12.
In the secondary storage saving process in the third embodiment, the write confirmation process is performed when data (updated record, update reflected data) is written in the secondary storage unit 26, so the writing speed is slow, and the database linkage apparatus 1 Throughput may be significantly reduced.
In the present embodiment, the secondary storage control unit 28 associates the identifier of the data cooperation processing unit 23 that is the write source, the secondary storage storage unit 26 that is the write destination, and the identifier of the data cooperation processing unit 23 that is the read target. Hold.
As a result, when the secondary storage control unit 28 requests the secondary storage storage unit 26 to write to the secondary storage storage unit 26, the secondary storage control unit 28 receives the identifier of the requesting data linkage processing unit 23, and the corresponding save area. Temporarily store data in
At this time, the write confirmation process to the disk of the secondary storage unit 26 is not performed, and the data to be written is retained on the memory of the secondary storage unit 26.
When there is a data read request from the data cooperation processing unit 23 of the database cooperation device 1, the secondary storage control unit 28 acquires the identifier of the data cooperation processing unit 23 that is the request source, and the corresponding secondary storage storage unit 26 is performed, and only the confirmation process is performed by the reading process from the secondary storage unit 26.
More specifically, when a relational database is used as the data saving device 27, the commit is performed when the data read request arrives at the secondary storage control unit 28 without committing at the time of data writing, and the confirmed data is read. It is configured to return to the request source.

Thus, in this embodiment, the secondary storage control unit 28 writes the updated record / update reflected data to the individual data storage unit 21 in the secondary storage storage unit 26 by the reception unit 11 / conversion unit 12. At the time of reading, the receiving unit 11 does not confirm the writing, and when the conversion unit 12 / CDW processing unit 13 reads the updated record / update reflected data from the individual data storage unit 21 in the secondary storage unit 26. / Confirm writing of updated record / update reflected data by the conversion unit 12.
For this reason, it is possible to suppress a decrease in throughput of the database linkage apparatus 1 due to repeated occurrence of the write confirmation process.

As described above, in the present embodiment,
The database cooperation system provided with the secondary storage storage part which stores data as a secondary storage, and the secondary storage control part was demonstrated.
More specifically,
Secondary storage controller
Control the writing process and reading process to the secondary storage unit,
Do not execute the data write confirmation process during the write process, perform the data confirm process during the read process,
It has been explained that the processing time of the database linkage apparatus with respect to the saving apparatus can be increased by reducing the number of execution times of the data confirmation process.

Finally, a hardware configuration example of the database linkage apparatus 1 shown in the first to fourth embodiments will be described.
FIG. 15 is a diagram illustrating an example of hardware resources of the database linkage apparatus 1 described in the first to fourth embodiments.
The configuration in FIG. 15 is merely an example of the hardware configuration of the database linkage device 1, and the hardware configuration of the database linkage device 1 is not limited to the configuration described in FIG. Also good.

In FIG. 15, the database linkage apparatus 1 includes a CPU 911 (also referred to as a central processing unit, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a processor) that executes a program.
The CPU 911 is connected to, for example, a ROM (Read Only Memory) 913, a RAM (Random Access Memory) 914, a communication board 915, a display device 901, a keyboard 902, a mouse 903, and a magnetic disk device 920 via a bus 912. Control hardware devices.
Further, the CPU 911 may be connected to an FDD 904 (Flexible Disk Drive), a compact disk device 905 (CDD), a printer device 906, and a scanner device 907. Further, instead of the magnetic disk device 920, a storage device such as an SSD (Solid State Drive), an optical disk device, or a memory card (registered trademark) read / write device may be used.
The RAM 914 is an example of a volatile memory. The storage media of the ROM 913, the FDD 904, the CDD 905, and the magnetic disk device 920 are an example of a nonvolatile memory. These are examples of the storage device.
The “individual data storage unit 21” described in the first to fourth embodiments is configured on the RAM 914 as a shared memory.
A communication board 915, a keyboard 902, a mouse 903, a scanner device 907, an FDD 904, and the like are examples of input devices.
The communication board 915, the display device 901, the printer device 906, and the like are examples of output devices.

The communication board 915 is connected to the network as shown in FIG.
For example, the communication board 915 may be connected to a LAN (local area network), the Internet, a WAN (wide area network), a SAN (storage area network), or the like.

The magnetic disk device 920 stores an operating system 921 (OS), a window system 922, a program group 923, and a file group 924.
The programs in the program group 923 are executed by the CPU 911 using the operating system 921 and the window system 922.

The RAM 914 temporarily stores at least part of the operating system 921 program and application programs to be executed by the CPU 911.
The RAM 914 stores various data necessary for processing by the CPU 911.

The ROM 913 stores a BIOS (Basic Input Output System) program, and the magnetic disk device 920 stores a boot program.
When the database linkage apparatus 1 is activated, the BIOS program in the ROM 913 and the boot program in the magnetic disk device 920 are executed, and the operating system 921 is activated by the BIOS program and the boot program.

The program group 923 stores programs for executing the functions described as “˜unit” (except for “individual data storage unit 21”, the same applies hereinafter) and “˜means” in the description of the first to fourth embodiments. Has been. The program is read and executed by the CPU 911.

In the file group 924, in the description of the first to fourth embodiments, “determination of”, “calculation of”, “comparison of”, “generation of”, “extraction of”, “search of” ”,“ Acquisition of ”,“ update of ”,“ setting of ”,“ registration of ”,“ selection of ”,“ input of ”,“ output of ”, etc. Information, data, signal values, variable values, and parameters indicating the results are stored as items “˜file” and “˜database”.
The “˜file” and “˜database” are stored in a recording medium such as a disk or a memory.
Information, data, signal values, variable values, and parameters stored in a storage medium such as a disk or memory are read out to the main memory or cache memory by the CPU 911 via a read / write circuit.
The read information, data, signal value, variable value, and parameter are used for CPU operations such as extraction, search, reference, comparison, calculation, calculation, processing, editing, output, printing, and display.
Information, data, signal values, variable values, and parameters are stored in the main memory, registers, cache memory, and buffers during the CPU operations of extraction, search, reference, comparison, calculation, processing, editing, output, printing, and display. It is temporarily stored in a memory or the like.
The arrows in the flowcharts described in Embodiments 1 to 4 mainly indicate input / output of data and signals.
Data and signal values are recorded on a recording medium such as a memory of the RAM 914, a flexible disk of the FDD 904, a compact disk of the CDD 905, a magnetic disk of the magnetic disk device 920, other optical disks, a mini disk, and a DVD.
Data and signals are transmitted online via a bus 912, signal lines, cables, or other transmission media.

In the description of Embodiments 1 to 4, what is described as “to part” and “to means” may be “to circuit”, “to device”, and “to device”. It may be “˜step”, “˜procedure”, or “˜process”.
That is, the data processing apparatus according to the present invention can be grasped as a method by the steps, procedures, and processes shown in the flowcharts described in the first to fourth embodiments.
Further, what is described as “˜unit” and “˜means” may be realized by firmware stored in the ROM 913.
Alternatively, it may be implemented only by software, or only by hardware such as elements, devices, substrates, and wirings, by a combination of software and hardware, or by a combination of firmware.
Firmware and software are stored as programs in a recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, and a DVD.
The program is read by the CPU 911 and executed by the CPU 911.
In other words, the program causes the computer to function as “to part” and “to means” in the first to fourth embodiments. Alternatively, the computer executes the procedures and methods of “to part” and “to means” in the first to fourth embodiments.

As described above, the database cooperation device 1 shown in the first to fourth embodiments includes a CPU as a processing device, a memory as a storage device, a magnetic disk, a keyboard as an input device, a mouse, a communication board, a display device as an output device, and a communication device. A computer including a board or the like.
As described above, the functions indicated as “˜unit” and “˜means” are realized by using these processing devices, storage devices, input devices, and output devices.

1 database linkage device, 2 linkage source system, 3 linkage source database, 4 linkage destination system, 5 linkage destination database, 6 data linkage design department, 7 data linkage design information, 8 integration linkage information, 9 linkage source table information, 10 linkage Destination table information, 11 receiving unit, 12 converting unit, 13 CDW processing unit, 14 data linkage interlocking buffer management unit, 15 update data arrival log, 16 data processing control unit, 17 linkage management information, 18 linkage memory management unit, 19 Data storage unit, 20 individual exclusive control unit, 21 individual data storage unit, 22 CDC, 23 data linkage processing unit, 24 linkage connection information.

Claims (15)

1. A data processing apparatus that reflects the update in the linkage source table in the linkage destination table,
   An updated record receiving unit for receiving an updated record updated in the cooperation source table;
   A linkage destination table update unit for updating the linkage destination table in units of records;
   Using the updated record, data that reflects the update in the cooperation source table, and an update reflection data generation unit that generates, as update reflection data, data that is at least part of the record in the cooperation destination table;
   An updated record buffer generating unit for generating an updated record buffer for delivering an updated record from the updated record receiving unit to the update reflected data generating unit;
   An update reflection data buffer generation unit that generates an update reflection data buffer for delivering update reflection data from the update reflection data generation unit to the cooperation destination table update unit;
   The updated record receiver is
   Write the received updated record to the updated record buffer,
   The update reflected data generation unit
   Asynchronously with the reception of the updated record by the updated record receiving unit, the updated record stored in the updated record buffer is read, and the update reflected data is generated using the read updated record. Write the update reflection data to the update reflection data buffer,
   The cooperation destination table update unit
   The update reflected data stored in the update reflected data buffer is read asynchronously with the reception of the updated record by the updated record receiving unit and the generation of the update reflected data by the update reflected data generating unit, and the read update reflected A data processing apparatus that updates a cooperation destination table using data.
2. The cooperation destination table update unit
   The data processing apparatus according to claim 1, wherein two or more records in the cooperation destination table are updated in one transaction using the update reflection data read from the update reflection data buffer.
3. The data processing device includes:
   It has a plurality of cooperation destination table update units with different target cooperation destination tables,
   The update reflected data buffer generation unit
   Generate an update reflection data buffer for each linkage destination table update unit,
   The update reflected data generation unit
   Write the generated update reflection data to the update reflection data buffer of the linkage destination table update unit specified as the transfer destination,
   Each linkage destination table update unit
   3. The data processing apparatus according to claim 1, wherein the update reflection data stored in the corresponding update reflection data buffer is read, and the linkage destination table is updated using the read update reflection data.
4. Each linkage destination table update unit
   The data processing according to claim 3, wherein the update reflection data is read and the cooperation destination table is updated in parallel with the update reflection data reading and the cooperation destination table update in another cooperation destination table update unit. apparatus.
5. The data processing device includes:
   It has a plurality of update reflection data generation units that are different target collaboration source table,
   The updated record buffer generation unit
   Generate an updated record buffer for each update reflection data generation unit,
   The updated record receiver is
   Write the received updated record to the updated record buffer of the update reflected data generation unit specified as the delivery destination,
   Each update reflection data generation unit
   Read the updated record from the corresponding update reflection data buffer, generate update reflection data using the read updated record, and update the update update data of the linkage destination table specified as the transfer destination. 5. The data processing device according to claim 3, wherein the data processing device writes in a data buffer.
6. The update reflected data buffer generation unit
   6. The data processing apparatus according to claim 5, wherein an update reflection data buffer is generated for each update reflection data generation unit that is a transfer source of update reflection data for each linkage destination table update unit.
7. The update reflected data buffer generation unit
   For each update reflection data generation unit of the update reflection data transfer source, the data length of the update reflection data transferred from the update reflection data generation unit to the cooperation destination table update unit, and the link source table targeted by the update reflection data generation unit The number of received records per unit time of the updated record and the number of records that the cooperation destination table update unit reflects the update reflection data are integrated, and an update reflection data buffer for the capacity obtained by the integration is generated. The data processing apparatus according to claim 6.
8. Each update reflection data generation unit
   8. The update reflected data is written into a corresponding update reflected data buffer among update reflected data buffers for each update reflected data generating unit of the linkage destination table updating unit specified as a transfer destination. The data processing apparatus described.
9. The data processing device further includes:
   9. The data processing apparatus according to claim 6, further comprising: an exclusive control unit that performs exclusive control for writing update reflected data in units of update reflected data buffers for each update reflected data generation unit.
10. Each update reflection data generation unit
    10. The update reflected data is written to a plurality of update reflected data buffers in parallel when there are a plurality of update reflected data buffers to which the update reflected data is written. Data processing device.
11. The updated record buffer generation unit
    For each update reflection data generation unit, calculate the capacity of the updated record buffer based on the number of received records per unit time from the target linkage source table and the record length of the target linkage source table. The data processing apparatus according to any one of claims 5 to 10, wherein:
12. The update reflected data buffer generation unit
    When the link destination table targeted by any link destination table update unit changes,
    Without affecting the operation of the update target data update unit other than the target update table other than the target update table and the update target data update unit that uses the target update table as the transfer destination of the update target data. And at least one of generating a new update reflection data buffer corresponding to the linkage destination table update unit and changing a capacity of an existing update reflection data buffer corresponding to the linkage destination table update unit. Item 12. The data processing device according to any one of Items 5 to 11.
13. The update reflected data buffer generation unit
    After the update reflected data buffer is generated in the main storage device of the data processing device, the update reflected data buffer in the main storage device is updated in the secondary storage device of the data processing device when it becomes a specific state. Create a reflection data buffer,
    Each update reflection data generation unit
    Before the update reflection data buffer is generated in the secondary storage device by the update reflection data buffer generation unit, the update reflection data buffer is written in the update reflection data buffer in the main storage device, and the update reflection data buffer generation unit 13. The update reflection data buffer is written in the update reflection data buffer in the secondary storage device after the update reflection data buffer is generated in the secondary storage device according to any one of claims 5 to 12. The data processing apparatus described.
14. The data processing device further includes:
    The update reflection data in the secondary storage device by the cooperation destination table update unit is not confirmed when the update reflection data is written to the update reflection data buffer in the secondary storage device by the update reflection data generation unit. The data processing apparatus according to claim 13, further comprising a secondary storage control unit that determines writing of the update reflected data by the update reflected data generation unit when reading the update reflected data from the buffer.
15. The updated record buffer generation unit
    After the updated record buffer is generated in the main storage device of the data processing device, when the updated record buffer in the main storage device is in a specific state, the update is performed in the secondary storage device of the data processing device Generate a completed record buffer,
    The updated record receiver is
    Before the updated record buffer is generated in the secondary storage device by the updated record buffer generation unit, the updated record buffer is written in the updated record buffer in the main storage device, and the updated record buffer generation unit 15. After the updated record buffer is generated in the secondary storage device by the above, the updated record is written to the updated record buffer in the secondary storage device. The data processing apparatus described.

Images