JP2018181042A - Record reflecting program, record reflecting device and record reflecting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a record reflecting program, record reflecting device and record reflecting method that can reduce the processing load of a processor for performing a specific operation for reflection to a database.SOLUTION: The record reflecting device retrieves deleting records indicating an operation of deleting data stored in a database from a plurality of records included in a difference log for each primary key of the database in an order of earlier generation of records, retrieves adding records indicating an operation of adding data in an order of later generation of records, and reflects a plurality of records to the database based on the deleting records and adding records retrieved first for each primary key.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a record reflection program, a record reflection device, and a record reflection method.

  In a business system (hereinafter also referred to as an information processing system) in which a database is used, for example, a business that provides a service to a user (hereinafter also referred to simply as a business) may, for example, Build a business system that includes a database that stores

  The above-described databases generally include a row-oriented database (hereinafter also referred to as a row format database) and a column-oriented database (hereinafter also referred to as a column format database). The row-oriented database is a database that manages each record included in the database, and is a database that can perform an operation on data of each record faster than a column-oriented database. On the other hand, a column-oriented database is a database that manages data corresponding to each item included in the database, and can refer to data of each item included in the database faster than a row-oriented database. It is a database.

  Therefore, for example, when it is predicted that operations on data for each record will be frequently performed along with provision of a service, the enterprise constructs a business system including a row-oriented database. Also, for example, when it is predicted that the data for each item included in the database will be frequently referred to along with the provision of the service, the enterprise constructs a business system including a column-oriented database (see FIG. See, for example, Patent Documents 1 to 3).

JP, 2015-064850, A JP, 2012-155498, A JP 9-101912 A

  The amount of data handled in connection with the provision of such services tends to increase year by year. Therefore, in the business system, the increase in the amount of data handled as the service is provided increases the possibility that the execution time of each process exceeds the constraint time. Therefore, in recent years, for example, even in the case of operating a business system using a column-oriented database, there is an increasing need to reduce the time required for data manipulation.

  Therefore, in a business system, for example, when a plurality of operations on data stored in a database are collectively performed, it is necessary to perform at least the database in order to ensure data consistency among the plurality of operations. Identify the operation. As a result, the business system does not have to perform all of the plurality of operations on the database, and it is possible to reduce the time required for the operation on data.

  However, since the specification of the minimum required operation to the database needs to refer to all the plurality of operations, the processing load on the business system may be increased. Therefore, in the business system, when the processing load involved in specifying the minimum required operation for the database is large, other processes performed in parallel in the business system may be affected by delays or the like.

  On the other hand, in a business system, for example, processing for specifying an operation that needs to be performed at least in a database may be shared by another processor (for example, an accelerator) different from a CPU (Central Processing Unit). .

  However, even if processing is shared among other processors, if the processing load on the other processors is large, the business system may not be able to execute each processing within the restricted time.

  Therefore, in one aspect, the present invention has an object to provide a record reflection program, a record reflection device, and a record reflection method, which can reduce the processing load on a processor that specifies an operation to be reflected in a database. Do.

  In one aspect of the embodiment, the computer is caused to read a plurality of records included in the difference log into a specific processor which is a processor different from the processor of the computer, and in the specific processor, for each primary key of the database, From the plurality of records, a deletion record indicating an operation to delete data stored in the database is searched in the order of early generation of the record, and an addition record indicating an operation to add the data is arranged in order of late generation of the record A process of reflecting the plurality of records on the database based on the deleted record and the additional record for each primary key searched first is searched.

  According to one aspect, it is possible to reduce the processing load of a processor that specifies an operation to be reflected in a database.

FIG. 1 is a diagram showing the configuration of an information processing system 10. FIG. 2 is a diagram showing a hardware configuration of the information processing apparatus 1. FIG. 3 is a functional block diagram of the information processing apparatus 1. FIG. 4 is a flowchart for explaining the outline of the DB reflection process according to the first embodiment. FIG. 5 is a diagram for explaining an outline of the DB reflection process in the first embodiment. FIG. 6 is a flowchart for explaining the details of the DB reflection process in the first embodiment. FIG. 7 is a flowchart illustrating the details of the DB reflection process according to the first embodiment. FIG. 8 is a flowchart for explaining the details of the DB reflection process in the first embodiment. FIG. 9 is a flowchart illustrating the details of the DB reflection process according to the first embodiment. FIG. 10 is a flowchart for explaining the details of the DB reflection process in the first embodiment. FIG. 11 is a flowchart for explaining the details of the DB reflection process in the first embodiment. FIG. 12 is a diagram for explaining a specific example of the difference log 131. FIG. 13 is a diagram for explaining a specific example of the difference log 131 to which the generation order is added. FIG. 14 is a diagram for explaining a specific example of the partial record. FIG. 15 is a diagram for explaining a specific example of the processes of S23 and S24 by the FPGA. FIG. 16 is a diagram for explaining a specific example of the processes of S23 and S24 by the FPGA. FIG. 17 is a diagram for explaining a specific example of the processes of S23 and S24 by the FPGA. FIG. 18 is a diagram showing a hardware configuration of the information processing device 1 in the second embodiment. FIG. 19 is a functional block diagram of the information processing device 1 in the second embodiment. FIG. 20 is a flowchart for explaining the DB reflection process in the second embodiment. FIG. 21 is a flowchart for explaining the DB reflection process in the second embodiment.

[Information processing system configuration]
First, the configuration of the information processing system 10 will be described. FIG. 1 is a diagram showing the configuration of an information processing system 10. An information processing system 10 shown in FIG. 1 includes, for example, an information processing apparatus 1 (hereinafter also referred to as a reflection apparatus 1), a copy source processing apparatus 2, and a copy source storage apparatus 2a (hereinafter also referred to as a copy source database 2a). And a copy destination processing device 3 and a copy destination storage device 3a (hereinafter, copy destination database 3a).

  In the copy source processing device 2, for example, the business operator performs processing for providing a service to the user. Then, the copy source processing device 2 stores, for example, data (record) used for providing the service to the user in the copy source database 2 a as needed. Further, the copy source processing device 2 refers to, for example, data stored in the copy source database 2 a as needed.

  The copy destination processing device 3 analyzes the data stored in the copy destination database 3a, for example, by referring to the data stored in the copy destination database 3a. The following description will be made on the assumption that the copy destination database 3a is a column oriented database.

  For example, before the information stored in the copy destination database 3a is analyzed, the information processing apparatus 1 may compare the content of the data stored in the copy source database 2a with the content of the data stored in the copy destination database 3a. Synchronize. Specifically, the information processing apparatus 1 refers to the copy source database 2a, and indicates the contents of the operation on the data performed on the copy source database 2a from the previous synchronization to the present (see below , Also called difference log). Then, the information processing device 1 performs each of the operations indicated by the plurality of records included in the acquired difference log on the data stored in the copy destination database 3a. Hereinafter, the content of the operation on data includes addition of data, deletion of data, and update of data.

  Here, there is a case where the information processing apparatus 1 needs to synchronize the content of the data stored in the copy source database 2a and the content of the data stored in the copy destination database 3a within a predetermined time. is there. Therefore, even if the copy destination database 3a is a column-oriented database, for example, the information processing apparatus 1 needs to reflect the contents of the difference log (the contents of the record) on the copy destination database 3a in a short time. There is a case.

  On the other hand, in the information processing apparatus 1, for example, when a plurality of operations for updating data stored in the copy destination database 3a are collectively performed, data consistency among the plurality of operations is ensured. In order to identify the minimum required operation to be performed on the copy destination database 3a. As a result, the information processing apparatus 1 does not have to perform all of the plurality of operations on the copy destination database 3a, and the time required to update data can be shortened.

  However, when the processing load (for example, the processing load of the CPU operating in the information processing apparatus 1) for identifying the operation that needs to be performed at least on the copy destination database 3a is large, Other processes to be performed may be affected by delays and the like.

  Therefore, the CPU of the information processing device 1 according to the present embodiment causes a specific processor that is a processor different from the CPU (processor of the information processing device 1) to read a plurality of records included in the difference log.

  Then, a specific processor of the information processing apparatus 1 indicates an operation for deleting data stored in the copy destination database 3a from a plurality of records included in the difference log for each primary key of the copy destination database 3a Hereinafter, the search is also called “deleted record” in the order of early generation of the record. In addition, a specific processor of the information processing device 1 is a record indicating an operation of adding data stored in the copy destination database 3a from a plurality of records included in the difference log for each primary key of the copy destination database 3a Hereinafter, additional records are also searched in the order in which the generation of the records is late.

  Specifically, in a particular processor, a record corresponding to data update is a record corresponding to deletion of data (a record corresponding to deletion of data before update) and a record corresponding to addition of data (data after update) Treat as including the records corresponding to the addition of Therefore, in a specific processor, each of the deletion record and the addition record includes a record corresponding to data update. Therefore, a particular processor searches for a plurality of records with the record corresponding to data deletion and data update as a deletion record. In addition, a specific processor searches a plurality of records by using records corresponding to data addition and data update as additional records.

  Thereafter, the CPU of the information processing device 1 reflects a plurality of records on the copy destination database 3a based on the deletion record and the addition record for each primary key which are searched first.

  That is, in the information processing apparatus 1, among the processes performed when reflecting a plurality of records included in the difference log in the copy destination database 3a, the specific processor different from the CPU searches for the deletion record and the addition record.

  Thus, the information processing apparatus 1 can reflect the contents indicated by the plurality of records included in the difference log on the copy destination database 3a while suppressing the influence on other processes executed by the CPU.

  Further, when there are a plurality of deletion records corresponding to a specific primary key, the information processing apparatus 1 (specific processor) reflects only one of the deletion records corresponding to the specific primary key in the copy destination database 3a. As a result, it is possible to delete data of the initial state corresponding to a specific primary key from the copy destination database 3a. Therefore, the information processing device 1, for example, reflects only the deletion record with the earliest generation order among the deletion records corresponding to the specific primary key, to the copy destination database 3a, so that other than the deletion record with the earliest generation order. There is no need to reflect deleted records.

  Further, when there are a plurality of additional records corresponding to a specific primary key, the information processing apparatus 1 reflects only the last additional record among the additional records corresponding to the specific primary key in the copy destination database 3a. The state of the copy destination database 3a can be made the same as the case where all of the additional records corresponding to the specific primary key are reflected. Therefore, the information processing apparatus 1 need not reflect additional records other than the slowest-ordered addition record by reflecting only the slowest-ordering additional record in the copy destination database 3a.

  As a result, when the information processing apparatus 1 reflects the contents of the difference log in the copy destination database 3a, it is not necessary to reflect all of the plurality of records included in the difference log in the copy destination database 3a. Therefore, the information processing apparatus 1 can reduce the processing load (processing load of a specific processor) required for reflecting the contents of the difference log on the copy destination database 3a.

  If the particular processor is a processor capable of performing pipeline processing at high speed (for example, an FPGA (Field Programmable Gate Array) which is a type of accelerator), the particular processor searches for deletion records and addition records. Can be performed at high speed by pipeline processing for each primary key.

[Hardware configuration of information processing system]
Next, the hardware configuration of the information processing system 10 will be described. FIG. 2 is a diagram showing a hardware configuration of the information processing apparatus 1.

  The information processing apparatus 1 includes a CPU 101 as a processor, a memory 102 (hereinafter also referred to as a first memory 102), an external interface (hereinafter also referred to as an I / O unit) 103, a storage medium 104 and an accelerator 105. A memory 106 (hereinafter also referred to as a second memory 106) is included. The respective parts are connected to one another via a bus 107. The following description will be made on the assumption that the specific processor is the accelerator 105.

  The storage medium 104 performs, for example, processing of reflecting a plurality of records included in the difference log in the copy destination database 3a in a program storage area (not shown) in the storage medium 104 (hereinafter also referred to as DB reflection processing). Program 110 is stored. Further, the storage medium 104 has, for example, an information storage area 130 (hereinafter, also referred to as a storage unit 130) for storing information used when performing the DB reflection process.

  When executing the program 110, the CPU 101 loads the program 110 from the storage medium 104 to the memory 102 and cooperates with the program 110 to perform DB reflection processing.

  The accelerator 105 is a processor responsible for sharing a part of the processing executed by the CPU 101, loads the program 110 (for example, a part of the program 110) from the storage medium 104 to the memory 106 when the program 110 is executed. In cooperation with 110, DB reflection processing (for example, part of DB reflection processing) is performed. Specifically, the accelerator 105 may be, for example, an FPGA or an application specific integrated circuit (ASIC).

  Further, the external interface 103 communicates with, for example, the copy source database 2a and the copy destination database 3a.

[Function of information processing system]
Next, the functions of the information processing system 10 will be described. FIG. 3 is a functional block diagram of the information processing apparatus 1.

  The CPU 101 of the information processing apparatus 1 operates as, for example, the difference log acquisition unit 111, the log management unit 112, the record storage unit 113, and the record reflection unit 114 by cooperating with the program 110. Further, the accelerator 105 of the information processing apparatus 1 operates as, for example, the record search unit 121 by cooperating with the program 110. Furthermore, in the information storage area 130, for example, a difference log 131, first generation order information 132, and second generation order information 133 are stored.

  For example, the difference log acquisition unit 111 of the CPU 101 acquires the difference log 131 from the copy source database 2a at regular timing. Specifically, when analysis of data stored in the copy destination database 3a is performed once a day, the difference log acquisition unit 111 performs acquisition of the difference log 131 from the copy source database 2a once a day. May be there. Then, the log management unit 112 of the CPU 101 stores, for example, the difference log 131 acquired by the difference log acquisition unit 111 in the information storage area 130.

  The record storage unit 113 of the CPU 101 causes the accelerator 105, which is a processor different from the CPU 101, to read the difference log 131. Specifically, the record storage unit 113 stores, for example, the difference log 131 in the memory 106 accessible by the accelerator 105.

  The record search unit 121 of the accelerator 105 searches for deleted records in the order of record generation from the plurality of records included in the difference log 131 stored in the memory 106, for each primary key of the copy destination database 3a. Then, the record search unit 121 specifies, for each of the primary keys of the copy destination database 3a, the deletion record first searched for in the plurality of records included in the difference log 131.

  In addition, the record search unit 121 searches for additional records in the order of late record generation, for example, from a plurality of records included in the difference log 131 stored in the memory 106 for each primary key of the copy destination database 3a. Then, the record search unit 121 specifies, for each of the primary keys of the copy destination database 3a, the first additional record searched for in the plurality of records included in the difference log 131.

  Furthermore, the record search unit 121 stores, for example, information indicating the identified deletion record and the additional record in the first memory 102. Specifically, the record search unit 121 stores, in the first memory 102, the first generation order information 132, which is information indicating the identified deletion record, and the second generation order information 133, which is information indicating the identified additional record. Do. The record search unit 121 may store, for example, the first generation order information 132 and the second generation order information 133 in the information storage area 130.

  Thereafter, the record reflecting unit 114 of the CPU 101 reflects a plurality of records on the copy destination database 3a based on the deletion record and the additional record indicated by the information stored in the first memory 102 by the record searching unit 121.

[Outline of the First Embodiment]
Next, an outline of the first embodiment will be described. FIG. 4 is a flowchart for explaining the outline of the DB reflection process according to the first embodiment. FIG. 5 is a diagram for explaining an outline of the DB reflection process in the first embodiment. The outline of the DB reflection process in the first embodiment shown in FIG. 4 will be described with reference to FIG.

  As shown in FIG. 4, the information processing apparatus 1 stands by until the difference reflection timing (NO in S1). The difference reflection timing is, for example, timing for synchronizing the content of data stored in the copy source database 2a with the content of data stored in the copy destination database 3a. Then, when the difference reflection timing comes (YES in S1), the information processing device 1 causes the accelerator 105, which is a processor different from the CPU 101, to read a plurality of records included in the difference log 131 as shown in FIG. S2).

  That is, in the information processing apparatus 1, the accelerator 105 is made to search for a deletion record and an addition record with a large processing load among the processes performed when reflecting a plurality of records included in the difference log in the copy destination database 3a.

  As a result, the information processing apparatus 1 can reflect the contents indicated by a plurality of records included in the difference log 131 on the copy destination database 3a while suppressing the influence on other processes executed by the CPU 101.

  Subsequently, as illustrated in FIG. 5, the information processing apparatus 1 deletes the delete records in the order of record generation from the plurality of records included in the difference log 131 in the accelerator 105 for each primary key of the copy destination database 3a. Search for (S3). In addition, as shown in FIG. 5, the information processing apparatus 1 causes the accelerator 105 to add additional records in the order of late record generation from the plurality of records included in the difference log 131 for each primary key of the copy destination database 3a. A search is performed (S4).

  Thereafter, as shown in FIG. 5, the information processing apparatus 1 reflects the plurality of records on the copy destination database 3a based on the deletion record and the addition record for each primary key searched first in the processing of S3 and S4. Perform (S5).

  That is, when there are a plurality of deletion records corresponding to a specific primary key, the information processing apparatus 1 (accelerator 105) reflects only one of the deletion records corresponding to the specific primary key in the copy destination database 3a. It is possible to delete data of the initial state corresponding to a specific primary key from the copy destination database 3a. Therefore, the information processing device 1, for example, reflects only the deletion record with the earliest generation order among the deletion records corresponding to the specific primary key, to the copy destination database 3a, so that other than the deletion record with the earliest generation order. There is no need to reflect deleted records.

  Further, when there are a plurality of additional records corresponding to a specific primary key, the information processing apparatus 1 reflects only the last additional record among the additional records corresponding to the specific primary key in the copy destination database 3a. The state of the copy destination database 3a can be made the same as the case where all of the additional records corresponding to the specific primary key are reflected. Therefore, the information processing apparatus 1 need not reflect additional records other than the slowest-ordered addition record by reflecting only the slowest-ordering additional record in the copy destination database 3a.

  As a result, when the information processing apparatus 1 reflects the contents of the difference log in the copy destination database 3a, it is not necessary to reflect all of the plurality of records included in the difference log in the copy destination database 3a. Therefore, the information processing apparatus 1 can reduce the processing load required for reflecting the contents of the difference log on the copy destination database 3a.

[Details of the First Embodiment]
Next, the details of the first embodiment will be described. 6 to 11 are flowcharts for explaining the details of the DB reflection process in the first embodiment. 12 to 17 are diagrams for explaining the details of the DB reflection process in the first embodiment. The details of the DB reflection process shown in FIGS. 6 to 11 will be described with reference to FIGS. 12 to 17.

[Differential log acquisition process]
First, the process of acquiring the difference log 131 from the copy source database 2a (hereinafter, difference log acquisition process) will be described. FIG. 6 is a flowchart illustrating the difference log acquisition process.

  The difference log acquisition unit 111 of the CPU 101 waits until the difference log acquisition timing comes (NO in S101). The difference log acquisition timing is, for example, a timing before the timing at which the content of data stored in the copy source database 2a and the content of data stored in the copy destination database 3a are synchronized. Specifically, the differential log acquisition timing may be, for example, a timing predetermined by the business operator.

  When the differential log acquisition timing has come (YES in S101), the differential log acquisition unit 111 acquires the differential log 131 from the copy source database 2a (S102). Then, the log management unit 112 of the CPU 101 stores the difference log 131 acquired in the process of S12 in the information storage area 130 (S103). Hereinafter, a specific example of the difference log 131 will be described.

[Specific example of difference log]
FIG. 12 is a diagram for explaining a specific example of the difference log 131. The difference log 131 shown in FIG. 12 includes an “item P” indicating an item as a primary key for specifying a record including data to be operated among the records stored in the copy destination database 3a, and an operation on the data And “content of operation” indicating the content of the item as an item. In "content of operation", "INSERT" indicating addition of data (record), "UPDATE" indicating update of data, or "DELETE" indicating deletion of data (record) are set.

  Furthermore, the difference log 131 illustrated in FIG. 12 includes “Item X” and “Item Y” indicating items corresponding to each data included in each record as items. If "DELETE" is set in "Content of operation" for "Item X" and "Item Y", the content of the record to be deleted is set, and "INSERT" is set in "Content of operation" If yes, the content of the record to be added is set. Further, when “UPDATE” is set in “content of operation” in “item X” and “item Y”, the content of the record including the data after update is set.

  Specifically, in the difference log 131 shown in FIG. 12, for example, the “item P” is “A1”, the “operation content” is “INSERT”, and the “item X” is “100”, A record in which "item Y" is "10" is included. Furthermore, in the difference log 131 illustrated in FIG. 12, for example, the "item P" is "A1", the "operation content" is "UPDATE", and the "item X" is "10" A record in which Y is "10" is included. The description of the other records included in FIG. 12 is omitted.

  When there are a plurality of primary keys of records stored in the copy destination database 3a, the difference log 131 shown in FIG. 12 includes the number of "items P" corresponding to the combinations of existing primary keys. May be there.

[DB reflection process in CPU (1)]
Next, processing performed by the CPU 101 in the DB reflection processing will be described. FIG. 7 is a flowchart illustrating the process performed by the CPU 101 in the DB reflection process.

  As shown in FIG. 7, the record storage unit 113 of the CPU 101 stands by until the difference reflection timing (NO in S11). Then, when the difference reflection timing comes (YES in S11), the record storage unit 113 acquires the difference log 131 stored in the information storage area 130 (S12).

  Subsequently, the record storage unit 113 adds the generation order of each record to each of the plurality of records included in the difference log 131 acquired in the process of S12 (S13). Specifically, for example, when the records included in the difference log 131 acquired in the process of S102 are arranged in the generation order, the record storage unit 113 adds the generation order according to the alignment order of the records. You may Further, for example, when the difference log 131 includes the generation time of each record, the record storage unit 113 may add the generation order according to the generation time. Hereinafter, a specific example of the difference log 131 to which the generation order is added will be described.

[Specific example of difference log with generation order added]
FIG. 13 is a diagram for explaining a specific example of the difference log 131 to which the generation order is added. The difference log 131 illustrated in FIG. 13 includes, in addition to the items described in FIG. 12, “generation order” indicating the generation order of each record included in the difference log 131 as an item.

  Specifically, in the difference log 131 illustrated in FIG. 13, for example, “1” is generated as “generation order” in a record in which “item P” is “A1” and “operation content” is “INSERT”. It is attached. Further, in the difference log 131 shown in FIG. 13, for example, “2” is added as “generation order” to a record in which “item P” is “A1” and “operation content” is “UPDATE”. ing. The description of the other records included in FIG. 13 will be omitted.

  Returning to FIG. 7, the record storage unit 113 stores the plurality of records to which the generation order is added in the process of S13 in the second memory 106 accessible by the accelerator 105. Also, in this case, the record storage unit 113 stores the information indicating the primary key of the difference log 131 (for example, the information set in “Item P” of the difference log 131 shown in FIG. 13) in the second memory 106 To do (S14).

  As a result, the information processing apparatus 1 can cause the accelerator 105 to perform part of the DB reflection process. Hereinafter, DB reflection processing in the accelerator 105 will be described. The processes of S15 and S16 in FIG. 7 will be described later.

[DB reflection process in accelerator]
FIGS. 8 to 11 are flowcharts for explaining the processing performed by the accelerator 105 in the DB reflection processing.

  The record search unit 121 of the accelerator 105 waits until a plurality of records included in the difference log 131 and information indicating the primary key of the difference log 131 are stored in the second memory 106 (NO in S21).

  When a plurality of records and the like are stored in the second memory 106 (YES in S21), the record search unit 121 selects a primary key from the plurality of records included in the difference log 131 stored in the second memory 106. A plurality of partial records including the content of the operation on the data and the generation order of the records are extracted (S22). That is, the record search unit 121 extracts a partial record including only the minimum necessary information for searching for a deletion record and an additional record.

  As a result, the record search unit 121 can suppress the processing load required for searching for the deletion record and the addition record. Hereinafter, specific examples of partial records will be described.

[Example of partial record]
FIG. 14 is a diagram for explaining a specific example of the partial record. The partial record shown in FIG. 14 includes “order of generation”, “item P”, and “content of operation” as the items included in the difference log 131 described with reference to FIG.

  Specifically, in the partial record shown in FIG. 14, for example, “A1” is set as “Item P” in the information in which “generation order” is “1”, and “INSERT” is set as “content of operation”. It is set. Further, in the partial record shown in FIG. 14, for example, “B1” is set as the “item P” and “DELETE” is set as the “content of operation” in the information in which the “generation order” is “4”. ing. The description of the other records included in FIG. 14 is omitted.

  The process of extracting partial records (S22) may be performed by the CPU 101. In this case, the CPU 101 may store the extracted partial record in the second memory 106, and the accelerator 105 may acquire the partial record stored in the second memory 106 and perform subsequent processing.

  Referring back to FIG. 8, the record search unit 121 searches for deletion records in the order of record generation from the plurality of partial records extracted in the process of S22 for each primary key of the copy destination database 3a, and generates the first generation order information 132 is identified (S23). The first generation order information 132 is information indicating the generation order included in each of the searched deletion records.

  Further, the record search unit 121 searches for additional records in order of late record generation from the plurality of partial records extracted in the process of S22 for each primary key of the copy destination database 3a, and specifies the second generation order information 133. To do (S24). The second generation order information 133 is information indicating the generation order included in each of the searched additional records. The details of the processes of S23 and S24 will be described below.

[Details of the process of S23]
First, the details of the process of S23 will be described. FIG. 9 is a flowchart for explaining the details of the process of S23.

  As shown in FIG. 9, the record search unit 121 refers to the information indicating the primary key stored in the second memory 106 by the CPU 101, and identifies one primary key of the difference log 131 (S31).

  Then, the record search unit 121 is a primary key specified in the process of S31, and until the partial record whose content of the operation is UPDATE or DELETE is first specified, a plurality of items extracted in the process of S22 The partial records are searched in order of generation (S32).

  Specifically, in the partial record described in FIG. 14, among the records in which “item P” is “A1” and “operation content” is “UPDATE” or “DELETE”, “generation order” The record with the smallest value set in is the record whose "generation order" is "2". Therefore, when “A1” is specified in the process of S31, the record search unit 121 specifies a record whose “generation order” is “2”. That is, in this case, the record search unit 121 refers to only the records in which the “generation order” is “1” and “2”, and the records in which the “generation order” has a value larger than “3” are set. Do not refer to

  After that, when the partial record is specified in the process of S32 (YES in S33), the record search unit 121 specifies the generation order of the partial record specified in the process of S32 as the first generation order information 132 ( S34). On the other hand, when the partial record is not specified in the process of S32 (NO in S33), the record search unit 121 does not perform the process of S34.

  Specifically, for example, when a record whose “generation order” is “2” is specified in the process of S32, the record search unit 121 specifies “2” as the first generation order information 132.

  Then, if all primary keys included in the plurality of partial records have been specified in the process of S31 (YES in S35), the record search unit 121 ends the process of S23. On the other hand, when all primary keys included in the plurality of partial records have not been identified in the process of S31 (NO in S35), the record search unit 121 performs the process of S31 and later again.

[Details of processing of S24]
Next, the details of the process of S24 will be described. FIG. 10 is a flowchart for explaining the details of the process of S24.

  The record search unit 121 specifies one primary key of the difference log 131 by referring to the information indicating the primary key stored in the second memory 106 by the CPU 101 as in the process of S31, as shown in FIG. (S41).

  Then, the record search unit 121 searches for a plurality of partial records extracted in the process of S22 in the reverse order of generation order until a partial record whose primary key is the primary key identified in the process of S41 is first specified. (S42). That is, when the record search unit 121 searches for an additional record, unlike the case where a search for a delete record is performed, the primary whose primary key is identified in the process of S41 does not depend on the information set in the “content of operation”. Identify the first partial record that is the key.

  Specifically, among the records in which "Item P" is "B1" in the partial record described in FIG. 14, the record with the largest value set in "generation order" has "6" in "generation order". Is a record. Therefore, when the "item P" specified in the process of S41 is "B1", the record search unit 121 specifies a record whose "generation order" is "6". That is, in this case, the record search unit 121 refers to only the records in which the “generation order” is “8” to “6”, and the records in which the “generation order” has a value smaller than “5” are set. Do not refer to

  Thereafter, when the partial record is specified in the process of S42 (YES in S43), the record search unit 121 determines whether the information set in the “operation content” of the specified partial record is “DELETE” It is judged whether or not it is not (S44). Then, if the information set in the “content of operation” of the identified partial record is not “DELETE” (NO in S44), the record search unit 121 generates the partial records identified in the process of S41 in the order of generation 2) Specify as the generation order information 133 (S45). On the other hand, when the partial record is not specified in the process of S42 (NO in S43), the record search unit 121 does not perform the processes in S44 and S45.

  Furthermore, the record search unit 121 does not perform the processes of S44 and S45 when the information set in the "content of operation" of the partial record specified in the process of S42 is "DELETE" (YES in S44).

  That is, even if a record in which "content of operation" is "UPDATE" or "INSERT" exists, the primary key set in "item P" later than the existing record is the same, and "operation When there is a record whose content is "DELETE", the record search unit 121 does not have to reflect the record whose "content of operation" is "UPDATE" or "INSERT" in the copy destination database 3a.

  Specifically, in the partial record described in FIG. 14, for example, a record in which the “item P” is “A1” and the “operation content” is “INSERT” or “UPDATE” (the “generation order” is “ There are records 1) and 2). However, in the partial record described in FIG. 14, the record in which the “item P” is “A1” and the “operation content” is “DELETE” after the records (“generation order” is “7” There is a record). Therefore, in this case, since the record whose “item P” is “A1” is finally deleted in this case, each of the records where “item P” is “A1” (“generation order” is It is determined that the records “1”, “2” and “7”) need not be reflected in the copy destination database 3a.

  On the other hand, in the partial record described in FIG. 14, for example, a record in which the “item P” is “B1” and the “operation content” is “INSERT” or “UPDATE” (“generation order” is “6” There is a record). Then, in the partial record described with reference to FIG. 14, there is no record after the records in which the “item P” is “A1” and the “operation content” is “DELETE”. Therefore, the record search unit 121 specifies “6” as the second generation order information 133 in this case.

  Returning to FIG. 10, when all the primary keys included in the plurality of partial records have been specified in the process of S41 (YES in S46), the record search unit 121 ends the process of S24. On the other hand, when all primary keys included in the plurality of partial records have not been specified in the process of S41 (NO in S46), the record search unit 121 performs the process of S41 and later again.

  As a result, when the accelerator 105 reflects the content of the difference log 131 acquired in the process of S12 on the copy destination database 3a, it is not necessary to perform all the operations indicated by the plurality of records included in the difference log 131. Therefore, the accelerator 105 can reduce the processing load required when reflecting the contents of the difference log 131 in the copy destination database 3a.

[Specific example of processing of S23 and S24 by FGPA]
Next, a specific example of the DB reflection process when the accelerator 105 is an FPGA will be described. FIGS. 15 to 17 are diagrams for explaining specific examples of the processes of S23 and S24 by the FPGA.

  The FPGA shown in FIG. 15 has a filter 105a for searching for a deleted record whose "item P" is "A1" and "operation content" is "DELETE" or "UPDATE", and "item P" is "A2" And a filter 105 b for searching for a deleted record whose “content of operation” is “DELETE” or “UPDATE”. Further, the FPGA shown in FIG. 15 has a filter 105c for searching for deletion records in which the "item P" is "B1" and the "operation content" is "DELETE" or "UPDATE", and the "item P" is " B2 and a filter 105 d for searching for a deleted record whose “operation content” is “DELETE” or “UPDATE”.

  Furthermore, the FPGA shown in FIG. 15 has a filter 105 e for searching for additional records in which “Item P” is “A1” and “Operation Content” is “INSERT” or “UPDATE”, and “Item P” is “A”. A2 ”, and a filter 105 f for searching for an additional record whose“ operation content ”is“ INSERT ”or“ UPDATE ”. Further, the FPGA shown in FIG. 15 has a filter 105g for searching for additional records in which the "item P" is "B1" and the "operation content" is "INSERT" or "UPDATE", and the "item P" is " B2 "and has a filter 105h for searching for additional records whose" operation content "is" INSERT "or" UPDATE ".

  That is, the FPGA shown in FIG. 15 has a filter for searching for deleted records by pipeline processing and a filter for searching for additional records by pipeline processing for each primary key of partial records described in FIG. 14. Have. Further, the FPGA shown in FIG. 15 performs a search for deleted records and a search for additional records in parallel.

  Specifically, when the FPGA shown in FIG. 15 searches for deletion records, the partial records described in FIG. 14 are input to the filter 105 a in the order of generation, and the “item P” is “A1” and “operation content Search for deleted records whose "is" DELETE "or" UPDATE ". Then, as shown in FIG. 16, the filter 105a outputs “2”, which is a value set in the “generation order” of the first deleted record that matches the condition, as the first generation order information 132.

  Subsequently, the FPGA shown in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105 b in the order of generation, and the “item P” is “A2” and the “operation content” is “DELETE” or “UPDATE” Search for a deletion record that is Then, as shown in FIG. 16, the filter 105 b outputs “3” which is a value set in the “generation order” of the first deleted record that matches the condition as the first generation order information 132.

  Subsequently, the FPGA shown in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105c in the order of generation, and the “item P” is “B1” and the “operation content” is “DELETE” or “UPDATE” Search for a deletion record that is Then, as shown in FIG. 16, the filter 105 c outputs “4”, which is a value set in the “generation order” of the first deleted record matching the condition, as the first generation order information 132.

  Subsequently, the FPGA shown in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105 d in the order of generation, and the “item P” is “B2” and the “operation content” is “DELETE” or “UPDATE” Search for a deletion record that is Then, the filter 105 d does not output the first generation order information 132 because there is no deleted record that matches the condition.

  Further, when searching for additional records, the FPGA shown in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105 e in the order of generation, and searches for records in which “Item P” is “A1”. Then, the filter 105 e first specifies a record in which the “item P” is “A1” and the “operation content” is “DELETE”, and the second generation order information 133 is not output.

  Subsequently, the FPGA illustrated in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105 f in the order of generation, and causes the search for records in which the “item P” is “A2”. Then, as shown in FIG. 17, the filter 105 f first identifies the record in which the “item P” is “A2” and the “operation content” is “UPDATE”, and the “generation order” of the identified records. The value “8”, which is the value set in “1”, is output as the second generation order information 133.

  Subsequently, the FPGA illustrated in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105 g in the order of generation, and causes the search for records in which “item P” is “B1”. Then, as shown in FIG. 17, the filter 105 g first identifies the record in which the “item P” is “B1” and the “operation content” is “INSERT”, and the “generation order” of the identified records. The value “6”, which is set to “1”, is output as the second generation order information 133.

  Subsequently, the FPGA illustrated in FIG. 15 inputs the partial records described in FIG. 14 to the filter 105 h in the order of generation, and causes the search for records in which “item P” is “B2”. Then, as shown in FIG. 17, the filter 105 h first identifies the record in which the “item P” is “B2” and the “operation content” is “INSERT”, and the “generation order” of the identified records. The value “5” set in is output as the second generation order information 133.

  As a result, the accelerator 105 can further reduce the processing load required when reflecting the contents of the difference log 131 in the copy destination database 3a.

  Returning to FIG. 8, the record search unit 121 stores, for example, the first generation order information 132 specified in the process of S23 and the second generation order information 133 specified in S24 in the first memory 102 (S25).

  Specifically, when the processes of S23 and S24 are performed on the partial record described in FIG. 14, the record search unit 121 sets “2”, “3” and “4” as the first generation order information 132. It is stored in the first memory 102. Also, in this case, the record search unit 121 stores “8”, “6” and “5” as the second generation order information 133 in the first memory 102. The processes of S15 and S16 in the DB reflection process in the CPU 101 will be described below.

[DB reflection processing in CPU (2)]
As shown in FIG. 7, the record reflecting unit 114 of the CPU 101 continues until the first memory 102 stores the first generation order information 132 and the second generation order information 133 output by the DB reflection process performed by the accelerator 105. It waits (NO of S15).

  Then, when the first generation order information 132 and the second generation order information 133 are stored in the first memory 102 (YES in S15), the record reflecting unit 114 stores the first generation order information stored in the first memory 102. A plurality of records are reflected on the copy destination database 3a based on 132 and the second generation order information 133 (S16). The details of the process of S16 will be described below.

[Details of processing of S16]
FIG. 11 is a flowchart for explaining the details of the process of S16.

  The record reflecting unit 114 deletes the record corresponding to the first generation order information 132 identified in the process of S34 among the plurality of records included in the difference log 131 acquired in the process of S12 from the copy destination database 3a ( S51).

  In addition, the record reflecting unit 114 adds the record corresponding to the second generation order information 133 identified in the process of S44 among the plurality of records included in the difference log 131 acquired in the process of S12 to the copy destination database 3a. (S52).

  Thus, the information processing apparatus 1 can reflect the plurality of records included in the difference log 131 on the copy destination database 3a while suppressing the influence on other processes executed by the CPU 101.

Second Embodiment
Next, a second embodiment will be described. In the DB reflection process in the second embodiment, the information processing apparatus 1 causes the CPU 101 to perform all of the DB reflection process without causing the accelerator 105 to perform a part of the DB reflection process.

[Hardware Configuration of Information Processing System in Second Embodiment]
First, the hardware configuration of the information processing system 10 according to the second embodiment will be described. FIG. 18 is a diagram showing a hardware configuration of the information processing device 1 in the second embodiment.

  An information processing apparatus 1 according to the second embodiment includes a CPU 101, a memory 102, an external interface 103, and a storage medium 104. The respective parts are connected to one another via a bus 107. The information processing apparatus 1 in the second embodiment may have the accelerator 105 and the memory 106 as in the information processing apparatus 1 in the first embodiment.

[Function of Information Processing System in Second Embodiment]
Next, functions of the information processing system 10 according to the second embodiment will be described. FIG. 19 is a functional block diagram of the information processing device 1 in the second embodiment.

  The CPU 101 of the information processing apparatus 1 according to the second embodiment is similar to the CPU 101 according to the first embodiment in that the difference log acquisition unit 111, the log management unit 112, the record storage unit 113, and the record reflection unit 114. Act as. Further, the CPU 101 of the information processing device 1 in the second embodiment also operates as the record search unit 115 unlike the CPU 101 in the first embodiment. The record search unit 115 performs the same process as the record search unit 121 described with reference to FIG.

[DB reflection processing in the second embodiment]
Next, DB reflection processing in the second embodiment will be described. FIG. 20 and FIG. 21 are flowcharts explaining the DB reflection processing in the second embodiment. The difference log acquisition process in the second embodiment is the same process as the difference log acquisition process in the first embodiment, and therefore the description thereof is omitted.

  As shown in FIG. 20, the record storage unit 113 waits until the difference reflection timing (NO in S201). When the difference reflection timing comes (YES in S201), the record storage unit 113 acquires the difference log 131 stored in the information storage area 130 (S202).

  Subsequently, the record storage unit 113 adds the generation order of each record to each of the plurality of records included in the difference log 131 acquired in the process of S202 (S203). Then, the record search unit 121 extracts a plurality of partial records including the primary key, the content of the operation on the data, and the generation order of the records from the records to which the generation order is added in the process of S203 (S204).

  Thereafter, as shown in FIG. 21, the record search unit 115 searches for deletion records in the order of record generation from the plurality of partial records extracted in the process of S204 for each primary key of the copy destination database 3a. 1) The generation order information 132 is specified (S211). Further, the record search unit 121 searches for additional records in order of late record generation from the plurality of partial records extracted in the process of S204 for each primary key of the copy destination database 3a, and specifies the second generation order information 133. (S212).

  That is, in the DB reflection process in the second embodiment, the information processing device 1 causes the CPU 101 to search for a deletion record and search for an additional record.

  Then, the record reflecting unit 114 reflects a plurality of records on the copy destination database 3a based on the first generation order information 132 and the second generation order information 133 specified in the processing of S211 and S212 (S213).

  Thus, even if the information processing apparatus 1 does not have the accelerator 105, for example, it is possible to reduce the processing load required for reflecting the content of the difference log 131 on the copy destination database 3a.

  The above-mentioned embodiment is summarized as the following remarks.

(Supplementary Note 1)
On the computer
Read a plurality of records included in the differential log into a specific processor that is different from the processor of the computer,
In the specific processor, for each primary key of the database, the plurality of records are searched for a deletion record indicating an operation to delete data stored in the database in order of record generation, and addition of the data is performed. Search for additional records indicating the operation to be performed in the order of late record generation,
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection program that executes processing.

(Supplementary Note 2)
In Appendix 1,
The particular processor is a processor that performs pipeline processing.
Record reflection program.

(Supplementary Note 3)
In Appendix 1,
The particular processor is an accelerator,
Record reflection program.

(Supplementary Note 4)
In Appendix 1,
In the search process, the deletion record and the additional record are searched from a plurality of partial records including the primary key generated from the plurality of records, the content of the operation on data, and the generation order of the records.
Record reflection program.

(Supplementary Note 5)
In Appendix 4,
The content of the operation includes deletion of the data, addition of the data and update of the data,
In the process of searching,
If there is the partial record in which the content of the operation is deletion of the data or update of the data, it is determined that the deletion record is present;
If the partial record whose content of the operation is addition of the data or update of the data exists, it is determined that the additional record exists.
Record reflection program.

(Supplementary Note 6)
In Appendix 5,
In the search process, when searching for the additional record for a specific primary key, the content of the operation is the content of the operation prior to the partial record that is the addition of the data or the update of the data. When the partial record which is the deletion of the data is searched, it is determined that the additional record for the specific primary key does not exist.
Record reflection program.

(Appendix 7)
In Appendix 5,
In the process of performing the reflection,
From the data stored in the database, delete the data indicated by the deletion record for each of the primary keys searched first.
Adding data indicated by the additional record for each of the primary keys searched first to the database;
Record reflection program.

(Supplementary Note 8)
In Appendix 7,
In the process of searching,
If it is determined that the deleted record exists, the generation order corresponding to the existing deleted record is specified as a first generation order,
When it is determined that the additional record exists, the generation order corresponding to the existing additional record is specified as a second generation order,
In the process of performing the reflection,
Data indicated by a record corresponding to the specified first generation order among the plurality of records is deleted from the data stored in the database,
The data indicated by the record corresponding to the specified second generation order among the plurality of records is added to the database.
Record reflection program.

(Appendix 9)
On the computer
An operation of searching for a deletion record indicating an operation of deleting data stored in the database from a plurality of records included in the difference log for each primary key of the database in the order of early record generation and adding the data Search for additional records indicating the order in which the record generation is late,
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection program that executes processing.

(Supplementary Note 10)
A record storage unit that causes a specific processor, which is a processor different from the processor of the computer, to read a plurality of records included in the difference log;
In the specific processor, for each primary key of the database, the plurality of records are searched for a deletion record indicating an operation to delete data stored in the database in order of record generation, and addition of the data is performed. A record search unit that searches additional records indicating an operation to perform in the order of late record generation;
A record reflecting unit that reflects the plurality of records on the database based on the deletion record and the additional record for each primary key initially searched;
Record reflection device.

(Supplementary Note 11)
An operation of searching for a deletion record indicating an operation of deleting data stored in the database from a plurality of records included in the difference log for each primary key of the database in the order of early record generation and adding the data A record search unit that searches for additional records indicating the order of generation of the records in the order of late
A record reflecting unit that reflects the plurality of records on the database based on the deletion record and the additional record for each primary key initially searched;
Record reflection device.

(Supplementary Note 12)
Read a plurality of records included in the differential log into a specific processor that is different from the processor of the computer,
In the specific processor, for each primary key of the database, the plurality of records are searched for a deletion record indicating an operation to delete data stored in the database in order of record generation, and addition of the data is performed. Search for additional records indicating the operation to
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection method.

(Supplementary Note 13)
An operation of searching for a deletion record indicating an operation of deleting data stored in the database from a plurality of records included in the difference log for each primary key of the database in the order of early record generation and adding the data Search for additional records indicating the order in which the record generation is late,
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection method.

1: Information processing device 2: Copy source processing device 2a: Copy source database 3: Copy destination processing device 3a: Copy destination database

Claims (12)

On the computer
Read a plurality of records included in the differential log into a specific processor that is different from the processor of the computer,
In the specific processor, for each primary key of the database, the plurality of records are searched for a deletion record indicating an operation to delete data stored in the database in order of record generation, and addition of the data is performed. Search for additional records indicating the operation to be performed in the order of late record generation,
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection program that executes processing. In claim 1,
The particular processor is a processor that performs pipeline processing.
Record reflection program. In claim 1,
In the search process, the deletion record and the additional record are searched from a plurality of partial records including the primary key generated from the plurality of records, the content of the operation on data, and the generation order of the records.
Record reflection program. In claim 3,
The content of the operation includes deletion of the data, addition of the data and update of the data,
In the process of searching,
If there is the partial record in which the content of the operation is deletion of the data or update of the data, it is determined that the deletion record is present;
If the partial record whose content of the operation is addition of the data or update of the data exists, it is determined that the additional record exists.
Record reflection program. In claim 4,
In the search process, when searching for the additional record for a specific primary key, the content of the operation is the content of the operation prior to the partial record that is the addition of the data or the update of the data. When the partial record which is the deletion of the data is searched, it is determined that the additional record for the specific primary key does not exist.
Record reflection program. In claim 4,
In the process of performing the reflection,
From the data stored in the database, delete the data indicated by the deletion record for each of the primary keys searched first.
Adding data indicated by the additional record for each of the primary keys searched first to the database;
Record reflection program. In claim 6,
In the process of searching,
If it is determined that the deleted record exists, the generation order corresponding to the existing deleted record is specified as a first generation order,
When it is determined that the additional record exists, the generation order corresponding to the existing additional record is specified as a second generation order,
In the process of performing the reflection,
Data indicated by a record corresponding to the specified first generation order among the plurality of records is deleted from the data stored in the database,
The data indicated by the record corresponding to the specified second generation order among the plurality of records is added to the database.
Record reflection program. On the computer
An operation of searching for a deletion record indicating an operation of deleting data stored in the database from a plurality of records included in the difference log for each primary key of the database in the order of early record generation and adding the data Search for additional records indicating the order in which the record generation is late,
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection program that executes processing. A record storage unit that causes a specific processor, which is a processor different from the processor of the computer, to read a plurality of records included in the difference log;
In the specific processor, for each primary key of the database, the plurality of records are searched for a deletion record indicating an operation to delete data stored in the database in order of record generation, and addition of the data is performed. A record search unit that searches additional records indicating an operation to perform in the order of late record generation;
A record reflecting unit that reflects the plurality of records on the database based on the deletion record and the additional record for each primary key initially searched;
Record reflection device. An operation of searching for a deletion record indicating an operation of deleting data stored in the database from a plurality of records included in the difference log for each primary key of the database in the order of early record generation and adding the data A record search unit that searches for additional records indicating the order of generation of the records in the order of late
A record reflecting unit that reflects the plurality of records on the database based on the deletion record and the additional record for each primary key initially searched;
Record reflection device. Read a plurality of records included in the differential log into a specific processor that is different from the processor of the computer,
In the specific processor, for each primary key of the database, the plurality of records are searched for a deletion record indicating an operation to delete data stored in the database in order of record generation, and addition of the data is performed. Search for additional records indicating the operation to
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection method. An operation of searching for a deletion record indicating an operation of deleting data stored in the database from a plurality of records included in the difference log for each primary key of the database in the order of early record generation and adding the data Search for additional records indicating the order in which the record generation is late,
The plurality of records are reflected on the database based on the deletion record and the additional record for each primary key searched first.
Record reflection method.

Images