WO2022193894A1 - Asynchronous persistency of replicated data changes in database accelerator - Google Patents
Asynchronous persistency of replicated data changes in database accelerator Download PDFInfo
- Publication number
- WO2022193894A1 WO2022193894A1 PCT/CN2022/076471 CN2022076471W WO2022193894A1 WO 2022193894 A1 WO2022193894 A1 WO 2022193894A1 CN 2022076471 W CN2022076471 W CN 2022076471W WO 2022193894 A1 WO2022193894 A1 WO 2022193894A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- database
- target database
- recovery
- target
- tables
- Prior art date
Links
- 238000011084 recovery Methods 0.000 claims abstract description 196
- 238000000034 method Methods 0.000 claims abstract description 88
- 238000003860 storage Methods 0.000 claims abstract description 75
- 230000002085 persistent effect Effects 0.000 claims abstract description 59
- 238000012545 processing Methods 0.000 claims description 36
- 238000004590 computer program Methods 0.000 claims description 10
- 230000001360 synchronised effect Effects 0.000 claims description 8
- 238000007726 management method Methods 0.000 description 41
- 238000005192 partition Methods 0.000 description 37
- 238000010586 diagram Methods 0.000 description 25
- 230000008569 process Effects 0.000 description 18
- 230000008859 change Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 11
- 230000000903 blocking effect Effects 0.000 description 10
- 230000003287 optical effect Effects 0.000 description 8
- 230000009471 action Effects 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000001902 propagating effect Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000012517 data analytics Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Definitions
- the invention relates generally to crash recovery for a database, and more specifically, to a computer-implemented method for a crash recovery for linked databases.
- the invention relates further to a linked database system with crash recovery for linked databases, and a computer program product.
- a full-blown row-based database management system may serve as an application endpoint for data manipulation language (DML) operations and query executions.
- DML data manipulation language
- the query optimizer may transparently decide whether a query should be executed on the source DBMS –especially online transaction processing –or offloaded to a target DBMS –especially, for online analytical processing.
- the target DBMS may be a full-blown column-based database management system holding a shadow copy of a selected set of the source DBMS tables.
- Strategies for creating shadow copies of the target DBMS may comprise transferring data stored in one or more tables of the source DBMS to the target DBMS at a given point in time.
- query data in the target DBMS might be out of date if the loading was performed some time ago and the corresponding source table has been modified in the meantime. Therefore, the contents of the tables are usually adapted incrementally as changes are recorded in the corresponding source database tables.
- Both database systems implement transaction mechanisms to guarantee ACID (atomic, consistent, isolation, durable) properties of the respective database. I.e., concurrent modifications are properly isolated via locking techniques, and consistency checks may guarantee that the database state is transferred from one consistent state to another one and, logging techniques such as write-ahead-logging may be implemented in order to guarantee atomicity and durability for transactional changes.
- ACID atomic, consistent, isolation, durable
- the target accelerator database simply mirrors a snapshot of the source database.
- the source database may act as the master of the data, or data manipulations via IUDs may be processed by the source database management system and, changes are replicated to the target database via an update technique.
- the accelerator database therefore, may act as a cache for storing partitions of the source database that may need fast analysis operations.
- a related system can include a tracking module that monitors first transactions from a database application to a source storage device to generate log entries having at least one marker indicating a known good state of the application.
- the system further includes a computer coupled to a target storage device comprising a database and log files. The computer processes the transactions based on the log entries, to replicate data to the target storage device, performs a first snapshot and replays data stored in the log files into the database.
- document US 2014 /0 279 930 A1 describes a distributed database system which implements fast crash recovery.
- a connection with one or more storage nodes of a distributed storage system storing data for a database implemented by the database head node may be established.
- that database may be made available for access, such as for various access requests.
- a computer-implemented method for a crash recovery for linked databases may be provided.
- the linked databases may comprise a source database and a related target database, and selected queries against a database management system comprising the source database may be transferred to a database management system comprising the target database for processing.
- the method may comprise synchronizing selected portions of content of tables of the source database with respective portions of content of tables of the target database, applying, during the synchronizing, changes to the source database to an in-memory target database portion of the database management system comprising the target database using a recovery log file of the source database, and storing persistently applied changes to the in-memory target database portion asynchronously to a persistent target database storage.
- the method may comprise restoring the in-memory target database portion with the latest snapshot available in the persistent target database storage, and applying, upon the database crash on the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage to the in-memory target database portion.
- a linked database system with crash recovery for linked databases may be provided.
- the linked databases may comprise a source database and a related target database, and selected portions of content of tables of the source database may be synchronized with respective portions of the content of the tables of the target database.
- the linked database system may comprise a processor and a memory, communicatively coupled to the processor, wherein the memory may store program code portions that, if executed, enable the processor, to synchronize selected portions of content of tables of the source database with respective portions of content of tables of the target database, apply, during the synchronizing, changes to the source database to an in-memory target database portion of the database management system comprising the target database using a recovery log file of the source database, and store persistently applied changes to the in-memory target database portion asynchronously to a persistent target database storage.
- the memory may also store program code portions that, if executed, enable the processor, to restore, upon a database crash on the target database system, the in-memory target database portion with the latest snapshot available in the persistent target database storage, and apply, upon the database crash on the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage to the in-memory target database portion.
- the proposed computer-implemented method for a crash recovery for linked databases may offer multiple advantages, technical effects, contributions and/or improvements:
- the update and change as well as the queries against the in-memory portion may continue without any slowdown by also storing the changes to the target database persistently.
- the logging and state storage of physical database data structures may be decoupled from processing IUDs inside database transactions. This means that the persistency phase is skipped when changes are performed on the target database –in particular in the in-memory portion –and update operations may continue without an additional delay. Therefore, records that would be required for crash recovery of the target database may be written asynchronously by a dedicated asynchronous data persistency service.
- Changes made by IUD transactions are just applied to the in-memory representation of the data, where they may be immediately processed by database queries that are offloaded to the target database, i.e., the accelerator, regardless of whether they are already stored on persistent media on not. This may also be interpreted as lifting the durability constraint of ACID properties of the database.
- the crash recovery may have to synchronize the potentially incomplete snapshot with the source database system by replaying missing changes.
- the missing portions may be recovered incrementally. That is, the target database state may be recovered from the latest asynchronously persistent snapshot of the target database, and, with the help of recovery metadata, determines, which additional changes may still be missing. I.e., which changes from the source database needs to be replayed by the target databases from the source database system’s transaction or recovery log file to finally restore the current consistent state of the target database.
- the asynchronous persistent storage of changes to the target database may also help to execute the logging more efficiently, I/O operations may be executed more efficiently –e.g., by a batch process –or self-compensating changes may be deleted altogether That is, a target record that has been changed in a way which may be revered before a next persistent storage step of changes of the target database can be deleted in a buffer between the target database and the persistent storage medium.
- the synchronizing may comprise reading entries of a recovery log file relating to the source database and applying the read entries to the target database. This may be performed by the management portion of the target database management system.
- other techniques for synchronization may be used, e.g. based on SQL (structured query language) or Q, i.e., a query language optimized for column-based databases.
- the source database may be optimized for transactions –i.e., online transaction processing (OLTP) –and/or wherein the source database is a row-oriented relational DBMS.
- OTP online transaction processing
- the source database is a row-oriented relational DBMS.
- This may represent and support efficiently the operational backbone of enterprise IT (Information Technology) operations.
- a row-oriented database may be optimized for a balanced fast read, write and change operations to the data in the database. It may also be good for generating reports.
- this type of database is often not optimal for analysis-type of operations.
- the target database may be optimized to analytical operations –i.e., online analytical processing (OLAP) –and/or the target database may be a column-oriented database.
- OLAP online analytical processing
- the method may also comprise delaying, in case of a crash of the target database, queries against the target database until a recovery of the target database may be finished.
- the recovery process may be transparent for a user. He may eventually experience a little delay in his analysis query.
- the response-time for complex analysis queries is comparably long anyway, the user may not be aware of the delay at all.
- queries targeted to the target database may be executed by the source database during the recovery process of the target database. This may slow down the source database a little bit during the transactions, however, for the overall user satisfaction this may be an acceptable compromise.
- metadata defining the selected tables may be part of the recovery log file.
- the general architecture of the in-memory target database may already be defined in the recovery log file of the source database. This may also be true for the portions of data of tables to be mirrored to the source database. Hence, a single source for data definitions may be used so that no conflicting situations may be provoked.
- the storing of persistently applied changes may comprise waiting until a predefined number of changes in the in-memory target database portion may have been completed.
- the predefined number may be configurable during the setup of the database and/or may also be altered during the operation of the database management system.
- the advantage of this approach may be that it does not slow down of the analytical operations of the target database.
- a group of updates to the target database may be extracted and persistently stored in one go.
- one may also consider to monitor the load to the target database and performing the persistent storage of changes to the target database during time periods in which the analysis load to the target database is comparatively low.
- This feature may be implemented inside a separate thread or process which may wait until a predefined number of changes are completed in the in-memory portion. For each change inside the set of completed changes since the last run, the changes are persisted to the persistent database portion of the target database. Additionally, also related metadata are persistently stored describing the recently persisted change (such as the last corresponding block record sequence number inside the source database system) for being able to identify change replay points during the recovery phase. From this, a point in time of the last consistent changed set that has been stored may be possible. This point in time may be helpful to recover the in-memory portion of the target database from the persistent portion of the target database and, incrementally recover the remaining IUDs of the in memory-portion of the target database from the recovery log file of the source database.
- the query processor may be informed by an “active again” signal that queries may be processed from the target database from then on.
- the restoring of tables –or the partitions thereof –of the in-memory target database portion may comprise a prioritizing or prioritization of the recovering or recovery according to one selected out of the group consisting of a data usage, a query priority and data priority.
- the general concept is to allow different optimization methods during the recovery process in order to allow an execution of queries although the target database “is under reconstruction” , i.e., being in recovery mode.
- These optimization options may be configurable and may –in general –be combinable in one implementation.
- the prioritizing of the recovery according to the data usage may comprise maintaining a counter for each table –or a partition thereof –in the target database.
- the counter value of the counter may be indicative of how many queries may be waiting for the related table, and restoring firstly the database table with the highest counter value first. Later on, the other tables may be recovered subsequently.
- This option may allow a quick recovery of those tables of the target database being in high demand.
- the users may be provisioned as soon as possible with a completely recovered portion of the database, i.e., the tables being in high demand. This option may be denoted as demand or data usage optimized recovery .
- the list of waiting queries may be considered for controlling the recovery process and to minimize the recovery time.
- a new database query arrives at the target database while recovery is in progress, its data accesses may be analyzed. If it does not access non-recovered data, it may be processed immediately. Otherwise, the query is registered in a list of waiting queries as part of the recovery state.
- the recovery stages are evaluated by a recovery schedule in order to derive a sequence of recovery actions, preferring those tables and/or table partitions that actually block queries. The most valuable recovery action may be scheduled via one of the available recovery strategies upon completion of the recovery, the recovery state may be updated and waiting queries for the recovered table and/or partition may be notified if they do not need to be blocked anymore.
- the crash recovery on the target database can be described as follows: firstly, the list of tables or table partitions that need to be recovered by using known recovery metadata on the target database. Secondly, the list of to-be-recovered tables or table partitions is stored inside the recovery state. Thirdly, while the to-be-recovered list is not empty, the following may be executed:
- the list of to-be-recovered tables and/or table partitions may be implemented by a priority queue, using the below described algorithm to dynamically calculate the next recovery item for the next recovery cycle.
- the steps mentioned under (iii) may be executed in parallel for multiple tables and/or table partitions in order to speed up the recovery process.
- the determination of the to-be-recovered tables and/or table partitions may be executed as follows:
- the prioritizing the recovery according to the query priority may comprise restoring firstly those database tables receiving queries with the highest priority.
- a priority value may be assigned to a database system —e.g., production database versus test database –or may also be assigned based on an individual query. This option may be denoted as query priority or simply data priority optimized recovery .
- the prioritizing the recovery according to the data priority may comprise maintaining (at least) two groups of database tables, each group relating to a separate group of users, and restoring firstly the database tables of the group having a higher configured group priority.
- Such a situation may happen in a multi-user/multi-group/multi-tenant environment in which one user, or group, or tenant may have assigned a higher priority for queries.
- one tenant may have a higher guaranteed availability of the database system. In such a case, this tenant may be assigned a higher priority.
- Such a scenario may work best in a multi-tenant database being operated in a cloud computing data center. This option may be denoted as customer priority optimized recovery .
- the method may also comprise determining the data volume to be recovered for next to-be-recovered tables, and recovering the table (s) using a recovery strategy depending on the volume to be recovered.
- the recovering strategy is an incremental update strategy or a bulk update strategy.
- it may be decided database table by database table (or group of database tables) which update strategy to be used in order to minimize the total recovery time of the target database. This option may be denoted as time-optimized recovery.
- embodiments may take the form of a related computer program product, accessible from a computer-usable or computer-readable medium providing program code for use, by, or in connection, with a computer or any instruction execution system.
- a computer-usable or computer-readable medium may be any apparatus that may contain means for storing, communicating, propagating or transporting the program for use, by, or in connection, with the instruction execution system, apparatus, or device.
- Fig. 1 shows a block diagram of an embodiment of the inventive computer-implemented method for a crash recovery for linked databases.
- Fig. 2 shows a block diagram of an embodiment of linked databases.
- Fig. 3 shows a block diagram of an embodiment of how the target database may be synchronized.
- Fig. 4 shows a block diagram of an embodiment of the proposed concept in a more implementation-near form.
- Fig. 5 shows a block diagram of an embodiment of the linked database system comprising components for a customer priority optimize recovery strategy.
- Fig. 6 shows a block diagram of an embodiment of the linked database system comprising components for a volume optimize recovery strategy.
- Fig. 7 shows a block diagram of an embodiment of the linked database system for a crash recovery for linked databases.
- Fig. 8 shows an embodiment of a computing system comprising the linked database system.
- 'crash recovery' may denote a process of reconstructing a state of a database before the crash has happened. If a crash happens, data may not be available or inconsistent.
- linked databases may denote at least two databases that are closely related to one another.
- linked databases may be denoted as those that may store, at least in part, identical data.
- the primary database may be optimized for other tasks than the secondary database in such a couple.
- the term 'source database' –or 'primary database’ – may denote a database being optimized for, e.g., fast transactions, i.e., online transaction processing.
- databases optimized in such a way –i.e., for fast read, write, update operations on the data – may be slow in performing complex queries involving a high number of tables or a large number of data like in the case of online analytical processing.
- the online analytical processing may slow down the online transaction processing. Therefore, highly optimized database management systems of the just mentioned types of databases may successfully work as a tandem.
- the term 'target database' –or 'secondary database' may denote the second database in such a tandem of databases being optimized for different tasks.
- the target database may be optimized for online analytical processing. It may store at least portions of the tables and portions of the data in the tables of the source database.
- the target database may also be comprised of two portions: an in-memory portion for a fast execution of complex and multi-dimensional queries, and a persistent portion which may store the tables and data of the in-memory portion of the target database on the longer-term storage, e.g., a hard disk or a flash memory. This way, the target database may be enabled to –in case of a crash of the target database –recover most of its content from the persistent storage.
- the term 'selected portions of content of tables' may denote the just mentioned portions or partitions of the data of portions of the tables of the primary database which may be copied and kept in sync in the target database.
- the term 'selected queries' may denote a specific type of queries addressed to the linked databases which may better be executed by one of the two databases –in particular, the target database –based on the nature of the query. E.g., if the query type relates to online analytical processing, the query may be forwarded to the target database and may not be executed by the source database.
- the term 'database management system' may denote a combination of an administration/management system, typically being implemented in a combination of hardware and software and at least one related database storing the data.
- the term 'in-memory target database portion' may denote the part of the target database which may hold almost all of its data in the main memory of the computer system.
- the database management system of the target database may comprise an in-memory portion and a persistent portion of the target database, wherein the persistent portion may be –apart from the last changes to the in-memory –a persistently stored copy of the in-memory portion.
- the term 'persistent target database storage' may describe that part of the target database management system enabled to store the data of the target database persistently, i.e., using a hard disk or flash memory instead of in-memory storage.
- the term 'latest snapshot' may denote a last consistent stored status of the target database.
- the term 'later timestamp' –in particular a change having a later timestamp – may denote e.g., a record of the recovery log file of the source database comprising a time indicator that may time-wise be created later than the latest snapshot stored by the persistent portion of the target database.
- the term 'recovery log file' may denote a sequential file protocolling operations being performed on a database, particularly, all those operations that modify the data, i.e., insert, update and delete operations.
- the recovery log file may be designed to allow a complete reconstruction of the database. Therefore, also the table definitions of the database may be part of the recovery log file.
- 'metadata' may denote data about data, in particular a definition of data of tables in a database and potentially relations between them.
- Fig. 1 shows a block diagram of a preferred embodiment of the computer-implemented method 100 for a crash recovery for linked databases –in particular, according to an IDAA architecture –wherein the provided, 102, linked databases comprise a source or primary database –in particular, optimized for transactions, e.g., a role-based database –and a related target or secondary database. This may be optimized for online analytical processing operations (OLAP) and may advantageously be organized column-based.
- OLAP online analytical processing operations
- Selected queries –in particular, those directed to analytics operations –against a database management system comprising the source database are transferred, i.e., offloaded, to a database management system comprising the target database for processing.
- the method 100 also comprises synchronizing, 104, selected portions of content of tables of the source database –in some databases only portions of data of portions of the tables and in other implementations a complete copy –with respective portions of content of tables of the target database, by applying, 106, during the synchronizing, changes to the source database to an in-memory target database portion of the database management system comprising the target database using a recovery log file of the source database.
- the method 100 comprises storing, 108, persistently applied changes to the in-memory target database portion asynchronously to a persistent target database storage portion, and, upon a database crash on the target database system, restoring, 110, the in-memory target database portion with the latest snapshot available in the persistent target database storage, and applying, 112, upon the database crash of the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage to the in-memory target database portion.
- the persistent portion of the target database may be updated in parallel.
- the persistent storage process or storage processor would need to be active to persistently store changes of the in-memory portion of the target database.
- the target database may be advantageous to bulk-load the target database from the source database, in particular during an initialization process. This can help to avoid the comparable slower incremental update or synchronization process because the source database may already have a larger number of entries and thus a larger number of entries in its recovery log file.
- the initialization may also work for selected tables or partitions thereof only.
- Fig. 2 shows a block diagram of an embodiment 200 of linked databases.
- the primary or source database 202 receives OLTP queries 210 and OLAP queries 214. Those queries that are identified as OLAP queries 214 are transferred or offloaded, 222, to the secondary or target database 206.
- the source database 202 comprises a plurality of tables 204 and related stored data.
- the target database 206 also comprises tables which represent at least a subset of the database tables 204 of the source database 202 and at least a subset of the data in the database tables 208.
- OLTP queries 210 are executed directly in the source database 202 and returned as OLTP output 212 to the requesting program or process.
- the OLTP or source database 202 may operate at its best performance because it is not slowed down by any resource intensive OLAP queries.
- the target database 206 may return, 224, OLAP results much faster than the source database 202.
- Fig. 3 shows a block diagram of an embodiment 300 of how the target database can be synchronized.
- the source database management system 302 controls the operation of the source database 202 and its related tables 204 (compare Fig. 2) .
- the source database management system 302 also maintains the recovery log file 306 for the source database 202.
- a log reader or log file reader 314 reading the recovery log file 306 provides these data to the applying unit 316 which applies the changes (i.e., insert, update, delete) performed to the source database also to the target database 206 for the selected tables and the selected set of data.
- the selected tables and the selected set of data may be a subset of those related tables and data in the source database 202 as defined beforehand.
- the applying unit 316 can optimize the application of the changes to the target database 206 depending on executed OLAP queries. For this, the log buffer 318 can be instrumental.
- a bulk load operation 312 can be performed for performance reasons from the source database 202 to the target database 206.
- Fig. 3 does–for reasons of clarity of the inventive concept –not yet show the split of the target database management system into an in-memory portion and a persistent portion of the target database 206 and its tables 208. This will be shown in the next figure.
- this synchronization mechanism may represent one of many synchronization techniques. Also other synchronization techniques may be applicable.
- Fig. 4 shows a block diagram of an embodiment of the proposed concept in a more implementation-near form 400.
- the elements of the source database management system in those used optionally to synchronize the source database with the target database (mainly the upper part of Fig. 4) will not be described again.
- the target database management system 308 comprises the in-memory portion 402 of the target database and the persistent portion 404 of the target database.
- the persistency service 406 writes the status of the in-memory target database portion 402 to the persistent database 404 during the operation.
- the query processor 408 directs the incoming queries to the in-memory portion 402 of the target database.
- the query processor 408 together with the recovery processor 410 analyzes the queries data accesses to determine a list of target tables and/or lists of table partitions that need to be available for the query using state-of-the-art query parsing a view resolution techniques;
- a blocking data list is determined by looking at all tables and/or table partitions of the data access list from the previous step within the recovery state is determined that has not been yet recovered;
- the latest consistent snapshot available in the persistent database 404 of the relevant tables and/or table partitions is loaded, 412, to the in-memory portion 402 of the target database, and the entries in the recovery log file 306 of the source database 202 having a timestamp later than the snapshot of the persistent database 404 are replayed to the in memory database 402.
- These are requested from the recovery processor 410 via the query processor 408 form the source DBMS 302, e.g., from the recovery log file 306 via the log reader 314 and the apply unit 316.
- Fig. 5 shows a block diagram of an embodiment 500 of the linked database system comprising components for a customer priority optimized recovery strategy.
- the source DBMS 542 has exemplary a first source database 504 of a first user or tenant (e.g., in a cloud computing environment) and a second source database 506 of a second user or tenant.
- first source database 504 of a first user or tenant e.g., in a cloud computing environment
- second source database 506 of a second user or tenant.
- separated source database systems for other users or tenants may be available.
- Separate data synchronization subsystems 510, 512 are implemented in the data synchronization system 508 in order to synchronize the source databases 504, 506 with the respective in-memory portions 518, 520 of the in-memory target database 516.
- the query process 522 receives the database queries for execution in the in-memory portion of target database (s) 516. These queries are typically the OLAP queries which have been offloaded from the source database (s) 504, 506.
- the recovery process or processor 524 of the target DBMS 514 also receives data from the query processor regarding queries to be registered and analyzed in the data access analyzer 526.
- the query analyzer determines the waiting queries 530 and the already recovered table partitions 532 in the recovery state management system 528 in order to determine –based on a priority of queries of a specific user –which tables shall be recovered first. This is finally determined and decided by the recovery schedule 534.
- the recovery schedule 534 is in constant data exchange with the recovery state management system 528 to check the actual recovery status and receives configuration data from the workload management system configuration storage.
- the recovery schedule 534 also exchanges data with the data synchronization system 508 in order to trigger the target database management system recovery database loading from the source database management system 502.
- this may be achieved by the following procedure: when the crash recovery process is initialized for the target database, for each tenant an empty histogram of blocked table/table partitions is created. When a new blocking query is registered in the recovery state the usage counter for each table/partition in the query’s blocking data list is increased by 1. Then, when a next to-be-recovered table/partition needs to be determined, a recovery priority of the table is determined based on the counter number of blocked query usage and the current tenants WLM (workload management system) configuration (i.e., its priority or importance) , the recovery item with the highest priority is selected. Finally, when the table/partition recovery action finishes, also the corresponding histogram entry is removed.
- WLM workload management system
- the calculation of the user or tenant-specific recovery priorities can exemplary be implemented as follows: if there is a strict priority between tenant pairings –e.g., a production system is more important than a test system (which may be specified by a simple configuration text) , the list of to-be-recovered tables/partitions are grouped by users/tenant; the list of groups is ordered by decreasing users/tenant priority; and the next table/partition is selected from the first non-empty group via the query usage counter selection algorithm.
- Fig. 6 shows a block diagram of an embodiment 600 of the linked database system comprising components for a volume optimized recovery strategy.
- the source DBMS 502 is shown with a source database 504 and a related recovery log file 604.
- the in-memory database portion 516 of the target DBMS 514 may be by bulk loaded via the bulk loader 602.
- the recovery process comprises at least three components: the recovery item selection unit 616, the change estimation unit 618 and the recovery schedule 620.
- the recovery processor 614 is in data exchange with the in-memory database 516 in order to collect status information about the tables in the in memory database portion 516 of the target DBMS 514.
- the target database 516 needs to be restored from the source database 504 either via a bulk load mechanism or incrementally, as already described above.
- the to-be-recovered table/table partitions are dynamically selected by the recovery process 614.
- the here proposed concept is extended by a change estimation component 618 that is responsible to estimate or determine the amount of data that needs to be restored during crash recovery. Therefore, this change estimation component 618 evaluates data change statistics in order to extrapolate how many data changes have accumulated in the source database since a recovery baseline timestamp. Based on this information, the recovery scheduler 620 selects the most efficient data synchronization method for restoring the data of to-be-recovered table/table partition.
- the data change statistics are maintained during regular updating processing, either incrementally or via bulk loading, of the target database (i.e., the in-memory database portion 516) .
- the change estimation 618 may be triggered by the recovery item selection component 616.
- the recovery scheduler 620 is also in data exchange with the bulk loader 602 and details of the incremental update process 606 in order to manage the recovery process of the in memory database portion 516. E.g., if the recovery of the in-memory database portion 516 is completed, a recovery completion notification is received from the bulk loader 602. On the other side, the recovery scheduler 620 requests change replays for specific tables in the in-memory database portion 516. Similar to the signal from the bulk loaders 602, the recovery scheduler 620 also receives a recovery completion notification from the incremental update process (or) 606. As already described in the context of Fig.
- the incremental update process 606 comprises the log reader (not shown here) adapted for reading the recovery log file entries from the recovery log file 604, as well as, the recovery log file applying unit (not shown here) which is adapted for incrementally updating the memory database portion 516 using the respective recovery log file entries from the source database 504.
- the recovery log file applying unit (not shown here) which is adapted for incrementally updating the memory database portion 516 using the respective recovery log file entries from the source database 504.
- data change statistics should be stored in the persistent metadata catalog comprised in the metadata 608 of the target database system 514 and be maintained when the target database system 514 is updated, i.e., via the incremental update or bulk loading strategy. It may store the following information: (i) timestamp when the update was processed; (ii) schema information of the database tables, e.g., column types, column widths, ... ; (iii) the amount of data changes per table/table partition that were affected by the update, i.e., inserted records, and deleted records, updated records; and aggregated metrics such as total number of records changed, total data volume that was applied, total execution time of the updates, etc.
- the statistics data can be linked to a time interval, e.g., the last x days. Additionally, the statistics data may be managed incrementally as part of each update cycle.
- the recovery baseline 610 is also maintained in the persistent metadata 608 (e.g., a metadata catalogue) of the target database system 514. It is required for estimating for determining the data volume that needs to be restored. Therefore, it is not necessary to determine an exact timestamp but interpolating the value is sufficient, e.g. : it may be maintained as heartbeat timestamp that is updated by the target database system in regular time intervals; it may be maintained as starting point of the recovery process after the crash; and it may be maintained as time of last successfully persisted target database snapshot. Thereby, the timestamp may be maintained per table in the target database.
- the persistent metadata 608 e.g., a metadata catalogue
- the crash recovery on the target database can be executed as follows: (i) firstly, the next to-be-recovered table/partition is determined; (ii) the data volume that needs to be recovered since the recovery baseline is estimated; (iii) the best recovery strategy is selected based on the estimated data volume and the recovery time is estimated; (iv) then, the recovery of the table with the selected strategy is scheduled; and (v) these steps are repeated in a loop until all data have been recovered.
- the estimation of the to-be-recovered data volumes may be executed as follows: (i) for a to-be-recovered table/partition the corresponding data change statistics 612 are looked up; (ii) the recovery baseline for the table/partition is determined; and (iii) the number of changes that need to be replicated via the incremental update process in the interval [recovery baseline, current recovery time] is interpolated.
- Fig. 7 shows a block diagram of an embodiment of the linked database system 700 for a crash recovery for linked databases.
- the linked databases 700 comprise a source database 706 and a related target database 708, wherein selected portions of content of tables of the source database are synchronized with respective portions of the content of the tables of the target database.
- the linked database system 700 comprises a processor 702 and a memory 704, communicatively coupled to the processor 702, wherein the memory 704 stores program code portions that, if executed, enable the processor, to synchronize –e.g. using a synchronization unit 714 –selected portions of content of tables of the source database 706 with respective portions of content of tables of the target database 708.
- the stored program code portions that, if executed, enable the processor 702, to apply –using an applying unit 716 –during the synchronizing, changes to the source database 706 to the in-memory target database portion 710 of the database management system comprising the target database, and store persistently –e.g. by storage processor 718 –applied changes to the in-memory target database portion 712 asynchronously to a persistent target database storage.
- the stored program code portions that, if executed, enable the processor 702 to restore –e.g., by a restore unit 720 –upon a database crash on the target database system, the in-memory target database portion 710 with the latest snapshot available in the persistent target database storage portion 712, and apply –e.g. by a second applying unit 722 –upon the database crash on the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage portion 712 to the in-memory target database portion 710.
- all modules and units of the linked database system 700 may be electrically interconnected for signal and/or data exchange. This may apply to the processor 702, the memory 704, the source database system 706, the target database system 708 including the in-memory portion 710 and the persistent portion 712, the synchronization unit 714, the applying unit 716, the storage processor 718, the restore unit 720 and the second applying unit 722. Instead of a 1: 1 connection between these modules and units they may also be connected to a linked database system internal bus system 724 for signaling data exchange.
- Embodiments of the invention may be implemented together with virtually any type of computer, regardless of the platform being suitable for storing and/or executing program code.
- Fig. 8 shows, as an example, a computing system 800 suitable for executing program code related to the proposed method.
- the computing system 800 is only one example of a suitable computer system, and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the invention described herein, regardless, whether the computer system 800 is capable of being implemented and/or performing any of the functionality set forth hereinabove.
- the computer system600 there are components, which are operational with numerous other general purpose or special purpose computing system environments or configurations.
- Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with computer system/server 800 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.
- Computer system/server 800 may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system 800.
- program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types.
- Computer system/server 800 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both, local and remote computer system storage media, including memory storage devices.
- computer system/server 800 is shown in the form of a general-purpose computing device.
- the components of computer system/server 800 may include, but are not limited to, one or more processors or processing units 802, a system memory 804, and a bus 806 that couple various system components including system memory 804 to the processor 802.
- Bus 806 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures.
- Computer system/server 800 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server 800, and it includes both, volatile and non-volatile media, removable and non-removable media.
- the system memory 804 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 808 and/or cache memory 810.
- Computer system/server 800 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
- a storage system 812 may be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a 'hard drive') .
- a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a 'floppy disk' )
- an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media
- each can be connected to bus 806 by one or more data media interfaces.
- memory 804 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
- the program/utility having a set (at least one) of program modules 816, may be stored in memory 804 by way of example, and not limiting, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating systems, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment.
- Program modules 816 generally carry out the functions and/or methodologies of embodiments of the invention, as described herein.
- the computer system/server 800 may also communicate with one or more external devices 818 such as a keyboard, a pointing device, a display 820, etc. ; one or more devices that enable a user to interact with computer system/server 800; and/or any devices (e.g., network card, modem, etc. ) that enable computer system/server 800 to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces 814. Still yet, computer system/server 800 may communicate with one or more networks such as a local area network (LAN) , a general wide area network (WAN) , and/or a public network (e.g., the Internet) via network adapter 822.
- LAN local area network
- WAN wide area network
- public network e.g., the Internet
- network adapter 822 may communicate with the other components of the computer system/server 800 via bus 806. It should be understood that, although not shown, other hardware and/or software components could be used in conjunction with computer system/server 800. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.
- linked database system 700 with crash recovery for linked databases may be attached to the bus system 806.
- the present invention may be embodied as a system, a method, and/or a computer program product.
- the computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
- the medium may be an electronic, magnetic, optical, electromagnetic, infrared or a semi-conductor system for a propagation medium.
- Examples of a computer-readable medium may include a semi-conductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM) , a read-only memory (ROM) , a rigid magnetic disk and an optical disk.
- Current examples of optical disks include compact disk-read only memory (CD-ROM) , compact disk-read/write (CD-R/W) , DVD and Blu-Ray-Disk.
- the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
- the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
- a non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , a static random access memory (SRAM) , a portable compact disk read-only memory (CD-ROM) , a digital versatile disk (DVD) , a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- SRAM static random access memory
- CD-ROM compact disk read-only memory
- DVD digital versatile disk
- memory stick a floppy disk
- a mechanically encoded device such as punch-cards or raised structures in a groove having instructions
- a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable) , or electrical signals transmitted through a wire.
- Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network.
- the network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.
- a network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
- Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
- the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
- the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) .
- electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA) , or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
- These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
- the computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatuses, or another device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatuses, or another device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
- each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function (s) .
- the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
- a computer-implemented method for a crash recovery for linked databases wherein the linked databases comprise a source database and a related target database, wherein selected queries against a database management system comprising the source database are transferred to a database management system comprising the target database for processing, the method comprising, the method comprising
- the restoring of tables of the in-memory target database portion comprises a prioritizing the recovering according to one selected out of the group consisting of a data usage, a query priority and data priority.
- recovering the table using a recovery strategy depending on the volume to be recovered, wherein the recovering strategy is an incremental update strategy or a bulk update strategy.
- a linked database system with crash recovery for linked databases wherein the linked databases comprise a source database and a related target database, wherein selected portions of content of tables of the source database are synchronized with respective portions of the content of the tables of the target database, the linked database system comprising,
- processors - a processor and a memory, communicatively coupled to the processor, wherein the memory stores program code portions that, if executed, enable the processor, to
- a computer program product for a linked database system with crash recovery for linked databases wherein the linked databases comprise a source database and a related target database, wherein selected portions of content of tables of the source database are synchronized with respective portions of the content of tables of the target database
- said computer program product comprising a computer readable storage medium having program instructions embodied therewith, said program instructions being executable by one or more computing systems or controllers to cause said one or more computing systems to
- processors - a processor and a memory, communicatively coupled to the processor, wherein the memory stores program code portions that, if executed, enable the processor, to
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (25)
- A computer-implemented method for a crash recovery for linked databases, wherein the linked databases comprise a source database and a related target database, wherein selected queries against a database management system comprising the source database are transferred to a database management system comprising the target database for processing, the method comprising:synchronizing selected portions of content of tables of the source database with respective portions of content of tables of the target database;applying, during the synchronizing, changes to the source database to an in-memory target database portion of the database management system comprising the target database;storing persistently applied changes to the in-memory target database portion asynchronously to a persistent target database storage;restoring, upon a database crash on the target database system, the in-memory target database portion with the latest snapshot available in the persistent target database storage; andapplying, upon the database crash on the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage to the in-memory target database portion.
- The method according to claim 1, wherein the synchronizing comprises:reading entries of a recovery log file relating to the source database and applying the read entries to the target database.
- The method according to claim 1, wherein the source database is optimized for transactions or wherein the source database is a row-oriented relational database management system.
- The method according to claim 1, wherein the target database is optimized to analytical operations or the target database is a column-oriented database.
- The method according to claim 1, also comprising:delaying, in case of a crash of the target database, queries against the target database until a recovery of the target database is finished.
- The method according to claim 1, wherein metadata defining the selected tables are part of the recovery log file.
- The method according to claim 1, wherein the storing persistently applied changes comprises:waiting until a predefined number of changes have been completed in the in-memory target database portion.
- The method according to claim 1, wherein the restoring of tables of the in-memory target database portion comprises a prioritizing the recovering according to one selected out of the group consisting of a data usage, a query priority and data priority.
- The method according to claim 8, wherein the prioritizing the recovery according to the data usage comprises:maintaining a counter for each table in the target database, the counter value of the counter being indicative of how many queries are waiting for the related table; andrestoring firstly the database table with the highest counter value first.
- The method according to claim 8, wherein the prioritizing the recovery according to the query priority comprises:restoring firstly the database tables receiving queries with the highest priority.
- The method according to claim 8, wherein the prioritizing the recovery according to the data priority comprises:maintaining two groups of database tables, each group relating to a separate group of users: andrestoring firstly the database tables of the group having a higher configured group priority.
- The method according to claim 1, also comprising:determining the data volume to be recovered for a next to be recovered tables:recovering the table using a recovery strategy depending on the volume to be recovered, wherein the recovering strategy is an incremental update strategy or a bulk update strategy.
- A linked database system with crash recovery for linked databases, wherein the linked databases comprise a source database and a related target database, wherein selected portions of content of tables of the source database are synchronized with respective portions of the content of the tables of the target database, the linked database system comprising:a processor and a memory, communicatively coupled to the processor, wherein the memory stores program code portions that, if executed, enable the processor, to:synchronize selected portions of the content of the tables of the source database with respective portions of the content of tables of the target database;apply, during the synchronizing, changes to the source database to an in-memory target database portion of the database management system comprising the target database; andstore persistently applied changes to the in-memory target database portion asynchronously to a persistent target database storage:restore, upon a database crash on the target database system, the in-memory target database portion with the latest snapshot available in the persistent target database storage; andapply, upon the database crash on the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage to the in-memory target database portion.
- The linked database system according to claim 13, wherein the program code portions enable the processor also to:read, for the synchronizing, the entries of a recovery log file relating to the source database and applying the read entries to the target database.
- The linked database system according to claim 13, wherein the source database is optimized for transactions or wherein the source database is a row-oriented relational database management system.
- The linked database system according to claim 13, wherein the target database is optimized to analytical operations or the target database is a column-oriented database.
- The linked database system according to claim 13, wherein the program code portions enable the processor also to:delay, in case of a crash of the target database, queries against the target database until a recovery of the target database is finished.
- The linked database system according to claim 13, wherein metadata defining the selected tables are part of the recovery log file.
- The linked database system according to claim 13, wherein the program code portions enable the processor also to:wait, for the storing persistently applied changes, until a predefined number of changes have been completed in the in-memory target database portion.
- The linked database system according to claim 13, wherein the restoring of tables of the in-memory target database portion comprises a prioritizing the recovering according to one selected out of the group consisting of a data usage, a query priority and data priority.
- The linked database system according to claim 20, wherein the program code portions enable the processor for the prioritizing the recovery according to the data usage also to:maintain a counter for each table in the target database, the counter value of the counter being indicative of how many queries are waiting for the related table; andrestore firstly the database table with the highest counter value first.
- The linked database system according to claim 20, wherein the program code portions enable the processor for the prioritizing the recovery according to the query priority also to:restore firstly the database tables receiving queries with the highest priority.
- The linked database system according to claim 20, wherein the program code portions enable the processor for the prioritizing the recovery according to the data priority also to:maintain two groups of database tables, each group relating to a separate group of users; andrestore firstly the database tables of the group having a higher configured group priority.
- The linked database system according to claim 13, wherein the program code portions enable the processor also to:determine the data volume to be recovered for a next to be recovered table; andrecover the table using a recovery strategy depending on the volume to be recovered, wherein the recovering strategy is an incremental update strategy or a bulk update strategy.
- A computer program product for a linked database system with crash recovery for linked databases, wherein the linked databases comprise a source database and a related target database, wherein selected portions of content of tables of the source database are synchronized with respective portions of the content of tables of the target database, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being executable by one or more computing systems or controllers to cause the one or more computing systems to:a processor and a memory, communicatively coupled to the processor, wherein the memory stores program code portions that, if executed, enable the processor, to:synchronize selected portions of content of tables of the source database with respective portions of content of tables of the target database;apply, during the synchronizing, changes to the source database to an in-memory target database portion of the database management system comprising the target database; andstore persistently applied changes to the in-memory target database portion asynchronously to a persistent target database storage;restore, upon a database crash on the target database system, the in-memory target database portion with the latest snapshot available in the persistent target database storage; andapply, upon the database crash on the target database system, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage to the in-memory target database portion.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202280022012.7A CN117083591A (en) | 2021-03-19 | 2022-02-16 | Asynchronous persistence of duplicate data changes in a database accelerator |
DE112022000767.2T DE112022000767T5 (en) | 2021-03-19 | 2022-02-16 | ASYNCHRONOUS PERSISTENCE OF REPLICATED DATA CHANGES IN A DATABASE ACCELERATOR |
JP2023553633A JP2024512335A (en) | 2021-03-19 | 2022-02-16 | Asynchronous persistence of replicated data changes in database accelerators |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/206,221 US11500733B2 (en) | 2021-03-19 | 2021-03-19 | Volatile database caching in a database accelerator |
US17/206,240 | 2021-03-19 | ||
US17/206,240 US11797570B2 (en) | 2021-03-19 | 2021-03-19 | Asynchronous persistency of replicated data changes in a database accelerator |
US17/206,221 | 2021-03-19 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022193894A1 true WO2022193894A1 (en) | 2022-09-22 |
Family
ID=83321517
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/076435 WO2022193893A1 (en) | 2021-03-19 | 2022-02-16 | Volatile database caching in a database accelerator |
PCT/CN2022/076471 WO2022193894A1 (en) | 2021-03-19 | 2022-02-16 | Asynchronous persistency of replicated data changes in database accelerator |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/076435 WO2022193893A1 (en) | 2021-03-19 | 2022-02-16 | Volatile database caching in a database accelerator |
Country Status (3)
Country | Link |
---|---|
JP (2) | JP2024512335A (en) |
DE (2) | DE112022000767T5 (en) |
WO (2) | WO2022193893A1 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7287043B2 (en) * | 2003-08-21 | 2007-10-23 | International Business Machines Corporation | System and method for asynchronous data replication without persistence for distributed computing |
US20140279930A1 (en) * | 2013-03-15 | 2014-09-18 | Amazon Technologies, Inc. | Fast crash recovery for distributed database systems |
US20170255528A1 (en) * | 2016-03-02 | 2017-09-07 | International Business Machines Corporation | Smart data replication recoverer |
CN112470130A (en) * | 2018-07-25 | 2021-03-09 | 微软技术许可有限责任公司 | Constant time database recovery |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6205449B1 (en) * | 1998-03-20 | 2001-03-20 | Lucent Technologies, Inc. | System and method for providing hot spare redundancy and recovery for a very large database management system |
US10803066B2 (en) | 2010-06-29 | 2020-10-13 | Teradata Us, Inc. | Methods and systems for hardware acceleration of database operations and queries for a versioned database based on multiple hardware accelerators |
CN104239476B (en) * | 2014-09-04 | 2018-09-25 | 上海天脉聚源文化传媒有限公司 | A kind of method, apparatus and system of database synchronization |
CN106815275B (en) * | 2015-12-02 | 2020-11-27 | 阿里巴巴集团控股有限公司 | Method and equipment for realizing synchronization of main database and standby database through standby database |
US10242050B2 (en) * | 2015-12-23 | 2019-03-26 | Sybase, Inc. | Database caching in a database system |
CN108052681B (en) * | 2018-01-12 | 2020-05-26 | 毛彬 | Method and system for synchronizing structured data between relational databases |
-
2022
- 2022-02-16 WO PCT/CN2022/076435 patent/WO2022193893A1/en active Application Filing
- 2022-02-16 WO PCT/CN2022/076471 patent/WO2022193894A1/en active Application Filing
- 2022-02-16 JP JP2023553633A patent/JP2024512335A/en active Pending
- 2022-02-16 JP JP2023553627A patent/JP2024510137A/en active Pending
- 2022-02-16 DE DE112022000767.2T patent/DE112022000767T5/en active Pending
- 2022-02-16 DE DE112022000492.4T patent/DE112022000492T5/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7287043B2 (en) * | 2003-08-21 | 2007-10-23 | International Business Machines Corporation | System and method for asynchronous data replication without persistence for distributed computing |
US20140279930A1 (en) * | 2013-03-15 | 2014-09-18 | Amazon Technologies, Inc. | Fast crash recovery for distributed database systems |
US20170255528A1 (en) * | 2016-03-02 | 2017-09-07 | International Business Machines Corporation | Smart data replication recoverer |
CN112470130A (en) * | 2018-07-25 | 2021-03-09 | 微软技术许可有限责任公司 | Constant time database recovery |
Also Published As
Publication number | Publication date |
---|---|
JP2024512335A (en) | 2024-03-19 |
DE112022000492T5 (en) | 2023-12-14 |
DE112022000767T5 (en) | 2023-11-09 |
JP2024510137A (en) | 2024-03-06 |
WO2022193893A1 (en) | 2022-09-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11360998B2 (en) | Database management system and method of operation | |
US10990610B2 (en) | Synchronization on reactivation of asynchronous table replication | |
US10762108B2 (en) | Query dispatching system and method | |
US10929398B2 (en) | Distributed system with accelerator and catalog | |
US11010262B2 (en) | Database system recovery using preliminary and final slave node replay positions | |
EP2590087B1 (en) | Database log parallelization | |
EP4283482A2 (en) | Data replication and data failover in database systems | |
US9069704B2 (en) | Database log replay parallelization | |
US8554726B2 (en) | Systems and methods for reslicing data in a relational database | |
EP2746971A2 (en) | Replication mechanisms for database environments | |
US9075858B2 (en) | Non-disruptive data movement and node rebalancing in extreme OLTP environments | |
US11182260B1 (en) | Avoiding recovery log archive access in database accelerator environments | |
US11797570B2 (en) | Asynchronous persistency of replicated data changes in a database accelerator | |
US10769034B2 (en) | Caching DML statement context during asynchronous database system replication | |
US11226878B1 (en) | Accelerator-based database recovery | |
US20230325404A1 (en) | Query execution on a replicated database with most recent data | |
WO2022193894A1 (en) | Asynchronous persistency of replicated data changes in database accelerator | |
US10678812B2 (en) | Asynchronous database transaction handling | |
US11500733B2 (en) | Volatile database caching in a database accelerator | |
Qu | On-Demand ETL for Real-Time Analytics | |
Arora et al. | Oracle® Streams for Near Real Time Asynchronous Replication | |
CN117033322A (en) | Big data storage method, system, electronic equipment and storage medium integrating stream and batch |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22770249 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2023553633 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202280022012.7 Country of ref document: CN Ref document number: 112022000767 Country of ref document: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22770249 Country of ref document: EP Kind code of ref document: A1 |