CN110232093B - Initial loading method and equipment based on flashback inquiry in database synchronization - Google Patents
Initial loading method and equipment based on flashback inquiry in database synchronization Download PDFInfo
- Publication number
- CN110232093B CN110232093B CN201910360528.2A CN201910360528A CN110232093B CN 110232093 B CN110232093 B CN 110232093B CN 201910360528 A CN201910360528 A CN 201910360528A CN 110232093 B CN110232093 B CN 110232093B
- Authority
- CN
- China
- Prior art keywords
- scn
- loaded
- database
- final
- flashback
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011068 loading method Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 claims abstract description 41
- 230000000694 effects Effects 0.000 claims description 15
- 238000004891 communication Methods 0.000 claims description 8
- 239000000126 substance Substances 0.000 claims 1
- 206010016754 Flashback Diseases 0.000 description 59
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000007704 transition Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The embodiment of the invention provides an initial loading method and equipment based on flashback inquiry in database synchronization. The method comprises the following steps: acquiring a first final SCN of a current log of a source database, and if an active transaction related to a system table exists in the current active transaction and an initial SCN of the related active transaction is smaller than the first final SCN, finishing submitting the related active transaction; acquiring a second final SCN, sending the second final SCN to a target end data synchronization service, associating the second final SCN with a table to be loaded, flashing back to inquire a data mirror image of the table to be loaded at the second final SCN according to the second final SCN, sending the data mirror image to the target end data synchronization service, synchronizing the data mirror image, and sending an operation of capturing the table to be loaded in a log stream to the target end data synchronization service; if the SCN of the operation is larger than the second final SCN, synchronizing the table to be loaded. The invention can ensure the consistency of the source end and the target end databases and the stability of the application of the source end database.
Description
Technical Field
The embodiment of the invention relates to the technical field of database synchronization, in particular to an initial loading method and equipment based on flashback inquiry in database synchronization.
Background
Among database data real-time synchronization technologies, the data real-time synchronization technology based on database log analysis is widely applied because of the characteristics of supporting heterogeneous operating systems and database platforms, having small invasion to source-end database systems, having low data synchronization delay, and the like. A real-time data synchronization technology based on log analysis generally captures incremental log content of a source end database, then sends the incremental log content to a target end through a network, recovers transaction operation SQL statements of the source end database by using an SQL reverse generation technology at the target end, and executes the recovered SQL statements through a general database access interface such as ODBC, OCI and the like to realize real-time data synchronization of the source end and the target end.
The real-time synchronization of the database data realized by the data real-time synchronization technology comprises the following steps: reading a source database log file in real time, and capturing transaction increment log content on a source database; analyzing the content of the incremental log in real time, extracting a transaction ID, an operation table ID, operation types of the transaction such as INSERT, UPDATE and DELETE, a transaction operation timestamp, a log LSN value, operation data and the like in the log, then assembling the transaction ID, the operation table ID, the operation types of the transaction into a message with a specific format, and storing the message in a sending queue; and finally, sending the message in the sending queue to a target end of the synchronous system through a TCP/IP network.
When the database data is synchronized in real time, first, data initialization operation needs to be performed on a target database to obtain a base point of data synchronization. After the data initialization operation is completed, real-time incremental data synchronization can be performed on the basic data. In a real database application, there may be many applications in the source database, and these applications may perform modification operations on the database in minutes, seconds, and some applications have very long transaction time span (i.e. long transaction), and often a transaction lasts for several hours. In this scenario, when data initialization from the source database to the target database is implemented, an S-lock on a table to be loaded on the source database is required, so as to obtain a clean data. However, the S lock on the source table may affect the database application at the source end, so that the application cannot modify the table, and when the table to be loaded is occupied by a long transaction, the loaded connection cannot acquire the S lock of the table in a short time, so that the time spent on loading is prolonged. Therefore, a method for ensuring the consistency of the source database and the target database and the stability of the application of the source database on the premise of avoiding the S lock on the table to be loaded is found, which is a technical problem to be solved in the industry.
Disclosure of Invention
In view of the above problems in the prior art, embodiments of the present invention provide an initialization loading method and device based on flashback query in database synchronization.
In a first aspect, an embodiment of the present invention provides a method for initializing and loading based on flashback query in database synchronization, including: acquiring a first final SCN of a current log of a source database, inquiring a system table of the source database, and if an active transaction related to the system table exists in the current active transaction and the initial SCN of the related active transaction is smaller than the first final SCN, finishing submitting the related active transaction; acquiring a second final SCN of a current log of a source database, sending the second final SCN to a target end data synchronization service, associating the second final SCN with a table to be loaded by the target end data synchronization service, flashing back to inquire a data mirror image of the table to be loaded at the second final SCN according to the second final SCN, sending the acquired data mirror image to the target end data synchronization service, synchronizing the data mirror image, and sending an operation of capturing the table to be loaded in a log stream to the target end data synchronization service; and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is greater than the second final SCN, synchronizing the table to be loaded.
Further, on the basis of the contents of the above method embodiments, in the method for initializing and loading based on flashback query in database synchronization provided in the embodiments of the present invention, the system table includes: v $ TRANSACTION and/or V $ LOCKED _ OBJECT.
Further, on the basis of the content of the foregoing method embodiment, the method for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention further includes, after sending the acquired data mirror to the target-end data synchronization service and before synchronizing the data mirror: and starting log analysis of the source database.
Further, on the basis of the content of the above method embodiment, the method for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention further includes: and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is less than or equal to the second final SCN, discarding the operation of the table to be loaded.
In a second aspect, an embodiment of the present invention provides an initial loading apparatus based on flashback query in database synchronization, including:
a related activity transaction submitting module, configured to obtain a first final SCN of a current log of a source database, query a system table of the source database, and if an activity transaction related to the system table exists in the current activity transaction and an initial SCN of the related activity transaction is smaller than the first final SCN, complete the submission of the related activity transaction;
a to-be-loaded table data mirror image acquisition module, configured to acquire a second final SCN of a current log of a source database, send the second final SCN to a target end data synchronization service, where the target end data synchronization service associates the second final SCN with a to-be-loaded table, and according to the second final SCN, perform a flash query on a data mirror image of the to-be-loaded table at the second final SCN, send the acquired data mirror image to the target end data synchronization service, synchronize the data mirror image, and send an operation of capturing the to-be-loaded table in a log stream to the target end data synchronization service;
and the table to be loaded synchronizing module is used for synchronizing the table to be loaded if the SCN of the operation captured by the table to be loaded in the log stream is greater than the second final SCN.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
at least one processor; and
at least one memory communicatively coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform a flashback-query-based initial loading method in database synchronization provided by any one of the various possible implementations of the first aspect.
In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a flashback query-based initial loading method in database synchronization provided in any one of various possible implementations of the first aspect.
According to the method and the device for initial loading based on the flashback query in the database synchronization, provided by the embodiment of the invention, the data mirror image of the database at a certain moment is obtained through a flashback query mechanism, and the transition from the initialization to the data synchronization is realized by combining the SCN in the log stream, so that the consistency of the source end database and the target end database and the application stability of the source end database can be ensured on the premise of avoiding an S lock on a table to be loaded.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description will be given below to the drawings required for the description of the embodiments or the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of an initial loading method based on flashback query in database synchronization according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an initialization loading apparatus based on flashback query in database synchronization according to an embodiment of the present invention;
fig. 3 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. In addition, technical features of various embodiments or individual embodiments provided by the invention can be arbitrarily combined with each other to form a feasible technical solution, but must be realized by a person skilled in the art, and when the technical solution combination is contradictory or cannot be realized, the technical solution combination is not considered to exist and is not within the protection scope of the present invention.
At present, a method for initializing data from a source database to a destination database without using an S-lock on a table to be loaded needs to be found, and the technical problem that the application of the source database is greatly affected when data is loaded in the prior art is solved. To achieve the object, an embodiment of the present invention provides an initial loading method based on flashback query in database synchronization, and referring to fig. 1, the method includes:
101. acquiring a first final SCN of a current log of a source database, inquiring a system table of the source database, and if an active transaction related to the system table exists in the current active transaction and the initial SCN of the related active transaction is smaller than the first final SCN, finishing submitting the related active transaction;
102. acquiring a second final SCN of a current log of a source database, sending the second final SCN to a target end data synchronization service, associating the second final SCN with a table to be loaded by the target end data synchronization service, flashing back to inquire a data mirror image of the table to be loaded at the second final SCN according to the second final SCN, sending the acquired data mirror image to the target end data synchronization service, synchronizing the data mirror image, and sending an operation of capturing the table to be loaded in a log stream to the target end data synchronization service; specifically, the Flashback Query (Flashback Query) provided in the embodiment of the present invention is one of four database Flashback functions provided by an ORACLE database (the four Flashback functions are: Flashback Query, Flashback deletion, Flashback archiving, and Flashback database respectively), the Flashback Query can be subdivided into a basic Flashback Query and a Flashback table, and the basic Flashback Query is mainly used for querying a database state in a past certain time period. The Oracle database will fetch the required revocation data (provided that revocation is available, i.e. the revocation data is not yet covered) for rollback, but this rollback is temporary and only visible for the current session.
The flashback table is used primarily to rollback a table to a point in time in the past. Similarly, the Oracle database queries the revocation segment first, extracts all changes after a certain time point passes, constructs an SQL statement for inverting the changes, and backs the SQL statement, and the flashback query is a single transaction. Specifically, enabling table flashback first supports row movement on the table, and flashback table operation may occur as follows: violating the database constraint, for example, if the user carelessly deletes the data in the child table, the user wants to perform rollback by using the flashback table technique, and just in the middle, the record corresponding to the data in the parent table is also deleted, in this case, the flashback table operation fails due to violating the foreign key constraint; revocation data fails, such as revocation data used to support a flashback operation is overwritten, in which case a flashback table operation will naturally fail; flashback cannot cross the DDL, i.e., there is an excessive change in the table structure between the flashback point and the current point, in which case the flashback operation will also fail.
103. And if the SCN of the operation of the table to be loaded, which is captured in the log stream, is greater than the second final SCN, synchronizing the table to be loaded.
Based on the content of the foregoing method embodiment, as an optional embodiment, in the method for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention, the system table includes: v $ TRANSACTION and/or V $ LOCKED _ OBJECT.
Based on the content of the foregoing method embodiment, as an optional embodiment, the method for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention further includes, after sending the acquired data mirror to the target-end data synchronization service and before synchronizing the data mirror: and starting log analysis of the source database.
Based on the content of the foregoing method embodiment, as an optional embodiment, the method for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention further includes: and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is less than or equal to the second final SCN, discarding the operation of the table to be loaded.
According to the flash query-based initialization loading method in database synchronization provided by the embodiment of the invention, the data mirror image of the database at a certain moment is obtained through a flash query mechanism, and the transition from initialization to data synchronization is realized by combining the SCN in the log stream, so that the consistency of a source end database and a target end database and the application stability of the source end database can be ensured on the premise of avoiding an S lock on a table to be loaded.
In order to more clearly illustrate the essence of the technical solution of the present invention, on the basis of the above-mentioned embodiments, a detailed embodiment is proposed, which shows the overall view of the technical solution of the present invention. It should be noted that the detailed embodiment is only for embodying the technical essence of the present invention, and is not intended to limit the scope of the present invention, and those skilled in the art can obtain any combination type technical solution meeting the essence of the technical solution of the present invention by combining technical features based on the various embodiments of the present invention, and as long as the combination type technical solution can be practically implemented, the combination type technical solution is within the scope of the present patent.
The detailed embodiment of the invention provides an initial loading method based on flashback inquiry in database synchronization, taking a loading T table as an example, and comprising the following steps:
s1, acquiring the final SCN of the current log of the source database: get _ SYSTEM _ CHANGE _ NUMBER FROM DUAL;
s2, by querying the system tables V $ TRANSACTION and V $ LOCKED _ OBJECT of the source database, judging whether the starting SCN of the active TRANSACTION related to the tables in the current active TRANSACTION has a smaller TRANSACTION than the SCN obtained in S1, if yes, waiting for the TRANSACTIONs to be submitted and then continuing;
the main purpose of the above steps is to ensure that the transaction log related to the T table in the current database system can be read from this moment, because after the log synchronization is started, the log file position where the current SCN value is located is set as the starting point of the synchronization reading and analysis, and if not, a certain transaction related to the T table starts before the current SCN, then its modification operation cannot be analyzed, and the modification snapback function of the transaction cannot be obtained, resulting in data loss.
S3, the maximum SCN of the current log of the source database is obtained again, the SCN is sent to the target end data synchronization service, and the target end data synchronization service associates the SCN with the table to be loaded.
This step is effective in that, after waiting for a long transaction to be committed, the SCN obtained in S1 may be a long-past value, and if a flashback query is performed using the SCN, the probability that the rollback section is cleaned during the flashback query is greatly increased, and the flashback query is a data image obtained from the database at a certain time based on the historical data in the rollback section, and the rollback section is cleaned up to cause a flashback query to report an error. Therefore, retrieving the latest SCN value in the current source database helps to reduce the probability of error reporting of the flashback query.
And S4, utilizing the SCN value obtained in S3 to carry out flash-back query on the data mirror image of the table to be loaded in the SCN, and sending the obtained data to the target end for synchronous data storage. Query SQL used SELECT FROM AS SCN (;
and S5, after the initialization is completed, starting the log analysis of the source end to perform data synchronization, and sending the operation acquired by the table in the log stream to the target end to perform data synchronization.
S6, after the data synchronization of the target end receives the commit operation of the transaction, whether the SCN in the commit operation of the transaction is larger than the flashback SCN related to the table in S3 needs to be judged, if so, the operation of the table is synchronized; otherwise, the operation of the table in the current transaction is discarded, the transition from loading to synchronization is realized, and the initialization work of data synchronization is ensured to be completed under the condition that the application of the source database is not influenced.
The principle of this step is that the flashback query can obtain all the data that has been committed before the table flashback SCN, while the uncommitted transactions are invisible to the flashback query, so all transactions in the log stream that commit an SCN greater than flashback SCN need to be synchronized.
The detailed embodiment provided by the invention can not load any lock (such as an S lock or an X lock) on a table to be loaded, reduces the influence on the application of the source-end database during the initial loading of data to the minimum, then utilizes a flash mechanism of the ORACLE database to obtain a data mirror image of the database at a certain moment, and combines an SCN value in a log stream to realize the transition from the data initialization to the data synchronization, thereby ensuring the consistency of the source-end database and the target-end database and ensuring the stable operation of the application of the source-end database.
The implementation basis of the various embodiments of the present invention is realized by programmed processing performed by a device having a processor function. Therefore, in engineering practice, the technical solutions and functions thereof of the embodiments of the present invention can be packaged into various modules. Based on this practical situation, on the basis of the foregoing embodiments, an embodiment of the present invention provides an apparatus for initializing and loading based on flashback inquiries in database synchronization, where the apparatus is used to execute a method for initializing and loading based on flashback inquiries in database synchronization in the foregoing method embodiments. Referring to fig. 2, the apparatus includes:
a related activity transaction submitting module 201, configured to obtain a first final SCN of a current log of a source database, query a system table of the source database, and if an activity transaction related to the system table exists in the current activity transaction and an initial SCN of the related activity transaction is smaller than the first final SCN, complete the submission of the related activity transaction;
a to-be-loaded table data mirror image obtaining module 202, configured to obtain a second final SCN of a current log of a source database, send the second final SCN to a target end data synchronization service, where the target end data synchronization service associates the second final SCN with a to-be-loaded table, according to the second final SCN, perform a flash query on a data mirror image of the to-be-loaded table at the second final SCN, send the obtained data mirror image to the target end data synchronization service, perform synchronization on the data mirror image, and send an operation of capturing the to-be-loaded table in a log stream to the target end data synchronization service;
a table to be loaded synchronizing module 203, configured to synchronize the table to be loaded if the SCN of the operation captured by the table to be loaded in the log stream is greater than the second final SCN.
According to the initialization loading device based on the flashback query in the database synchronization, provided by the embodiment of the invention, the related active transaction submitting module, the data mirror image acquisition module of the table to be loaded and the synchronization module of the table to be loaded are adopted, the data mirror image of the database at a certain moment is acquired through a flashback query mechanism, and the transition from initialization to data synchronization is realized by combining with the SCN in the log stream, so that the consistency of a source end database and a target end database and the application stability of the source end database can be ensured on the premise of avoiding an S lock on the table to be loaded.
It should be noted that, the apparatus in the apparatus embodiment provided by the present invention may be used for implementing methods in other method embodiments provided by the present invention, except for implementing methods in the above method embodiments, and the difference is only that a corresponding function module is provided, and the principle of the apparatus embodiment is basically the same as that of the above apparatus embodiment provided by the present invention. For example:
based on the content of the foregoing device embodiment, as an optional embodiment, the device for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention further includes: and the log analysis starting module is used for starting the log analysis of the source database.
Based on the content of the foregoing device embodiment, as an optional embodiment, the device for initializing and loading based on flashback query in database synchronization provided in the embodiment of the present invention further includes: and the operation module for discarding the table to be loaded is used for discarding the operation of the table to be loaded if the SCN of the operation captured by the table to be loaded in the log stream is less than or equal to the second final SCN.
The method of the embodiment of the invention is realized by depending on the electronic equipment, so that the related electronic equipment is necessarily introduced. To this end, an embodiment of the present invention provides an electronic apparatus, as shown in fig. 3, including: at least one processor (processor)301, a communication Interface (Communications Interface)304, at least one memory (memory)302 and a communication bus 303, wherein the at least one processor 301, the communication Interface 304 and the at least one memory 302 are configured to communicate with each other via the communication bus 303. The at least one processor 301 may call logic instructions in the at least one memory 302 to perform the following method: acquiring a first final SCN of a current log of a source database, inquiring a system table of the source database, and if an active transaction related to the system table exists in the current active transaction and the initial SCN of the related active transaction is smaller than the first final SCN, finishing submitting the related active transaction; acquiring a second final SCN of a current log of a source database, sending the second final SCN to a target end data synchronization service, associating the second final SCN with a table to be loaded by the target end data synchronization service, flashing back to inquire a data mirror image of the table to be loaded at the second final SCN according to the second final SCN, sending the acquired data mirror image to the target end data synchronization service, synchronizing the data mirror image, and sending an operation of capturing the table to be loaded in a log stream to the target end data synchronization service; and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is greater than the second final SCN, synchronizing the table to be loaded.
Furthermore, the logic instructions in the at least one memory 302 may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. Examples include: acquiring a first final SCN of a current log of a source database, inquiring a system table of the source database, and if an active transaction related to the system table exists in the current active transaction and the initial SCN of the related active transaction is smaller than the first final SCN, finishing submitting the related active transaction; acquiring a second final SCN of a current log of a source database, sending the second final SCN to a target end data synchronization service, associating the second final SCN with a table to be loaded by the target end data synchronization service, flashing back to inquire a data mirror image of the table to be loaded at the second final SCN according to the second final SCN, sending the acquired data mirror image to the target end data synchronization service, synchronizing the data mirror image, and sending an operation of capturing the table to be loaded in a log stream to the target end data synchronization service; and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is greater than the second final SCN, synchronizing the table to be loaded. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. Based on this recognition, each block in the flowchart or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In this patent, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (7)
1. An initial loading method based on flashback inquiry in database synchronization is characterized by comprising the following steps:
acquiring a first final SCN of a current log of a source database, inquiring a system table of the source database, and if an active transaction related to the system table exists in the current active transaction and the initial SCN of the related active transaction is smaller than the first final SCN, finishing submitting the related active transaction;
acquiring a second final SCN of a current log of a source database, sending the second final SCN to a target end data synchronization service, associating the second final SCN with a table to be loaded by the target end data synchronization service, flashing back to inquire a data mirror image of the table to be loaded at the second final SCN according to the second final SCN, sending the acquired data mirror image to the target end data synchronization service, synchronizing the data mirror image, and sending an operation of capturing the table to be loaded in a log stream to the target end data synchronization service;
and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is greater than the second final SCN, synchronizing the table to be loaded.
2. The method for initial loading based on flashback query in database synchronization according to claim 1, wherein the system table includes:
v $ TRANSACTION and/or V $ LOCKED _ OBJECT.
3. The method for initial loading based on flashback query in database synchronization according to claim 1, further comprising, after the sending the obtained data image to the target-end data synchronization service and before the synchronizing the data image:
and starting log analysis of the source database.
4. The method for initial loading based on flashback inquiry in database synchronization according to claim 1, further comprising:
and if the SCN of the operation of the table to be loaded, which is captured in the log stream, is less than or equal to the second final SCN, discarding the operation of the table to be loaded.
5. An initial loading device based on flashback inquiry in database synchronization is characterized by comprising:
a related activity transaction submitting module, configured to obtain a first final SCN of a current log of a source database, query a system table of the source database, and if an activity transaction related to the system table exists in the current activity transaction and an initial SCN of the related activity transaction is smaller than the first final SCN, complete the submission of the related activity transaction;
a to-be-loaded table data mirror image acquisition module, configured to acquire a second final SCN of a current log of a source database, send the second final SCN to a target end data synchronization service, where the target end data synchronization service associates the second final SCN with a to-be-loaded table, and according to the second final SCN, perform a flash query on a data mirror image of the to-be-loaded table at the second final SCN, send the acquired data mirror image to the target end data synchronization service, synchronize the data mirror image, and send an operation of capturing the to-be-loaded table in a log stream to the target end data synchronization service;
and the table to be loaded synchronizing module is used for synchronizing the table to be loaded if the SCN of the operation captured by the table to be loaded in the log stream is greater than the second final SCN.
6. An electronic device, comprising:
at least one processor, at least one memory, a communication interface, and a bus; wherein the content of the first and second substances,
the processor, the memory and the communication interface complete mutual communication through the bus;
the memory stores program instructions executable by the processor, the processor calling the program instructions to perform the method of any of claims 1 to 4.
7. A non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910360528.2A CN110232093B (en) | 2019-04-30 | 2019-04-30 | Initial loading method and equipment based on flashback inquiry in database synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910360528.2A CN110232093B (en) | 2019-04-30 | 2019-04-30 | Initial loading method and equipment based on flashback inquiry in database synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110232093A CN110232093A (en) | 2019-09-13 |
| CN110232093B true CN110232093B (en) | 2021-04-13 |
Family
ID=67861030
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910360528.2A Active CN110232093B (en) | 2019-04-30 | 2019-04-30 | Initial loading method and equipment based on flashback inquiry in database synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110232093B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111241094B (en) * | 2019-12-31 | 2021-06-08 | 武汉达梦数据库有限公司 | Database deleted column synchronization method and device based on log analysis |
| CN113220788A (en) * | 2021-05-31 | 2021-08-06 | 上海英方软件股份有限公司 | Snapshot selection method and device based on Oracle database full synchronization |
| CN113282643B (en) * | 2021-06-04 | 2023-03-21 | 上海英方软件股份有限公司 | Table comparison method and system under ORACLE database synchronization environment |
| CN113515353A (en) * | 2021-06-04 | 2021-10-19 | 深圳奥哲网络科技有限公司 | Long transaction processing method, system, electronic device and storage medium |
| CN116541470B (en) * | 2023-07-07 | 2024-02-13 | 深圳创维智慧科技有限公司 | Synchronization method, device, equipment and medium of database read-only standby library |
| CN117033398B (en) * | 2023-10-09 | 2024-01-16 | 之江实验室 | Data flashback query method and device based on dolt database |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221573A (en) * | 2003-04-30 | 2008-07-16 | 甲骨文国际公司 | Method for restoring storage database to the original state |
| CN104573056A (en) * | 2015-01-22 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Database large data volume online migration method based on oracle |
| CN105068893A (en) * | 2015-08-26 | 2015-11-18 | 国家电网公司 | Database state restoration method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7281023B2 (en) * | 2003-12-15 | 2007-10-09 | At&T Knowledge Ventures, L.P. | Architecture of database application with robust online recoverability |
-
2019
- 2019-04-30 CN CN201910360528.2A patent/CN110232093B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221573A (en) * | 2003-04-30 | 2008-07-16 | 甲骨文国际公司 | Method for restoring storage database to the original state |
| CN104573056A (en) * | 2015-01-22 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Database large data volume online migration method based on oracle |
| CN105068893A (en) * | 2015-08-26 | 2015-11-18 | 国家电网公司 | Database state restoration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110232093A (en) | 2019-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232093B (en) | Initial loading method and equipment based on flashback inquiry in database synchronization | |
| CN109656934B (en) | Source Oracle database DDL synchronization method and device based on log analysis | |
| EP3602341B1 (en) | Data replication system | |
| US20070255763A1 (en) | Database replication method and system | |
| US10503699B2 (en) | Metadata synchronization in a distrubuted database | |
| US8655836B2 (en) | Method, device and system for implementing data synchronization between source database and target database | |
| US7702698B1 (en) | Database replication across different database platforms | |
| EP2052338B1 (en) | Dynamic bulk-to-brick transformation of data | |
| CN108121827B (en) | Full data synchronization method and device | |
| US6873995B2 (en) | Method, system, and program product for transaction management in a distributed content management application | |
| CN109271452B (en) | DB2 database data synchronous updating method and device | |
| US20230092911A1 (en) | Tracking change data in a database | |
| US10467192B2 (en) | Method and apparatus for updating data table in keyvalue database | |
| WO2011158387A1 (en) | Data processing failure recovery method, system and program | |
| CN109298978B (en) | Recovery method and system for database cluster of specified position | |
| CN107870982B (en) | Data processing method, system and computer readable storage medium | |
| CN109634975B (en) | Data synchronization method and device, electronic equipment and computer readable storage medium | |
| CN109446262B (en) | Data aggregation method and device | |
| EP1131756B1 (en) | Protocol for synchronizing parallel processors in a mobile communications system | |
| US20090157775A1 (en) | Archiving method and system | |
| US7599903B2 (en) | Systems and methods for extracting data sets from an online relational database into a data warehouse | |
| CN113986942B (en) | Message queue management method and device based on man-machine conversation | |
| CN114661823A (en) | Data synchronization method and device, electronic equipment and readable storage medium | |
| CN115658815A (en) | CDC (control data center) -based data synchronization method | |
| CN115033578A (en) | Method for updating service data, related device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Sun Feng Inventor after: Yu Yuanlan Inventor after: Zhao Jiawei Inventor before: Fu Quan Inventor before: Sun Feng Inventor before: Yu Yuanlan Inventor before: Zhao Jiawei |
|
| CB03 | Change of inventor or designer information | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Patentee after: Wuhan dream database Co.,Ltd. Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Patentee before: WUHAN DAMENG DATABASE Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220905 Address after: 430073 16-19 / F, building C3, future science and technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province Patentee after: Wuhan dream database Co.,Ltd. Patentee after: HUAZHONG University OF SCIENCE AND TECHNOLOGY Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Patentee before: Wuhan dream database Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230802 Address after: 16-19/F, Building C3, Future Science and Technology Building, No. 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province, 430206 Patentee after: Wuhan dream database Co.,Ltd. Address before: 430073 16-19 / F, building C3, future science and technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan City, Hubei Province Patentee before: Wuhan dream database Co.,Ltd. Patentee before: HUAZHONG University OF SCIENCE AND TECHNOLOGY |
|
| TR01 | Transfer of patent right |