CN109241174B - Data synchronization method and device, readable storage medium and electronic equipment - Google Patents
Data synchronization method and device, readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN109241174B CN109241174B CN201810671890.7A CN201810671890A CN109241174B CN 109241174 B CN109241174 B CN 109241174B CN 201810671890 A CN201810671890 A CN 201810671890A CN 109241174 B CN109241174 B CN 109241174B
- Authority
- CN
- China
- Prior art keywords
- synchronized
- data
- database
- change
- data table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure relates to a data synchronization method, a data synchronization device, a readable storage medium and electronic equipment, which belong to the field of data storage, can reduce the maintenance cost of a database, improve the efficiency and the real-time performance of database synchronization, and neither require all PDBs to be in an open state nor require the PDBs not to have the same table name under the same user, nor require manual participation. The method comprises the following steps: acquiring metadata and a change log of a source database; analyzing the obtained change log by using the obtained metadata to capture data change operations about the source database in the change log; analyzing the captured data change operation by using the acquired metadata to generate change data of the source database; the generated change data is synchronized to the target database.
Description
Technical Field
The present disclosure relates to the field of data storage, and in particular, to a data synchronization method, apparatus, readable storage medium, and electronic device.
Background
Currently, there are generally two methods to implement real-time data synchronization of Oracle 12C multi-tenant databases. One approach is to analyze the changed data of the source database through an Oracle ONLINE dictionary (e.g., DBMS log nr, dit FROM ONLINE CATALOG ONLINE dictionary), and then synchronize the target database using the analyzed changed data. The drawback of this approach is that all Pluggable Databases (PDBs) must be guaranteed to be open, and there cannot be the same table name for the same user between PDBs. The other method is to analyze the changed data of the source database through the offline dictionary file and then synchronize the target database by utilizing the analyzed changed data. The method has the defects that the dictionary file needs to be manually written and imported, the maintenance cost of the database is high, and the real-time performance is poor.
Disclosure of Invention
The purpose of the present disclosure is to provide a data synchronization method, apparatus, readable storage medium, and electronic device, which neither require all PDBs to be in an open state nor require the PDBs to have the same table name under the same user, and do not require manual involvement, thereby reducing the maintenance cost of the database and improving the efficiency and real-time performance of database synchronization.
According to a first embodiment of the present disclosure, there is provided a data synchronization method including: acquiring metadata and a change log of a source database; analyzing the obtained change log by using the obtained metadata to capture data change operations about the source database in the change log; analyzing the captured data change operation by using the acquired metadata to generate change data of the source database; the generated change data is synchronized to the target database.
Optionally, the obtaining metadata and a change log of the source database includes: acquiring metadata of the source database according to the pluggable database which needs to be synchronized and the configuration of the data table which needs to be synchronized of the pluggable database which needs to be synchronized; and acquiring a change log of the source database.
Optionally, the obtained metadata includes a unique identifier of a pluggable database that needs to be synchronized of the source database and a unique identifier of a data table that needs to be synchronized of the pluggable database that needs to be synchronized; the analyzing the obtained change log with the obtained metadata to capture data change operations in the change log with respect to the source database includes: and analyzing the obtained change log by using the unique identification of the pluggable database to be synchronized and the unique identification of the data table to be synchronized of the pluggable database to be synchronized to capture the data change operation of the data table to be synchronized in the change log.
Optionally, the method further comprises: acquiring the unique identifier of the pluggable database to be synchronized by using the view for recording the information of the pluggable database of the source database; and acquiring the unique identifier of the data table to be synchronized by using the view of the information of the data table of the pluggable database recording the source database.
Optionally, the analyzing the obtained change log by using the unique identifier of the pluggable database to be synchronized and the unique identifier of the data table to be synchronized of the pluggable database to be synchronized to capture the data change operation in the change log with respect to the data table to be synchronized includes: capturing data change operations in the change log about the data table needing to be synchronized by recording a view of an analysis result for the obtained change log and utilizing the unique identification of the pluggable database needing to be synchronized and the unique identification of the data table needing to be synchronized of the pluggable database needing to be synchronized.
Optionally, the obtained metadata includes column information of the data table that needs to be synchronized; the analyzing the captured data change operation by using the acquired metadata to generate change data of the source database includes: and analyzing the captured data change operation by using the column information of the data table needing to be synchronized, and generating change data of the data table needing to be synchronized.
Optionally, the method further comprises: and acquiring the column information of the data table to be synchronized by using the view of the column information of the data table of the pluggable database recording the source database.
According to a second embodiment of the present disclosure, there is provided a data synchronization apparatus including: the acquisition module is used for acquiring metadata and a change log of a source database; the acquisition module is used for analyzing the acquired change log by using the acquired metadata to acquire data change operation related to the source database in the change log; the generating module is used for analyzing the captured data change operation by using the acquired metadata to generate change data of the source database; a synchronization module to synchronize the generated change data to a target database.
Optionally, the obtaining module is configured to obtain a change log of the source database, and obtain the metadata of the source database according to the configuration of the pluggable database that needs to be synchronized and the data table that needs to be synchronized of the pluggable database that needs to be synchronized.
Optionally, the metadata acquired by the acquisition module includes a unique identifier of a pluggable database of the source database that needs to be synchronized and a unique identifier of a data table of the pluggable database that needs to be synchronized; the capture module is used for analyzing the obtained change log by using the unique identifier of the pluggable database to be synchronized and the unique identifier of the data table to be synchronized of the pluggable database to be synchronized, so as to capture the data change operation of the data table to be synchronized in the change log.
Optionally, the obtaining module is configured to obtain the unique identifier of the pluggable database that needs to be synchronized by using the view that records the information of the pluggable database of the source database, and obtain the unique identifier of the data table that needs to be synchronized by using the view that records the information of the data table of the pluggable database of the source database.
Optionally, the capture module is configured to capture, by recording a view of an analysis result for the obtained change log, a data change operation in the change log regarding the data table that needs to be synchronized, using the unique identifier of the pluggable database that needs to be synchronized and the unique identifier of the data table that needs to be synchronized of the pluggable database that needs to be synchronized.
Optionally, the metadata acquired by the acquisition module includes column information of the data table to be synchronized; the generating module is used for analyzing the captured data change operation by utilizing the column information of the data table needing to be synchronized, and generating change data of the data table needing to be synchronized.
Optionally, the obtaining module is further configured to obtain the column information of the data table that needs to be synchronized by using the view of the column information of the data table of the pluggable database that records the source database.
According to a third embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to the first embodiment of the present disclosure.
According to a fourth embodiment of the present disclosure, there is provided an electronic apparatus including: a memory having a computer program stored thereon; a processor for executing the computer program in the memory to carry out the steps of the method according to the first embodiment of the disclosure.
By adopting the above technical solution, since the obtained change log can be analyzed by using the obtained metadata to capture the data change operation of the source database in the change log, and then the captured data change operation is analyzed by using the obtained metadata to generate the change data of the source database, the data synchronization method according to the embodiment of the present disclosure does not require all PDBs to be in an open state, but only requires the PDBs that need to be synchronized to be in an open state to synchronize the change data of the source database to the target database, and the PDBs may have the same table name under the same user. In addition, according to the data synchronization method disclosed by the embodiment of the disclosure, the dictionary file is not required to be manually written and imported, so that the maintenance cost of the database is greatly reduced, and the synchronization efficiency and the real-time performance of the database are improved.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
fig. 1 shows a flow chart of a data synchronization method according to an embodiment of the present disclosure.
Fig. 2 shows a schematic block diagram of a data synchronization apparatus according to an embodiment of the present disclosure.
FIG. 3 is a block diagram illustrating an electronic device in accordance with an example embodiment.
FIG. 4 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
Fig. 1 illustrates a flowchart of a data synchronization method according to an embodiment of the present disclosure, and as shown in fig. 1, the method may include the following steps S11 to S14.
In step S11, metadata and a change log of the source database are acquired.
In the present disclosure, the source database may be a database having one or more PDB databases, for example, an Oracle 12C multi-tenant database.
In step S12, the obtained change log is analyzed using the obtained metadata to capture data change operations in the change log with respect to the source database.
In step S13, the captured data change operation is analyzed using the acquired metadata, and change data of the source database is generated.
In step S14, the generated change data is synchronized to the target database.
By adopting the above technical solution, since the obtained change log can be analyzed by using the obtained metadata to capture the data change operation of the source database in the change log, and then the captured data change operation is analyzed by using the obtained metadata to generate the change data of the source database, the data synchronization method according to the embodiment of the present disclosure does not require all PDBs to be in an open state, but only requires the PDBs that need to be synchronized to be in an open state to synchronize the change data of the source database to the target database, and the PDBs may have the same table name under the same user. In addition, according to the data synchronization method disclosed by the embodiment of the disclosure, the dictionary file is not required to be manually written and imported, so that the maintenance cost of the database is greatly reduced, and the synchronization efficiency and the real-time performance of the database are improved.
In a possible implementation, the obtaining metadata and change log of the source database in step S11 may include the following steps S11a and S11b, and steps S11a and S11b may be executed simultaneously or sequentially, for example, S11a may be executed first and then S11b is executed or S11b may be executed first and then S11a is executed.
In step S11a, the metadata of the source database is obtained according to the pluggable database that needs to be synchronized and the configuration of the data table that needs to be synchronized of the pluggable database that needs to be synchronized.
In practical application scenarios, database maintenance personnel will typically configure when to synchronize the changed data of which data tables under which PDBs of the source database to the target database. Therefore, when acquiring metadata of the source database, the acquisition of metadata of the source database is also performed based on such a configuration. For example, assume that a maintenance person of the database makes such a configuration: during a first time period, data table 1 and data table 2 under PDB1 and data table 3 and data table 5 under PDB2 of the source database are synchronized to the target database. Based on such a configuration, when the first time period starts, the data synchronization method according to the embodiment of the present disclosure starts to acquire metadata about PDB1 that needs to be synchronized and data table 1 and data table 2 that need to be synchronized below it, and PDB2 that needs to be synchronized and data table 3 and data table 5 that need to be synchronized below it.
In step S11b, a change log of the source database is acquired.
Through the technical scheme, the metadata of the whole source database does not need to be acquired, but only the metadata of the PDB needing to be synchronized and the metadata of the data table needing to be synchronized below the PDB need to be acquired, so that the amount of the metadata needing to be acquired can be greatly reduced, the synchronization efficiency and the real-time performance are improved, and the power consumption is reduced.
In one possible embodiment, the obtained metadata includes a unique identification of the PDB of the source database that needs to be synchronized and a unique identification of the data table of the PDB that needs to be synchronized. Then, the analyzing the obtained change log with the obtained metadata to capture the data change operation of the source database in the change log in step S12 may include the following steps: analyzing the obtained change log by using the unique identification of the PDB needing to be synchronized and the unique identification of the data table needing to be synchronized of the PDB needing to be synchronized to capture the data change operation of the data table needing to be synchronized in the change log. In particular, the data change operation of the data table which needs to be synchronized in the change log can be captured by recording the view of the analysis result of the obtained change log and utilizing the unique identification of the pluggable database which needs to be synchronized and the unique identification of the data table which needs to be synchronized of the pluggable database which needs to be synchronized.
That is, when data change operations are acquired, in a view recording analysis results for the acquired change log, by using a unique identifier of a PDB that needs to be synchronized and a unique identifier of a data table that needs to be synchronized under the PDB that needs to be synchronized, which data change operations are for the data table that needs to be synchronized under the PDB that needs to be synchronized are identified in the analysis results for the change log, and then the identified data change operations are captured. In this way, only data change operation of the data table which needs to be synchronized currently can be captured, and therefore synchronization efficiency and real-time performance are greatly improved.
Taking an Oracle 12C multi-tenant database as an example, the obtained change log may be analyzed by using a logmnr tool provided by Oracle, and then in a view recording an analysis result for the obtained change log, that is, in a V $ logmnr _ contents view, using a unique identifier CON _ UID of a PDB to be synchronized and a unique identifier OBJECT _ ID of a data table to be synchronized under the PDB to be synchronized, which data change operations are for the data table to be synchronized under the PDB to be synchronized, and then capturing the identified data change operations.
In addition, in one possible implementation, the unique identification of the PDB that needs to be synchronized can be obtained by using a view that records information of PDBs of the source database. Taking the Oracle 12C multi-tenant database as an example, the dba _ PDBs view can be utilized to obtain the unique identity CON _ UID of the PDB that needs to be synchronized. Likewise, the view of the information of the data table of the PDB recording the source database can be utilized to obtain the unique identification of the data table that needs to be synchronized. Still taking the example of an Oracle 12C multi-tenant database, the dba _ OBJECTs view can be utilized to obtain the unique identification OBJECT _ ID of the data table that needs to be synchronized.
In one possible embodiment, the obtained metadata may include column information of the data table that needs to be synchronized. The column information may include the data type, data length, data precision, and the like of the data in each column. Specifically, the view of the column information of the data table of the PDB recording the source database may be utilized to obtain the column information of the data table that needs to be synchronized. Taking an Oracle 12C multi-tenant database as an example, the user _ tab _ columns view can be utilized to obtain the column information of the data table that needs to be synchronized. Moreover, in the case that the obtained metadata includes the unique identification of the PDB of the source database that needs to be synchronized, the unique identification of the data table of the PDB that needs to be synchronized, and the column information of the data table that needs to be synchronized, which data tables need to be synchronized can be uniquely determined.
Alternatively, in the case that the obtained metadata includes column information of a data table that needs to be synchronized, the parsing the captured data change operation by using the obtained metadata to generate change data of the source database in step S13 may include: and analyzing the captured data change operation by using the column information of the data table needing to be synchronized, and generating change data of the data table needing to be synchronized. Thus, instead of using the ONLINE dictionary sys, dbms, logmnr, DICT FROM ONLINE CATALOG, as in the prior art, the byte code of the captured data change operation can be parsed by the column information of the data table to be synchronized to generate change data about the data table to be synchronized.
The following describes an operation flow of the data synchronization method according to an embodiment of the present disclosure by taking an Oracle 12C multi-tenant database as an example. Among these, what needs to be synchronized is the incremental data of data table 1 under user 1, e.g., multi-tenant PDB 1.
First, the unique identifier CON _ UID of the PDB1 to be synchronized is obtained from the dba _ PDBs view of the information of the PDB of the record source database, the unique identifier OBJECT _ ID of the data table 1 under user 1 of the PDB1 to be synchronized is obtained from the dba _ OBJECTs of the information of the data table of the PDB of the record source database, and all the column information of the data table 1 to be synchronized, including the data type, data length, data accuracy, etc. of the data in each column, is obtained from the view user _ tab _ columns of the column information of the data table of the PDB of the record source database. In addition, a change log of the source database is also obtained.
The obtained change log is then analyzed in real time using the logmnr tool provided by Oracle, after which the data change operation of the data table 1 to be synchronized is captured from the view V $ logmnr _ contents recording the analysis result for the obtained change log, using the unique identification CON _ UID of the PDB1 to be synchronized and the unique identification OBJECT _ ID of the data table 1 to be synchronized under the PDB1 to be synchronized.
Then, the bytecode of the captured data change operation is parsed using the column information of the data table 1 included in the acquired metadata to generate change data about the data table 1 that needs to be synchronized.
The generated change data is then synchronized to the target database.
Therefore, the synchronization of the source database to the target database is efficiently completed in real time, all PDBs are not required to be in an open state, only the PDBs 1 which need to be synchronized are required to be in an open state, and the PDBs can have the same table name under the same user. In addition, the dictionary file does not need to be manually written and imported, and the maintenance cost of the database is greatly reduced.
Fig. 2 shows a schematic block diagram of a data synchronization apparatus according to an embodiment of the present disclosure, and as shown in fig. 2, the apparatus may include:
an obtaining module 21, configured to obtain metadata and a change log of a source database;
a capture module 22, configured to analyze the obtained change log by using the obtained metadata to capture a data change operation in the change log regarding the source database;
the generating module 23 is configured to analyze the captured data change operation by using the obtained metadata, and generate change data of the source database;
a synchronization module 24 for synchronizing the generated change data to the target database.
Optionally, the obtaining module is configured to obtain a change log of the source database, and obtain the metadata of the source database according to the configuration of the pluggable database that needs to be synchronized and the data table that needs to be synchronized of the pluggable database that needs to be synchronized.
Optionally, the metadata acquired by the acquisition module includes a unique identifier of a pluggable database of the source database that needs to be synchronized and a unique identifier of a data table of the pluggable database that needs to be synchronized; the capture module is used for analyzing the obtained change log by using the unique identifier of the pluggable database to be synchronized and the unique identifier of the data table to be synchronized of the pluggable database to be synchronized, so as to capture the data change operation of the data table to be synchronized in the change log.
Optionally, the obtaining module is configured to obtain the unique identifier of the pluggable database that needs to be synchronized by using the view that records the information of the pluggable database of the source database, and obtain the unique identifier of the data table that needs to be synchronized by using the view that records the information of the data table of the pluggable database of the source database.
Optionally, the capture module is configured to capture, by recording a view of an analysis result for the obtained change log, a data change operation in the change log regarding the data table that needs to be synchronized, using the unique identifier of the pluggable database that needs to be synchronized and the unique identifier of the data table that needs to be synchronized of the pluggable database that needs to be synchronized.
Optionally, the metadata acquired by the acquisition module includes column information of the data table to be synchronized; the generating module is used for analyzing the captured data change operation by utilizing the column information of the data table needing to be synchronized, and generating change data of the data table needing to be synchronized.
Optionally, the obtaining module is further configured to obtain the column information of the data table that needs to be synchronized by using the view of the column information of the data table of the pluggable database that records the source database.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 3 is a block diagram illustrating an electronic device 700 according to an example embodiment. As shown in fig. 3, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.
The processor 701 is configured to control the overall operation of the electronic device 700, so as to complete all or part of the steps in the data synchronization method. The memory 702 is used to store various types of data to support operation at the electronic device 700, such as instructions for any application or method operating on the electronic device 700 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and the like. The Memory 702 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 703 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 702 or transmitted through the communication component 705. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding Communication component 705 may include: Wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic Device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the data synchronization method described above.
In another exemplary embodiment, there is also provided a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the data synchronization method described above. For example, the computer readable storage medium may be the memory 702 described above including program instructions that are executable by the processor 701 of the electronic device 700 to perform the data synchronization method described above.
Fig. 4 is a block diagram illustrating an electronic device 1900 according to an example embodiment. For example, the electronic device 1900 may be provided as a server. Referring to fig. 4, an electronic device 1900 includes a processor 1922, which may be one or more in number, and a memory 1932 for storing computer programs executable by the processor 1922. The computer program stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processor 1922 may be configured to execute the computer program to perform the data synchronization method described above.
Additionally, electronic device 1900 may also include a power component 1926 and a communication component 1950, the power component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to enable communication, e.g., wired or wireless communication, of the electronic device 1900. In addition, the electronic device 1900 may also include input/output (I/O) interfaces 1958. The electronic device 1900 may operate based on an operating system, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, etc., stored in memory 1932.
In another exemplary embodiment, there is also provided a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the data synchronization method described above. For example, the computer readable storage medium may be the memory 1932 described above that includes program instructions that are executable by the processor 1922 of the electronic device 1900 to perform the data synchronization method described above.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, various possible combinations will not be separately described in this disclosure.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.
Claims (7)
1. A method for synchronizing data, the method comprising:
acquiring metadata and a change log of a source database, wherein the source database is a database with one or more pluggable databases;
analyzing the obtained change log by using the obtained metadata to capture data change operations about the source database in the change log;
analyzing the captured data change operation by using the acquired metadata to generate change data of the source database;
synchronizing the generated change data to a target database;
wherein, the obtaining metadata of the source database includes: acquiring metadata of the source database according to the pluggable database which needs to be synchronized of the source database and the configuration of the data tables which need to be synchronized of the pluggable database which needs to be synchronized, wherein the configuration comprises when to synchronize the changed data of which data tables under which pluggable databases of the source database into the target database;
the acquired metadata comprises the unique identification of the pluggable database which needs to be synchronized of the source database and the unique identification of the data table which needs to be synchronized of the pluggable database which needs to be synchronized; then, the analyzing the obtained change log by using the obtained metadata to capture the data change operation of the source database in the change log includes: analyzing the obtained change log by using the unique identification of the pluggable database to be synchronized and the unique identification of the data table to be synchronized of the pluggable database to be synchronized to capture the data change operation of the data table to be synchronized in the change log;
wherein the obtained metadata comprises column information of the data table to be synchronized; then, the analyzing the captured data change operation by using the obtained metadata to generate change data of the source database includes: analyzing the byte code of the captured data change operation by using the column information of the data table needing to be synchronized, and generating change data of the data table needing to be synchronized.
2. The method of claim 1, further comprising:
acquiring the unique identifier of the pluggable database to be synchronized by using the view for recording the information of the pluggable database of the source database;
and acquiring the unique identifier of the data table to be synchronized by using the view of the information of the data table of the pluggable database recording the source database.
3. The method of claim 1, wherein said analyzing the obtained change log with the unique identification of the pluggable database to be synchronized and the unique identification of the data table to be synchronized of the pluggable database to be synchronized to capture the data change operation in the change log with respect to the data table to be synchronized comprises:
capturing data change operations in the change log about the data table needing to be synchronized by recording a view of an analysis result for the obtained change log and utilizing the unique identification of the pluggable database needing to be synchronized and the unique identification of the data table needing to be synchronized of the pluggable database needing to be synchronized.
4. The method of claim 1, further comprising:
and acquiring the column information of the data table to be synchronized by using the view of the column information of the data table of the pluggable database recording the source database.
5. A data synchronization apparatus, comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring metadata and a change log of a source database, and the source database is a database with one or more pluggable databases;
the acquisition module is used for analyzing the acquired change log by using the acquired metadata to acquire data change operation related to the source database in the change log;
the generating module is used for analyzing the captured data change operation by using the acquired metadata to generate change data of the source database;
a synchronization module for synchronizing the generated change data to a target database;
wherein, the obtaining module obtains the metadata of the source database, and includes: acquiring metadata of the source database according to the pluggable database which needs to be synchronized of the source database and the configuration of the data tables which need to be synchronized of the pluggable database which needs to be synchronized, wherein the configuration comprises when to synchronize the changed data of which data tables under which pluggable databases of the source database into the target database;
the acquired metadata comprises the unique identification of the pluggable database which needs to be synchronized of the source database and the unique identification of the data table which needs to be synchronized of the pluggable database which needs to be synchronized; then, the capturing module analyzes the obtained change log by using the obtained metadata to capture the data change operation of the source database in the change log, including: analyzing the obtained change log by using the unique identification of the pluggable database to be synchronized and the unique identification of the data table to be synchronized of the pluggable database to be synchronized to capture the data change operation of the data table to be synchronized in the change log;
wherein the obtained metadata comprises column information of the data table to be synchronized; then, the generating module analyzes the captured data change operation by using the obtained metadata, and generates change data of the source database, including: analyzing the byte code of the captured data change operation by using the column information of the data table needing to be synchronized, and generating change data of the data table needing to be synchronized.
6. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.
7. An electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810671890.7A CN109241174B (en) | 2018-06-26 | 2018-06-26 | Data synchronization method and device, readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810671890.7A CN109241174B (en) | 2018-06-26 | 2018-06-26 | Data synchronization method and device, readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109241174A CN109241174A (en) | 2019-01-18 |
| CN109241174B true CN109241174B (en) | 2021-04-16 |
Family
ID=65071984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810671890.7A Active CN109241174B (en) | 2018-06-26 | 2018-06-26 | Data synchronization method and device, readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109241174B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109933594A (en) * | 2019-02-15 | 2019-06-25 | 北京大米科技有限公司 | Obtain method, apparatus, electronic equipment and the medium of data |
| CN110196793B (en) * | 2019-04-30 | 2023-05-12 | 武汉达梦数据库股份有限公司 | Log analysis method and device for plug-in database |
| CN110543413A (en) * | 2019-07-22 | 2019-12-06 | 深圳壹账通智能科技有限公司 | Business system testing method, device, equipment and storage medium |
| CN110457181B (en) * | 2019-08-02 | 2023-05-16 | 武汉达梦数据库股份有限公司 | Log optimization analysis method and device for database |
| CN112527807A (en) * | 2019-09-18 | 2021-03-19 | 银联数据服务有限公司 | Data structure operation method and device |
| CN113515569B (en) * | 2020-04-09 | 2023-12-26 | 阿里巴巴集团控股有限公司 | Data synchronization method, device and system |
| CN113449043A (en) * | 2021-07-21 | 2021-09-28 | 中国人民解放军61932部队 | Data synchronization method and device, computer equipment and storage medium |
| CN113742420B (en) * | 2021-08-09 | 2024-02-02 | 广州市易工品科技有限公司 | Data synchronization method and device |
| CN117171262A (en) * | 2023-08-09 | 2023-12-05 | 飞算数智科技(深圳)有限公司 | Data synchronization method and device, storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| CN107169108A (en) * | 2017-05-19 | 2017-09-15 | 郑州云海信息技术有限公司 | A kind of method and apparatus that the fast quick-recovery of MySQL data is realized based on Oracle |
| CN107491558A (en) * | 2017-09-08 | 2017-12-19 | 北京奇艺世纪科技有限公司 | Metadata updates method and device |
| CN107590277A (en) * | 2017-09-28 | 2018-01-16 | 泰康保险集团股份有限公司 | Method of data synchronization, device, electronic equipment and storage medium |
| CN107783975A (en) * | 2016-08-24 | 2018-03-09 | 北京京东尚科信息技术有限公司 | The method and apparatus of distributed data base synchronization process |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9767171B2 (en) * | 2015-04-03 | 2017-09-19 | Oracle International Corporation | Method and system for implementing an operating system hook in a log analytics system |
-
2018
- 2018-06-26 CN CN201810671890.7A patent/CN109241174B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| CN107783975A (en) * | 2016-08-24 | 2018-03-09 | 北京京东尚科信息技术有限公司 | The method and apparatus of distributed data base synchronization process |
| CN107169108A (en) * | 2017-05-19 | 2017-09-15 | 郑州云海信息技术有限公司 | A kind of method and apparatus that the fast quick-recovery of MySQL data is realized based on Oracle |
| CN107491558A (en) * | 2017-09-08 | 2017-12-19 | 北京奇艺世纪科技有限公司 | Metadata updates method and device |
| CN107590277A (en) * | 2017-09-28 | 2018-01-16 | 泰康保险集团股份有限公司 | Method of data synchronization, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109241174A (en) | 2019-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241174B (en) | Data synchronization method and device, readable storage medium and electronic equipment | |
| CN106874389B (en) | Data migration method and device | |
| CN110737594B (en) | Database standard conformance testing method and device for automatically generating test cases | |
| US20160171505A1 (en) | Extract, transform, and load (etl) processing | |
| US20160283357A1 (en) | Call stack relationship acquiring method and apparatus | |
| CN105205053A (en) | Method and system for analyzing database incremental logs | |
| US20190325021A1 (en) | Method and Device for Creating Hyperlink | |
| US11164567B2 (en) | Systems and methods for extraction of user intent from speech or text | |
| WO2019056494A1 (en) | Chart generation method, device, computer apparatus, and storage medium | |
| US9953640B2 (en) | Systems and methods of interpreting speech data | |
| CN110688544A (en) | Method, device and storage medium for querying database | |
| CN111046036A (en) | Data synchronization method, device, system and storage medium | |
| CN111400170B (en) | Data authority testing method and device | |
| US20160019266A1 (en) | Query generating method and query generating device | |
| CN110888756A (en) | Diagnostic log generation method and device | |
| CN111324510B (en) | Log processing method and device and electronic equipment | |
| CN107357619B (en) | Method, device, equipment and storage medium for generating card configuration file | |
| CN106682210B (en) | Log file query method and device | |
| CN110825802A (en) | Multi-type database data backup method, device, equipment and storage medium | |
| US10970337B1 (en) | Method and device for outputting result of operations using data sources of different types | |
| WO2016095716A1 (en) | Fault information processing method and related device | |
| CN111209195B (en) | Method and device for generating test case | |
| CN117033309A (en) | Data conversion method and device, electronic equipment and readable storage medium | |
| US9588998B2 (en) | Protecting storage data during system migration | |
| CN115048359A (en) | Universal data processing architecture design method for multi-type access data sources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |