CN116955369A - Database-based data fusion method and device, electronic equipment and storage medium - Google Patents
Database-based data fusion method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116955369A CN116955369A CN202210397603.4A CN202210397603A CN116955369A CN 116955369 A CN116955369 A CN 116955369A CN 202210397603 A CN202210397603 A CN 202210397603A CN 116955369 A CN116955369 A CN 116955369A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- fused
- fusion
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 55
- 230000004927 fusion Effects 0.000 claims abstract description 188
- 238000000034 method Methods 0.000 claims abstract description 45
- 230000002159 abnormal effect Effects 0.000 claims description 36
- 238000006243 chemical reaction Methods 0.000 claims description 30
- 230000001360 synchronised effect Effects 0.000 claims description 21
- 230000008859 change Effects 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 15
- 230000005856 abnormality Effects 0.000 claims description 6
- 238000004519 manufacturing process Methods 0.000 abstract description 15
- 238000012545 processing Methods 0.000 abstract description 15
- 238000012544 monitoring process Methods 0.000 description 11
- 230000007246 mechanism Effects 0.000 description 10
- 230000010076 replication Effects 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 238000012546 transfer Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000011084 recovery Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000012550 audit Methods 0.000 description 3
- 238000007499 fusion processing Methods 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
- G06F16/2386—Bulk updating operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/278—Data partitioning, e.g. horizontal or vertical partitioning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of data processing, and provides a data fusion method, a device, electronic equipment and a storage medium based on a database, wherein the method comprises the following steps: based on the data fusion request, synchronizing the data to be fused in the source database to the intermediate database; performing fusion operation for fusing the data to be fused in the intermediate database to the target database; the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table of the data to be fused in the target database. According to the application, the related processing operation for the data to be fused is realized by using the intermediate database as the transit buffer zone, so that the actual production of the source database and the target database is not influenced, the data fusion of a plurality of non-homologous databases is supported, and the simultaneous accessibility of the source database and the target database is maintained, thereby realizing the purpose of large-scale complex data fusion on the premise of not influencing the actual production.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a database-based data fusion method, apparatus, electronic device, and storage medium.
Background
At present, most databases have corresponding data backup and replication mechanisms, but data backup and replication in the form of export import files are usually supported, so as to solve the problems of data backup, disaster recovery, recovery and the like, and therefore, data transfer between databases is more and more common.
In the related art, the GoldenGate software is based on the capability of providing data disaster recovery and data replication, supporting the upgrading and migration of the database with no downtime, and realizing the real-time synchronization of sub-second data, the GoldenGate software is generally used for carrying out data replication on a homologous database or databases in different regions of the same service, so that partial database tables with similar names and contents in a source end and a target end are fused.
However, when the names of the library tables in the source end and the destination end are different and the structures are different, the library tables need to be processed at the source end, so that actual production is affected, and large-scale complex data fusion is not suitable.
Disclosure of Invention
The application provides a data fusion method, a device, electronic equipment and a storage medium based on a database, which are used for solving the defects that the actual production is influenced and large-scale complex data fusion is not suitable for being carried out due to the fact that a database table needs to be processed at a source end in the prior art, and achieving the purpose that the method is suitable for large-scale complex data fusion on the premise that the actual production is not influenced.
The application provides a data fusion method based on a database, which comprises the following steps:
based on the data fusion request, synchronizing the data to be fused in the source database to the intermediate database;
performing fusion operation for fusing the data to be fused in the intermediate database to the target database;
the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
According to the database-based data fusion method provided by the application, the fusion operation of the data to be fused in the intermediate database to the target database is executed, and the method comprises the following steps:
and synchronizing the data to be fused in the intermediate database to the target database when the target database is not provided with the data to be fused.
According to the database-based data fusion method provided by the application, when the number of the target databases is a plurality of, the fusion operation of the data to be fused in the intermediate database to the target databases is executed, and the method comprises the following steps:
when the target database is determined to have no data to be fused, synchronizing the data to be fused into a temporary source database of the intermediate database for splitting operation to obtain a plurality of data to be fused;
Transferring the data to be fused to a temporary target library of the intermediate database;
and executing fusion operation of fusing each data to be fused in the temporary target database into the corresponding target database.
According to the database-based data fusion method provided by the application, the fusion operation of the data to be fused in the intermediate database to the target database is executed, and the method comprises the following steps:
when a first library table corresponding to the data to be fused exists in a target database, executing data conversion operation for the data to be fused in the intermediate database based on the difference between the data to be fused and the first library table;
and synchronizing the target data to be fused obtained by the data conversion operation to the first library table of the target database.
According to the database-based data fusion method provided by the application, when the number of the data to be fused is a plurality of data to be fused and a preset relationship exists among the plurality of data to be fused, the fusion operation for fusing the data to be fused in the intermediate database to the target database is executed, and the method comprises the following steps:
When determining that a second library table corresponding to a plurality of data to be fused exists in a target database, executing merging operation aiming at the plurality of data to be fused in the intermediate database to obtain an intermediate library table;
performing a data conversion operation for the intermediate library table in the intermediate database based on the difference between the intermediate library table and the second library table;
and synchronizing the target data to be fused obtained by the data conversion operation to the second library table of the target database.
According to the database-based data fusion method provided by the application, the fusion operation of the data to be fused in the intermediate database to the target database is executed, and the method comprises the following steps:
triggering to generate an incremental data fusion request when determining that the trigger collects change data representing changes of database table data in a source database;
and acquiring incremental data to be fused corresponding to the change data, and executing incremental fusion operation for fusing the incremental data to be fused to the target database based on the incremental data fusion request.
According to the database-based data fusion method provided by the application, the executing the incremental fusion operation for fusing the incremental data to be fused to the target database comprises the following steps:
Acquiring the real-time requirement of the incremental data to be fused, and selecting a target incremental synchronous interface matched with the real-time requirement from preset incremental synchronous interfaces;
and executing the delta fusion operation for fusing the delta data to be fused to the target database based on the target delta synchronous interface.
According to the database-based data fusion method provided by the application, after the fusion operation of the data to be fused in the intermediate database to the target database is performed, the method further comprises:
acquiring task detail data of a current fusion task recorded when the fusion operation is finished;
and updating a preset fusion task history table and a task data detail history table based on the current fusion task and the task detail data.
According to the database-based data fusion method provided by the application, after the fusion operation of the data to be fused in the intermediate database to the target database is performed, the method further comprises:
when first abnormal information representing the abnormality of the trigger is determined to be monitored, the first abnormal information is recorded and alarming prompt is carried out.
According to the database-based data fusion method provided by the application, after the fusion operation of the data to be fused in the intermediate database to the target database is performed, the method further comprises:
and when the second abnormal information representing the abnormal fusion operation is determined to be monitored, outputting alarm indication information, wherein the alarm indication information is used for indicating manual intervention to process the abnormal information and supporting the re-fusion operation aiming at the data to be fused corresponding to the second abnormal information.
According to the data fusion method based on the database, the method further comprises the following steps:
when the source database and the target database are determined to be in the same database table and the same primary key, executing an adding operation for adding a new preset field to the same database table;
and determining a joint primary key based on the same primary key and the preset field obtained by the adding operation.
The application also provides a data fusion device based on the database, which comprises:
the synchronization module is used for synchronizing the data to be fused in the source database to the intermediate database based on the data fusion request;
the fusion module is used for executing fusion operation for fusing the data to be fused in the intermediate database to the target database;
The intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
The application also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the database-based data fusion method as described in any one of the above when executing the program.
The application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a database-based data fusion method as described in any of the above.
The application also provides a computer program product comprising a computer program which, when executed by a processor, implements a database-based data fusion method as described in any of the above.
According to the data fusion method, the device, the electronic equipment and the storage medium based on the database, the data to be fused in the source database is synchronized to the intermediate database based on the data fusion request, and then fusion operation for fusing the data to be fused in the intermediate database to the target database is executed, so that relevant processing operation for the data to be fused is realized by using the intermediate database as a transit buffer zone, and actual production of the source database and the target database is not influenced; furthermore, the intermediate database is used for representing attribute information of data to be fused and attribute information of a database table of the data to be fused in the target database, so that the intermediate database can be used for processing data fusion operation under various conditions of different names, different structures, different fields, different database tables and the like of the database table in the source database and the target database, supporting data fusion of a plurality of non-homologous databases, also maintaining simultaneous accessibility of the source database and the target database, realizing data online fusion, and realizing the purpose of large-scale complex data fusion on the premise of not influencing actual production.
Drawings
In order to more clearly illustrate the application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a database-based data fusion method according to the present application;
FIG. 2 is a logic diagram of a data flow of the database-based data fusion method provided by the application;
FIG. 3 is a schematic diagram of a database-based data fusion device according to the present application;
fig. 4 is a schematic structural diagram of an electronic device provided by the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Most database products at present have corresponding backup and replication mechanisms, but standardized products are provided, and a generalized solution is provided for scenes such as data replication, data backup and the like, but in the actual production process, the situation of gradual fusion of a plurality of databases is required to be processed besides the scenes of data replication and data backup.
Because the most typical and mature product in the market is GoldenGate software of Oracle (Oracle) company, the GoldenGate software is mainly used for data replication operation between homologous databases or regional databases with different services, although the GoldenGate software can provide the functions of data disaster tolerance and data replication, support the upgrade and migration of databases with zero downtime and realize the real-time synchronization of data with sub-second level. For a database of two different services originally, when the names and contents of most of the database tables are similar, the following problems cannot be handled: (1) When completely different library tables exist at the source end and the target end, certain library tables need to be deleted or added, and automatic processing cannot be performed; (2) When the source end library table and the target end library table have different fields, certain fields need to be added or deleted, and automatic processing cannot be performed; (3) When the primary key conflict exists in the source end library table and the target end library table and the data are different, only the data can be selected to be covered or discarded, the corresponding two data cannot be completely stored, and the data source cannot be recorded; (4) no backtracking of history logs of data synchronization is supported; (5) Support for complex scenarios where multiple non-homologous databases are fused is lacking.
Based on the above problems, the present application provides a data fusion method, device, electronic device and storage medium based on a database, wherein an execution subject of the data fusion method based on the database may be a terminal device containing an active database, the terminal device may be a personal computer (Personal Computer, PC), a portable device, a notebook computer, a smart phone, a tablet computer, a portable wearable device and other electronic devices, and the client has at least a data packet receiving and playing function, a data packet capturing function, a storage function, an analysis function and a statistical analysis function. The application is not limited to the specific form of the client.
It should be noted that, the following method embodiments are described by taking an execution subject as a terminal device, and the following method embodiments describe the fusion method in detail by taking a target database, which is a source database of a group customer service integrated operation platform (Enterprise Service Operation Platform, ESOP) library and a target database of a customer relationship management (Customer Relationship Management, CRM) library, as examples. The database-based data fusion method, apparatus, electronic device and storage medium of the present application are described below with reference to fig. 1 to 4.
Referring to fig. 1, a flow chart of a database-based data fusion method provided by the application, as shown in fig. 1, includes the following steps:
and 110, synchronizing the data to be fused in the source database to the intermediate database based on the data fusion request.
The intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
Specifically, when the data to be fused is the data of at least one library table in the source database, the attribute information of the data to be fused may include the library table structure and the data in the at least one library table, and the data fusion request is a full-scale data fusion request for the at least one library table; and the attribute information of the database table in the target database for fusing the data to be fused can comprise the database table structure of the database table, namely, when the intermediate database comprises a temporary source database and a temporary target database, the temporary source database keeps consistent with the structure of the database table in which the data to be fused is located and the data in the table, and the temporary target database keeps consistent with the database table structure of the corresponding database table in the target database. Based on the data fusion request, the data to be fused in the source database is synchronized into the intermediate database, and the data to be fused can be synchronously transferred to the temporary source database of the intermediate database by using a reliable and efficient GoldenGate technology.
Step 120, performing a fusion operation for fusing the data to be fused in the intermediate database to the target database.
Specifically, when the data to be fused in the source database is synchronously transferred to the temporary source database of the intermediate database, the fusion operation of the data to be fused to the target database can be performed based on a reliable and efficient GoldenGate technology. As shown in fig. 2, the TABLE1, TABLE2, TABLE3 and TABLE4 in the source database may be synchronously transferred to the temporary source database, and the TABLE1, TABLE2, TABLE3 and TABLE4 in the temporary source database are synchronously transferred to the temporary target database, and both synchronous transfers use GoldenGate technology, if the operations of data splitting, data conversion and the like need to be performed on the data to be fused, they may also be completed in the intermediate database, so as to ensure that the influence on the source database and the target database is minimized, and thus the smooth butt joint synchronization of the data can also be ensured.
According to the data fusion method based on the database, based on the data fusion request, the data to be fused in the source database is synchronized into the intermediate database, and then fusion operation for fusing the data to be fused in the intermediate database to the target database is executed, so that relevant processing operation for the data to be fused is realized by using the intermediate database as a transfer buffer zone, and the actual production of the source database and the target database is ensured not to be influenced; furthermore, the intermediate database is used for representing attribute information of data to be fused and attribute information of a database table of the data to be fused in the target database, so that the intermediate database can be used for processing data fusion operation under various conditions of different names, different structures, different fields, different database tables and the like of the database table in the source database and the target database, supporting data fusion of a plurality of non-homologous databases, also maintaining simultaneous accessibility of the source database and the target database, realizing data online fusion, and realizing the purpose of large-scale complex data fusion on the premise of not influencing actual production.
Optionally, the implementation procedure of step 120 may include:
and synchronizing the data to be fused in the intermediate database to the target database when the target database is not provided with the data to be fused.
Specifically, when it is determined that the target database does not have a database table in which the data to be fused is located and the target database does not need to split the database table according to a preset identifier, the data to be fused is not required to be converted and migrated at this time, instead, the database table is newly built in the target database to import the data to be fused, and the GoldenGate technology can also be adopted to directly transfer the data to be fused in the temporary source database of the intermediate database to the target database synchronously, so that names, structures and field meanings of the database tables in which the data to be fused stored in the source database, the intermediate database and the target database are located are consistent. The preset identifier includes, but is not limited to, an area identifier.
According to the database-based data fusion method provided by the application, when the target database has no database table in which the data to be fused is located, the operability of data fusion is improved by directly and synchronously transferring the data to be fused in the intermediate database to the target database, so that the actual production of the source database and the target database is not influenced by utilizing the intermediate database.
Optionally, when the number of the target databases is plural, the implementation process of step 120 may include:
firstly, synchronizing the data to be fused into a temporary source library of the intermediate database for splitting operation when the target database is determined to have no data to be fused, so as to obtain a plurality of data to be fused; then, transferring the data to be fused to a temporary target library of the intermediate database; and finally, executing fusion operation of fusing each data to be fused in the temporary target database into the corresponding target database.
Specifically, when it is determined that the target database has no library table in which the data to be fused is located and the target database needs to split the library table according to the preset identifier, splitting and transferring the data to be fused in the temporary source library of the intermediate database is needed at this time, namely, splitting the library table in which the data to be fused is located into a plurality of library tables, further exporting the plurality of library tables split in the temporary source library into the temporary target library by adopting a DUMP (DUMP) and INSERT (INSERT) mode, and finally synchronously transferring the plurality of library tables in the temporary target library into the corresponding target database respectively by using a GoldenGate technology.
According to the data fusion method based on the database, when the target database has no database table of the data to be fused and the number of the database tables is a plurality of, the database tables of the data to be fused are split in the intermediate database and then synchronously transferred to the corresponding target database, so that the flexibility and the reliability of data fusion are improved by presetting various data conversion operations and mapping relations between the intermediate database temporary source database and the temporary target database, the purpose of data online fusion can be realized without modifying the existing business logic, the difficulty that the personalized requirements cannot be met in the traditional data synchronization and fusion scene is overcome, and the fact that the actual production of the source database and the target database is not influenced can be realized by utilizing the intermediate database.
Optionally, the implementation of step 120 may further include
Firstly, when a first library table corresponding to the data to be fused exists in a target database, based on the difference between the data to be fused and the first library table, executing data conversion operation for the data to be fused in the intermediate database; and then, synchronizing the target data to be fused obtained by the data conversion operation into the first library table of the target database.
Specifically, when it is determined that a first library table corresponding to the library table where the data to be fused is located exists in the target database, at least differences such as inconsistent fields and/or inconsistent library table structures exist between the library table where the data to be fused is located and the first library table, at this time, the library table where the data to be fused in the temporary source library of the intermediate database is first transferred to the temporary target library synchronously, and then the temporary target database is instructed to perform data conversion operations such as field modification, field addition and/or library table structure conversion on the library table where the data to be fused which is transferred synchronously are unified, so that the library table structure of the library table where the data to be fused is located, which is the target obtained by the data conversion operation, for example, the field name is modified to be the name. And then, synchronously transferring the target data to be fused obtained by the data conversion operation into a first library table of a target database by adopting a GoldenGate technology.
According to the data fusion operation based on the database, when the first library table corresponding to the library table where the data to be fused is located exists in the target database, the data fusion operation under various situations such as different names, different structures, different fields, different library tables and the like of the source database and the target database is processed through the intermediate database, and the purpose of large-scale complex data fusion can be achieved by synchronously transferring the data to be fused to the temporary target database and then carrying out data conversion operation on the data to be fused in the temporary target database.
Optionally, when the number of the data to be fused is a plurality of data to be fused and a preset relationship is provided between the plurality of data to be fused, the implementation process of step 120 may further include:
firstly, when a second library table corresponding to a plurality of data to be fused exists in a target database, executing merging operation aiming at the plurality of data to be fused in the intermediate database to obtain an intermediate library table; secondly, based on the difference between the intermediate library table and the second library table, performing a data conversion operation for the intermediate library table in the intermediate database; and finally, synchronizing the target data to be fused obtained by the data conversion operation into the second library table of the target database.
The preset relationship may represent that the library tables of the plurality of data to be fused are substantially the same and have slight differences, for example, the library tables of the data to be fused may be a "girl detailed information table" and a "girl detailed information table" when the library tables of the data to be fused are two.
Specifically, when the source database has a table in which the data to be fused representing a plurality of different user data are located, that is, when the number of tables in which the data to be fused are located is a plurality of, if a second table corresponding to the table in which the data to be fused are located is present in the target database, for example, the second table may be a "personnel detail information table", at this time, the tables in which the data to be fused are located may be synchronously transferred to the temporary source database by using the GoldenGate technology, and the temporary source database is instructed to merge the tables in which the data to be fused are located, and when the number of the target database is a plurality of tables, the temporary target database may be instructed to execute the data conversion operation with respect to the intermediate table obtained by merging; when the number of the target databases is multiple, the temporary source database can be instructed to execute splitting operation firstly for the merged intermediate database table, then instruct the temporary target database to execute data conversion operation, and finally synchronously transfer the target fusion data obtained by the data conversion operation to a second database table of the target database. The splitting operation and the data conversion operation may refer to the foregoing method embodiments, and are not described herein again.
According to the database-based data fusion method provided by the application, when the second database table corresponding to the database tables of the data to be fused exists in the target database, the database tables of the data to be fused are combined in the intermediate database, and then the data conversion operation is executed, so that the operations of splitting operation, data conversion, field addition, field modification and the like of the database tables involved in the data fusion process are realized by utilizing the intermediate database, and the problem that complex data fusion is difficult to realize is solved.
In the actual processing process, in addition to the full data fusion, there is incremental data fusion, that is, after the full data fusion is completed, the incremental data to be fused can be obtained based on the changed data in the source database and the incremental data fusion can be performed, that is, the application supports the incremental data fusion in the full data fusion process. Based on this, the implementation procedure of step 120 may include:
firstly, triggering to generate an incremental data fusion request when determining that a trigger collects change data representing changes of database table data in a source database; and then, acquiring incremental data to be fused corresponding to the change data, and executing incremental fusion operation for fusing the incremental data to be fused to the target database based on the incremental data fusion request.
Specifically, since the source database and the target database can operate simultaneously and the device is operated simultaneously, the source database can continuously generate new business data, so that the data in the corresponding database table in the source database is changed, therefore, in order to ensure that the changed data can be accurately and timely fused into the corresponding database table in the target database, a trigger can be respectively arranged for each database table in the source database, each trigger is used for collecting the changed change data of the corresponding database table in the source database, the change data characterizes the corresponding database table data in the source database to be changed due to business processing or to be changed due to manual modification data, and when the trigger is determined to collect the changed change data characterizing the database table data in the source database, the incremental data fusion request can be automatically triggered and generated. And, newly adding a fusion task list, a task data list, a fusion task history list and a task data list history list in the source database in advance, and correspondingly modifying the fusion task list and the task data list based on the change data so as to enable the fusion task list and the task data list to reach the current latest edition.
Further, the incremental data to be fused is read from the current latest version of fusion task table and task data detail table, the incremental data to be fused comprises a task to be fused corresponding to the changed data and task detail data of the task to be fused, and based on the incremental data fusion request, incremental fusion operation for fusing the incremental data to be fused to the target database is executed.
It should be noted that, the triggers set for each library table in the source database may be new/UPDATE/DELETE (DML) triggers, and the DML triggers are used to trigger data fusion operation, and establish an audit comparison mechanism for fusing task tables and task data detail tables, so as to ensure data consistency. If the data is found to be abnormal, backtracking and verification can be carried out by fusing the task history table and the task data detail history table.
According to the data fusion method based on the database, the incremental fusion operation for fusing the incremental data to be fused to the target database is executed aiming at the incremental data fusion request generated after the completion of the full-volume data fusion and the acquired incremental data to be fused, so that the purpose of storing the incremental data fusion task and task detail data thereof in real time aiming at the change data after the completion of the full-volume data fusion is realized, the target database is ensured to fuse the change data in the source database timely and effectively, and a synchronous log recording function and a data synchronous backtracking function are provided.
Optionally, when generating the incremental data fusion request, the implementation process of step 120 may further include:
firstly, acquiring real-time requirements of the incremental data to be fused, and selecting a target incremental synchronous interface matched with the real-time requirements from preset incremental synchronous interfaces; and then, based on the target increment synchronization interface, executing an increment fusion operation for fusing the increment data to be fused in the intermediate database to the target database.
Specifically, aiming at key information which does not allow errors, such as arrearage state, balance, account number and the like, in a source database, the key information has high real-time requirements, so when incremental data to be fused are fused, whether the incremental data to be fused is data with high real-time requirements is judged first, and if the incremental data to be fused has low real-time requirements, a file interface can be used as a target incremental synchronous interface; if the requirement of the incremental data to be fused on the real-time performance is high, a real-time interface can be adopted as a target incremental synchronous interface. And then, based on the target increment synchronous interface, executing increment fusion operation for fusing the increment data to be fused to the target database.
It should be noted that an incremental synchronization interface for incremental fusion operation, such as a file interface and a real-time interface, may be preset in the source database, where the file interface is used for synchronously transferring data with low practicality requirements, and the real-time interface is used for synchronously transferring data with high instantaneity requirements, that is, the original application corresponding to the library table where the incremental data to be fused is located is not offline, the original application is also the azimuth source database, the data fused to the target database is also only a data copy of the original application, the real-time requirements are not high, and at this time, the file interface may be selected for incremental data synchronization; in the gradual online process of the new application, partial data still can be generated in the offline process of the original application corresponding to the library table where the incremental data to be fused are located, in order to ensure the accuracy of the data of the new application in the gradual online process, the incremental data fusion is required, and aiming at the query service with high requirement on partial real-time performance, the real-time interface is required to be used for carrying out the incremental data fusion. At this time, since the database table data in the source database and the target database are inconsistent, after the incremental data to be fused is read from the current latest version of the fusion task table and the task data detail table, the incremental data to be fused can be split and/or transferred to the target database after data conversion processing according to the mapping relationship between the database table in the source database and the database table in the target source database.
According to the data fusion method based on the database, the target increment synchronous interface matched with the real-time requirement of the incremental data to be fused is selected, so that the incremental data to be fused can be accurately and timely synchronously transferred to the target database, and the reliability and the stability of the incremental data fusion are improved.
Optionally, for incremental data fusion, the implementation process of step 120 may include:
firstly, acquiring a current fusion task recorded when the fusion operation is finished and task detail data of the current fusion task; and updating a preset fusion task history table and a task data detail history table based on the current fusion task and the task detail data.
Specifically, whether the current fusion task is aimed at full data fusion or incremental data fusion, the current fusion task and task detail data thereof are recorded in a preset fusion task list and task data detail list, and the recorded current fusion task and task detail data thereof are moved into a corresponding fusion task history list and task data detail history list, so that the fusion task history list and task data detail history list are updated in real time. The specific structures of the fusion task table, the task data detail table, the fusion task history table and the task data detail history table are as follows:
TABLE 1 fusion task sheet
TABLE 2 task data detail Table
Table 3 fusion task history table
TABLE 4 task data detail history table
According to the database-based data fusion method, the recorded current fusion task and task detail data thereof are moved into the corresponding fusion task history table and task data detail history table when each fusion operation is finished, and the fusion task history table and the task data detail history table are updated in real time, so that full data support is provided for follow-up audit and backtracking.
In the actual processing process, when the data in the source database, the intermediate database or the target database changes, database logs are correspondingly generated and stored, based on the fact, before step 110, timing tasks can be set in the source database, the intermediate database or the target database, all the database tables needing full data fusion, namely the full table of the cut-over range, are preset, so that when the quantity in the task detail data table and the task data detail history table is abnormal through statistical comparison, the database logs can be searched for repairing, and when the data to be fused, which is synchronously transferred to the target database, is determined to have a data missing problem, for example, when the structures of the database tables to be fused in the source database and the target database are consistent, the data to be fused in the source database has 100 ten thousand, and is synchronously transferred to the target database by 1 ten thousand, and at the moment, the data missing problem can be considered to exist; for example, for whether field loss or field meaning error occurs after the field of the name in the database table of the data to be fused in the source database is synchronously transferred to the target database, field sampling comparison can be performed for the problem of data loss, the problem of data loss can be caused by trigger abnormality or synchronous transfer abnormality, and manual intervention repair can be performed by searching each database log; in addition, aiming at important data in the production process, such as balance of numbers, customer basic information, user attribute information and other core data which cannot be made mistakes, the core data can be set in a core library as core table data, daily gain data are acquired by adopting newly generated business every day for auditing and checking, and aiming at non-important data in the production process, such as data in a work order and log record intermediate driving process and data on a business flow, the data volume is usually larger, and checking can be omitted; for the trigger correspondingly arranged on each database table in the source database, when a certain trigger fails, the change data can be obtained according to each database log file to carry out manual comparison and manual restoration, each database log file can be set by an administrator to preserve duration and configuration of the database log file, when a certain trigger is abnormal, fusion operation can be triggered manually, and the trigger can be flexibly configured based on the database log files. For example, when the trigger a is abnormal at the time a, the update data of the trigger a after the time a can be obtained by searching each database log file, so that the update data generated in the time period from the failure to the repair of the trigger a can also participate in audit accounting later.
Optionally, after step 120, the method may further include:
when first abnormal information representing the abnormality of the trigger is determined to be monitored, the first abnormal information is recorded and alarming prompt is carried out.
Specifically, the trigger corresponding to each library table in the source database not only has a monitoring mechanism, but also has a recovery mechanism, so in order to ensure that the fusion operation is not affected when the monitoring mechanism of the trigger is abnormal, a daemon process for real-time monitoring of the trigger can be preset, so that the purpose of real-time monitoring of the state of the trigger is realized, the daemon process is an independent program and can be set on the terminal equipment, and can access various databases, such as the source database, the intermediate database and the target database. When the daemon monitors that an abnormality occurs in a certain trigger, the abnormal first abnormal information is recorded for the trigger, and an alarm is timely sent out, meanwhile, the abnormal trigger can start a self recovery mechanism to automatically recover a monitoring mechanism, and the monitoring mechanism can be recovered by manually modifying a program setting mode, and the method is not particularly limited.
According to the data fusion method based on the database, the daemon process is arranged to monitor whether the trigger is abnormal or not, so that the abnormal trigger is discovered timely, the log detail of the fusion task can be saved, backtracking is facilitated, data consistency can be ensured, normal execution of data fusion operation and effective recording of new business data can be ensured when the trigger is abnormal due to failure or other reasons, normal operation of online data fusion is ensured, and data consistency and accuracy of data backtracking are effectively improved.
Optionally, after step 120, the method may further include:
and when the second abnormal information representing the abnormal fusion operation is determined to be monitored, outputting alarm indication information, wherein the alarm indication information is used for indicating manual intervention to process the abnormal information and supporting the re-fusion operation aiming at the data to be fused corresponding to the second abnormal information.
The abnormal fusion operation may be backlog of queued data to be fused caused by failure of synchronous transfer of the data to be fused to the target database, and the backlog of queued data to be fused may be that the number of queued data to be fused exceeds a preset number threshold.
Specifically, for a preset fusion task table or fusion task history table, a monitoring process may be set, where the monitoring process is used to monitor whether the fusion task is backlogged, that is, monitor whether queued data to be fused is backlogged, for example, when there is too much data to be fused in a short time, the fusion operation may fail until backlogged occurs. Based on the above, when the monitoring process monitors the second abnormal information representing the backlog of the queued data to be fused, it can be considered that the fusion operation failure for a certain data to be fused causes the subsequent data to be fused to be queued one by one until backlog or a large number of fusion operations fail, for example, the queued data to be fused exceeds one thousand or ten thousand times to consider the backlog of the queued data to be fused, the fusion operation failure can include abnormal failure and normal failure, the normal failure does not need alarming and manual intervention processing, and the abnormal failure (such as system timeout) times need alarming and manual intervention processing. That is, for the second anomaly information, the alarm indication information may be output while the second anomaly information is recorded, where the alarm indication information is used to indicate that the manual intervention is abnormal and support the re-fusion operation for the data to be fused corresponding to the second anomaly information, so that the fusion operation is re-performed for the queued data to be fused when the manual intervention is abnormal.
According to the database-based data fusion method provided by the application, the data fusion task backlog monitoring mechanism is established by setting the mode that the monitoring process monitors whether the fusion operation backlog, so that each data fusion task can be normally executed without influencing actual production, and the stability and reliability of the data fusion operation are further improved.
Optionally, the method may further include:
firstly, when the source database and the target database are determined to be in the same database table and the same primary key, executing an adding operation for adding a preset field to the same database table; then, based on the same primary key and the preset field obtained by the increment operation, a joint primary key is determined.
Specifically, when different source databases and target databases are fused, if the source databases and the target databases are stored in the same database table and the same primary key, preset fields can be added for the same database table respectively, for example, the newly added preset fields can include, but are not limited to, a data source field, and a combined primary key is reestablished by using the original same primary key and the preset fields, wherein the combined primary key is a primary key formed by using 2 or more fields, and when fields contained in the combined primary key are used as primary keys, the primary key is unique in the source databases and the target databases, and a combined primary key index can be added, so that the purpose of storing the data of the source databases and the target databases is achieved.
The data fusion method based on the database solves the problem of primary key conflict by reconstructing the combined primary key aiming at the same database table in the source database and the target database, can also completely save the data of the source database and the target database, and reduces the influence on the operation of a service system, thereby realizing the purposes of ensuring the data integrity and recording the data source when the primary key conflict exists in the source database and the target database and the data are different.
The database-based data fusion device provided by the application is described below, and the database-based data fusion device described below and the database-based data fusion method described above can be referred to correspondingly.
Fig. 3 is a schematic structural diagram of a database-based data fusion device according to the present application, and as shown in fig. 3, the database-based data fusion device 300 includes:
the synchronization module 310 is configured to synchronize data to be fused in the source database to the intermediate database based on the data fusion request; a fusion module 320, configured to perform a fusion operation for fusing the data to be fused in the intermediate database to the target database;
The intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
Optionally, the fusing module 320 may be specifically configured to synchronize the data to be fused in the intermediate database to the target database when it is determined that the target database does not have the data to be fused.
Optionally, the fusion module 320 may be further specifically configured to synchronize the data to be fused to a temporary source library of the intermediate database for splitting operation when it is determined that the target database does not have the data to be fused, so as to obtain a plurality of data to be fused; transferring the data to be fused to a temporary target library of the intermediate database; and executing fusion operation of fusing each data to be fused in the temporary target database into the corresponding target database.
Optionally, the fusion module 320 may be further specifically configured to execute, when it is determined that a first library table corresponding to the data to be fused exists in the target database, a data conversion operation for the data to be fused in the intermediate database based on a difference between the data to be fused and the first library table; and synchronizing the target data to be fused obtained by the data conversion operation to the first library table of the target database.
Optionally, the fusion module 320 may be further specifically configured to execute a merging operation for the plurality of data to be fused in the intermediate database to obtain an intermediate library table when determining that a second library table corresponding to the plurality of data to be fused exists in the target database; performing a data conversion operation for the intermediate library table in the intermediate database based on the difference between the intermediate library table and the second library table; and synchronizing the target data to be fused obtained by the data conversion operation to the second library table of the target database.
Optionally, the fusion module 320 may be further specifically configured to trigger generation of an incremental data fusion request when determining that the trigger collects the change data that characterizes the change of the database table data in the source database; and acquiring incremental data to be fused corresponding to the change data, and executing incremental fusion operation for fusing the incremental data to be fused to the target database based on the incremental data fusion request.
Optionally, the fusion module 320 may be further specifically configured to obtain a real-time requirement of the incremental data to be fused, and select a target incremental synchronization interface matched with the real-time requirement from preset incremental synchronization interfaces; and executing the delta fusion operation for fusing the delta data to be fused to the target database based on the target delta synchronous interface.
Optionally, the device further comprises an updating module, configured to obtain a current fusion task recorded when the fusion operation ends and task detail data of the current fusion task; and updating a preset fusion task history table and a task data detail history table based on the current fusion task and the task detail data.
Optionally, the device further includes an anomaly alarm module, configured to record first anomaly information when first anomaly information characterizing anomaly of the trigger is monitored, and perform alarm prompt.
Optionally, the anomaly alarm module may be further configured to output alarm indication information when determining that second anomaly information characterizing anomaly of fusion operations is monitored, where the alarm indication information is used to indicate that a human intervention is abnormal and support a re-fusion operation for data to be fused corresponding to the second anomaly information.
Optionally, the device further includes a determining module, configured to determine that, when the source database and the target database are in the same library table and the same primary key, perform an adding operation for adding a preset field to the same library table; and determining a joint primary key based on the same primary key and the preset field obtained by the adding operation.
Fig. 4 illustrates a physical schematic diagram of an electronic device, and as shown in fig. 4, the electronic device 400 may include: processor 410, communication interface 420, memory 430 and communication bus 440, wherein processor 410, communication interface 420 and memory 430 communicate with each other through communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a database-based data fusion method comprising:
based on the data fusion request, synchronizing the data to be fused in the source database to the intermediate database;
performing fusion operation for fusing the data to be fused in the intermediate database to the target database;
the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the database-based data fusion method provided by the methods above, the method comprising:
based on the data fusion request, synchronizing the data to be fused in the source database to the intermediate database;
performing fusion operation for fusing the data to be fused in the intermediate database to the target database;
the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the database-based data fusion method provided by the above methods, the method comprising:
based on the data fusion request, synchronizing the data to be fused in the source database to the intermediate database;
Performing fusion operation for fusing the data to be fused in the intermediate database to the target database;
the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (15)
1. A database-based data fusion method, comprising:
based on the data fusion request, synchronizing the data to be fused in the source database to the intermediate database;
performing fusion operation for fusing the data to be fused in the intermediate database to the target database;
the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
2. The database-based data fusion method of claim 1, wherein the performing a fusion operation for the data to be fused in the intermediate database to the target database comprises:
And synchronizing the data to be fused in the intermediate database to the target database when the target database is not provided with the data to be fused.
3. The database-based data fusion method according to claim 1, wherein when the number of the target databases is plural, the performing a fusion operation for the data to be fused in the intermediate database to the target databases includes:
when the target database is determined to have no data to be fused, synchronizing the data to be fused into a temporary source database of the intermediate database for splitting operation to obtain a plurality of data to be fused;
transferring the data to be fused to a temporary target library of the intermediate database;
and executing fusion operation of fusing each data to be fused in the temporary target database into the corresponding target database.
4. The database-based data fusion method of claim 1, wherein the performing a fusion operation for the data to be fused in the intermediate database to the target database comprises:
when a first library table corresponding to the data to be fused exists in a target database, executing data conversion operation for the data to be fused in the intermediate database based on the difference between the data to be fused and the first library table;
And synchronizing the target data to be fused obtained by the data conversion operation to the first library table of the target database.
5. The database-based data fusion method according to claim 1, wherein when the number of the data to be fused is plural and a predetermined relationship is provided between the plural data to be fused, the performing the fusion operation for the data to be fused in the intermediate database to the target database includes:
when determining that a second library table corresponding to a plurality of data to be fused exists in a target database, executing merging operation aiming at the plurality of data to be fused in the intermediate database to obtain an intermediate library table;
performing a data conversion operation for the intermediate library table in the intermediate database based on the difference between the intermediate library table and the second library table;
and synchronizing the target data to be fused obtained by the data conversion operation to the second library table of the target database.
6. The database-based data fusion method of claim 1, wherein the performing a fusion operation for the data to be fused in the intermediate database to the target database comprises:
Triggering to generate an incremental data fusion request when determining that the trigger collects change data representing changes of database table data in a source database;
and acquiring incremental data to be fused corresponding to the change data, and executing incremental fusion operation for fusing the incremental data to be fused to the target database based on the incremental data fusion request.
7. The database-based data fusion method of claim 6, wherein the performing an incremental fusion operation for the incremental data to be fused to the target database comprises:
acquiring the real-time requirement of the incremental data to be fused, and selecting a target incremental synchronous interface matched with the real-time requirement from preset incremental synchronous interfaces;
and executing the delta fusion operation for fusing the delta data to be fused to the target database based on the target delta synchronous interface.
8. The database-based data fusion method of claim 1, wherein after the performing a fusion operation for the data to be fused in the intermediate database to the target database, the method further comprises:
Acquiring task detail data of a current fusion task recorded when the fusion operation is finished;
and updating a preset fusion task history table and a task data detail history table based on the current fusion task and the task detail data.
9. The database-based data fusion method of claim 1, wherein after the performing a fusion operation for the data to be fused in the intermediate database to the target database, the method further comprises:
when first abnormal information representing the abnormality of the trigger is determined to be monitored, the first abnormal information is recorded and alarming prompt is carried out.
10. The database-based data fusion method of claim 1, wherein after the performing a fusion operation for the data to be fused in the intermediate database to the target database, the method further comprises:
and when the second abnormal information representing the abnormal fusion operation is determined to be monitored, outputting alarm indication information, wherein the alarm indication information is used for indicating manual intervention to process the abnormal information and supporting the re-fusion operation aiming at the data to be fused corresponding to the second abnormal information.
11. The database-based data fusion method of claim 1, further comprising:
when the source database and the target database are determined to be in the same database table and the same primary key, executing an adding operation for adding a new preset field to the same database table;
and determining a joint primary key based on the same primary key and the preset field obtained by the adding operation.
12. A database-based data fusion apparatus, comprising:
the synchronization module is used for synchronizing the data to be fused in the source database to the intermediate database based on the data fusion request;
the fusion module is used for executing fusion operation for fusing the data to be fused in the intermediate database to the target database;
the intermediate database is used for representing attribute information of the data to be fused and attribute information of a database table fusing the data to be fused in the target database.
13. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the database-based data fusion method of any one of claims 1 to 11 when the program is executed by the processor.
14. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the database-based data fusion method according to any one of claims 1 to 11.
15. A computer program product comprising a computer program which, when executed by a processor, implements a database-based data fusion method according to any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210397603.4A CN116955369A (en) | 2022-04-15 | 2022-04-15 | Database-based data fusion method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210397603.4A CN116955369A (en) | 2022-04-15 | 2022-04-15 | Database-based data fusion method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116955369A true CN116955369A (en) | 2023-10-27 |
Family
ID=88441411
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210397603.4A Pending CN116955369A (en) | 2022-04-15 | 2022-04-15 | Database-based data fusion method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116955369A (en) |
-
2022
- 2022-04-15 CN CN202210397603.4A patent/CN116955369A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019154394A1 (en) | Distributed database cluster system, data synchronization method and storage medium | |
| EP3435235B1 (en) | Method and device for executing data recovery operation | |
| CN109901949B (en) | Application disaster recovery system and method for double-activity data center | |
| US7200626B1 (en) | System and method for verification of a quiesced database copy | |
| WO2017177941A1 (en) | Active/standby database switching method and apparatus | |
| WO2016173179A1 (en) | Method and device for switching primary database and secondary database | |
| CN111427898A (en) | Continuous data protection system and method based on analysis of Oracle log | |
| WO2011158387A1 (en) | Data processing failure recovery method, system and program | |
| CN109298978B (en) | Recovery method and system for database cluster of specified position | |
| CN110011853B (en) | Cross fault troubleshooting method and device for multiple platforms and clusters | |
| US20120324279A1 (en) | Method and Apparatus of Backing up Subversion Repository | |
| US9612921B2 (en) | Method and system for load balancing a distributed database providing object-level management and recovery | |
| WO2021082925A1 (en) | Transaction processing method and apparatus | |
| CN111404737B (en) | Disaster recovery processing method and related device | |
| CN112925676A (en) | Method for realizing recovery of distributed database cluster at any time point based on WAL | |
| CN105007172A (en) | Method for realizing HDFS high-availability scheme | |
| CN116955369A (en) | Database-based data fusion method and device, electronic equipment and storage medium | |
| US20130290385A1 (en) | Durably recording events for performing file system operations | |
| CN113190531A (en) | Database migration method, device, equipment and storage medium | |
| CN112685230A (en) | Method for realizing backup and restoration of assigned time point by distributed database | |
| JP2010152707A (en) | Backup method of database and database system | |
| JPH07114495A (en) | Multiplexing file managing system | |
| US7644306B2 (en) | Method and system for synchronous operation of an application by a purality of processing units | |
| CN117971984A (en) | Method and system for synchronizing table data among distributed database clusters | |
| CN117349384B (en) | Database synchronization method, system and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |