WO2019100638A1 - Procédé, dispositif et équipement de synchronisation de données, et support d'informations - Google Patents

Procédé, dispositif et équipement de synchronisation de données, et support d'informations Download PDF

Info

Publication number
WO2019100638A1
WO2019100638A1 PCT/CN2018/082270 CN2018082270W WO2019100638A1 WO 2019100638 A1 WO2019100638 A1 WO 2019100638A1 CN 2018082270 W CN2018082270 W CN 2018082270W WO 2019100638 A1 WO2019100638 A1 WO 2019100638A1
Authority
WO
WIPO (PCT)
Prior art keywords
library
source
target
metadata
target library
Prior art date
Application number
PCT/CN2018/082270
Other languages
English (en)
Chinese (zh)
Inventor
付军
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2019100638A1 publication Critical patent/WO2019100638A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273—Asynchronous replication or reconciliation

Definitions

  • the present application relates to the field of data synchronization technologies, and in particular, to a data synchronization method, apparatus, device, and storage medium.
  • the scripts and synchronization scripts in the above process are all developed by the developer through manual development.
  • the development process is complicated, the efficiency is low, and the error rate is high, which greatly reduces the efficiency of data synchronization.
  • the purpose of the present application is to provide a data synchronization method, apparatus, device, and storage medium, which are intended to solve the problem of creating a table script and a synchronization script in a data synchronization process in the prior art.
  • the development process is complicated, the efficiency is low, the error rate is high, and the defect of data synchronization efficiency is greatly reduced.
  • a data synchronization method includes the following steps:
  • the table type of the first target library table is one of an incremental table, a pipeline table, or a full scale table
  • the table type of the second target library table is an incremental table, a pipeline table, or a full scale table.
  • the step of acquiring the source library information including the source library name, the source library table name, and the source library table type, and the step of parsing the source library information corresponding to the metadata included in the source library to obtain the source table structure includes:
  • the metadata information table of the obtained metadata is parsed, and the source table structure is obtained according to the metadata information table.
  • the step of acquiring the table type of the first target library table and correspondingly generating a synchronization script for synchronizing metadata from the source library to the second target library table and the second target library table to the target library includes: ;
  • the first sqoop synchronization script and the hive program are generated correspondingly; the first sqoop synchronization script and the hive program are used to synchronize metadata from the source library to the first target.
  • the metadata in the first target library table is deduplicated according to the source table deduplication field and then stored in the second target library table.
  • the step of acquiring a table type of the first target library table, and correspondingly generating a synchronization script for synchronizing metadata from the source library to the first target library table and the second target library table to the target library also includes:
  • the second sqoop synchronization script and the hive program are generated correspondingly; the second sqoop synchronization script and the hive program are used to synchronize the metadata from the source library to the first target library.
  • the metadata in the first target library table is stored in the second target library table.
  • the step of acquiring a table type of the first target library table, and correspondingly generating a synchronization script for synchronizing metadata from the source library to the first target library table and the second target library table to the target library also includes:
  • the third sqoop synchronization script and the hive program are generated correspondingly; the third sqoop synchronization script and the hive program are used to synchronize the metadata from the source library to the first target library.
  • the table then stores the metadata in the first target library table into the second target library table.
  • the execution periods of the first sqoop synchronization script and the hive program, the second sqoop synchronization script, the hive program, the third sqoop synchronization script, and the hive program are both 24 hours.
  • the metadata information table corresponding to the metadata includes at least a table owner, a table name, a table comment, a column name, a column comment, and a column order.
  • a data synchronization device comprising:
  • the input parsing module is configured to obtain source library information including at least a source library name, a source library table name, and a source library table type, and parse the source library information corresponding to the metadata included in the source library to obtain a source table structure;
  • a table creation script module configured to generate a first target library table for establishing temporary storage data in the target library according to the source table structure, and to establish a second target library table for storing the same data in the target library as the source library Table script
  • the synchronization script generation module is configured to obtain a table type of the first target library table, and correspondingly generate a synchronization script for synchronizing the metadata from the source library to the target library through the first target library table and the second target library table.
  • a data synchronization device includes: a processor, a memory, a communication bus; and the memory stores a computer readable program executable by the processor;
  • the communication bus implements connection communication between the processor and the memory
  • the processor implements the steps in the data synchronization method described above when the computer readable program is executed.
  • a storage medium wherein the storage medium stores one or more programs, the one or more programs being executable by one or more processors to implement the steps in the data synchronization method described above.
  • the present application automatically generates a table creation script and a synchronization script by configuring a limited number of entries, so that the data synchronization operation is simplified, the development efficiency is improved, and human error is reduced.
  • FIG. 1 is a flowchart of a data synchronization method provided by the present application.
  • FIG. 2 is a flowchart of step S100 in the data synchronization method provided by the present application.
  • FIG. 3 is a flowchart of step S100 in the data synchronization method provided by the present application.
  • FIG. 4 is a schematic diagram of an operating environment of a preferred embodiment of a data synchronization device provided by the present application.
  • FIG. 5 is a functional block diagram of a preferred embodiment of the data synchronization procedure of the present application.
  • FIG. 6 is a structural block diagram of a data synchronization system provided by the present application.
  • the table creation script and the synchronization script in the data synchronization process are all developed by the developer through manual development, the development process is complicated, the efficiency is low, the error rate is high, and the efficiency of data synchronization is greatly reduced.
  • the purpose is to provide a data synchronization method, device, device and storage medium, and automatically generate a table creation script and a synchronization script by configuring a limited number of entries, thereby simplifying data synchronization operation, improving development efficiency, and reducing human error.
  • the data synchronization method includes the following steps:
  • Step S100 Obtain source library information that includes at least a source library name, a source library table name, and a source library table type, and parse the source library information corresponding to the metadata included in the source library to obtain a source table structure.
  • the synchronization tool when it is required to synchronize the metadata from the source library to the target library, it is only necessary to input the source in the interface of the synchronization tool (the synchronization tool is an application) designed by the data synchronization method.
  • the source library information of the library name, the source library table name, the source library table type, the source table update field, the source table deduplication field, and the target library name may be used. After the source library information is entered, the build script and the synchronization script are automatically generated later, which makes the data synchronization operation simple.
  • the source library initially stored by the metadata is an oracle database (the oracle database is also named Oracle) RDBMS, or Oracle for short, is a relational database management system from Oracle), MySQL (MySQL is an open source small relational database management system) database or PostgreSQL (PostgreSQL) Is a free object-relational database server) database, the target library is hive (hive is a data warehouse tool based on Hadoop) database.
  • Oracle is also named Oracle
  • MySQL MySQL is an open source small relational database management system
  • PostgreSQL PostgreSQL Is a free object-relational database server
  • the target library is hive (hive is a data warehouse tool based on Hadoop) database.
  • Step S200 Generate a first target library table for establishing temporary storage data in the target library according to the source table structure, and a table construction script for establishing a second target library table storing the same data as the source library in the target library.
  • the source table structure is automatically parsed according to the entered source library information, and the table creation script is automatically generated according to the source table structure and the preset table construction rules.
  • the preset build rules In order to more clearly understand the process of automatically generating the build script, the following describes the preset build rules.
  • the source library's table construction rules are as follows.
  • the source table in the source library is divided into three types, namely the source table delta table (also called the source table table delta table) and the source table flow table (also called the source).
  • the library table flow table) and the source table full scale also known as the source library table full scale three.
  • the data in the source table delta table is continuously updated and added, and the historical data before the day is updated on the same day; the data in the source table flow table is continuously added, and the historical data before the day is not updated on the same day; the source table is full.
  • the amount of data in the table is small, such as some configuration tables or dimension tables.
  • a first target library table corresponding to the source table ie, a src table, that is, a source file table, a source table
  • a second target library table ie, an ods table
  • Operational Data Store table operation data storage table
  • the src table is used as a temporary table, which is divided into src increment table (by day partition), src flow table (by day partition), src full scale (no partition required).
  • the ods table is consistent with the source table data, which is divided into ods delta tables (no partitioning, ods delta table can be used to remove all data in the src delta table), ods flow meter (by day partition, not divided) Heavy), ods full scale (no partitioning, no weight removal).
  • the source including the source library name, the source library table name, the source library table type, the source table update field, the source table deduplication field, and the target library name is entered.
  • Library information is ok, which simplifies operation.
  • Step S300 Obtain a table type of the first target library table, and generate a synchronization script for synchronizing the metadata from the source library to the target library through the first target library table and the second target library table.
  • the table type of the first target library table is one of a delta table, a pipeline table, or a full scale table
  • the table type of the second target library table is an increment table, a flow table, or a full amount.
  • One of the tables; and the table type of the first target library table is the same as the source library table type, and the table type of the second target library table is the same as the source library table type. In this way, according to the source library table type of the source library information, the table type of the first target library table is obtained, and the table type of the second target library table is obtained.
  • the step S100 specifically includes:
  • Step S101 Obtain source library information that includes the source library name, the source library table name, the source library table type, the source table update field, the source table deduplication field, and the target library name.
  • n_bas_pc_sop n_bas_pc_sop is also just a source database table name in a source database, the data included is metadata
  • pa_nesop pa_nesop is the target library name of the hive type
  • Source table deduplication field id
  • Target library name pa_nesop.
  • Step S102 Obtain metadata from a source library corresponding to the source library name in the source library information.
  • the metadata information is mainly various information of the front-end business system, such as various identity and deposit information of the customers of the banking system, and the metadata information is embodied by the table owner, table name, table comment, column name, column comment and Column order and other parts of the metadata information table.
  • Step S103 Parse the metadata information table of the acquired metadata, and obtain the source table structure according to the metadata information table.
  • the table structure defines the fields, types, primary keys, foreign keys, and indexes of a table. These basic attributes form the table structure of the database.
  • the source table structure that is, the source table corresponding to the table structure in the source library.
  • the change of the table structure is mainly reflected in the change of the column name, such as adding or modifying the gender information column of the customer, the income status information column, etc., so in this embodiment, the table mainly focuses on the metadata information.
  • the owner's column names are compared.
  • the step S300 includes:
  • Step S301 Obtain a table type of the first target library table, and determine that the table type of the first target library table is an incremental table, a flow table, or a full scale table;
  • Step S302 when the table type of the first target library table is an incremental table, correspondingly generating a first sqoop synchronization script and a hive program; the first sqoop synchronization script and the hive program are used to synchronize metadata from the source library to In the specified partition of the first target library table, the metadata in the first target library table is deduplicated according to the source table deduplication field and then stored in the second target library table.
  • the first target library table (n_bas_pc_sop_src) and the second target library table (n_bas_pc_sop_ods) are automatically generated.
  • the program automatically generates the first sqoop synchronization script and the hive program, and the data is The source library is synchronized to a new partition of the first target library table (n_bas_pc_sop_src), and then all the data in the first target library table (n_bas_pc_sop_src) is deduplicated according to the source table deduplication field (id) and stored in the second target library table. (n_bas_pc_sop_ods).
  • the execution period of the first sqoop synchronization script and the hive program is 24h (
  • Hive stores the separator between fields in the file in hdfs;
  • --lines-terminated-by “ ⁇ n”– Hive is the separator between each line in the file stored in hdfs;
  • the synchronization script can be automatically generated according to the source library information entered and the preset table construction rules.
  • the step S300 further includes:
  • Step S303 when the table type of the first target library table is a pipeline table, correspondingly generating a second sqoop synchronization script and a hive program; the second sqoop synchronization script and the hive program are used to synchronize metadata from the source library to the first In the specified partition of the target library table, the metadata in the first target library table is stored in the second target library table.
  • the program automatically generates a second sqoop synchronization script and a hive program, and synchronizes the data from the source library to the first day.
  • the program In a new partition of the target library table (n_bas_pc_sop_src), all the data in the first target library table (n_bas_pc_sop_src) is directly stored in the second target library table (n_bas_pc_sop_ods) according to the source table without deduplication.
  • the execution period of the second sqoop synchronization script and the hive program is 24h (that is, once a day).
  • the step S300 further includes:
  • Step S304 when the table type of the first target library table is a full scale, corresponding to generating a third sqoop synchronization script and a hive program; the third sqoop synchronization script and the hive program are used to synchronize metadata from the source library to the first A target library table, and then storing the metadata in the first target library table into the second target library table.
  • the program automatically generates a third sqoop synchronization script and a hive program, and the data is not removed from the full table of the source library every day.
  • the data is directly synchronized to the first target library table (n_bas_pc_sop_src), and then all the data in the first target library table (n_bas_pc_sop_src) is directly stored in the second target library table (n_bas_pc_sop_ods).
  • the execution period of the third sqoop synchronization script and the hive program is 24h (that is, once a day).
  • FIG. 4 is a schematic diagram showing the internal structure of a computer device in an embodiment.
  • the computer device may be a terminal or a server, wherein the terminal may be a communication device, such as a smart phone, a tablet computer, a notebook computer, or a desktop computer.
  • the server can be a standalone server or a server cluster consisting of multiple servers.
  • the computer device includes a processor, a non-volatile storage medium, an internal memory, and a network interface connected by a system bus.
  • the non-volatile storage medium of the computer device can store an operating system and a computer readable program that, when executed, can cause the processor to perform a method of verifying the difficulty prediction.
  • the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • the internal memory can store a computer readable program that, when executed by the processor, causes the processor to perform a data synchronization method.
  • the network interface of the computer device is used for network communication, such as sending assigned tasks. It will be understood by those skilled in the art that the structure shown in FIG. 4 does not constitute a limitation on the computer device to which the present application is applied, and the specific computer device may include more or less components than those shown in the figure. , or combine some components, or have different component arrangements.
  • the application also provides a data synchronization device including a processor 10, a memory 20, and a display 30.
  • Figure 4 shows only some of the components of the data synchronization device, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.
  • the memory 20, in some embodiments, may be an internal storage unit of components of the data synchronization device, such as a hard disk or memory of a data synchronization device.
  • the memory 20 may also be an external storage device of each component of the data synchronization device in other embodiments, such as a plug-in hard disk equipped on each component of the data synchronization device, and a smart memory card (Smart Media Card, SMC), Secure Digital (SD) card, flash card (Flash) Card) and so on.
  • the memory 20 may also include both an internal storage unit of the data synchronization device and an external storage device.
  • the memory 20 is configured to store application software and various types of data installed on the data synchronization device, such as the program code of the installation data synchronization device.
  • the memory 20 can also be used to temporarily store data that has been output or is about to be output.
  • a memory synchronization program 40 is stored on the memory 20, and the data synchronization program 40 can be executed by the processor 10 to implement the modified source database table structure method of various embodiments of the present application.
  • the processor 10 is configured to execute program code or process data stored in the memory 20, for example, to execute the rights authentication method and the like.
  • the display 30 is for displaying information processed in the WeChat customer behavior feedback device and a user interface for displaying visualizations, such as an assignment information interface, an authentication report interface, and the like.
  • the components 10-30 of the WeChat customer behavior feedback device communicate with one another via a system bus.
  • the various steps of the data synchronization method described above are implemented when the processor 10 executes the data synchronization program 40 in the memory 20.
  • FIG. 5 is a functional block diagram of a data synchronization apparatus for implementing the data synchronization method of the present application.
  • the data synchronization device can be divided into a record resolution module 31, a table creation script generation module 32, and a synchronization script generation module 33:
  • the input parsing module 31 is configured to obtain source library information that includes at least a source library name, a source library table name, and a source library table type, and parse the source library information corresponding to the metadata included in the source library to obtain a source table structure;
  • a table creation script module 32 configured to generate, according to the source table structure, a first target library table for establishing temporary storage data in the target library, and a second target library table for storing the same data in the target library as that of the source library Table creation script;
  • the synchronization script generating module 33 is configured to obtain a table type of the first target library table, and correspondingly generate a synchronization script for synchronizing the metadata from the source library to the target library through the first target library table and the second target library table.
  • the table type of the first target library table is one of an incremental table, a pipeline table, or a full scale table
  • the table type of the second target library table is an incremental table, a pipeline table, or a full scale table.
  • the step of acquiring the source library information including the source library name, the source library table name, and the source library table type, and the step of parsing the source library information corresponding to the metadata included in the source library to obtain the source table structure includes:
  • the metadata information table of the obtained metadata is parsed, and the source table structure is obtained according to the metadata information table.
  • the steps include:
  • the first sqoop synchronization script and the hive program are generated correspondingly; the first sqoop synchronization script and the hive program are used to synchronize metadata from the source library to the first target.
  • the metadata in the first target library table is deduplicated according to the source table deduplication field and then stored in the second target library table.
  • the steps also include:
  • the second sqoop synchronization script and the hive program are generated correspondingly; the second sqoop synchronization script and the hive program are used to synchronize the metadata from the source library to the first target library.
  • the metadata in the first target library table is stored in the second target library table.
  • the steps also include:
  • the third sqoop synchronization script and the hive program are generated correspondingly; the third sqoop synchronization script and the hive program are used to synchronize the metadata from the source library to the first target library.
  • the table then stores the metadata in the first target library table into the second target library table.
  • the execution periods of the first sqoop synchronization script and the hive program, the second sqoop synchronization script, the hive program, the third sqoop synchronization script, and the hive program are both 24 hours.
  • the metadata information table corresponding to the metadata includes at least a table owner, a table name, a table comment, a column name, a column comment, and a column order.
  • the present application further provides a data synchronization system.
  • a plurality of source databases 110, a target database 120, and a data synchronization device 130 are included.
  • the metadata of the plurality of source databases 110 are processed by the data synchronization device 130 and uploaded to the target database 120 by the automatically generated table creation script and synchronization script.
  • the present application also provides a storage medium accordingly.
  • the storage medium stores one or more programs that can be executed by one or more processors to implement the various steps of the data synchronization method described above.
  • the computer program when executed, may include the flow of an embodiment of the methods as described above.
  • the foregoing computer readable storage medium can be a magnetic disk, an optical disk, or a read-only storage memory (Read-Only) Non-volatile storage media such as Memory, ROM).
  • the present application automatically generates a table creation script and a synchronization script by configuring a limited number of entries, thereby simplifying data synchronization operations, improving development efficiency, and reducing human error.
  • a computer program to instruct related hardware (such as a processor, a controller, etc.), and the program may be stored in a computer.
  • the program may include the flow of the method embodiments as described above when executed.
  • the computer readable storage medium described therein may be a memory, a magnetic disk, an optical disk, or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention concerne un procédé, un dispositif et un équipement de synchronisation de données, et un support d'informations lisible. Le procédé comprend : l'acquisition d'informations de bibliothèque source entrées qui comprennent au moins un nom de bibliothèque source, un nom de table de bibliothèque source et un type de table de bibliothèque source, et l'analyse de métadonnées comprises dans une bibliothèque source correspondant aux informations de bibliothèque source afin d'obtenir une structure de table source ; la génération, selon la structure de table source, d'une première table de bibliothèque cible pour établir des données de stockage temporaires dans une bibliothèque cible et d'un script de création de table pour établir une seconde table de bibliothèque cible qui stocke les mêmes données que la bibliothèque source dans la bibliothèque cible ; et l'acquisition du type de table de la première table de bibliothèque cible, et la génération de manière correspondante d'un script de synchronisation pour synchroniser en séquence des métadonnées de la bibliothèque source vers la bibliothèque cible au moyen de la première table de bibliothèque cible et de la seconde table de bibliothèque cible.
PCT/CN2018/082270 2017-11-22 2018-04-09 Procédé, dispositif et équipement de synchronisation de données, et support d'informations WO2019100638A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711175125.8A CN107967316A (zh) 2017-11-22 2017-11-22 一种数据同步方法、设备及计算机可读存储介质
CN201711175125.8 2017-11-22

Publications (1)

Publication Number Publication Date
WO2019100638A1 true WO2019100638A1 (fr) 2019-05-31

Family

ID=62000385

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/082270 WO2019100638A1 (fr) 2017-11-22 2018-04-09 Procédé, dispositif et équipement de synchronisation de données, et support d'informations

Country Status (2)

Country Link
CN (1) CN107967316A (fr)
WO (1) WO2019100638A1 (fr)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109241026B (zh) * 2018-07-18 2021-10-15 创新先进技术有限公司 数据管理的方法、装置及系统
CN109241184B (zh) * 2018-08-20 2024-03-15 中国平安人寿保险股份有限公司 数据同步方法、装置、计算机设备和存储介质
CN109558448B (zh) * 2018-10-10 2021-04-06 北京海数宝科技有限公司 数据处理方法、装置、计算机设备和存储介质
CN109657002B (zh) * 2018-11-09 2022-04-08 山东中创软件商用中间件股份有限公司 一种多表批量数据同步方法、装置及设备
CN109684407A (zh) * 2018-11-23 2019-04-26 武汉达梦数据库有限公司 一种dml操作的数据同步方法
CN109614446A (zh) * 2018-11-23 2019-04-12 金色熊猫有限公司 数据同步方法、装置、电子设备及存储介质
CN110020840B (zh) * 2019-01-04 2023-09-22 创新先进技术有限公司 数据传输方法及其系统
CN109977157A (zh) * 2019-02-27 2019-07-05 深圳点猫科技有限公司 一种基于数据平台将数据导入目标目录的方法和电子设备
CN110059134A (zh) * 2019-03-18 2019-07-26 深圳市买买提信息科技有限公司 一种数据同步至云平台的方法、相关装置和设备
CN110209680A (zh) * 2019-04-25 2019-09-06 深圳壹账通智能科技有限公司 基于Hive外部表的数据更新方法、装置、及电子装置
CN110933144A (zh) * 2019-11-09 2020-03-27 许继集团有限公司 一种变电站主站系统及数据库同步方法
CN111125254A (zh) * 2019-12-23 2020-05-08 北京懿医云科技有限公司 数据库同步方法、装置、电子设备及计算机可读介质
CN111209282A (zh) * 2020-01-10 2020-05-29 深圳前海环融联易信息科技服务有限公司 数据存储方法、装置、计算机设备及存储介质
CN111324610A (zh) * 2020-02-19 2020-06-23 深圳市融壹买信息科技有限公司 一种数据同步的方法及装置
CN111367883A (zh) * 2020-02-25 2020-07-03 平安科技（深圳）有限公司 数据同步方法、装置、设备及计算机可读存储介质
CN111400397B (zh) * 2020-02-29 2023-04-11 平安科技（深圳）有限公司 数据同步方法、装置、设备和计算机存储介质
CN111259068A (zh) * 2020-04-28 2020-06-09 成都四方伟业软件股份有限公司 一种基于数据仓库的数据开发方法及系统
CN111767267B (zh) * 2020-06-18 2024-05-10 杭州数梦工场科技有限公司 元数据处理方法、装置、电子设备
CN111858760B (zh) * 2020-07-13 2024-03-22 中国工商银行股份有限公司 一种异构数据库的数据处理方法及装置
CN112364049B (zh) * 2020-11-10 2024-05-17 中国平安人寿保险股份有限公司 数据同步脚本生成方法、系统、终端及存储介质
CN112269788A (zh) * 2020-11-13 2021-01-26 中盈优创资讯科技有限公司 提升ClickHouse数据入库性能的方法及装置
CN112597150A (zh) * 2020-12-04 2021-04-02 光大科技有限公司 数据采集方法、装置、可读存储介质和电子装置
CN112817934A (zh) * 2021-01-21 2021-05-18 厦门熵基科技有限公司 一种数据迁移方法、装置、设备和计算机可读存储介质
CN113076314B (zh) * 2021-03-30 2024-04-19 深圳市酷开网络科技股份有限公司 数据表的存储方法、装置及计算机可读存储介质
CN113127448A (zh) * 2021-04-23 2021-07-16 深圳市酷开网络科技股份有限公司 一种地域维度表生成方法、装置、服务器及存储介质
CN114780641B (zh) * 2022-05-07 2023-07-14 湖南长银五八消费金融股份有限公司 多库多表同步方法、装置、计算机设备和存储介质
CN116089537B (zh) * 2023-04-07 2023-08-04 江西省智能产业技术创新研究院 增量数据同步方法、系统、计算机及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102495910A (zh) * 2011-12-28 2012-06-13 畅捷通信息技术股份有限公司 实现异构系统数据定时同步的装置和方法
CN103440273A (zh) * 2013-08-06 2013-12-11 北京航空航天大学 一种数据跨平台迁移方法及装置
CN103744949A (zh) * 2013-12-31 2014-04-23 金蝶软件（中国）有限公司 数据集成方法和系统
CN104317843A (zh) * 2014-10-11 2015-01-28 上海瀚之友信息技术服务有限公司 一种数据同步etl系统
US20170147672A1 (en) * 2015-11-25 2017-05-25 International Business Machines Corporation Determining Data Replication Cost for Cloud Based Application

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8639657B2 (en) * 2008-10-29 2014-01-28 International Business Machines Corporation Reorganizing table-based data objects
CN102346775A (zh) * 2011-09-26 2012-02-08 苏州博远容天信息科技有限公司 一种基于日志的异构多源数据库同步方法
US9245249B2 (en) * 2013-03-12 2016-01-26 Labtech Llc General, flexible, resilent ticketing interface between a device management system and ticketing systems
CN106919697B (zh) * 2017-03-07 2020-09-25 浪潮云信息技术股份公司 一种将数据同时导入多个Hadoop组件的方法
CN107330003A (zh) * 2017-06-12 2017-11-07 上海藤榕网络科技有限公司 数据同步方法、系统、存储器及数据同步设备
CN107301250B (zh) * 2017-07-27 2020-06-26 南京南瑞集团公司 一种多源数据库协同备份方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102495910A (zh) * 2011-12-28 2012-06-13 畅捷通信息技术股份有限公司 实现异构系统数据定时同步的装置和方法
CN103440273A (zh) * 2013-08-06 2013-12-11 北京航空航天大学 一种数据跨平台迁移方法及装置
CN103744949A (zh) * 2013-12-31 2014-04-23 金蝶软件（中国）有限公司 数据集成方法和系统
CN104317843A (zh) * 2014-10-11 2015-01-28 上海瀚之友信息技术服务有限公司 一种数据同步etl系统
US20170147672A1 (en) * 2015-11-25 2017-05-25 International Business Machines Corporation Determining Data Replication Cost for Cloud Based Application

Also Published As

Publication number Publication date
CN107967316A (zh) 2018-04-27

Similar Documents

Publication Publication Date Title
WO2019100638A1 (fr) Procédé, dispositif et équipement de synchronisation de données, et support d'informations
WO2018107610A1 (fr) Procédé, système et dispositif de traitement de données de service, et support d'informations lisible par ordinateur
WO2019037396A1 (fr) Procédé, dispositif et équipement de suppression de compte et support d'informations
WO2018205376A1 (fr) Procédé d'interrogation d'informations d'association, terminal, système de gestion de serveur et support de stockage lisible par ordinateur
WO2019041832A1 (fr) Procédé, serveur et système de modification de structure de table de base de données source, et support de stockage
WO2019104876A1 (fr) Procédé et système de poussée de produit d'assurance, terminal, terminal client et support d'informations
WO2019024219A1 (fr) Procédé et appareil de génération automatique de document, générateur automatique de document et support
WO2019061613A1 (fr) Procédé de criblage d'habilitation pour un prêt, dispositif et support de stockage lisible par ordinateur
WO2019165691A1 (fr) Procédé, appareil et dispositif de production automatique de cas de test, et support de stockage lisible par ordinateur
WO2018188196A1 (fr) Procédé de commande de version de données, contrôleur de version de données, dispositif et support de stockage lisible par ordinateur
WO2019061612A1 (fr) Procédé et appareil de promotion de produit de prêt et support d'informations lisible par ordinateur
WO2019000801A1 (fr) Procédé, appareil et dispositif de synchronisation de données, et support d'informations lisible par ordinateur
WO2018233370A1 (fr) Procédé, système, dispositif et support de stockage lisible par ordinateur de synchronisation d'une image
WO2019174375A1 (fr) Procédé, appareil et dispositif de test d'interface, et support de stockage lisible par ordinateur
WO2015158297A1 (fr) Procédé, appareil et système de commande de tâche de distribution dans une plate-forme de réseautage social
WO2017143692A1 (fr) Téléviseur intelligent et son procédé de commande vocale
WO2019080247A1 (fr) Procédé, appareil et dispositif pour générer une approbation de police d'assurance, et support d'informations lisible par ordinateur
WO2018227880A1 (fr) Procédé, appareil, et dispositif de comparaison de données, et support d'informations lisible
WO2019169814A1 (fr) Procédé, appareil et dispositif de génération automatique d'annotation en chinois, et support d'informations
WO2019075973A1 (fr) Procédé et dispositif de test de programme d'application
WO2018120430A1 (fr) Procédé de construction de page, terminal, support de stockage lisible par ordinateur et dispositif de construction de page
WO2014026526A1 (fr) Procédé de réglage d'informations de personne naturelle et dispositif électronique
WO2018120429A1 (fr) Procédé de mise à jour de ressources, terminal, support de stockage lisible par ordinateur et dispositif de mise à jour de ressources
WO2019114262A1 (fr) Procédé de chargement d'interface utilisateur, téléviseur intelligent, et support de stockage lisible par ordinateur
WO2019000800A1 (fr) Procédé, appareil et dispositif de préparation de justificatifs d'identité, et support de stockage lisible par ordinateur

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 02/10/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18881860

Country of ref document: EP

Kind code of ref document: A1