WO2019000629A1 - Procédé et système de synchronisation de données de multiples sources de données, serveur d'application et support de stockage lisible par ordinateur - Google Patents

Procédé et système de synchronisation de données de multiples sources de données, serveur d'application et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2019000629A1
WO2019000629A1 PCT/CN2017/100051 CN2017100051W WO2019000629A1 WO 2019000629 A1 WO2019000629 A1 WO 2019000629A1 CN 2017100051 W CN2017100051 W CN 2017100051W WO 2019000629 A1 WO2019000629 A1 WO 2019000629A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
source
type
task
synchronized
Prior art date
Application number
PCT/CN2017/100051
Other languages
English (en)
Chinese (zh)
Inventor
付军
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2019000629A1 publication Critical patent/WO2019000629A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H04L67/141—Setup of application sessions

Definitions

  • the present invention relates to the field of computer information technology, and in particular, to a multi-data source data synchronization method, system, application server, and computer readable storage medium.
  • the existing multi-data source data synchronization operation is very complicated, developers need to develop a large number of scripts, and need to go through development, testing, deployment and other processes, long development cycle, large amount of tasks, and error-prone. Therefore, the data synchronization operation of multiple data sources in the prior art is not reasonable enough and needs to be improved.
  • the present invention provides a multi-data source data synchronization method, system, application server and computer readable storage medium, which eliminates the development, testing and deployment process of the developer, and the entire data synchronization process realizes fully automatic operation, It saves development costs and reduces the risk of error for developers.
  • the present invention provides an application server including a memory, a processor, and a multi-data source data synchronization system stored on the memory and operable on the processor,
  • the multi-data source data synchronization system is implemented by the processor to implement the following steps:
  • the built-up task and the data synchronization task corresponding to each generated data source are sent to a preset workflow scheduling engine, and the preset workflow scheduling engine executes the table-building task and the data synchronization task corresponding to each data source.
  • the present invention further provides a multi-data source data synchronization method, which is applied to an application server, and the method includes:
  • the built-up task and the data synchronization task corresponding to each generated data source are sent to a preset workflow scheduling engine, and the preset workflow scheduling engine executes the table-building task and the data synchronization task corresponding to each data source.
  • the present invention also provides a computer readable storage medium storing a data source data synchronization system, the multiple data source data synchronization system being configurable by at least one processor Execute to implement the following steps:
  • the built-up task and the data synchronization task corresponding to each generated data source are sent to a preset workflow scheduling engine, and the preset workflow scheduling engine executes the table-building task and the data synchronization task corresponding to each data source.
  • the present invention further provides a multi-data source data synchronization system, the system comprising:
  • connection module configured to establish a first connection between the data platform and the at least one data source, and establish a second connection between the data platform and the application server;
  • the task generation module is configured to receive a source table and a data source type that are selected by the user and need to be synchronized, and generate a table construction task and a data synchronization task corresponding to each data source according to the source table structure of each data source obtained in advance;
  • a data synchronization module configured to: publish a table-building task and a data synchronization task corresponding to each generated data source to a preset workflow scheduling engine, and execute a table corresponding to each data source by using the preset workflow scheduling engine Task and data synchronization tasks.
  • the application server, the multi-data source data synchronization method, the system, and the computer-readable storage medium provided by the present invention automatically generate corresponding data sources according to the source table and the data source type selected by the user to be synchronized.
  • the data synchronization task is automatically published to the preset workflow scheduling engine, and the preset workflow scheduling engine is used to execute the execution center.
  • the task of creating tables and data synchronization tasks realizes one-stop data synchronization service, which eliminates the development, testing and deployment process of developers.
  • the entire data synchronization process realizes fully automatic operation, which saves development costs and reduces developers. The risk of error.
  • FIG. 1 is a schematic diagram of an optional application environment of each embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an optional hardware architecture of the application server of FIG. 1;
  • FIG. 3 is a schematic diagram of functional modules of an embodiment of a multi-data source data synchronization system according to the present invention.
  • FIG. 4 is a schematic diagram of an implementation process of an embodiment of a multi-data source data synchronization method according to the present invention.
  • first, second and the like in the present invention are for the purpose of description only, and are not to be construed as indicating or implying their relative importance or implicitly indicating the number of indicated technical features. .
  • features defining “first” and “second” may include at least one of the features, either explicitly or implicitly.
  • the technical solutions between the various embodiments may be combined with each other, but must be based on the realization of those skilled in the art, and when the combination of the technical solutions is contradictory or impossible to implement, it should be considered that the combination of the technical solutions does not exist. It is also within the scope of protection required by the present invention.
  • FIG. 1 it is a schematic diagram of an optional application environment according to various embodiments of the present invention.
  • the present invention is applicable to an application environment including, but not limited to, an application server 2, a network 3, a data platform 4, and a data source 5.
  • the application server 2 may be a computing device such as a rack server, a blade server, a tower server, or a rack server.
  • the application server 2 may be an independent server or a server cluster composed of multiple servers.
  • the network 3 may be an intranet, an Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, Wireless or wired networks such as 5G networks, Bluetooth, Wi-Fi, etc.
  • GSM Global System of Mobile communication
  • WCDMA Wideband Code Division Multiple Access
  • 4G Wireless or wired networks
  • 5G networks Bluetooth, Wi-Fi, etc.
  • the application server 2 is communicably connected to the data platform 4 via the network 3, and the data platform 4 is communicably connected to each data source 5 through the network 3.
  • a first connection (or a first type of connection) is established between the data platform 4 and each data source 5, so that the data platform 4 can perform data transmission with each data source 5 through the first connection.
  • a second connection (or a second type of connection) is established between the data platform 4 and the application server 2, so that the data platform 4 can perform data transmission and interaction with the application server 2 through the second connection. .
  • the application server 2 may include, but is not limited to, a memory 21, a processor 22, and a network interface 23 that can communicate with each other through a system bus. It is noted that FIG. 2 only shows the application server 2 with components 21-23, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.
  • the memory 21 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), and a random access memory (RAM). , static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, and the like.
  • the memory 21 may be an internal storage unit of the application server 2, such as a hard disk or memory of the application server 2.
  • the memory 21 may also be an external storage device of the application server 2, such as a plug-in hard disk equipped on the application server 2, a smart memory card (SMC), and a secure digital number. (Secure Digital, SD) card, flash card, etc.
  • the memory 21 may also include both the internal storage unit of the application server 2 and its external storage device.
  • the memory 21 is generally used to store an operating system installed in the application server 2 and various types of application software, such as program codes of the multi-data source data synchronization system 20.
  • the memory 21 can also be used to temporarily store various types of data that have been output or are to be output.
  • the processor 22 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments.
  • the processor 22 is typically used to control the overall operation of the application server 2, such as performing control and processing related to data interaction or communication with the application server 2.
  • the processor 22 is configured to run program code or process data stored in the memory 21, such as running the multi-data source data synchronization system 20 and the like.
  • the network interface 23 may comprise a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the application server 2 and other electronic devices.
  • the network interface 23 is mainly used to connect the application server 2 to the data platform 4 through the network 3, and establish data transmission between the application server 2 and the data platform 4. Channel and communication connections.
  • the present invention proposes a multiple data source data synchronization system 20.
  • the multi-data source data synchronization system 20 may be divided into one or more modules, the one or more modules being stored in the memory 21 and being processed by one or more processors ( This embodiment is executed by the processor 22) to complete the present invention.
  • the multi-data source data synchronization system 20 can be divided into a connection module 201, a task generation module 202, and a data synchronization module 203.
  • the functional module referred to in the present invention refers to a series of computer program instruction segments capable of performing a specific function, and is more suitable than the program to describe the execution process of the multi-data source data synchronization system 20 in the application server 2.
  • the functions of the respective function modules 201-203 will be described in detail below.
  • the connection module 201 is configured to establish a first connection (or a first type of connection) between the data platform 4 and the at least one data source 5, and establish the data platform 4 and an application server 2 (such as a resolution server).
  • the second connection (or the second type of connection).
  • the data platform 4 adopts a Hadoop data platform, and establishes a first connection between the data platform 4 and each data source 5 through a data transmission tool in the data platform 4.
  • the data transfer tool is a Sqoop tool set in Hadoop.
  • the data platform and data transfer tool may be of other types.
  • the task generation module 202 is configured to receive a source table (partial or all) and a data source type that are selected by the user to be synchronized, and generate a table-building task corresponding to each data source according to the source table structure of each data source obtained in advance. Synchronize tasks with data (or as data extraction tasks).
  • the data source type includes, but is not limited to, a first type of data source (such as an MS SQL Server data source), and a second type of data source (such as an Oracle data source), Three types of data sources (such as MySQL data sources).
  • a first type of data source such as an MS SQL Server data source
  • a second type of data source such as an Oracle data source
  • Three types of data sources such as MySQL data sources.
  • the number of data source types may also be increased or decreased according to the requirements of different data platforms. For example, a user may choose to synchronize a portion of the source table in the first type of data source.
  • the source table structure of the data source 5 includes, but is not limited to, a data type of each data source 5, field information included, and the like.
  • the application server 2 is provided with a corresponding parsing script file of each data source 5, and the data transfer tool in the data platform 4 is controlled to execute each parsing script file in parallel through a parallel scheduling mechanism.
  • the source table structure parsing task parsing the source table structure of each data source, and returning the source table structure of each data source obtained by the parsing to the application server 2 through the second connection.
  • the generating a table-building task corresponding to each data source includes: generating a source table with the selected source table according to a source table (partial or all) and a source table type that are selected by the user to be synchronized.
  • the target table corresponding to the type.
  • the source table type includes, but is not limited to, a first type dimension table (ie, a small amount of data, a source table used by the associated configuration), and a second type task table (ie, the data in the table is continuously updated and added, and the same day Update the historical data except the day), the third type of flow meter (that is, the data in the table is constantly added, and the historical data except the day will not be updated on the same day).
  • the source table type selected by the user to be synchronized is the first type dimension table (that is, the source table is small, the source table used by the association configuration)
  • the source table structure corresponding to the first type dimension table is generated.
  • the second type task table is generated.
  • the second type target table corresponding to the source table structure, wherein the number of the second type target table is determined according to the number of the second type source table that the user selects to synchronize.
  • the source table type selected by the user to be synchronized is the third type of flow table (that is, the data in the table is continuously added, and the historical data except the day is not updated on the day)
  • the source of the third type of the flow table is generated.
  • the third type target table corresponding to the table structure, wherein the number of the third type target table is determined according to the number of the third type source table selected by the user to be synchronized.
  • the generating a data synchronization task corresponding to each data source includes:
  • the resource configuration (such as CPU, memory, disk, etc.) and resource management nodes required for data synchronization of each task are determined, wherein the resource configuration can be listed by means of a list.
  • the resource management node may be implemented by using a Hadoop resource representation model and a resource allocation model;
  • the source table type selected by the user to be synchronized is the first type dimension table
  • Corresponding relationship between the corresponding field of the first type dimension table that needs to be synchronized and the corresponding field of each generated first type target table, and according to the corresponding relationship between the first type dimension table and the corresponding field of the first type target table Generating a data extraction task corresponding to each first type dimension table
  • the source table type that is selected by the user to be synchronized is the second type task table
  • the corresponding relationship between the corresponding field of the second type task table that needs to be synchronized and the corresponding field of each generated second type target table is established. And generating a data extraction task corresponding to each second type task table according to the correspondence between the second type task table and the corresponding field of the second type target table;
  • the source table type that is selected by the user to be synchronized is the third type of flow table
  • the corresponding relationship between the corresponding field of the third type of flow table that needs to be synchronized and the corresponding field of each generated third type target table is established. And generating a data extraction task corresponding to each third type of flow table according to the correspondence between the third type of flow table and the corresponding field of the third type target table.
  • the data synchronization module 203 is configured to advertise the built-up task and the data synchronization task corresponding to each generated data source to a preset workflow scheduling engine (such as an oozie scheduling engine), and schedule the workflow through the preset workflow.
  • the engine executes the task of creating a table and data synchronization corresponding to each data source.
  • the performing of the data synchronization task includes:
  • the data corresponding to each data source is started to synchronize each task, the resource configuration required for each task of data synchronization is acquired, and the data corresponding to the corresponding field is extracted from each source table (metadata information) ), and the extracted data is stored in the corresponding fields of each target table.
  • the source table type that is selected by the user to be synchronized is the first type dimension table
  • the first data corresponding to the corresponding field is respectively extracted from each first type dimension table, and the extracted first data is separately stored.
  • the source table type that is selected by the user to be synchronized is the second type task table
  • the second data corresponding to the corresponding field is respectively extracted from each second type task table, and the extracted second data is separately stored in each second type.
  • the source table type selected by the user to be synchronized is the third type of flow table
  • the third data corresponding to the corresponding field is respectively extracted from each third type of flow table, and the extracted third data is separately stored in each third type.
  • the following steps are also implemented:
  • the multi-data source data synchronization system 20 of the present invention automatically generates a table-building task and a data synchronization task corresponding to each data source according to the source table and the data source type selected by the user. And automatically generating the table-building task and the data synchronization task corresponding to the generated data source to a preset workflow scheduling engine, and scheduling, by using the preset workflow scheduling engine, the table-building task and the data synchronization task,
  • a one-stop data synchronization service eliminates the development, testing, and deployment process of developers, and the entire data synchronization process enables fully automated operations. It saves development costs and reduces the risk of error for developers.
  • the present invention also proposes a multi-data source data synchronization method.
  • FIG. 4 it is a schematic flowchart of an implementation manner of an embodiment of a multi-data source data synchronization method according to the present invention.
  • the order of execution of the steps in the flowchart shown in FIG. 4 may be changed according to different requirements, and some steps may be omitted.
  • Step S41 establishing a first connection (or a first type of connection) between the data platform 4 and the at least one data source 5, and establishing a second connection between the data platform 4 and an application server 2 (such as a resolution server) ( Or call it the second type of connection).
  • an application server 2 such as a resolution server
  • the data platform 4 adopts a Hadoop data platform, and establishes a first connection between the data platform 4 and each data source 5 through a data transmission tool in the data platform 4.
  • the data transfer tool is a Sqoop tool set in Hadoop.
  • the data platform and data transfer tool may be of other types.
  • Step S42 receiving a source table (partial or all) and a data source type that are selected by the user to be synchronized, and generating a table construction task and a data synchronization task corresponding to each data source according to the source table structure of each data source obtained by pre-parsing (or Call it a data extraction task).
  • the data source type includes, but is not limited to, a first type of data source (such as an MS SQL Server data source), a second type of data source (such as an Oracle data source), and a third type of data.
  • a first type of data source such as an MS SQL Server data source
  • a second type of data source such as an Oracle data source
  • a third type of data such as MySQL data source
  • Source such as MySQL data source
  • the number of data source types may also be increased or decreased according to the requirements of different data platforms. For example, a user may choose to synchronize a portion of the source table in the first type of data source.
  • the source table structure of the data source 5 includes, but is not limited to, a data type of each data source 5, field information included, and the like.
  • the application server 2 is provided with a corresponding parsing script file of each data source 5, and the data transfer tool in the data platform 4 is controlled to execute each parsing script file in parallel through a parallel scheduling mechanism.
  • the source table structure parsing task parsing the source table structure of each data source, and returning the source table structure of each data source obtained by the parsing to the application server 2 through the second connection.
  • the generating a table-building task corresponding to each data source includes: generating a source table with the selected source table according to a source table (partial or all) and a source table type that are selected by the user to be synchronized.
  • the target table corresponding to the type.
  • the source table type includes, but is not limited to, a first type dimension table (ie, a small amount of data, a source table used by an associated configuration), and a second type task table (ie, data in the table continuously) There are updates and additions, and the historical data except the day will be updated on the same day), and the third type of flow table (that is, the data in the table is continuously added, and the historical data except the day is not updated on the same day).
  • a first type dimension table ie, a small amount of data, a source table used by an associated configuration
  • a second type task table ie, data in the table continuously
  • the source table type selected by the user to be synchronized is the first type dimension table (that is, the source table is small, the source table used by the association configuration)
  • the source table structure corresponding to the first type dimension table is generated.
  • the number of source tables that need to be synchronized is determined by the number of source tables.
  • the second type task table is generated.
  • the second type target table corresponding to the source table structure, wherein the number of the second type target table is determined according to the number of the second type source table that the user selects to synchronize.
  • the source table type selected by the user to be synchronized is the third type of flow table (that is, the data in the table is continuously added, and the historical data except the day is not updated on the day)
  • the source of the third type of the flow table is generated.
  • the third type target table corresponding to the table structure, wherein the number of the third type target table is determined according to the number of the third type source table selected by the user to be synchronized.
  • the generating a data synchronization task corresponding to each data source includes:
  • the resource configuration (such as CPU, memory, disk, etc.) and resource management nodes required for data synchronization of each task are determined, wherein the resource configuration can be listed by means of a list.
  • the resource management node may be implemented by using a Hadoop resource representation model and a resource allocation model;
  • the source table type that the user selects to synchronize is the first type dimension table
  • the corresponding field of the first type dimension table that needs to be synchronized and the corresponding field of each generated first type target table are established.
  • the source table type that is selected by the user to be synchronized is the second type task table
  • the corresponding relationship between the corresponding field of the second type task table that needs to be synchronized and the corresponding field of each generated second type target table is established. And generating a data extraction task corresponding to each second type task table according to the correspondence between the second type task table and the corresponding field of the second type target table;
  • the source table type that is selected by the user to be synchronized is the third type of flow table
  • the corresponding relationship between the corresponding field of the third type of flow table that needs to be synchronized and the corresponding field of each generated third type target table is established. And generating a data extraction task corresponding to each third type of flow table according to the correspondence between the third type of flow table and the corresponding field of the third type target table.
  • Step S43 Publish the built-up task and the data synchronization task corresponding to the generated data sources to a preset workflow scheduling engine (such as an oozie scheduling engine), and execute corresponding data sources by using the preset workflow scheduling engine.
  • a preset workflow scheduling engine such as an oozie scheduling engine
  • the performing of the data synchronization task includes:
  • the data corresponding to each data source is started to synchronize each task, the resource configuration required for each task of data synchronization is acquired, and the data corresponding to the corresponding field is extracted from each source table (metadata information) ), and the extracted data is stored in the corresponding fields of each target table.
  • the source table type that is selected by the user to be synchronized is the first type dimension table
  • the first data corresponding to the corresponding field is respectively extracted from each first type dimension table, and the extracted first data is separately stored.
  • the source table type that is selected by the user to be synchronized is the second type task table
  • the second data corresponding to the corresponding field is respectively extracted from each second type task table, and the extracted second data is separately stored in each second type.
  • the source table type selected by the user to be synchronized is the third type of flow table
  • the third data corresponding to the corresponding field is respectively extracted from each third type of flow table, and the extracted third data is separately stored in each third type.
  • the method further comprises the steps of:
  • the multi-data source data synchronization method proposed by the present invention automatically generates a table-building task and a data synchronization task corresponding to each data source according to the source table and the data source type selected by the user to be synchronized, and
  • the generated table-building task and the data synchronization task corresponding to each generated data source are automatically advertised to a preset workflow scheduling engine, and the preset workflow task scheduling engine executes the table-building task and the data synchronization task to implement one
  • the station-based data synchronization service eliminates the developer's development, testing, and deployment processes, and the entire data synchronization process enables fully automated operations, saving development costs and reducing developer error risk.
  • the present invention further provides a computer readable storage medium (such as a ROM/RAM, a magnetic disk, an optical disk), the computer readable storage medium storing a multiple data source data synchronization system 20,
  • the multiple data source data synchronization system 20 can be executed by at least one processor 22 to cause the at least one processor to perform the steps of the multiple data source data synchronization method as described above.
  • the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and can also be implemented by hardware, but in many cases, the former is A better implementation.
  • the technical solution of the present invention which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk,
  • the optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

L'invention concerne un procédé et un système de synchronisation de données de multiples sources de données, ainsi qu'un serveur d'application et un support de stockage lisible par ordinateur. Le procédé comprend les étapes consistant : à établir une première connexion entre une plate-forme de données (4) et au moins une source de données (5), et à établir une seconde connexion entre la plateforme de données (4) et le serveur d'application (2) (S41) ; à recevoir une table source et des types de sources de données, sélectionnées par un utilisateur et devant être synchronisées, et à générer des tâches d'établissement de table et des tâches de synchronisation de données correspondant à diverses sources de données selon des structures de table source des diverses sources de données obtenues par résolution préalable (S42) ; et à publier les tâches d'établissement de table et les tâches de synchronisation de données générées correspondant aux diverses sources de données vers un moteur de planification de flux de travail prédéfini, et à exécuter les tâches d'établissement de table et les tâches de synchronisation de données correspondant aux diverses sources de données au moyen du moteur de planification de flux de travail prédéfini (S43). Le procédé décrit peut réaliser une opération entièrement automatique d'un flux de synchronisation de données entier, ce qui permet de réduire les flux de développement, de test et de déploiement pour les développeurs.
PCT/CN2017/100051 2017-06-25 2017-08-31 Procédé et système de synchronisation de données de multiples sources de données, serveur d'application et support de stockage lisible par ordinateur WO2019000629A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710490443.7A CN107689982B (zh) 2017-06-25 2017-06-25 多数据源数据同步方法、应用服务器及计算机可读存储介质
CN201710490443.7 2017-06-25

Publications (1)

Publication Number Publication Date
WO2019000629A1 true WO2019000629A1 (fr) 2019-01-03

Family

ID=61152726

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/100051 WO2019000629A1 (fr) 2017-06-25 2017-08-31 Procédé et système de synchronisation de données de multiples sources de données, serveur d'application et support de stockage lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN107689982B (fr)
WO (1) WO2019000629A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111258742A (zh) * 2020-02-17 2020-06-09 杭州依图医疗技术有限公司 一种数据同步的方法、系统、计算设备及存储介质
CN111694840A (zh) * 2020-04-29 2020-09-22 平安科技（深圳）有限公司 数据同步方法、装置、服务器及存储介质

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109656999B (zh) * 2018-10-11 2024-03-15 平安科技（深圳）有限公司 大数据量的数据同步方法、设备、存储介质及装置
CN109299150B (zh) * 2018-10-24 2022-01-28 万惠投资管理有限公司 一种可配置多数据源适配规则引擎解决方法
CN109710603B (zh) * 2018-12-28 2020-11-24 江苏满运软件科技有限公司 数据清洗方法、系统、存储介质及电子设备
CN109871407A (zh) * 2019-01-23 2019-06-11 深圳壹账通智能科技有限公司 备付金数据实时上报方法、装置、计算机设备和存储介质
CN110689245B (zh) * 2019-09-17 2022-07-12 上海易点时空网络有限公司 一种分析大数据工作流调用关系的方法及系统
CN113542326B (zh) * 2020-04-17 2023-04-25 上海际链网络科技有限公司 分布式系统的数据缓存方法及装置、服务器、存储介质
CN111582824B (zh) * 2020-05-08 2023-03-24 北京青云科技股份有限公司 云资源同步方法、装置、设备及存储介质
CN111815200A (zh) * 2020-07-31 2020-10-23 深圳壹账通智能科技有限公司 任务调度方法及装置、计算机设备、存储介质
CN113645075A (zh) * 2021-08-11 2021-11-12 杭州安恒信息技术股份有限公司 一种网络链路故障定位方法、装置、设备及可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101094231A (zh) * 2007-07-10 2007-12-26 北京易路联动技术有限公司 一种分布式可同步的多源数据的控制方法和装置
CN101105793A (zh) * 2006-07-11 2008-01-16 阿里巴巴公司 数据仓库中的数据处理方法及系统
CN102368261A (zh) * 2011-10-14 2012-03-07 成都勤智数码科技有限公司 一种可扩展的运维报表生成方法
CN103699580A (zh) * 2013-12-03 2014-04-02 中铁程科技有限责任公司 数据库同步方法及装置
EP2891994A1 (fr) * 2013-11-04 2015-07-08 Guangdong Electronics Industry Institute Ltd. Méthode permettant de réaliser une synchronisation automatique de ressources de données hétérogènes à multiples sources

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104915341B (zh) * 2014-03-10 2018-06-26 中国科学院沈阳自动化研究所 可视化多数据库etl集成方法和系统
CN105095327A (zh) * 2014-05-23 2015-11-25 深圳市珍爱网信息技术有限公司 一种分布式etl系统及调度方法
CN106528070B (zh) * 2015-09-15 2019-09-03 阿里巴巴集团控股有限公司 一种数据表生成方法及设备
CN106383901B (zh) * 2016-09-29 2019-05-14 航天科工智慧产业发展有限公司 一种多数据源之间单向数据同步方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101105793A (zh) * 2006-07-11 2008-01-16 阿里巴巴公司 数据仓库中的数据处理方法及系统
CN101094231A (zh) * 2007-07-10 2007-12-26 北京易路联动技术有限公司 一种分布式可同步的多源数据的控制方法和装置
CN102368261A (zh) * 2011-10-14 2012-03-07 成都勤智数码科技有限公司 一种可扩展的运维报表生成方法
EP2891994A1 (fr) * 2013-11-04 2015-07-08 Guangdong Electronics Industry Institute Ltd. Méthode permettant de réaliser une synchronisation automatique de ressources de données hétérogènes à multiples sources
CN103699580A (zh) * 2013-12-03 2014-04-02 中铁程科技有限责任公司 数据库同步方法及装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111258742A (zh) * 2020-02-17 2020-06-09 杭州依图医疗技术有限公司 一种数据同步的方法、系统、计算设备及存储介质
CN111258742B (zh) * 2020-02-17 2023-08-04 杭州依图医疗技术有限公司 一种数据同步的方法、系统、计算设备及存储介质
CN111694840A (zh) * 2020-04-29 2020-09-22 平安科技（深圳）有限公司 数据同步方法、装置、服务器及存储介质
CN111694840B (zh) * 2020-04-29 2023-05-30 平安科技（深圳）有限公司 数据同步方法、装置、服务器及存储介质

Also Published As

Publication number Publication date
CN107689982A (zh) 2018-02-13
CN107689982B (zh) 2020-11-24

Similar Documents

Publication Publication Date Title
WO2019000629A1 (fr) Procédé et système de synchronisation de données de multiples sources de données, serveur d'application et support de stockage lisible par ordinateur
JP6695984B2 (ja) マルチタスクスケジューリング方法、システム、アプリケーションサーバ及びコンピュータの読み取り可能な記憶媒体
WO2019085334A1 (fr) Procédé de stockage distribué, serveur de commande et support de stockage lisible par ordinateur
US8060892B2 (en) Executing business logic extensions on a client computing system
WO2019061991A1 (fr) Procédé de modélisation de plateforme de modèle universel à éléments multiples, dispositif électronique, et support d'informations lisible par ordinateur
WO2019028992A1 (fr) Procédé de construction de relation de dépendance de version multi-module, dispositif, serveur et support de stockage
WO2017143844A1 (fr) Procédé d'opération de données et serveur de gestion de données
WO2019134340A1 (fr) Procédé de calcul scalaire, serveur d'application et support de stockage lisible par ordinateur
US20170052884A1 (en) Generic test automation for restful web services applications
US10671376B2 (en) Server program hot upgrading method and device
CN106874281B (zh) 实现数据库读写分离的方法和装置
CN112286664B (zh) 任务调度方法、装置、计算机设备及可读存储介质
US9380001B2 (en) Deploying and modifying a service-oriented architecture deployment environment model
US10033737B2 (en) System and method for cross-cloud identity matching
WO2016078326A1 (fr) Procédé, appareil, et système, d'affichage de noms de machine virtuelle
CN109582386B (zh) 服务启动处理方法、装置、电子设备及可读存储介质
CN111431951B (zh) 一种数据处理方法、节点设备、系统及存储介质
CN110908644A (zh) 状态节点的配置方法、装置、计算机设备和存储介质
KR102156656B1 (ko) 가상머신 스택 생성 시스템 및 방법
JPWO2015190080A1 (ja) 関連情報作成装置、関連情報作成方法、及び、関連情報作成プログラム
WO2019205345A1 (fr) Procédé de synchronisation d'informations d'utilisateur, dispositif, ordinateur et support d'enregistrement
US20190035012A1 (en) Connector leasing for long-running software operations
CN112988806A (zh) 一种数据处理的方法及装置
CN113129049B (zh) 用于模型训练和应用的文件配置方法和系统
CN111125744A (zh) 代码分支合并方法、系统、计算机设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17915599

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 20.05.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17915599

Country of ref document: EP

Kind code of ref document: A1