WO2016138859A1 - Procédé de synchronisation de données et nœud de grappe - Google Patents

Procédé de synchronisation de données et nœud de grappe Download PDF

Info

Publication number
WO2016138859A1
WO2016138859A1 PCT/CN2016/075327 CN2016075327W WO2016138859A1 WO 2016138859 A1 WO2016138859 A1 WO 2016138859A1 CN 2016075327 W CN2016075327 W CN 2016075327W WO 2016138859 A1 WO2016138859 A1 WO 2016138859A1
Authority
WO
WIPO (PCT)
Prior art keywords
source data
cluster node
data
complete
generates
Prior art date
Application number
PCT/CN2016/075327
Other languages
English (en)
Chinese (zh)
Inventor
唐文奎
Original Assignee
北京京东尚科信息技术有限公司
北京京东世纪贸易有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东尚科信息技术有限公司, 北京京东世纪贸易有限公司 filed Critical 北京京东尚科信息技术有限公司
Publication of WO2016138859A1 publication Critical patent/WO2016138859A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/178—Techniques for file synchronisation in file systems

Definitions

  • Embodiments of the present invention relate to the field of computer technologies, and, more particularly, to a data synchronization method and a cluster node.
  • Impala is a new query system for distributed system infrastructure (such as Hadoop) that provides SQL (Structured Query Language) semantics.
  • SQL Structured Query Language
  • Impala can query PB-level big data stored in Hadoop's HDFS (Hadoop Distributed File System) with low latency. Decisions based on data analysis provide more efficient support.
  • the distributed system Hadoop uses Hive to manually import data in a file manner, and then judges the validity or integrity of the data. Therefore, not only the labor cost is large but also inefficient.
  • the embodiment of the invention provides a data synchronization method and a cluster node, which can effectively reduce labor costs and improve data synchronization efficiency.
  • a data synchronization method comprising: verifying integrity of the source data on a cluster node that generates source data; and if the source data is determined to be complete, acquiring the source Data; updating the data table according to the source data; wherein the source data is a complete process indicating that the cluster node that generates the source data has completed generating the source data, and the data table is used for recording The meta information of the source data.
  • the generating source data Verifying the integrity of the source data on the cluster node includes determining that the source data is complete when polling the cluster node that generates the source data to obtain a complete tag; or when receiving Determining that the source data is complete when the complete tag submitted by the cluster node of the source data is generated; wherein the complete tag is generated by the cluster node that generates the source data after the source data is generated Produced later.
  • the verifying the integrity of the source data on the cluster node that generates the source data includes: When the size of the source data does not change within a predetermined time range, it is determined that the source data is complete.
  • determining that the source data is complete includes: detecting the size of the source data multiple times, each time interval,
  • the predetermined time range refers to a time interval between the first time and the last time detecting the size of the source data; if the size of the source data detected multiple times is consistent, indicating that the size of the source data is predetermined There is no change in the time range, and the source data is determined to be complete.
  • the method before the updating the data table according to the source data, the method further includes: The source data is filtered to filter out invalid data.
  • the method further includes: partitioning to create the source data a table structure that records a mapping relationship from the cluster node that generates the source data to a storage location on a target cluster node, where the target cluster node refers to a cluster node that acquires the source data.
  • the data table adopts a full scale, a time partition table or a zipper table. form.
  • a data synchronization method comprising: generating source data;
  • the target cluster node After the target cluster node determines that the source data is complete, uploading the source data to the target cluster node, so that the target cluster node updates the data table according to the source data; wherein the source data is complete Representing that the cluster node that generated the source data has completed the process of generating the source data, the data table is used to record meta information of the source data.
  • the method before the source data is uploaded to the target cluster node, the method further includes: after the source data is generated, generating a complete mark, the complete A flag is used to indicate that the source data is complete; the complete tag is submitted to the target cluster node.
  • a data synchronization cluster node includes: a verification module, configured to verify integrity of the source data on a cluster node that generates source data; and acquire a module for The verification module determines that the source data is complete, and acquires the source data; and the update module is configured to update the data table according to the source data acquired by the obtaining module; wherein the source data is a complete representation
  • the cluster node that generates the source data has completed the process of generating the source data, and the data table is used to record meta information of the source data.
  • the verification module is specifically configured to determine the source data when a complete tag is obtained by polling the cluster node that generates the source data
  • the verification module is specifically configured to: when receiving the complete tag submitted by the cluster node that generates the source data, determine that the source data is complete; wherein the complete tag is generated by a The cluster node of the source data is generated after the source data generation is completed.
  • the verification module is specifically configured to: when the size of the source data does not change within a predetermined time range, determine that the source data is complete.
  • the verification module is specifically configured to: detect the size of the source data multiple times, each time being at a certain interval Time, if the size of the source data detected multiple times is consistent, indicating that the size of the source data has not changed within a predetermined time range, determining that the source data is complete, wherein the predetermined The time range refers to the time interval between the first and last detection of the size of the source data.
  • the updating module is further configured to: filter the source data to filter out invalid data.
  • the cluster node further includes: a creating module, configured to create a table structure of the source data, The table structure records a mapping relationship from the cluster node that generates the source data to a storage location on a target cluster node, where the target cluster node refers to a cluster node that acquires the source data.
  • a fourth aspect provides a cluster node for generating source data, where the cluster node includes: a generating module, configured to generate source data; and an uploading module, configured to: after the target cluster node determines that the source data is complete, The target cluster node uploads the source data generated by the generating module, so that the target cluster node updates the data table according to the source data; wherein the source data is a complete representation indicating that the source data is generated.
  • the cluster node has completed the process of generating the source data, and the data table is used to record meta information of the source data.
  • the generating module is further configured to: after the source data is generated, generate a complete tag, where the complete tag is used to indicate the The source data is complete; the uploading module is further configured to: submit the complete tag generated by the generating module to the target cluster node.
  • the source data integrity is verified by using the cluster node that generates the source data.
  • the source data is directly obtained from the cluster node that generates the source data, and correspondingly
  • the data table in which the data element information is recorded is updated to implement data synchronization. Therefore, it is not necessary to manually import the source data by means of Hive to complete the judgment of data integrity and validity, thereby reducing the labor cost and improving the efficiency of data synchronization.
  • FIG. 1 is a schematic flow chart of a process of a data synchronization method according to an embodiment of the present invention.
  • FIG. 2 is a schematic flow chart of a process of a data synchronization method according to an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of a data synchronization cluster node of the present invention.
  • FIG. 4 is a schematic structural diagram of a cluster node for generating source data according to the present invention.
  • FIG. 5 is a schematic structural diagram of another data synchronization cluster node of the present invention.
  • FIG. 6 is a schematic structural diagram of another cluster node that generates source data according to the present invention.
  • FIG. 1 is a schematic flow chart of a process of a data synchronization method according to an embodiment of the present invention.
  • the method of Figure 1 can be performed by a cluster node of data synchronization, including:
  • Step 101 Verify the integrity of the source data on the cluster node that generates the source data.
  • the verification of the integrity of the source data is to determine whether the cluster node that generates the source data has completed the process of generating the source data, or can be considered as a judgment on whether the source data is missing.
  • Step 102 If it is determined that the source data is complete, the source data is obtained.
  • the source data is a complete process that indicates that the cluster node that generated the source data has completed generating the source data.
  • Step 103 Update the data table according to the source data.
  • the data table is used to record meta information of the source data (also referred to as "metadata", which represents data describing the source data).
  • the source data integrity is verified by using the cluster node that generates the source data.
  • the source data is directly obtained from the cluster node that generates the source data, and correspondingly
  • the data table in which the data element information is recorded is updated to implement data synchronization. Therefore, it is not necessary to manually import the source data by means of Hive to complete the judgment of data integrity and validity, thereby reducing the labor cost and improving the efficiency of data synchronization.
  • the cluster node that generates the source data may generate a complete tag after completing the process of generating the source data, where the complete tag is used to indicate the source data. Is complete, can be polled by the cluster node that generates the source data, when polling the complete tag, it is determined that the source data is finished Integral, actively perform data acquisition.
  • the integrity of the source data may be verified by whether the size of the source data changes within a predetermined time range.
  • the cluster node that acquires the source data ie, the target cluster node
  • the predetermined time range here can be expressed as the time interval between the first and last detection of the source data. If the size of the source data detected multiple times is consistent, it indicates that the size of the source data has not changed within a predetermined time range, and it is determined that the source data is complete, and the data acquisition function can be performed.
  • the validity of the source data may be verified on the cluster node that generates the source data.
  • the validity of the source data may be verified after the source data is acquired.
  • the source data may be filtered according to a compression format to filter out invalid data. For example, verify source data (data files), exclude files that do not conform to the specified compression format, such as non-data files such as log (log files) generated by task scheduling.
  • the source data can be obtained by connecting to the Hive of the target cluster through JDBC (Java Data Base Connectivity) (for example, using the data copy function of HDFS).
  • JDBC Java Data Base Connectivity
  • the table structure of the source data may be partitioned, and the table structure records the local (that is, the cluster node that generates the source data) to the node of the target cluster (the cluster node that obtains the source data in the present invention, The mapping relationship of the storage locations on the cluster node as the method of FIG. 1 is performed.
  • the data table is in the form of a full scale table, a time partition table or a zipper table.
  • the data table can be All data recorded in the data table is updated, that is, it is generated in full; when the data table is represented by time partition, the data table can be updated according to a predetermined rule (for example, in units of days, weeks, or months); when the data table is a zipper table
  • the data table can be fully updated or some of the data in the data table can be updated (such as overwriting or appending). It should be understood that the embodiment of the present invention does not limit the form of the data table and the manner of updating the data table.
  • FIG. 2 is a schematic flow chart of a process of a data synchronization method according to an embodiment of the present invention.
  • the method of FIG. 2 may be performed by a cluster node (also referred to as a source data generator) and corresponds to the method of FIG. 1, and thus the description overlapping with the embodiment of FIG. 1 will be omitted as appropriate.
  • the method includes:
  • step 201 source data is generated.
  • Step 202 After the target cluster node determines that the source data is complete, upload the source data to the target cluster node, so that the target cluster node updates the data table according to the source data.
  • the source data is a complete process indicating that the cluster node generating the source data has completed generating the source data, and the data table is used to record the meta information of the source data.
  • the target cluster node determines that the source data is complete
  • the generated source data is uploaded to the target cluster node
  • the target cluster node updates the data table of the recorded data meta information to implement data synchronization. Therefore, it is not necessary to manually import the source data by means of Hive to complete the judgment of data integrity and validity, thereby reducing the labor cost and improving the efficiency of data synchronization.
  • the source data may be generated.
  • a complete tag is generated that indicates that the source data is complete; a complete tag is submitted to the target cluster node.
  • the target cluster node can know that the source data is complete through complete markup, and can actively obtain the source data from the cluster node that generates the source data (the data copy function of HDFS can be used).
  • the data synchronization cluster node 300 includes a verification module 301, an acquisition module 302, and an update module 303.
  • the verification module 301 is configured to verify the integrity of the source data on the cluster node that generates the source data;
  • the obtaining module 302 is configured to obtain source data if the verification module 301 determines that the source data is complete;
  • the update module 303 is configured to update the data table according to the source data acquired by the obtaining module 302.
  • the source data is a complete process indicating that the cluster node generating the source data has completed generating the source data, and the data table is used to record the meta information of the source data.
  • the source data integrity is verified by using the cluster node that generates the source data.
  • the source data is directly obtained from the cluster node that generates the source data, and correspondingly
  • the data table in which the data element information is recorded is updated to implement data synchronization. Therefore, it is not necessary to manually import the source data by means of Hive to complete the judgment of data integrity and validity, thereby reducing the labor cost and improving the efficiency of data synchronization.
  • the data synchronization cluster node 300 can implement the operations related to the cluster node in the foregoing embodiment, and therefore, in order to avoid repetition, details are not described in detail.
  • the verification module 301 may be specifically configured to: when the cluster node that generates the source data is polled, obtain the complete tag, determine that the source data is complete; or the verification module 301 may specifically use On: when receiving the generated source data When the complete tag submitted by the cluster node determines that the source data is complete; the complete tag is generated by the cluster node that generated the source data after the source data is generated.
  • the verification module 301 may be specifically configured to: when the size of the source data does not change within a predetermined time range, determine that the source data is complete.
  • the verification module 301 may be specifically configured to: detect the size of the source data multiple times, each time interval, if the size of the source data detected multiple times is consistent, indicating that the size of the source data is predetermined. There is no change in the time range, and the source data is determined to be complete, wherein the predetermined time range refers to the time interval between the first and last detection of the size of the source data.
  • the update module 303 is further configured to: filter the source data to filter out invalid data.
  • the cluster node 300 may further include a creating module 304.
  • the creating module 304 is configured to create a table structure of the source data, and the table structure records the mapping relationship from the cluster node that generates the source data to the storage location on the target cluster node, where the target cluster node refers to the cluster node that obtains the source data.
  • the cluster node 400 that generates the source data includes a generation module 401 and an upload module 402.
  • the generating module 401 is configured to generate source data.
  • the uploading module 402 is configured to: after the target cluster node determines that the source data is complete, upload the source data generated by the generating module 401 to the target cluster node, so that the target cluster node updates the data table according to the source data;
  • the cluster node that generated the source data has completed the number of generated sources.
  • the data table is used to record the meta information of the source data.
  • the target cluster node determines that the source data is complete
  • the generated source data is uploaded to the target cluster node
  • the target cluster node updates the data table of the recorded data meta information to implement data synchronization. Therefore, it is not necessary to manually import the source data by means of Hive to complete the judgment of data integrity and validity, thereby reducing the labor cost and improving the efficiency of data synchronization.
  • the cluster node 400 that generates the source data can implement the operations related to the cluster node in the above embodiment, and therefore, in order to avoid redundancy, detailed description will not be given.
  • the generating module 401 is further configured to: after the source data is generated, generate a complete tag, where the complete tag is used to indicate that the source data is complete; and the uploading module 402 is further configured to: target The cluster node submits the full tag.
  • the target cluster node can know that the source data is complete through complete markup, and can actively obtain the source data from the cluster node that generates the source data (the data copy function of HDFS can be used).
  • Node 500 includes a processor 501, a memory 502, and a transceiver 503.
  • the processor 501 controls the operation of the device 500.
  • Memory 502 can include read only memory and random access memory and provides instructions and data to processor 501.
  • a portion of the memory 502 may also include non-volatile line random access memory (NVRAM).
  • the processor 501, the memory 502, and the transceiver 503 are coupled together by a bus system 510.
  • the bus system 510 includes a power bus, a control bus, and a status signal bus in addition to the data bus. However, for clarity of description, various buses are labeled as bus system 510 in the figure.
  • the processor 501 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 501 or an instruction in a form of software.
  • the processor 501 can be a general purpose processor, including a CPU (Central Processing Unit), NP (Network) Processor, Network Processor, etc.; can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present invention may be implemented or carried out.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • Node 600 includes a processor 601, a memory 602, and a transceiver 603.
  • the processor 601 controls the operation of the device 600.
  • each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 601 or an instruction in a form of software.
  • FIG. 6 For the function of the device shown in FIG. 6, reference may be made to the description of FIG. 5, and details are not described herein again.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the units/modules is only one logical function division, and may be further divided in actual implementation, such as multiple units/modules or Components can be combined or integrated into another system, or some features can be ignored or not executed.
  • each functional unit/module in various embodiments of the present invention may be integrated in one In the processing unit, each unit/module may exist physically separately, or two or more units/modules may be integrated into one unit/module.
  • the functions, if implemented in the form of software functions and sold or used as separate products, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including
  • the instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a disk or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

L'invention concerne un procédé de synchronisation de données et un nœud de grappe. Le procédé comporte les étapes consistant à: confirmer, sur un nœud de grappe générant des données source, l'intégrité des données source (101); si l'intégrité des données source est assurée, obtenir les données source, l'intégrité des données source indiquant que le nœud de grappe générant les données source a achevé un processus de génération des données source (102); et mettre à jour une table de données d'après les données source, la table de données étant utilisée pour enregistrer des méta-informations des données source (103). En confirmant l'intégrité des données source au niveau du nœud de grappe générant les données source, les données peuvent être synchronisées lorsque l'intégrité des données source est assurée. Ainsi, l'intégrité et la validité des données peuvent être déterminées sans introduire manuellement les données source dans un système Hive, réduisant ainsi les coûts de main-d'œuvre et améliorant un rendement de synchronisation des données.
PCT/CN2016/075327 2015-03-02 2016-03-02 Procédé de synchronisation de données et nœud de grappe WO2016138859A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510092950.6 2015-03-02
CN201510092950.6A CN104699771B (zh) 2015-03-02 2015-03-02 数据同步方法和集群节点

Publications (1)

Publication Number Publication Date
WO2016138859A1 true WO2016138859A1 (fr) 2016-09-09

Family

ID=53346892

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/075327 WO2016138859A1 (fr) 2015-03-02 2016-03-02 Procédé de synchronisation de données et nœud de grappe

Country Status (2)

Country Link
CN (1) CN104699771B (fr)
WO (1) WO2016138859A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107818106A (zh) * 2016-09-13 2018-03-20 腾讯科技（深圳）有限公司 一种大数据离线计算数据质量校验方法和装置

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104699771B (zh) * 2015-03-02 2019-09-20 北京京东尚科信息技术有限公司 数据同步方法和集群节点
CN106708835A (zh) * 2015-08-11 2017-05-24 阿里巴巴集团控股有限公司 一种数据表的分类方法和装置
CN105069128B (zh) * 2015-08-14 2018-11-09 北京京东尚科信息技术有限公司 数据同步方法和装置
CN105447172A (zh) * 2015-12-07 2016-03-30 北京先进数通信息技术股份公司 一种Hadoop平台下的数据处理方法和系统
CN105554135B (zh) * 2015-12-23 2019-03-05 北京奇虎科技有限公司 数据节点间同步元信息的方法及装置
CN108874825B (zh) * 2017-05-12 2021-11-02 北京京东尚科信息技术有限公司 一种异常数据的校验方法和装置
CN112765135B (zh) * 2021-01-29 2023-12-05 北京达佳互联信息技术有限公司 数据处理方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101183993A (zh) * 2007-12-21 2008-05-21 亿阳信通股份有限公司 一种网络管理系统和性能数据处理方法
US20090271653A1 (en) * 2006-04-10 2009-10-29 Huawei Technologies Co., Ltd. Method and system for data synchronization
CN102355497A (zh) * 2011-09-28 2012-02-15 中国铁道科学研究院电子计算技术研究所 数据同步方法
CN102682096A (zh) * 2012-04-27 2012-09-19 北京航空航天大学 一种仿真资源信息与模型源码协同管理装置及其方法
CN104699771A (zh) * 2015-03-02 2015-06-10 北京京东尚科信息技术有限公司 数据同步方法和集群节点

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101207492A (zh) * 2006-12-21 2008-06-25 中兴通讯股份有限公司 避免下载错误文件的文件下载方法及装置
CN101499011A (zh) * 2008-01-29 2009-08-05 中兴通讯股份有限公司 实现移动通信基站系统usb启动和升级的方法及装置
CN101334797B (zh) * 2008-08-04 2010-06-02 中兴通讯股份有限公司 一种分布式文件系统及其数据块一致性管理的方法
US20100162230A1 (en) * 2008-12-24 2010-06-24 Yahoo! Inc. Distributed computing system for large-scale data handling
CN102065137B (zh) * 2010-12-21 2013-08-14 山东浪潮齐鲁软件产业股份有限公司 一种网络数据库表数据应用程序同步的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090271653A1 (en) * 2006-04-10 2009-10-29 Huawei Technologies Co., Ltd. Method and system for data synchronization
CN101183993A (zh) * 2007-12-21 2008-05-21 亿阳信通股份有限公司 一种网络管理系统和性能数据处理方法
CN102355497A (zh) * 2011-09-28 2012-02-15 中国铁道科学研究院电子计算技术研究所 数据同步方法
CN102682096A (zh) * 2012-04-27 2012-09-19 北京航空航天大学 一种仿真资源信息与模型源码协同管理装置及其方法
CN104699771A (zh) * 2015-03-02 2015-06-10 北京京东尚科信息技术有限公司 数据同步方法和集群节点

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107818106A (zh) * 2016-09-13 2018-03-20 腾讯科技（深圳）有限公司 一种大数据离线计算数据质量校验方法和装置

Also Published As

Publication number Publication date
CN104699771A (zh) 2015-06-10
CN104699771B (zh) 2019-09-20

Similar Documents

Publication Publication Date Title
WO2016138859A1 (fr) Procédé de synchronisation de données et nœud de grappe
US20200412526A1 (en) Method and apparatus for verifying smart contracts in blockchain, and storage medium
US8260742B2 (en) Data synchronization and consistency across distributed repositories
US8938430B2 (en) Intelligent data archiving
KR101780340B1 (ko) 가상 데이터베이스를 제공하기 위한 유효성 검증 데이터베이스 스냅샷의 생성 방법, 시스템 및 컴퓨터 판독가능 저장 매체
US10140351B2 (en) Method and apparatus for processing database data in distributed database system
US9600393B2 (en) Testing of application service versions on live data
US8977646B2 (en) Leveraging graph databases in a federated database system
US9876696B2 (en) Method and system for processing browser crash information
WO2017080431A1 (fr) Procédé et dispositif de réplication de base de données fondés sur une analyse de journal
US9904722B1 (en) Log-based distributed transaction management
WO2023050705A1 (fr) Procédé et appareil de gestion de données de surveillance, dispositif électronique et support de stockage
US10747776B2 (en) Replication control using eventually consistent meta-data
CN111026568B (zh) 数据和任务关系构建方法、装置、计算机设备及存储介质
WO2018176812A1 (fr) Procédé et dispositif d'émission de ressources statiques
WO2017028690A1 (fr) Procédé et système de traitement de fichiers basés sur l'etl
GB2500085A (en) Determining whether a standby database is synchronized with a primary database
US10423495B1 (en) Deduplication grouping
WO2021078139A1 (fr) Système hybride sur chaîne de blocs et son procédé de gestion de documents de construction
CN107491558B (zh) 元数据更新方法及装置
WO2016101759A1 (fr) Procédé d'acheminement de données, dispositif de gestion de données et système de stockage distribué
TW201335779A (zh) 檔案同步系統及方法
EP3495962A1 (fr) Systèmes et procédés d'utilisation de documents liés
JP2018514886A (ja) 多数の重複するソースからの大量の時間データの集計
US20210397599A1 (en) Techniques for generating a consistent view of an eventually consistent database

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16758474

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 14/12/2017)

122 Ep: pct application non-entry in european phase

Ref document number: 16758474

Country of ref document: EP

Kind code of ref document: A1