CN112765275A - Data synchronous exchange method, equipment and medium based on single time coordinate system - Google Patents
Data synchronous exchange method, equipment and medium based on single time coordinate system Download PDFInfo
- Publication number
- CN112765275A CN112765275A CN202110105257.3A CN202110105257A CN112765275A CN 112765275 A CN112765275 A CN 112765275A CN 202110105257 A CN202110105257 A CN 202110105257A CN 112765275 A CN112765275 A CN 112765275A
- Authority
- CN
- China
- Prior art keywords
- data
- time
- change
- time stamp
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Abstract
The present disclosure relates to the field of data synchronization technologies, and more particularly, to a method, a device, and a medium for data synchronization exchange based on a single time coordinate system, where the method includes: first, a time stamp t is initialized0(ii) a Second, obtaining the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source; thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum; the fourth step is to get t0Is updated to tmaxAnd returning to the second step. The method disclosed by the invention adopts a single time coordinate system, does not generate a plurality of time sources, ensures the uniqueness of the time stamp source, and can effectively solve the problem of the uniqueness of the time stamp sourceThe time of the data source side is inconsistent with that of the data target side, and the data is lost and the data exchange is failed.
Description
Technical Field
The present disclosure relates to the field of data synchronization technologies, and in particular, to a method, device, and medium for data synchronization exchange based on a single time coordinate system.
Background
The technology of data synchronization has been widely used and is essential in the construction of information systems. Data synchronization generally refers to the transfer of data from source data to destination data in an information system. The data synchronization technology can be used for data exchange of modules in an information system, data transmission between systems and integration of project implementation and an existing system.
There are two main ways of data synchronization that are commonly used: the first is full synchronization, which means that the full amount of data in the data table is synchronized to the target source at one time each time, and the full synchronization has the advantages of simple execution, but large data amount and poor performance each time. The second is incremental synchronization, which means that only the changed incremental part data is acquired when data is synchronized each time, and has the advantages that only the required data needs to be synchronized each time, the performance is better, and the defect is that the processing complexity is higher when the multi-target source is targeted.
In the traditional timestamp increment synchronization mode, after the time server completes one synchronization, the local time t is usedATo update t0As a time credential for the next incremental synchronization. And time t when data source serverBAnd synchronizing server time tAWhen there is a time difference (t)A＝tB+ x), data update time stamp t of data source serverchangeIs t isBAs a reference. When t is acquired on the data source serverchange>＝t0T is inevitably lostBTo tBData updated within + x time.
The present application therefore proposes an improved method to at least partially solve the above technical problem.
Disclosure of Invention
The scheme is provided for solving the technical problems of inconsistent time of a data source party and a data target party, data loss and data exchange failure in the prior art.
In order to achieve the technical purpose, the present disclosure provides a data synchronous exchange method based on a single time coordinate system, including the following steps:
first, a time stamp t is initialized0；
Second, obtaining the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source;
thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum;
the fourth step is to get t0Is updated to tmaxAnd returning to the second step to trigger periodically.
In particular, the initialization timestamp t0At first synchronization, time stamp t0Set to null.
Specifically, the time stamp t in the data source is obtainedchange>＝t0Data d oftimeThe method comprises the following steps:
sending a request signal to a data source;
receiving data returned by data sourcetime。
Specifically, the sending of the request signal to the data source is to determine whether the data source has the changed timestamp t by the synchronization systemchangeGreater than t0If the condition is satisfied, the next operation is performed.
Further, the data d are obtained through a preset rule function fntimeChange time stamp t in (1)maxSpecifically, the time stamp t is firstchangeSorting, and using the time with the maximum absolute value as the change time stamp tmax。
Preferably, the target data source may use the data source as the base data.
The present disclosure provides a data synchronous exchange method based on a single time coordinate system, which is applied to a data source, and comprises the following steps:
the first step, receiving a request signal sent by a synchronous system;
second, all time stamps t are queriedchange>＝t0And as incremental change data dtime；
Thirdly, the increment is changed into data dtimeAnd sending back to the synchronization system.
Specifically, a time-type timestamp data column is set in the data source, the default value of the time-type timestamp data column is the current system time, and the time-type timestamp data column is not allowed to be empty.
Further, the data source updates the data row timestamp t simultaneously when updating the datachange＝tnow。
Preferably, the data source is a database server.
In the present disclosure, the term "single time coordinate system" means that the time coordinate is the same as the data change time coordinate of the data source, and is based on the time of the same server.
The present disclosure also provides a computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of: first, a time stamp t is initialized0(ii) a Second, obtaining the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source; thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum; the fourth step is to get t0Is updated to tmaxAnd returning to the second step to trigger periodically.
The present disclosure also provides a storage medium having stored thereon computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of: first, a time stamp t is initialized0(ii) a Second, get the time in the data sourceTimestamp tchange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source; thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum; the fourth step is to get t0Is updated to tmaxAnd returning to the second step to trigger periodically.
The beneficial effect of this disclosure does:
the method adopts a single time coordinate system, does not generate a plurality of time sources, ensures the uniqueness of the time stamp source, and can effectively solve the problems of inconsistent time, data loss and data exchange failure of a data source party and a data target party.
Drawings
Fig. 1 shows a schematic flow diagram of embodiment 1 of the present disclosure;
fig. 2 shows a schematic flow diagram of embodiment 2 of the present disclosure;
fig. 3 shows a flow diagram of embodiment 3 of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure. It will be apparent to one skilled in the art that the present disclosure may be practiced without one or more of these details. In other instances, well-known features of the art have not been described in order to avoid obscuring the present disclosure.
It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the disclosure. As used herein, the singular is intended to include the plural unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Exemplary embodiments according to the present disclosure will now be described in more detail with reference to the accompanying drawings. These exemplary embodiments may, however, be embodied in many different forms and should not be construed as limited to only the embodiments set forth herein. The figures are not drawn to scale, wherein certain details may be exaggerated and omitted for clarity. The shapes of various regions, layers, and relative sizes and positional relationships therebetween shown in the drawings are merely exemplary, and deviations may occur in practice due to manufacturing tolerances or technical limitations, and a person skilled in the art may additionally design regions/layers having different shapes, sizes, relative positions, as actually required.
Example 1:
the present disclosure provides a data synchronous exchange method based on a single time coordinate system, as shown in fig. 1, the steps are as follows:
first, a time stamp t is initialized0；
Second, obtaining the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source;
thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum;
the fourth step is to get t0Is updated to tmaxAnd returning to the second step to trigger periodically.
Specifically, theThe initialization time stamp t0At first synchronization, time stamp t0Set to null.
Specifically, the time stamp t in the data source is obtainedchange>＝t0Data d oftimeThe method comprises the following steps:
sending a request signal to a data source;
receiving data returned by data sourcetime。
Specifically, the sending of the request signal to the data source is to determine whether the data source has the changed timestamp t by the synchronization systemchangeGreater than t0If the condition is satisfied, the next operation is performed.
Further, the data d are obtained through a preset rule function fntimeChange time stamp t in (1)maxSpecifically, the time stamp t is firstchangeSorting, and using the time with the maximum absolute value as the change time stamp tmax。
Preferably, the target data source may use the data source as basic data, the data source may select a certain database server, and the target data source may be a client application program; the data source may be a table, and the target data source may be a data object based on the table by means of an index, a trigger, a view, a stored procedure, and the like of the table.
As an alternative implementation, this embodiment may comprise a calculation module for determining the timestamp tchangeWhether t is greater than or equal to0And meanwhile, the method is also used for calculating the preset rule function.
As a convertible implementation, the present embodiment may further include a write module, configured to synchronously transmit the changed data in the data source to the target data source.
Example 2:
the present disclosure provides a data synchronous exchange method based on a single time coordinate system, which is applied to a data source, as shown in fig. 2, and includes the following steps:
the first step, receiving a request signal sent by a synchronous system;
second, all time stamps t are queriedchange>＝t0And as incremental change data dtime；
Thirdly, the increment is changed into data dtimeAnd sending back to the synchronization system.
Specifically, a time-type timestamp data column is set in the data source, the default value of the time-type timestamp data column is the current system time, and the time-type timestamp data column is not allowed to be empty.
Further, the data source updates the data row timestamp t simultaneously when updating the datachange＝tnow。
Preferably, the data source is a database server, such as MSSQL or Oracle.
Preferably, the data source may be provided with a query module, an update module, and the like.
Example 3:
the present disclosure provides a data synchronous exchange method based on a single time coordinate system, as shown in fig. 3, including a data source and a synchronization system, the steps are as follows:
synchronization system initialization timestamp t0First time of synchronization, time stamp t0Setting to be null;
the synchronous system accesses the data source and judges tchangeWhether t is greater than or equal to0；
tchange>＝t0Time, data source returns time stamp t to synchronous systemchange>＝t0Data d oftimeThe synchronization system will synchronize said data dtimeWriting a target data source;
the synchronous system obtains the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum;
the synchronous system will t0Is updated to tmaxContinuously accessing the data source to obtain the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting to the target data source to form the periodic policy trigger.
Wherein the data row time stamp t is updated simultaneously when the data source updates the datachange＝tnow。
Additionally, in one embodiment, a computer device is presented, the computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions that, when executed by the processor, perform the steps of: first, a time stamp t is initialized0(ii) a Second, obtaining the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source; thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum; the fourth step is to get t0Is updated to tmaxAnd returning to the second step to trigger periodically.
In addition, in one embodiment, a storage medium is presented having computer-readable instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform the steps of: first, a time stamp t is initialized0(ii) a Second, obtaining the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source; thirdly, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum; the fourth step is to get t0Is updated to tmaxAnd returning to the second step to trigger periodically.
The embodiments of the present disclosure have been described above. However, this is for illustrative purposes only and is not intended to limit the scope of the present disclosure. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (10)
1. A data synchronous exchange method based on a single time coordinate system is characterized by comprising the following steps:
s1, initializing time stamp t0；
S2, acquiring the time stamp t in the data sourcechange>＝t0Data d oftimeThe data d is processedtimeWriting a target data source;
s3, acquiring the data d through a preset rule function fntimeChange time stamp t in (1)maxWherein the change time stamp tmaxRepresents the data dtimeMiddle time stamp tchangeThe time when the absolute value of the middle value is maximum;
s4, mixing t0Is updated to tmaxReturning to step S2.
2. The method of claim 1, wherein the initialization time stamp t0At first synchronization, time stamp t0Set to null.
3. The method of claim 1, wherein obtaining the timestamp t in the data sourcechange>＝t0Data d oftimeThe method comprises the following steps:
sending a request signal to a data source;
receiving data returned by data sourcetime。
4. Method according to claim 3, wherein said sending of a request signal to the data source, in particular the synchronization system, determines whether the data source has a changed timestamp tchangeGreater than t0If the condition is satisfied, the next operation is performed.
5. Method according to claim 1, characterized in that said data d are obtained by a predetermined regular function fntimeChange time stamp t in (1)maxSpecifically, the time stamp t is firstchangeSorting, and using the time with the maximum absolute value as the change time stamp tmax。
6. A data synchronous exchange method based on a single time coordinate system is applied to a data source and is characterized by comprising the following steps:
s1, receiving a request signal sent by the synchronous system;
s2, inquiring all time stamps tchange>＝t0And as incremental change data dtime；
S3, changing the increment into data dtimeAnd sending back to the synchronization system.
7. The method according to claim 6, wherein the data source is provided with a time type time stamp data column, the default value of which is the current system time and is not allowed to be empty.
8. The method of claim 6, wherein the data source updates the data row timestamp t while updating the datachange＝tnow。
9. A computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions which, when executed by the processor, cause the processor to carry out the steps of the single time coordinate system based data synchronous exchange method according to any one of claims 1 to 5.
10. A storage medium storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the method for synchronous exchange of data based on a single time coordinate system as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110105257.3A CN112765275A (en) | 2021-01-26 | 2021-01-26 | Data synchronous exchange method, equipment and medium based on single time coordinate system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110105257.3A CN112765275A (en) | 2021-01-26 | 2021-01-26 | Data synchronous exchange method, equipment and medium based on single time coordinate system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112765275A true CN112765275A (en) | 2021-05-07 |
Family
ID=75705825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110105257.3A Pending CN112765275A (en) | 2021-01-26 | 2021-01-26 | Data synchronous exchange method, equipment and medium based on single time coordinate system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112765275A (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004046302A (en) * | 2002-07-08 | 2004-02-12 | Nec Corp | Data synchronization method between two or more pieces of information apparatus, information processor which performs this method and program for performing this method |
| US20070016626A1 (en) * | 2005-07-12 | 2007-01-18 | International Business Machines Corporation | Ranging scalable time stamp data synchronization |
| CN101030157A (en) * | 2007-04-20 | 2007-09-05 | 北京搜狗科技发展有限公司 | Method and system for updating user vocabulary synchronouslly |
| CN105450718A (en) * | 2014-09-30 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Interface updating method and client end |
| CN105740361A (en) * | 2016-01-26 | 2016-07-06 | 上海晶赞科技发展有限公司 | Detection method and apparatus for integrity of total data |
| CN106844689A (en) * | 2017-01-23 | 2017-06-13 | 北京皮尔布莱尼软件有限公司 | A kind of paged data synchronous method, apparatus and system |
| CN107741994A (en) * | 2017-11-09 | 2018-02-27 | 校脸科技（北京）有限公司 | A kind of data-updating method and device |
| GB201816808D0 (en) * | 2018-10-16 | 2018-11-28 | Palantir Technologies Inc | Data storage method and system |
| CA3045675A1 (en) * | 2018-06-07 | 2019-12-07 | Alexander Sheung Lai Wong | System and method for decentralized digital structured data storage, management, and authentication using blockchain |
| CN110633323A (en) * | 2019-09-16 | 2019-12-31 | 腾讯科技（深圳）有限公司 | Business data storage method, device, storage medium and computer equipment |
| CN110858197A (en) * | 2018-08-21 | 2020-03-03 | 北京京东尚科信息技术有限公司 | Method and device for synchronizing data |
| CN111245548A (en) * | 2020-01-14 | 2020-06-05 | 平安国际智慧城市科技股份有限公司 | Data synchronization method and device based on timestamp and computer equipment |
| CN111868707A (en) * | 2018-03-13 | 2020-10-30 | 谷歌有限责任公司 | Including transaction commit timestamps in a primary key of a relational database |
| CN112214547A (en) * | 2020-09-24 | 2021-01-12 | 百度国际科技（深圳）有限公司 | Data processing method, data server, electronic device and storage medium |
-
2021
- 2021-01-26 CN CN202110105257.3A patent/CN112765275A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004046302A (en) * | 2002-07-08 | 2004-02-12 | Nec Corp | Data synchronization method between two or more pieces of information apparatus, information processor which performs this method and program for performing this method |
| US20070016626A1 (en) * | 2005-07-12 | 2007-01-18 | International Business Machines Corporation | Ranging scalable time stamp data synchronization |
| CN101030157A (en) * | 2007-04-20 | 2007-09-05 | 北京搜狗科技发展有限公司 | Method and system for updating user vocabulary synchronouslly |
| CN105450718A (en) * | 2014-09-30 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Interface updating method and client end |
| CN105740361A (en) * | 2016-01-26 | 2016-07-06 | 上海晶赞科技发展有限公司 | Detection method and apparatus for integrity of total data |
| CN106844689A (en) * | 2017-01-23 | 2017-06-13 | 北京皮尔布莱尼软件有限公司 | A kind of paged data synchronous method, apparatus and system |
| CN107741994A (en) * | 2017-11-09 | 2018-02-27 | 校脸科技（北京）有限公司 | A kind of data-updating method and device |
| CN111868707A (en) * | 2018-03-13 | 2020-10-30 | 谷歌有限责任公司 | Including transaction commit timestamps in a primary key of a relational database |
| CA3045675A1 (en) * | 2018-06-07 | 2019-12-07 | Alexander Sheung Lai Wong | System and method for decentralized digital structured data storage, management, and authentication using blockchain |
| CN110858197A (en) * | 2018-08-21 | 2020-03-03 | 北京京东尚科信息技术有限公司 | Method and device for synchronizing data |
| GB201816808D0 (en) * | 2018-10-16 | 2018-11-28 | Palantir Technologies Inc | Data storage method and system |
| CN110633323A (en) * | 2019-09-16 | 2019-12-31 | 腾讯科技（深圳）有限公司 | Business data storage method, device, storage medium and computer equipment |
| CN111245548A (en) * | 2020-01-14 | 2020-06-05 | 平安国际智慧城市科技股份有限公司 | Data synchronization method and device based on timestamp and computer equipment |
| CN112214547A (en) * | 2020-09-24 | 2021-01-12 | 百度国际科技（深圳）有限公司 | Data processing method, data server, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107888657B (en) | Low latency distributed storage system | |
| CN103268318B (en) | A kind of distributed key value database system of strong consistency and reading/writing method thereof | |
| CN109379432B (en) | Data processing method, device, server and computer readable storage medium | |
| US11138061B2 (en) | Method and apparatus to neutralize replication error and retain primary and secondary synchronization during synchronous replication | |
| US11036423B2 (en) | Dynamic recycling algorithm to handle overlapping writes during synchronous replication of application workloads with large number of files | |
| US8671151B2 (en) | Maintaining item-to-node mapping information in a distributed system | |
| US11061924B2 (en) | Multi-region, multi-master replication of database tables | |
| US9307011B2 (en) | Synchronous mirroring of NVLog to multiple destinations (architecture level) | |
| US20140156586A1 (en) | Big-fast data connector between in-memory database system and data warehouse system | |
| CN103838770A (en) | Logic data partition method and system | |
| US20190347167A1 (en) | Primary Node-Standby Node Data Transmission Method, Control Node, and Database System | |
| CN103399894A (en) | Distributed transaction processing method on basis of shared storage pool | |
| CN112307119A (en) | Data synchronization method, device, equipment and storage medium | |
| CN105069152A (en) | Data processing method and apparatus | |
| CN105956032A (en) | Cache data synchronization method, system and apparatus | |
| CN111522811B (en) | Database processing method and device, storage medium and terminal | |
| JP6079876B2 (en) | Distributed processing system | |
| CN110543472B (en) | Data reconciliation method and related device | |
| CN109726211B (en) | Distributed time sequence database | |
| CN105302489A (en) | Heterogeneous multi-core remote embedded memory system and method | |
| JP4612714B2 (en) | Data processing method, cluster system, and data processing program | |
| CN112765275A (en) | Data synchronous exchange method, equipment and medium based on single time coordinate system | |
| CN109542872B (en) | Data reading method and device, computer equipment and storage medium | |
| US10970177B2 (en) | Methods and systems of managing consistency and availability tradeoffs in a real-time operational DBMS | |
| CN111428114A (en) | Index creating method and device for Elasticissearch search engine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |