CN116578647A - Data synchronization method, device, system and computer readable storage medium - Google Patents
Data synchronization method, device, system and computer readable storage medium Download PDFInfo
- Publication number
- CN116578647A CN116578647A CN202310618397.XA CN202310618397A CN116578647A CN 116578647 A CN116578647 A CN 116578647A CN 202310618397 A CN202310618397 A CN 202310618397A CN 116578647 A CN116578647 A CN 116578647A
- Authority
- CN
- China
- Prior art keywords
- merging
- language sentences
- data
- manipulation language
- data manipulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 34
- 230000008569 process Effects 0.000 claims abstract description 14
- 230000001105 regulatory effect Effects 0.000 claims abstract description 4
- 238000004590 computer program Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000008676 import Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 238000007790 scraping Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application relates to a data synchronization method, a device, a system and a computer readable storage medium in the technical field of big data processing, which comprises the following steps: delivering the data manipulation language sentences of the same list to the same queue, and acquiring two groups of similar data manipulation language sentences in the same queue; judging whether the two groups of data manipulation language sentences are inserted or deleted, if so, judging whether the two groups of data manipulation language sentences are of the same operation type; based on the judgment results of the same/different operation types, the merging/non-merging counting processing is carried out, and the opening and closing of a merging submitting switch are regulated according to the result of the counting processing, so that the problem of time delay in the existing batch submitting and non-batch submitting methods in the data synchronization process is solved.
Description
Technical Field
The application relates to the technical field of big data processing, in particular to a data synchronization method, a device, a system and a computer readable storage medium.
Background
In the real-time copying process of the database, the source database continuously generates new DML data, and each change statement needs to be delivered to the target for execution. If the source end generates a large amount of data change or data import, a large amount of logs can be generated in a short time, the duplicate link sends an execution statement to the target end, the performance is very slow, and the link delay is increased. Batch writing can be achieved by opening the batch, but the batch means that the number of batches waiting for the batch reaches or exceeds a threshold value to trigger commit, which in turn can lead to artificially increased latency of the replicated link for normal synchronization (non-batch change) scenarios.
The optimization of batch data change in a replication link is mainly to provide batch submission capability, when a source library generates batch data change to cause link delay, a user or a product side needs to open a batch submission switch through a background, which is a hysteresis processing mode, the link delay already occurs before opening, and a queue to-be-submitted statement is generated and cannot be recombined. When the batch statement is executed after opening, the delay of the copy link is artificially increased due to a batch scraping mechanism under the normal synchronous (non-batch change) scene. Between the piece-by-piece submission and batch collection, the batch submission is turned on or off by manual setting, and is not suitable for a dynamic production data scene.
Disclosure of Invention
Aiming at the defects in the prior art, the application provides a data synchronization method, a device, a system and a computer readable storage medium, which solve the problem of time delay in the existing batch submission and non-batch submission methods in the data synchronization process.
In order to solve the technical problems, the application is solved by the following technical scheme:
a data synchronization method, comprising the steps of:
delivering the data manipulation language sentences of the same list to the same queue, and acquiring two groups of similar data manipulation language sentences in the same queue;
judging whether the two groups of data manipulation language sentences are inserted or deleted, if so, judging whether the two groups of data manipulation language sentences are of the same operation type, otherwise, performing uncombined counting processing;
and carrying out merging/non-merging counting processing based on the judging results of the same/different operation types, and regulating the opening and closing of a merging submitting switch according to the counting processing result.
Optionally, the merge/not merge count process includes the steps of:
when the two groups of data manipulation language sentences are of the same operation type, the merging count is added with 1;
and when the two groups of data manipulation language sentences are of the type of no-use operation, processing of adding 1 to the merging count is not performed, and meanwhile, the merging count is cleared.
Optionally, adjusting the closing of the merge commit switch includes the steps of:
setting a starting merging threshold value, and judging whether the merging count is larger than the starting merging threshold value or not;
if yes, the merging submitting switch is opened, meanwhile, the merging counting zero clearing setting is not carried out, and otherwise, the merging submitting switch is not adjusted.
Optionally, adjusting the closing of the merge commit switch further comprises the steps of:
setting a closing merging threshold value, and judging whether the non-merging count is larger than the closing merging threshold value or not;
if yes, closing the merging submitting switch, and resetting the merging count, otherwise, not adjusting the merging submitting switch.
Optionally, the data manipulation language statements include insert, update, and delete operations.
Optionally, the method further comprises the following steps:
judging all the data manipulation language sentences according to the table names, and collecting the data manipulation language sentences with the same table names into the same list according to the sequence.
Optionally, when performing the merge count process, a plurality of adjacent and identical data manipulation language sentences in the same list are subjected to the merge process.
A data synchronization device for performing the data synchronization method according to any one of the above claims, comprising a source, a merge counter, a no merge counter, a merge commit switch, and a destination;
the source end is used for sending data manipulation language sentences;
the merging counter is used for carrying out merging counting processing according to the operation types of two groups of similar data operation language sentences;
the non-merging counter is used for performing non-merging counting processing according to the operation types of two groups of similar data operation language sentences;
the merging submitting switch is used for adjusting the opening and closing according to the numerical value of the merging counter/the numerical value of the non-merging counter;
the target end is used for receiving the data manipulation language statement sent by the source end.
A data synchronization system comprises a queue allocation unit, a type judgment unit and a merging and adjusting unit;
the queue allocation unit is used for delivering the data manipulation language sentences of the same list to the same queue and acquiring two similar groups of data manipulation language sentences in the same queue;
the type judging unit is used for judging whether the two groups of data manipulation language sentences are inserted or deleted, if yes, judging whether the two groups of data manipulation language sentences are of the same operation type, and otherwise, performing non-merging counting processing;
the merging and adjusting unit performs merging/non-merging counting processing based on the judgment results of the same/different operation types, and adjusts the opening and closing of the merging submitting switch according to the result of the counting processing.
A computer readable storage medium storing a computer program which, when executed by a processor, implements the data synchronization method of any one of the above.
Compared with the prior art, the technical scheme provided by the application has the following beneficial effects:
by designing two counting functions in the replication link, after logs are obtained from a source end and put into a queue, the next log is put into after counting is completed, so that when the starting merging threshold value is exceeded, the records can be merged and inserted into the queue, a merging submitting switch is automatically opened, and when the following inserted data exceeds the closing merging threshold value and cannot be merged, the merging submitting switch is automatically closed, and an automatic switching mechanism is realized; meanwhile, the automatic switching mechanism effectively avoids the problem of delay of emergency opening after data delay, and also avoids the increase of artificial delay caused by the fact that a batch-scraping threshold is not triggered in the normal synchronization process.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the application, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
Fig. 1 is a flowchart of a data synchronization method according to a first embodiment.
Detailed Description
The present application will be described in further detail with reference to the following examples, which are illustrative of the present application and are not intended to limit the present application thereto.
Example 1
As shown in fig. 1, a data synchronization method is used for judging whether a source end is batch change or not, and completing statement merging before copying a link to submit statement, so as to reduce network interaction between the link and a target library, and when the batch change statement is executed, automatically switching back to a normal statement-by-statement submitting mode, wherein the maximum of general relational data synchronization is DML (Data Manipulation Language ) statements, which are divided into three operations of insert (insert), update (delete) and delete (delete), and the business may randomly generate the three operations, or may generate batch data change under the situations of data import, batch data correction, historical data deletion and the like, wherein the source end generates batch data import or historical data batch deletion operation, and the periodic import or delete data is a main scene of batch data change.
Further, firstly judging all data manipulation language sentences according to table names, tracking the data manipulation language sentences with the same table names to the same List according to the sequence, then delivering the data manipulation language sentences of the same List to the same queue, acquiring two similar groups of data manipulation language sentences in the same queue, and particularly, before copying any DML, scattering the DML into a List named as 'table name and table name' according to the table names, wherein the DML belonging to the same table is together.
Then, judging whether the two groups of data manipulation language sentences are inserted or deleted, if so, judging whether the two groups of data manipulation language sentences are of the same operation type, namely, whether the two groups of data manipulation language sentences are inserted or deleted, if so, the two groups of data manipulation language sentences are of the same operation type, otherwise, the two groups of data manipulation language sentences are of different operation types, and performing non-merging counting processing, specifically, when the two groups of data manipulation language sentences are of the same operation type, merging counting is added with 1; when two groups of data manipulation language sentences are of the type without operation, the processing of adding 1 to the merging count is not performed, meanwhile, the merging count is cleared, more specifically, whether the two DML operations are Insert/Delete is judged according to the latest two DML operations, if the two DML operations are Insert or Delete, the two DML operations can be merged, the merging count (canEnableMerge) is added with 1, otherwise, the non-merging count (notEnableMerge) is added with 1, and the merging count is cleared, so that when the sentences which cannot be merged are encountered once, the merging counter is accumulated again, and a large-batch operation scene with less occurrence probability is screened according to stricter requirements.
Further, it is necessary to perform a merge/not merge count process based on the determination results of the same/different operation types, and adjust the opening/closing of the merge commit switch according to the result of the count process, wherein the closing of the merge commit switch is adjusted, including the steps of: setting a starting merging threshold value, and judging whether the merging count is larger than the starting merging threshold value or not; if yes, opening a merging and submitting switch, and meanwhile, setting zero clearing of merging and counting, otherwise, not adjusting the merging and submitting switch; setting a closing merging threshold value, and judging whether the non-merging count is larger than the closing merging threshold value or not; if yes, closing the merging submitting switch, and resetting the merging count, otherwise, not adjusting the merging submitting switch.
Specifically, when the merge count reaches a certain threshold, that is, when the merge count reaches a start merge threshold, the merge switch may be started, and in this embodiment, the start merge threshold may be set to 10, and at this time, more than 10 sentences are merged into one operation sentence, so as to achieve an obvious write optimization effect; on the other hand, when the merge count (canEnableMerge) is accumulated to exceed 10, it indicates that the batches are delivered identically, and the merge commit switch may be turned on.
After the batch operation is completed, the merge switch needs to be automatically detected and closed, so that a closing merge threshold is set, when the accumulated value of the non-merge count exceeds the closing merge threshold, the merge switch can be closed, for example, when 10 data operation sentences which cannot be merged appear in an accumulated way, the merge switch is closed, and the merge count is cleared to 0, so that the next detected batch merger operation can be counted again and merge can be started.
Therefore, each DML operation put into the queue can immediately count and calculate, then the next item is put into, and further, after the combinable records exceeding the starting combination threshold value are inserted into the queue, the combination submitting switch is automatically opened, and when the subsequent data operation has the incapability of being combined exceeding the closing combination threshold value, the combination submitting switch is automatically closed, and finally, the aim of accurately controlling statement combination is achieved.
When the merging and counting process is performed, if there are multiple adjacent and identical data manipulation language sentences in the same list, the merging process may be performed to improve efficiency.
Example two
A data synchronization apparatus for performing the data synchronization method according to any one of the embodiments, comprising a source, a merge counter, a non-merge counter, a merge commit switch, and a destination; the source end is used for sending data manipulation language sentences; the merging counter is used for carrying out merging counting processing according to the operation types of the two groups of similar data operation language sentences; the non-merging counter is used for performing non-merging counting processing according to the operation types of two groups of similar data operation language sentences; the merging submitting switch is used for adjusting the opening and closing according to the numerical value of the merging counter/the numerical value of the non-merging counter; the target end is used for receiving the data manipulation language statement sent by the source end.
It should be noted that, the present embodiment is a hardware device for executing the data synchronization method described in the first embodiment, and the execution steps are the same as those of the first embodiment, so detailed description is omitted in the present embodiment.
Example III
A data synchronization system comprises a queue allocation unit, a type judgment unit and a merging and adjusting unit; the queue allocation unit is used for delivering the data manipulation language sentences of the same list to the same queue and acquiring two similar groups of data manipulation language sentences in the same queue; the type judging unit is used for judging whether the two groups of data manipulation language sentences are inserted or deleted, if yes, judging whether the two groups of data manipulation language sentences are of the same operation type, and otherwise, performing non-merging counting processing; the merging and adjusting unit performs merging/non-merging counting processing based on the judgment results of the same/different operation types, and adjusts the opening and closing of the merging submitting switch according to the result of the counting processing.
Since the data synchronization system of the present embodiment performs the data synchronization method described in the first embodiment, in the present embodiment, the process of performing is not repeated.
A computer readable storage medium storing a computer program which, when executed by a processor, performs the data synchronization method according to any one of the embodiments.
More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wire segments, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and the division of modules, or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units, modules, or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed.
The units may or may not be physically separate, and the components shown as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via a communication portion, and/or installed from a removable medium. The above-described functions defined in the method of the present application are performed when the computer program is executed by a Central Processing Unit (CPU). The computer readable medium of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing is merely illustrative of specific embodiments of the present application, and the scope of the present application is not limited thereto, but any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of data synchronization comprising the steps of:
delivering the data manipulation language sentences of the same list to the same queue, and acquiring two groups of similar data manipulation language sentences in the same queue;
judging whether the two groups of data manipulation language sentences are inserted or deleted, if so, judging whether the two groups of data manipulation language sentences are of the same operation type, otherwise, performing uncombined counting processing;
and carrying out merging/non-merging counting processing based on the judging results of the same/different operation types, and regulating the opening and closing of a merging submitting switch according to the counting processing result.
2. A data synchronization method according to claim 1, characterized in that the merge/not merge count process comprises the steps of:
when the two groups of data manipulation language sentences are of the same operation type, the merging count is added with 1;
and when the two groups of data manipulation language sentences are of the type of no-use operation, processing of adding 1 to the merging count is not performed, and meanwhile, the merging count is cleared.
3. A method of data synchronization according to claim 2, wherein adjusting the closing of the merge commit switch comprises the steps of:
setting a starting merging threshold value, and judging whether the merging count is larger than the starting merging threshold value or not;
if yes, the merging submitting switch is opened, meanwhile, the merging counting zero clearing setting is not carried out, and otherwise, the merging submitting switch is not adjusted.
4. A method of data synchronization according to claim 2, wherein the closing of the merge commit switch is adjusted, further comprising the steps of:
setting a closing merging threshold value, and judging whether the non-merging count is larger than the closing merging threshold value or not;
if yes, closing the merging submitting switch, and resetting the merging count, otherwise, not adjusting the merging submitting switch.
5. A method of synchronizing data according to claim 1, wherein the data manipulation language statements comprise insert, update and delete operations.
6. A method of data synchronization according to claim 1, further comprising the steps of:
judging all the data manipulation language sentences according to the table names, and collecting the data manipulation language sentences with the same table names into the same list according to the sequence.
7. The method of claim 6, wherein a plurality of adjacent and identical data manipulation language sentences in the same list are subjected to the merge processing at the time of the merge count processing.
8. A data synchronization device, wherein the data synchronization device is configured to perform the data synchronization method according to any one of claims 1 to 7, and includes a source, a merge counter, a do not merge counter, a merge commit switch, and a destination;
the source end is used for sending data manipulation language sentences;
the merging counter is used for carrying out merging counting processing according to the operation types of two groups of similar data operation language sentences;
the non-merging counter is used for performing non-merging counting processing according to the operation types of two groups of similar data operation language sentences;
the merging submitting switch is used for adjusting the opening and closing according to the numerical value of the merging counter/the numerical value of the non-merging counter;
the target end is used for receiving the data manipulation language statement sent by the source end.
9. The data synchronization system is characterized by comprising a queue distribution unit, a type judgment unit and a merging and regulating unit;
the queue allocation unit is used for delivering the data manipulation language sentences of the same list to the same queue and acquiring two similar groups of data manipulation language sentences in the same queue;
the type judging unit is used for judging whether the two groups of data manipulation language sentences are inserted or deleted, if yes, judging whether the two groups of data manipulation language sentences are of the same operation type, and otherwise, performing non-merging counting processing;
the merging and adjusting unit performs merging/non-merging counting processing based on the judgment results of the same/different operation types, and adjusts the opening and closing of the merging submitting switch according to the result of the counting processing.
10. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the data synchronization method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310618397.XA CN116578647A (en) | 2023-05-29 | 2023-05-29 | Data synchronization method, device, system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310618397.XA CN116578647A (en) | 2023-05-29 | 2023-05-29 | Data synchronization method, device, system and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116578647A true CN116578647A (en) | 2023-08-11 |
Family
ID=87535657
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310618397.XA Pending CN116578647A (en) | 2023-05-29 | 2023-05-29 | Data synchronization method, device, system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116578647A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101971158A (en) * | 2008-03-11 | 2011-02-09 | 微软公司 | Synchronization of disconnected/offline data processing/entry |
| US20150149409A1 (en) * | 2013-11-22 | 2015-05-28 | Joo Yeon Lee | Dml replication with logical log shipping |
| CN105338051A (en) * | 2014-08-15 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Data synchronization method and equipment |
| US20170046377A1 (en) * | 2015-08-11 | 2017-02-16 | International Business Machines Corporation | Reducing the cost of update, delete, and append-only insert operations in a database |
| CN106844507A (en) * | 2016-12-27 | 2017-06-13 | 星环信息科技（上海）有限公司 | A kind of method and apparatus of data batch processing |
| CN109885617A (en) * | 2019-01-29 | 2019-06-14 | 中国工商银行股份有限公司 | The method of data synchronization and device of Distributed Heterogeneous Database system |
| CN112783574A (en) * | 2021-02-02 | 2021-05-11 | 北京百度网讯科技有限公司 | Application program development method, device, equipment and storage medium |
| CN114595242A (en) * | 2022-03-04 | 2022-06-07 | 北京字节跳动网络技术有限公司 | Data operation method and device, computer equipment and storage medium |
| CN115757612A (en) * | 2022-10-19 | 2023-03-07 | 粤港澳国际供应链(广州)有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN116069859A (en) * | 2023-01-09 | 2023-05-05 | 北京人大金仓信息技术股份有限公司 | Incremental data synchronization method of database, storage medium and computer equipment |
-
2023
- 2023-05-29 CN CN202310618397.XA patent/CN116578647A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101971158A (en) * | 2008-03-11 | 2011-02-09 | 微软公司 | Synchronization of disconnected/offline data processing/entry |
| US20150149409A1 (en) * | 2013-11-22 | 2015-05-28 | Joo Yeon Lee | Dml replication with logical log shipping |
| CN105338051A (en) * | 2014-08-15 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Data synchronization method and equipment |
| US20170046377A1 (en) * | 2015-08-11 | 2017-02-16 | International Business Machines Corporation | Reducing the cost of update, delete, and append-only insert operations in a database |
| CN106844507A (en) * | 2016-12-27 | 2017-06-13 | 星环信息科技（上海）有限公司 | A kind of method and apparatus of data batch processing |
| CN109885617A (en) * | 2019-01-29 | 2019-06-14 | 中国工商银行股份有限公司 | The method of data synchronization and device of Distributed Heterogeneous Database system |
| CN112783574A (en) * | 2021-02-02 | 2021-05-11 | 北京百度网讯科技有限公司 | Application program development method, device, equipment and storage medium |
| CN114595242A (en) * | 2022-03-04 | 2022-06-07 | 北京字节跳动网络技术有限公司 | Data operation method and device, computer equipment and storage medium |
| CN115757612A (en) * | 2022-10-19 | 2023-03-07 | 粤港澳国际供应链(广州)有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN116069859A (en) * | 2023-01-09 | 2023-05-05 | 北京人大金仓信息技术股份有限公司 | Incremental data synchronization method of database, storage medium and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2011256912B2 (en) | Systems and methods for providing increased scalability in deduplication storage systems | |
| US10210187B2 (en) | Removal of garbage data from a database | |
| US7958088B2 (en) | Dynamic data reorganization to accommodate growth across replicated databases | |
| WO2021238701A1 (en) | Data migration method and device | |
| US11474990B2 (en) | Priority queue for exclusive locks | |
| US10929397B2 (en) | Forecasting query access plan obsolescence | |
| WO2018201832A1 (en) | Data migration method and system for distributed cache database | |
| US8892526B2 (en) | Deduplication seeding | |
| US8380675B1 (en) | Mailbox archiving using adaptive patterns | |
| AU2014209697A1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN111858760B (en) | Data processing method and device for heterogeneous database | |
| US20130268498A1 (en) | Prioritization mechanism for deletion of chunks of deduplicated data objects | |
| CN113420026B (en) | Database table structure changing method, device, equipment and storage medium | |
| CN108876330B (en) | Multi-time zone daily cutting method and device | |
| CN113946628A (en) | Data synchronization method and device based on interceptor | |
| CN112434003B (en) | SQL optimization method and device, computer equipment and storage medium | |
| CN116578647A (en) | Data synchronization method, device, system and computer readable storage medium | |
| US20210006619A1 (en) | Email synchronization method and apparatus, and computer readable storage medium | |
| WO2021057515A1 (en) | Date switching data offloading method, apparatus, device, and medium | |
| US11226878B1 (en) | Accelerator-based database recovery | |
| CN110109970B (en) | Data query processing method and device | |
| EP3905064A1 (en) | Method and apparatus for synchronously replicating database | |
| CN103391318A (en) | Data processing method, client side, server side and device | |
| US20210117096A1 (en) | Method, device and computer program product for backuping data | |
| CN113568930A (en) | Method and equipment for optimizing distributed memory data query |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |