CN106407321A - Data synchronization method and device - Google Patents
Data synchronization method and device Download PDFInfo
- Publication number
- CN106407321A CN106407321A CN201610792235.8A CN201610792235A CN106407321A CN 106407321 A CN106407321 A CN 106407321A CN 201610792235 A CN201610792235 A CN 201610792235A CN 106407321 A CN106407321 A CN 106407321A
- Authority
- CN
- China
- Prior art keywords
- data
- source data
- target
- database
- caching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data synchronization method and device. The method comprises the steps of obtaining source data; matching the source data with target data in a cache, thereby obtaining a matching result of the source data, wherein the target data in the cache is pre-loaded to the cache from a target database; determining an database operation needing to be carried out on the source data according to the matching result; and submitting the database operation needing to be carried out on the source data to the target database for performance according to a preset condition. A purpose of improving the data synchronization performance greatly is realized.
Description
Technical field
A kind of it relates to data processing field, in particular it relates to method and device of synchrodata.
Background technology
In data processing field, the source data of source database, after extracting, changing, needs to be synchronized to target database
In.Before source data is synchronized to target database, need first to be inquired about in target database one by one, this source data with
In the case of the existing record matching of target data, this source data is updated in target database, in absence of such a match, will
This source data is inserted in target database.
However, because the matching performance of data base is not high, leading to data synchronization efficiency too low it is impossible to meet at mass data
The needs of reason.
Content of the invention
In view of this, present disclose provides a kind of method and device of synchrodata, to realize improving data synchronization efficiency
Purpose.
In the one side of the embodiment of the present disclosure, there is provided a kind of method of synchrodata.The method includes：Acquisition source
Data；Described source data is mated with the target data in caching, is obtained the matching result of described source data, wherein, institute
Stating the target data in caching is to be loaded previously into described caching from target database；According to described matching result, determine and need
The database manipulation that described source data is carried out；According to pre-conditioned it would be desirable to database manipulation that described source data is carried out
Submit to described target database execution.
Alternatively, described acquisition source data includes：Data is read thread and is read the source data in source database；Send described source
Data is to data buffer storage；Accordingly, described by described source data and caching in target data carry out mating including：Obtain described
Described source data in data buffer storage, described source data is mated with the target data in Hash caching, obtains described
Join result.
Alternatively, the data in described Hash caching is from the Partial key field of the no major-key table of target database in advance
It is loaded into described Hash caching.
Alternatively, the described database manipulation described source data being carried out according to described matching result, determination needs, bag
Include：When described source data is mated with the target data in caching, determine that the database manipulation that described source data is carried out is number
Update operation according to storehouse, or；When target data in described source data with caching mismatches, determine and described source data is carried out
Database manipulation be database insert operation.
Alternatively, described according to pre-conditioned it would be desirable to described in database manipulation that described source data is carried out submits to
Target database executes, including：When the data volume of the described source data that the match is successful reaches batch updating threshold value, using described
The source data that the match is successful, the corresponding data record of the target data that in target database described in batch updating, the match is successful, or
Person；When the data volume mating unsuccessful described source data reaches batch insertion threshold value, by all couplings unsuccessful source number
According to being inserted into described target database.
In the other side of the embodiment of the present disclosure, there is provided a kind of device of synchrodata.This device includes：Obtain
Module, is configured to obtain source data.Matching module, is configured to carry out described source data with the target data in caching
Join, obtain the matching result of described source data, wherein, the target data in described caching is to be loaded previously into from target database
Described caching.Operation determining module, is configured to, according to described matching result, determine the data needing that described source data is carried out
Storehouse operates.Module is submitted in operation to, is configured to according to pre-conditioned it would be desirable to the database manipulation that described source data is carried out carries
Give described target database execution.
Alternatively, described acquisition module is configured to trigger data reading thread and reads the source data in source database；Send
Described source data is to data buffer storage.Described matching module is configured to obtain the described source data in described data buffer storage, by institute
State source data to be mated with the target data in Hash caching, obtain described matching result.
Alternatively, the data in described Hash caching is from the Partial key field of the no major-key table of target database in advance
It is loaded into described Hash caching.
Alternatively, described operation determining module is configured to when described source data is mated with the target data in caching,
Determine that the database manipulation that described source data is carried out operates for database update, or；In described source data with caching
When target data mismatches, determine that the database manipulation that described source data is carried out is database insert operation.
Alternatively, described operation submits to the data volume that module is configured as the described source data that the match is successful to reach batch
When updating threshold value, using the described source data that the match is successful, the number of targets that in target database described in batch updating, the match is successful
According to corresponding data record, or；When the data volume mating unsuccessful described source data reaches batch insertion threshold value, by institute
The unsuccessful source data of coupling is had to be inserted into described target database.
By disclosure technique scheme, first the target data of target database is entered row cache, then by caching
High-performance to carry out mating of source data and target data, thus improve matching efficiency, needs are determined according to matching result
After the database manipulation that source data is carried out, further according to pre-conditioned it would be desirable to the database manipulation that source data is carried out carries
Give described target database execution, it is achieved thereby that the purpose that data syn-chronization performance is substantially improved.
Other feature and advantage of the disclosure will be described in detail in subsequent specific embodiment part.
Brief description
Accompanying drawing is used to provide further understanding of the disclosure, and constitutes the part of description, with following tool
Body embodiment is used for explaining the disclosure together, but does not constitute restriction of this disclosure.In the accompanying drawings：
Fig. 1 is the implementation environment schematic diagram according to an exemplary embodiment.
Fig. 2 is the flow chart of a kind of method of synchrodata according to one example embodiment of the present invention.
Fig. 3 is the flow chart of a kind of method of synchrodata illustrating in accordance with an alternative illustrative embodiment of the present invention.
Fig. 4 is a kind of block diagram of the device of synchrodata according to one example embodiment of the present invention.
Fig. 5 is a kind of block diagram of the device of synchrodata illustrating in accordance with an alternative illustrative embodiment of the present invention.
Specific embodiment
It is described in detail below in conjunction with accompanying drawing specific embodiment of this disclosure.It should be appreciated that this place is retouched
The specific embodiment stated is merely to illustrate and explains the disclosure, is not limited to the disclosure.
Fig. 1 is the implementation environment schematic diagram according to an exemplary embodiment.This implementation environment includes：Apply this public affairs
Open embodiment offer the data acquisition unit 101 of the method for synchrodata, target database 102, source database 103.Wherein, number
Server side can be arranged according to harvester 101, target database 102.For example, data acquisition unit 101, can be from source database
103 crawl source datas, source data is synchronized to target database 102.Wherein, this source data can through needs conversion it
Afterwards, it is re-synchronised to target database 102.
It is understood that implementation environment shown in Fig. 1 is only used for illustrating method provided in an embodiment of the present invention, do not constitute
Restriction to the embodiment of the present invention.
Fig. 2 is the flow chart of a kind of method of synchrodata according to one example embodiment of the present invention.The party
Method can include：
In step 210, obtain source data.
In a step 220, by described source data with caching in target data mated, obtain described source data
Join result.
It should be noted that in step 210 described in the present embodiment, described source data can once obtain a plurality of it is also possible to
Once obtain one.In described step 220, can be step 210 obtain a source data after i.e. with caching in number of targets
According to mated or after step 210 obtains a plurality of source data again with caching in target data mated.This reality
Apply example to this and to be not limited.
Wherein, the target data in described caching is to be loaded previously into described caching from target database.
For example, in a kind of possible embodiment, the source data in thread reading source database can be read by data, send
Described source data is to data buffer storage.Described source data in the thread described data buffer storage of acquisition is write by data, by described source data
Mated with the target data in Hash caching, obtained described matching result.Target data in described Hash caching is pre-
First it is loaded into described Hash caching from target database.Because Hash caching mainly to build caching rope using hash algorithm
Draw, there is very high efficiency data query, therefore, search efficiency be can further improve by the high-performance caching by Hash,
Such that it is able to further improve data syn-chronization performance.
It is understood that the operation of embodiment of the present disclosure acquisition source data and matched data, submission source data are to mesh
The operation of mark data base can be executed it is also possible to as executed by different threads respectively in embodiment above by same thread,
The disclosure to this and is not limited.It is understood that in the case of being executed by different threads respectively, can more one progress
Improve the efficiency of data syn-chronization.
In application scenes, for example, in data acquisition scenarios, the number of data to be stored usually no major-key table
According to.It is impossible to data is inquired about by major key for the data of no major-key table, can only by mate critical field data Lai
Determine in data base, whether there is corresponding record.In order to reduce the waste of memory space, in Hash caching described in the disclosure
Target data be to be loaded previously into described Hash caching from the Partial key field of the no major-key table of target database.Its
In, target data can place into caching through overcompression.For example, for the target database for storing people information,
Data in critical field can be name and identification card number.In conjunction with the embodiment putting into Hash caching, can by name and
Identification card number is spliced, and carries out Hash calculation to spliced name and identification card number, will be used for representing this name and identity
The cryptographic Hash of card number is stored in Hash caching.When needing to be mated, the data of critical field corresponding in source data can be entered
Row Hash calculation, obtains the cryptographic Hash of described source data, the cryptographic Hash according to described source data directly to Hash caching in
Join target data and whether there is respective record in target database to determine.It can be seen that, with respect to prestrain mesh in caching in Hash
For the total data of mark data base, the data cryptographic Hash of target database Partial key field is loaded previously into Hash and delays
Deposit, the memory space of occupancy is less.
For example, described data buffer storage can be realized by internal memory it is also possible to lead to as the Hash in embodiment above caches
Cross NoSQL database realizing.For example, described data buffer storage can be respectively adopted different NoSQL data bases from described Hash caching
Realize, it would however also be possible to employ same NoSQL database realizing.In the case of using same NoSQL database realizing, can pass through
Prefix or suffix is added on key name to distinguish two cachings.
In step 230, according to described matching result, determine the database manipulation needing that described source data is carried out.
For example, when described source data is mated with the target data in caching it may be determined that carrying out to described source data
Database manipulation operates for database update, or；When target data in described source data with caching mismatches, it is right to determine
The database manipulation that described source data is carried out is database insert operation.
In step 240, according to pre-conditioned it would be desirable to described in database manipulation that described source data is carried out submits to
Target database executes.
For example, it is possible to for updating with update, setting one is pre-conditioned jointly.As, if away from last submit to update and
The interval duration of update reaches the duration threshold value of a setting, if or, obtaining after last submission renewal and update
The bar number of the source data taking reaches the bar number threshold value of a setting, then will need the data base that these source datas are carried out respectively more
New operation and one piece of update batch submit to described target database execution.
Again for example, it is possible to for update and update be respectively provided with different pre-conditioned.In this embodiment, described
According to pre-conditioned it would be desirable to the database manipulation that described source data is carried out submits to described target database execution, permissible
Including：When the data volume of the described source data that the match is successful reaches batch updating threshold value, using the described source number that the match is successful
According to, the corresponding data record of the target data that in target database described in batch updating, the match is successful, or；Unsuccessful when mating
Described source data data volume reach batch insertion threshold value when, by all coupling unsuccessful source data be inserted into described target
Data base.Wherein, batch updating threshold value and batch insertion threshold value can be arranged according to actual needs, e.g., can be respectively provided with
For 500 data.
It can be seen that, by disclosed technique scheme, first the target data of target database is entered row cache, then by caching
High-performance improving matching efficiency, thus improve matching efficiency, being determined according to matching result needs source data is carried out
After database manipulation, further according to pre-conditioned it would be desirable to the database manipulation that source data is carried out submits to described number of targets
According to storehouse execution, it is achieved thereby that the purpose that data syn-chronization performance is substantially improved.Furthermore it is possible to utilize high performance batch of data base
Amount updates and insertion is to improve data writing capability, improves data syn-chronization performance further.
Fig. 3 is the flow chart of a kind of method of synchrodata illustrating in accordance with an alternative illustrative embodiment of the present invention.Should
Method can include：
In the step 310, obtain source data.
In step 320, by described source data with caching in target data mated, obtain described source data
Join result.
Wherein, the data in described caching is to be loaded previously into described caching from target database.
In a step 330, according to described matching result, determine the database manipulation needing that described source data is carried out.
In step 340, according to pre-conditioned it would be desirable to described in database manipulation that described source data is carried out submits to
Target database executes.
In step 350, according to pre-conditioned it would be desirable to the database manipulation that described source data is carried out is in described caching
Middle execution.
In the present embodiment, can in advance from target database loaded targets data to caching, can add when starting first
Carry, make caching synchronous with target database subsequently through this step 350, need not be again from target database loaded targets number
According to caching, decrease the transmission of data, further improve the efficiency of data syn-chronization.
Fig. 4 is a kind of block diagram of the device 400 of the synchrodata according to one example embodiment of the present invention.This dress
Put and can include：Module 440 is submitted in acquisition module 410, matching module 420, operation determining module 430 and operation to.
This acquisition module 410, can be configured to obtain source data.
This matching module 420, can be configured to be mated described source data with the target data in caching, obtain
The matching result of described source data.
Wherein, the target data in described caching is to be loaded previously into described caching from target database.
This operation determining module 430, can be configured to, according to described matching result, determine and need described source data is entered
The database manipulation of row.
Module 440 is submitted in this operation to, can be configured to according to pre-conditioned it would be desirable to number that described source data is carried out
Submit to described target database execution according to storehouse operation.
It can be seen that, by disclosed technique scheme, first the target data of target database is entered row cache, then by caching
High-performance improving matching efficiency, thus improve matching efficiency, being determined according to matching result needs source data is carried out
After database manipulation, further according to pre-conditioned it would be desirable to the database manipulation that source data is carried out submits to described number of targets
According to storehouse execution, it is achieved thereby that the purpose that data syn-chronization performance is substantially improved.
It is understood that the operation of embodiment of the present disclosure acquisition source data and matched data, submission source data are to mesh
The operation of mark data base can be executed by same thread it is also possible to be executed by different threads respectively, and the disclosure is not entered to this
Row limits.For example, described acquisition module 410 can be configured to trigger data reading thread and read the source data in source database；
Send described source data to data buffer storage.Described matching module 420 can be configured to trigger data and write the thread described number of acquisition
According to the described source data in caching, described source data is mated with the target data in Hash caching, is obtained described coupling
Result.
In application scenes, for example, in data acquisition scenarios, the number of data to be stored usually no major-key table
According to.It is impossible to data is inquired about by major key for the data of no major-key table, can only by mate critical field data Lai
Determine in data base, whether there is corresponding record.In order to reduce the waste to memory space, the disclosure is loaded previously into slow
Target data in depositing is to be loaded previously into described Hash caching from the Partial key field of the no major-key table of target database.
Wherein, target data can place into caching through overcompression.It can be seen that, whole with respect to preload target data base in the buffer
For data, the cryptographic Hash of the data of target database Partial key field is loaded previously into Hash caching, takies storage empty
Between less.
It should be noted that update can jointly arrange a batch submission condition it is also possible to set respectively with update
Put different batches and submit condition to.For example, described operation determining module 430 can be configured in described source data with caching
Target data coupling when, determine that database manipulation that described source data is carried out is database update operation, or；Described
When target data in source data and caching mismatches, determine that the database manipulation that described source data is carried out is that data base inserts
Operation.Described operation submits to module 440 can be configured to reaching batch when the data volume of the described source data that the match is successful
When updating threshold value, using the described source data that the match is successful, the number of targets that in target database described in batch updating, the match is successful
According to corresponding data record, or；When the data volume mating unsuccessful described source data reaches batch insertion threshold value, by institute
The unsuccessful source data of coupling is had to be inserted into described target database.
In a kind of possible embodiment, first load data into caching from target database, can add when starting first
Carry, subsequently make caching and target database synchronized update, caching need not be loaded data into from target database again, reduce
The transmission of data, further improves the efficiency of data syn-chronization.Specifically, as shown in figure 5, described device can also be wrapped
Include：Buffer update module 450, can be configured to according to pre-conditioned it would be desirable to the data base that described source data is carried out grasps
Make to execute in described caching.
Describe the preferred implementation of the disclosure above in association with accompanying drawing in detail, but, the disclosure is not limited to above-mentioned reality
Apply the detail in mode, in the range of the technology design of the disclosure, multiple letters can be carried out with technical scheme of this disclosure
Monotropic type, these simple variant belong to the protection domain of the disclosure.
It is further to note that each particular technique feature described in above-mentioned specific embodiment, in not lance
In the case of shield, can be combined by any suitable means.In order to avoid unnecessary repetition, the disclosure to various can
The compound mode of energy no longer separately illustrates.
Additionally, combination in any can also be carried out between the various different embodiment of the disclosure, as long as it is without prejudice to this
Disclosed thought, it equally should be considered as disclosure disclosure of that.
Claims (10)
1. a kind of method of synchrodata is it is characterised in that include：
Obtain source data；
Described source data is mated with the target data in caching, is obtained the matching result of described source data, wherein, described
Target data in caching is to be loaded previously into described caching from target database；
According to described matching result, determine the database manipulation needing that described source data is carried out；
According to pre-conditioned it would be desirable to the database manipulation that described source data is carried out submits to described target database execution.
2. method according to claim 1 is it is characterised in that described acquisition source data includes：
Data is read thread and is read the source data in source database；
Send described source data to data buffer storage；
Accordingly, described by described source data and caching in target data carry out mating including：
Obtain the described source data in described data buffer storage, the target data in described source data and Hash caching is carried out
Join, obtain described matching result.
3. method according to claim 2 is it is characterised in that the target data in described Hash caching is from target data
The Partial key field of the no major-key table in storehouse is loaded previously into described Hash caching.
4. method according to claim 1 it is characterised in that described according to described matching result, determine and need to described
The database manipulation that source data is carried out, including：
When described source data is mated with the target data in caching, determine that the database manipulation that described source data is carried out is number
Update operation according to storehouse, or；
When target data in described source data with caching mismatches, determine that the database manipulation that described source data is carried out is
Database insert operation.
5. method according to claim 4 it is characterised in that described according to pre-conditioned it would be desirable to described source data
The database manipulation carrying out submits to described target database execution, including：
When the data volume of the described source data that the match is successful reaches batch updating threshold value, using the described source number that the match is successful
According to, the corresponding data record of the target data that in target database described in batch updating, the match is successful, or；
When the data volume mating unsuccessful described source data reaches batch insertion threshold value, by all couplings unsuccessful source number
According to being inserted into described target database.
6. a kind of device of synchrodata is it is characterised in that include：
Acquisition module, is configured to obtain source data；
Matching module, is configured to be mated described source data with the target data in caching, obtains described source data
Matching result, wherein, the target data in described caching is to be loaded previously into described caching from target database；
Operation determining module, is configured to according to described matching result, determines the data base behaviour needing described source data is carried out
Make；
Module is submitted in operation to, is configured to according to pre-conditioned it would be desirable to the database manipulation that described source data is carried out is submitted to
To the execution of described target database.
7. device according to claim 6 reads thread reading it is characterised in that described acquisition module is configured to trigger data
Take the source data in source database；Send described source data to data buffer storage；
Described matching module is configured to obtain the described source data in described data buffer storage, and described source data is cached with Hash
In target data mated, obtain described matching result.
8. device according to claim 7 is it is characterised in that the data in described Hash caching is from target database
The Partial key field of no major-key table is loaded previously into described Hash caching.
9. device according to claim 6 is it is characterised in that described operation determining module is configured in described source data
When mating with the target data in caching, determine that the database manipulation that described source data is carried out operates for database update, or
Person；When target data in described source data with caching mismatches, determine that the database manipulation that described source data is carried out is
Database insert operation.
10. the match is successful it is characterised in that described operation submits to module to be configured as device according to claim 9
The data volume of described source data when reaching batch updating threshold value, using the described source data that the match is successful, described in batch updating
The corresponding data record of target data that in target database, the match is successful, or；When the unsuccessful described source data of coupling
When data volume reaches batch insertion threshold value, the unsuccessful source data of all couplings is inserted into described target database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610792235.8A CN106407321B (en) | 2016-08-31 | 2016-08-31 | Method and device for synchronizing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610792235.8A CN106407321B (en) | 2016-08-31 | 2016-08-31 | Method and device for synchronizing data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106407321A true CN106407321A (en) | 2017-02-15 |
| CN106407321B CN106407321B (en) | 2019-12-06 |
Family
ID=58000784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610792235.8A Active CN106407321B (en) | 2016-08-31 | 2016-08-31 | Method and device for synchronizing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106407321B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108062399A (en) * | 2017-12-21 | 2018-05-22 | 新华三大数据技术有限公司 | Data processing method and device |
| CN108304560A (en) * | 2018-02-08 | 2018-07-20 | 山东汇贸电子口岸有限公司 | A kind of data verification method based on critical field |
| CN108629002A (en) * | 2018-05-03 | 2018-10-09 | 山东汇贸电子口岸有限公司 | A kind of big data comparison method and device based on kettle |
| CN108829747A (en) * | 2018-05-24 | 2018-11-16 | 新华三大数据技术有限公司 | Data load method and device |
| CN109271394A (en) * | 2018-08-27 | 2019-01-25 | 武汉达梦数据库有限公司 | A kind of batch data insertion update implementation method based on ID caching |
| CN109558448A (en) * | 2018-10-10 | 2019-04-02 | 北京海数宝科技有限公司 | Data processing method, device, computer equipment and storage medium |
| CN111841021A (en) * | 2020-07-23 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Account matching method, device, equipment and storage medium |
| CN112463786A (en) * | 2020-12-10 | 2021-03-09 | 上海携宁计算机科技股份有限公司 | Data synchronization method, system, server and storage medium |
| CN112675538A (en) * | 2020-12-28 | 2021-04-20 | 腾讯科技（深圳）有限公司 | Data synchronization method, device, equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063500A (en) * | 2011-01-04 | 2011-05-18 | 北京凯铭风尚网络技术有限公司 | Data migration method and device |
| CN103294702A (en) * | 2012-02-27 | 2013-09-11 | 上海淼云文化传播有限公司 | Data processing method, device and system |
| US20150347410A1 (en) * | 2014-06-03 | 2015-12-03 | Sap Ag | Cached Views |
| CN105426419A (en) * | 2015-11-03 | 2016-03-23 | 用友网络科技股份有限公司 | System and method for data promotion among heterogeneous systems |
-
2016
- 2016-08-31 CN CN201610792235.8A patent/CN106407321B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063500A (en) * | 2011-01-04 | 2011-05-18 | 北京凯铭风尚网络技术有限公司 | Data migration method and device |
| CN103294702A (en) * | 2012-02-27 | 2013-09-11 | 上海淼云文化传播有限公司 | Data processing method, device and system |
| US20150347410A1 (en) * | 2014-06-03 | 2015-12-03 | Sap Ag | Cached Views |
| CN105426419A (en) * | 2015-11-03 | 2016-03-23 | 用友网络科技股份有限公司 | System and method for data promotion among heterogeneous systems |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108062399A (en) * | 2017-12-21 | 2018-05-22 | 新华三大数据技术有限公司 | Data processing method and device |
| CN108304560A (en) * | 2018-02-08 | 2018-07-20 | 山东汇贸电子口岸有限公司 | A kind of data verification method based on critical field |
| CN108629002A (en) * | 2018-05-03 | 2018-10-09 | 山东汇贸电子口岸有限公司 | A kind of big data comparison method and device based on kettle |
| CN108829747A (en) * | 2018-05-24 | 2018-11-16 | 新华三大数据技术有限公司 | Data load method and device |
| CN109271394A (en) * | 2018-08-27 | 2019-01-25 | 武汉达梦数据库有限公司 | A kind of batch data insertion update implementation method based on ID caching |
| CN109558448A (en) * | 2018-10-10 | 2019-04-02 | 北京海数宝科技有限公司 | Data processing method, device, computer equipment and storage medium |
| CN111841021A (en) * | 2020-07-23 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Account matching method, device, equipment and storage medium |
| CN112463786A (en) * | 2020-12-10 | 2021-03-09 | 上海携宁计算机科技股份有限公司 | Data synchronization method, system, server and storage medium |
| CN112463786B (en) * | 2020-12-10 | 2021-08-13 | 上海携宁计算机科技股份有限公司 | Data synchronization method, system, server and storage medium |
| CN112675538A (en) * | 2020-12-28 | 2021-04-20 | 腾讯科技（深圳）有限公司 | Data synchronization method, device, equipment and medium |
| CN112675538B (en) * | 2020-12-28 | 2023-08-01 | 腾讯科技（深圳）有限公司 | Data synchronization method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106407321B (en) | 2019-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106407321A (en) | Data synchronization method and device | |
| CN108961052B (en) | Verification method, storage method, device, equipment and medium of block chain data | |
| Fu et al. | Toward efficient multi-keyword fuzzy search over encrypted outsourced data with accuracy improvement | |
| EP3399434B1 (en) | Short link processing method, device and server | |
| US10740366B2 (en) | Method and system for searching encrypted data | |
| JP6244592B2 (en) | Column-oriented database processing method and processing device | |
| CN103428242B (en) | A kind of method of increment synchronization, Apparatus and system | |
| CN108153757A (en) | A kind of method and apparatus of Hash table management | |
| CN103810224B (en) | information persistence and query method and device | |
| US10402386B2 (en) | Method and apparatus for generating index for encrypted field in database | |
| CN108460041B (en) | Data processing method and device | |
| JP6449093B2 (en) | Concealed database system and concealed data management method | |
| US20180341653A1 (en) | Method for efficient primary key based queries using atomic rdma reads on cache friendly in-memory hash index | |
| CN111209591B (en) | Storage structure sorted according to time and quick query method | |
| Qin et al. | Applying private information retrieval to lightweight bitcoin clients | |
| WO2014021978A4 (en) | Aggregating data in a mediation system | |
| US9465954B1 (en) | Method and system for tracking masking of data | |
| CN103077208A (en) | Uniform resource locator matching processing method and device | |
| CN103226591B (en) | A kind of support multiple key quick access method and device | |
| CN106326499A (en) | Data processing method and device | |
| JP5558514B2 (en) | Method and apparatus for optimally processing N-sort queries in multi-range scanning | |
| CN110046062A (en) | Distributed data processing method and system | |
| US20150039558A1 (en) | Database management method, database system and medium | |
| Camacho et al. | A cloud-oriented integrity verification system for audio forensics | |
| KR101358793B1 (en) | Method of forming index file, Method of searching data and System for managing data using dictionary index file, Recoding medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |