CN108121757A - A kind of method of data synchronization, device, system, computing device and storage medium - Google Patents
A kind of method of data synchronization, device, system, computing device and storage medium Download PDFInfo
- Publication number
- CN108121757A CN108121757A CN201711108537.XA CN201711108537A CN108121757A CN 108121757 A CN108121757 A CN 108121757A CN 201711108537 A CN201711108537 A CN 201711108537A CN 108121757 A CN108121757 A CN 108121757A
- Authority
- CN
- China
- Prior art keywords
- data
- field
- database
- value
- sentence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method of data synchronization, device, system, computing device and storage mediums.Wherein, this method includes：The field name of the field of the first data in source database and field value are stored in intermediate store in the form of key-value pair；Based on the key-value pair in intermediate store, and according to intended conversion rule, generation is inserted into sentence suitable for the data of target database, and data insertion sentence restriction will be inserted into the field name and field value of the field of the second data of target database；And the second data are inserted into target database using data insertion sentence.The program can realize that the data between distinct type data-base are synchronous.
Description
Technical field
The present invention relates to a kind of methods of data synchronization between big data technical field more particularly to database.
Background technology
Database is to come tissue, storage and the warehouse for managing data according to data structure, and data are synchronously two differences
Database between data manipulation, by each data element mappings in a database into another database.Often
Data synchronization means has datax, sqoop, flume, embulk etc., and the application method of various data synchronization means respectively has not
Together, application scenarios are also had nothing in common with each other, and learning cost is higher, need to be understood its underlying principles and be debugged.It is applied for a certain kind
Scene is most suitable for most efficiently using which kind of instrument, and the technical staff to lack experience is often difficult to judge.
Therefore, there is still a need for a kind of general data synchronization solutions.
The content of the invention
The present invention provides a kind of general method of data synchronization, apparatus and system, can realize distinct type data-base it
Between data it is synchronous.
According to an aspect of the invention, there is provided a kind of method of data synchronization, including：By the first data in source database
Field field name and field value intermediate store is stored in the form of key-value pair；Based on the key-value pair in intermediate store,
And according to intended conversion rule, generation is inserted into sentence suitable for the data of target database, wherein, data insertion sentence limits will
It is inserted into the field name and field value of the field of the second data of target database；And sentence is inserted by the second data using data
It is inserted into target database.
Wherein, source database and target database are opposite that special data synchronization means is not required by this method,
It is synchronous data can be carried out to different types of database.
Preferably, the field name of data line in source database and field value can be write one in the form of key-value pair
Data structure cell, a data parenthesis sentence correspond to a data structure cell, with the insertion in target database or more
New data line.
In this way, a data parenthesis sentence is by reading the corresponding word of data line stored in a data structure cell
Section name and field value, by the field value of corresponding data line according in insertion target database after intended conversion rule conversion.
Preferably, intended conversion rule can be predetermined corresponding pass of the field with the field of target database of source database
System.The field value of the field of the second data can be according to predetermined correspondence from the key in intermediate store in data insertion sentence
It is worth centering to obtain.
And/or intended conversion rule can be the respective data of corresponding field of source database and target database
Form.The field value of the field of the second data can turn from the field value of the first data by data format in data insertion sentence
Become and obtain.
And/or intended conversion rule, which can also be, to be needed to be added in target database and is not belonging to source database
Field field name and field value.
And/or intended conversion rule can also be to be existed and data line in source database in target database
, it is necessary to the field name and field value of newer field in the case of corresponding data line.
And/or intended conversion rule can also apply to the form letter of the data insertion sentence of target database
Breath.
Preferably, this method further includes：Configuration interface is provided, for user configuration intended conversion rule.
Furthermore it is also possible in the configuration interface configuration data source information of offer and target database information.
Preferably, this method further includes：JSON character strings are configured, to limit intended conversion rule wherein；It and will
JSON character strings are converted into executable intended conversion rule.
Preferably, the method for data synchronization can be performed using Producer-consumer problem pattern.
Wherein, the field name of the field of the first data and field value are stored in centre by producer's module in the form of key-value pair
Memory.
Consumer module is inserted into sentence based on key-value pair and the generation of intended conversion rule suitable for the data of target database,
The second data are inserted into target database using data insertion sentence.
Can so multi-thread concurrent be used to handle synchrodata, improve the efficiency of data synchronization.
According to the second aspect of the invention, a kind of data synchronization unit is provided, can be included：
Transit module, for by the field name of the field of the first data in source database and field value with the shape of key-value pair
Formula is stored in intermediate store.
Generation module, for the key-value pair being stored in based on transit module, according to intended conversion rule, generation is suitable for target
The data insertion sentence of database, data insertion sentence restriction will be inserted into the field name of the field of the second data of target database
And field value.And
Module is inserted into, the second data are inserted into target database by the data insertion sentence for generation module to be used to generate.
By above device, data can be moved to from a database batch automatically according to intended conversion rule another
Special data synchronization means is not required in a database, realizes that the data between distinct type data-base are synchronous.
Preferably, which can also include：
Configuration module, for providing configuration interface, for user configuration intended conversion rule.
Preferably, which can include：Dispensing unit, for configuring JSON character strings, to limit predetermined turn wherein
Change rule；And converting unit, for the JSON character strings of configuration of described dispensing unit to be converted into executable intended conversion rule.
According to the third aspect of the invention we, a kind of data synchronous system is provided, first database, the second number can be included
According to storehouse and intermediate store.
Wherein it is possible to one in first database and the second database for source database, with first database and
Another in two databases is target database, and first database and the second database are realized by above-mentioned method of data synchronization
Between data it is synchronous.
According to the fourth aspect of the invention, a kind of computing device is provided, including：Processor；And memory, it deposits thereon
Executable code is contained, when executable code is executed by processor, processor is made to perform above-mentioned method of data synchronization.
According to the fifth aspect of the invention, a kind of non-transitory machinable medium is additionally provided, is stored thereon with
Executable code when executable code is performed by the processor of computing device, makes processor perform above-mentioned data synchronization side
Method.
By the present invention program, by weblication interface configurations transformation rule, by data from a database batch
Another database is moved to, can realize that the data between distinct type data-base are synchronous.
Description of the drawings
Disclosure illustrative embodiments are described in more detail in conjunction with the accompanying drawings, the disclosure above-mentioned and its
Its purpose, feature and advantage will be apparent, wherein, in disclosure illustrative embodiments, identical reference number
Typically represent same parts.
Fig. 1 shows the schematic flow chart of method of data synchronization according to an embodiment of the invention.
Fig. 2 shows the structure diagram of data synchronization unit according to an embodiment of the invention.
Fig. 3 shows the structure diagram of data synchronization unit according to an embodiment of the invention.
Fig. 4 shows the structure diagram of data synchronous system according to an embodiment of the invention.
Specific embodiment
The preferred embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although the disclosure is shown in attached drawing
Preferred embodiment, however, it is to be appreciated that may be realized in various forms the disclosure without the embodiment party that should be illustrated here
Formula is limited.On the contrary, these embodiments are provided so that the disclosure is more thorough and complete, and can be by the disclosure
Scope is intactly communicated to those skilled in the art.
As described above, in order to realize the synchronization of the data between distinct type data-base, the present invention provides a kind of general numbers
According to synchronous method, the data in source database are synchronized to target database by generating data insertion sentence.
Below with reference to the accompanying drawings with embodiment detailed description of the present invention technical solution.
Fig. 1 shows the schematic flow chart of method of data synchronization according to an embodiment of the invention.
As shown in Figure 1, in the step s 100, by the field name of the field of the first data in source database and field value with
The form deposit intermediate store of key-value pair.
Above-mentioned source database can be any type of database, such as HIVE, MYSQL, ACCESS, ORACLE, DB2,
Different databases may store different data structures, common data structure have array, chained list (list), queue, stack,
Tree, figure (map) etc., wherein, field name and field value, that is, map data structures of key-value pair form preserve the numbers of mapping relations
According to i.e. key and value, corresponding value can be taken out by specified key.Wherein, intermediate store can be memory or
External memory, convenient for different types of database access data.
In step s 110, based on the key-value pair in intermediate store, according to intended conversion rule, generation is suitable for target
The data insertion sentence of database, data insertion sentence restriction will be inserted into the field name of the field of the second data of target database
And field value.
Key-value pair in intermediate store can determine to need in addition to the field name specified in source database another
The outer field name and field value being added in target database.It may then pass through configuration intended conversion rule generation data insertion
Sentence.
Wherein, the form of data insertion sentence can be：Insert into table names (field 1, field 2, field 3)
Values (value 1, value 2, value 3).If specifying ON DUPLICATE KEY UPDATE at INSERT sentences end, and insert
It can cause to duplicate value in a UNIQUE index or PRIMARY KEY after entering a profession, then be performed in the row for duplicating value
UPDATE；If without repetition values, newline is inserted into.Target database is updated when being repeatedly inserted into.
It in a preferred embodiment, can be by the field name of data line in source database and field value with key-value pair
Form write a data structure cell, data parenthesis sentence corresponds to a data structure cell, in number of targets
According to inserted or updated data line in storehouse.
For example, every data line in specified source database table can be stored in map by the form of field name and field value
In data structure, i.e. intermediate store, when a data parenthesis sentence reads the first data from intermediate store, it can obtain
The corresponding field value of field name of the first data of a line.
In a preferred embodiment, a configuration interface can be provided, for user configuration intended conversion rule.
On configuration interface, the source database information and target that can be read with configuration data synchronous task information, needs
Database information, for example, can be with configuration task ID, source database type, target database type, library name, table name, source number
According to information such as list of fields, creation time and the renewal times in storehouse.
In a preferred embodiment, intended conversion rule can be configured by the following method：
First, for example, can based on user configuration interface configuration information, to configure JSON character strings, to limit wherein
Determine intended conversion rule.
Then, JSON character strings are converted into executable intended conversion rule.
For example, when program reads the JSON character strings of configuration, character string can be converted into transformation rule
(convertrule) example of class, and the description of specific each field can be converted into field mapping description (fieldmapdesc)
The example collection object of class, and the collection object is stored, to be read during program use.Because class is to things
Description, and instantiate and be equivalent to open one piece of memory for this description, thus it is possible to vary each attribute in this block region, with
Just actual use is carried out.
In a preferred embodiment, intended conversion rule can be the field of source database and the word of target database
The predetermined correspondence of section.The field value of the field of the second data can be according to predetermined correspondence therefrom in data insertion sentence
Between obtain in key-value pair in memory.
For example, when configuring intended conversion rule, source database field name corresponding with target database is specified, such as
{“hiveFn”：" dt ", " mysql "：" dt " }, { " hiveFn "：" channel ", " mysql "：" channel ",
{“hiveFn”：" ver ", " mysql "：" ver ", { " hiveFn "：" node ", " mysql "：" node ", { " hiveFn "：
" json_tree ", " mysql "：”json_tree”}.In two databases corresponding field name may as it is above-mentioned be it is identical,
It is also likely to be different, for example, working as the value of typemap is specified then to be converted into the value read in source database for " date "
The date format specified in toDt writes target database again.The field and target of source database can be determined by above-mentioned configuration
The predetermined correspondence of the field of database.
And/or intended conversion rule can be the respective data of corresponding field of source database and target database
Form.The field value of the field of the second data can turn from the field value of the first data by data format in data insertion sentence
Become and obtain.
For example, when configuring intended conversion rule, it is possible to specify the value of typemap is " int ", then will be read in source database
The value taken is converted into the value of numeric type in write-in target database.
And/or intended conversion rule can need to be added in target database and be not belonging to source database
The field name and field value of field.
For example, when configuring JSON character strings, by configuring " appendfield "：“create_time,update_
time”,”appendval”:" now (), now () ", specify the field name and field needed to be added additionally in target database
Value.
And/or intended conversion rule can be existed and data line pair in source database in target database
, it is necessary to the field name and field value of newer field in the case of the data line answered.
For example, when configuring JSON character strings, can configure " appenddka "：" update_time=now (),
“dka”：" json_tree ", it is necessary to newer field name and field in the presence of needing to be inserted into the data in target database
Value, uses the field name and field value in source database.
And/or intended conversion rule can also apply to the form letter of the data insertion sentence of target database
Breath.
For example, when configuring JSON character strings, by configuring " dkasplit "：", ", which specifies to work as, to be needed to be inserted into number of targets
According in the presence of the data in storehouse, it is necessary to used when updating multiple fields comma as separator or when need additionally incorporate
Field name and field value into target database are separated when having multiple with comma.
After being inserted into sentence suitable for the data of target database by above method generation, in the step s 120, use
Data are inserted into sentence and the second data are inserted into target database.
For example, sentence taking-up can be inserted into according to data needs the value of incoming sentence, carried out according to the order of transformation rule
It is incoming, the inserted or updated operation of target database is carried out by routine call.
In a preferred embodiment, the method for data synchronization can be performed using Producer-consumer problem pattern.Cause
This, as shown in Fig. 2, the data synchronization unit 200 can include producer's module 210 and consumer module 220, producer's module
210 include transit module 201, and consumer module 220 includes generation module 202 and insertion module 203.
During the field name of the field of first data and field value can be stored in by producer's module 210 in the form of key-value pair
Between memory.
For example, producer's module 210 can read the table specified and field according to the source database information of configuration, it will be every
Data line is closed according in the form deposit map of field name corresponding field value with preserving the mapping of the first data in source database
System.In the multiple map deposits list that multirow data can be formed, a batch output data bag as the producer is stored in number
It is used according in queue for consumer.
Consumer module 220 can be inserted based on key-value pair and intended conversion rule generation suitable for the data of target database
Enter sentence, the second data are inserted into target database using data insertion sentence.
For example, the intended conversion rule that consumer module 220 takes out from context, needs according to intended conversion rule assembling
The SQL statement of execution, then the batch of data bag that the producer produces is taken out from data queue, it will be each according to intended conversion rule
After field value is converted, using, the application program batch that can perform SQL statement is inserted or updated to target database.
In general, Producer-consumer problem pattern can set a buffering area, i.e. intermediate store, producer's module 210
Data are put into intermediate store, and consumer module 220 takes out data from intermediate store.In this way, producer's module 210 and consumption
Person's module 220 can be two independent concurrent main bodys, and program handles synchrodata using multi-thread concurrent, can improve data
Synchronous speed.
Therefore, producers and consumers' pattern can solve the problems, such as most concurrent that the pattern passes through line balancing
Journey and the ability to work of consuming thread improve the speed of the disposed of in its entirety data of program.
Further, this method can also include task scheduling, for example, by data source, transformation rule, exporting destination etc.
Configuration information configures a running example as a task for a task.It in this way, can be periodically from different relationship types
Database reads different data, and data are synchronized in target database by specified transformation rule, and records operation shape
State and run time.
Fig. 3 shows the structure diagram of data synchronization unit 200 according to an embodiment of the invention.
As shown in figure 3, the data synchronization unit 200 can include：Transit module 201, generation module 202 and insertion module
203。
Transit module 201 can be by the field name of the field of the first data in source database and field value with key-value pair
Form is stored in intermediate store.
The key-value pair that generation module 202 can be stored in based on transit module 201, according to intended conversion rule, generation is applicable in
Sentence is inserted into the data of target database, data insertion sentence restriction will be inserted into the field of the second data of target database
Field name and field value.
Second data are inserted into number of targets by the data insertion sentence that insertion module 203 can use generation module 202 to generate
According to storehouse.
By above device, data can be moved to from a database batch automatically according to intended conversion rule another
A database realizes that the rapid data between disparate databases is synchronous.
In a preferred embodiment, data synchronization unit 200 further includes configuration module (showing), can provide configuration
Interface, for user configuration intended conversion rule.
In configuration interface, user can be believed with configuration data synchronous task information, source database information, target database
Breath.Wherein, data synchronous task information can include task ID, task names, creation time, modification time etc., source database
Information can include：Database-name, table name, list of fields, creation time, modification time etc., target database information can
To include target database type, divide library name prefix, divide table prefix etc..
In a preferred embodiment, which can include dispensing unit and converting unit.
Dispensing unit can limit intended conversion in the configuration interface of offer by configuring conversion parameter JSON character strings
Rule.
The JSON character strings of configuration of described dispensing unit can be converted into executable transformation rule by converting unit.
Wherein, the form of JSON character strings can be { key1:value1,key2:Value2 ... }, in data transmission mistake
Cheng Zhong, JSON are transferred in the form of character string, and JS operations is JSON objects, so needing JSON character strings being converted into
The example of JSON objects, i.e. class, that is, it is converted into executable transformation rule.
For example, when configuring intended conversion rule, JSON character strings can be converted into the example of convertrule classes,
The description of each field is converted into the example of fieldmapdesc classes.
Fig. 4 shows the structure diagram of data synchronous system 300 according to an embodiment of the invention.
As shown in figure 4, data synchronous system 300 can include first database 301, intermediate store 302 and second counts
According to storehouse 303.
Wherein, with one in 301 and second database 303 of first database for source database, with first data
Another in 301 and second database 302 of storehouse is target database.
Intermediate store 302 can be main storage (memory) or additional storage (external memory), can be according to the position specified
It puts deposit and reads information.
Data synchronization scenario according to the present invention above is described in detail by reference to attached drawing.
Through the above scheme, special data synchronization means is not required, database can be directly acted on, is closed by mapping
Join source database and target database, write data insertion sentence, data are read from source database, it is inserted or updated to target
Database can quickly realize that the data between disparate databases are synchronous.
The present invention also provides a kind of computing device, including：Processor；And memory, it is stored thereon with executable generation
Code when executable code is executed by processor, makes processor perform method of data synchronization.
For example, it may be a computing device or intermediate server between source database and target database
For performing the method for data synchronization of the present invention.Intermediate store can on the computing device, can also outside it with its
Connection.
In addition, the method according to the invention is also implemented as a kind of computer program or computer program product, the meter
Calculation machine program or computer program product include performing the calculating of the above steps limited in the above method of the invention
Machine program code instruction.
Alternatively, the present invention can also be embodied as a kind of (or the computer-readable storage of non-transitory machinable medium
Medium or machine readable storage medium), executable code (or computer program or computer instruction code) is stored thereon with,
When the executable code (or computer program or computer instruction code) is by electronic equipment (or computing device, server
When) processor perform when, the processor is made to perform each step of the above method according to the present invention.
Those skilled in the art will also understand is that, with reference to the described various illustrative logical blocks of disclosure herein, mould
Block, circuit and algorithm steps may be implemented as the combination of electronic hardware, computer software or both.
Flow chart and block diagram in attached drawing show that the possibility of the system and method for multiple embodiments according to the present invention is real
Existing architectural framework, function and operation.In this regard, each box in flow chart or block diagram can represent module, a journey
A part for sequence section or code, as defined in the part of the module, program segment or code is used to implement comprising one or more
The executable instruction of logic function.It should also be noted that at some as in the realization replaced, the function of being marked in box also may be used
To be occurred with being different from the order marked in attached drawing.For example, two continuous boxes can essentially perform substantially in parallel,
They can also be performed in the opposite order sometimes, this is depending on involved function.It is also noted that block diagram and/or stream
The combination of each box in journey figure and the box in block diagram and/or flow chart can use functions or operations as defined in performing
Dedicated hardware based system realize or can be realized with the combination of specialized hardware and computer instruction.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.In the case of without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes will be apparent from for the those of ordinary skill in art field.The selection of term used herein, purport
It is best explaining the principle of each embodiment, practical application or to the improvement of the technology in market or is making the art
Other those of ordinary skill are understood that each embodiment disclosed herein.
Claims (12)
1. a kind of method of data synchronization, including：
The field name of the field of the first data in source database and field value are stored in intermediate store in the form of key-value pair；
Based on the key-value pair in the intermediate store, according to intended conversion rule, generation is suitable for the data of target database
Sentence is inserted into, the data insertion sentence restriction will be inserted into the field name and field of the field of the second data of target database
Value；And
Second data are inserted into target database using data insertion sentence.
2. according to the method described in claim 1, wherein, by the field name of data line in source database and field value with key assignments
To form write a data structure cell, data parenthesis sentence corresponds to a data structure cell,
With the inserted or updated data line in the target database.
3. according to the method described in claim 2, wherein, the intended conversion rule includes at least one of following：
The predetermined correspondence of the field of source database and the field of target database, the data are inserted into the second data in sentence
The field value of field be to be obtained according to the predetermined correspondence from the key-value pair in the intermediate store；
The respective data format of the corresponding field of source database and target database, the data are inserted into the second data in sentence
The field value of field is the field value for changing by data format and obtaining from the field value of first data；
Need to be added to the field name and field value for the field for being not belonging to the source database in the target database；
In the target database exist data line corresponding with data line in the source database in the case of, it is necessary to
The field name and field value of newer field；
Suitable for the format information of the data insertion sentence of target database.
4. it according to the method described in claim 3, further includes：
Configuration interface is provided, for intended conversion rule described in user configuration.
5. it according to the method described in claim 4, further includes：
JSON character strings are configured, to limit the intended conversion rule wherein；And
The JSON character strings are converted into executable intended conversion rule.
6. according to the method any one of claim 1-5, wherein, perform the number using Producer-consumer problem pattern
According to synchronous method, wherein,
The field name of the field of first data and field value are stored in intermediate storage by producer's module in the form of key-value pair
Device；
Consumer module is inserted into based on the key-value pair and the generation of intended conversion rule suitable for the data of target database
Second data are inserted into target database by sentence using data insertion sentence.
7. a kind of data synchronization unit, including：
Transit module, for the field name of the field of the first data in source database and field value to be deposited in the form of key-value pair
Enter intermediate store；
Generation module, for the key-value pair being stored in based on the transit module, according to intended conversion rule, generation is suitable for target
The data insertion sentence of database, the data insertion sentence restriction will be inserted into the word of the field of the second data of target database
Section name and field value；And
Module is inserted into, second data are inserted into target data by the data insertion sentence for being generated using the generation module
Storehouse.
8. device according to claim 7, further includes：
Configuration module, for providing configuration interface, for intended conversion rule described in user configuration.
9. device according to claim 8, further includes：
Dispensing unit, for configuring JSON character strings, to limit the intended conversion rule wherein；And
Converting unit is advised for the JSON character strings of the configuration of described dispensing unit to be converted into executable intended conversion
Then.
10. a kind of data synchronous system, including first database, the second database and intermediate store, wherein, with described first
One in database and the second database is source database, using in the first database and the second database another as
Target database realizes the first database and described the by method according to any one of claim 1 to 6
Data between two databases are synchronous.
11. a kind of computing device, including：
Processor；And
Memory is stored thereon with executable code, when the executable code is performed by the processor, makes the processing
Device performs the method as any one of claim 1 to 6.
12. a kind of non-transitory machinable medium, is stored thereon with executable code, when the executable code is counted
When calculating the processor execution of equipment, the processor is made to perform such as method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711108537.XA CN108121757A (en) | 2017-11-10 | 2017-11-10 | A kind of method of data synchronization, device, system, computing device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711108537.XA CN108121757A (en) | 2017-11-10 | 2017-11-10 | A kind of method of data synchronization, device, system, computing device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108121757A true CN108121757A (en) | 2018-06-05 |
Family
ID=62227609
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711108537.XA Pending CN108121757A (en) | 2017-11-10 | 2017-11-10 | A kind of method of data synchronization, device, system, computing device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108121757A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920592A (en) * | 2018-06-26 | 2018-11-30 | 杭州骑轻尘信息技术有限公司 | Database field correlating method and device |
| CN109165200A (en) * | 2018-08-10 | 2019-01-08 | 北京奇虎科技有限公司 | Method of data synchronization, calculates equipment and computer storage medium at device |
| CN109284334A (en) * | 2018-09-05 | 2019-01-29 | 拉扎斯网络科技（上海）有限公司 | Real-time data base synchronous method, device, electronic equipment and storage medium |
| CN109885614A (en) * | 2019-01-04 | 2019-06-14 | 平安科技（深圳）有限公司 | A kind of method and apparatus that data are synchronous |
| CN110334141A (en) * | 2019-05-30 | 2019-10-15 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium of data conversion |
| CN110765069A (en) * | 2019-09-29 | 2020-02-07 | 北京淇瑀信息科技有限公司 | Data migration method, device and system |
| CN110851511A (en) * | 2019-10-09 | 2020-02-28 | 上海易点时空网络有限公司 | Data synchronization method and device |
| CN110909075A (en) * | 2019-10-30 | 2020-03-24 | 北京达佳互联信息技术有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN111090640A (en) * | 2019-11-13 | 2020-05-01 | 山东中磁视讯股份有限公司 | ETL data cleaning method and system |
| CN111259104A (en) * | 2020-01-22 | 2020-06-09 | 山东浪潮通软信息科技有限公司 | Report data synchronization method and device capable of defining rules |
| CN111459943A (en) * | 2020-04-03 | 2020-07-28 | 中国建设银行股份有限公司 | Data processing method, device, system, equipment and storage medium |
| CN112817990A (en) * | 2021-01-28 | 2021-05-18 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and readable storage medium |
| CN113377871A (en) * | 2021-06-22 | 2021-09-10 | 特赞（上海）信息科技有限公司 | Data synchronization method, device and storage medium |
| CN116909688A (en) * | 2023-09-14 | 2023-10-20 | 中移(苏州)软件技术有限公司 | Database calling method and device, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102542007A (en) * | 2011-12-13 | 2012-07-04 | 中国电子科技集团公司第十五研究所 | Method and system for synchronization of relational databases |
| CN102999537A (en) * | 2011-09-19 | 2013-03-27 | 阿里巴巴集团控股有限公司 | System and method for data migration |
| US20130117227A1 (en) * | 2011-11-07 | 2013-05-09 | Empire Technology Development, Llc | Cache based key-value store mapping and replication |
| US20130332415A1 (en) * | 2012-06-07 | 2013-12-12 | Wal-Mart Stores, Inc. | Throttling mechanism |
| CN106339498A (en) * | 2016-09-05 | 2017-01-18 | 北京微播易科技股份有限公司 | Method, device and system for synchronizing data |
| CN106649378A (en) * | 2015-11-02 | 2017-05-10 | 北大方正集团有限公司 | Data synchronizing method and device |
-
2017
- 2017-11-10 CN CN201711108537.XA patent/CN108121757A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999537A (en) * | 2011-09-19 | 2013-03-27 | 阿里巴巴集团控股有限公司 | System and method for data migration |
| US20130117227A1 (en) * | 2011-11-07 | 2013-05-09 | Empire Technology Development, Llc | Cache based key-value store mapping and replication |
| CN102542007A (en) * | 2011-12-13 | 2012-07-04 | 中国电子科技集团公司第十五研究所 | Method and system for synchronization of relational databases |
| US20130332415A1 (en) * | 2012-06-07 | 2013-12-12 | Wal-Mart Stores, Inc. | Throttling mechanism |
| CN106649378A (en) * | 2015-11-02 | 2017-05-10 | 北大方正集团有限公司 | Data synchronizing method and device |
| CN106339498A (en) * | 2016-09-05 | 2017-01-18 | 北京微播易科技股份有限公司 | Method, device and system for synchronizing data |
Non-Patent Citations (3)
| Title |
|---|
| 周红波: ""基于XML的数据库交换技术研究"", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
| 梁合功: ""胜利油田异构数据迁移系统设计与实现"", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
| 陈健: ""卓越亚马逊新旧系统的数据迁移实现"", 《万方学位论文》 * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920592B (en) * | 2018-06-26 | 2020-11-10 | 杭州一骑轻尘信息技术有限公司 | Database field association method and device |
| CN108920592A (en) * | 2018-06-26 | 2018-11-30 | 杭州骑轻尘信息技术有限公司 | Database field correlating method and device |
| CN109165200A (en) * | 2018-08-10 | 2019-01-08 | 北京奇虎科技有限公司 | Method of data synchronization, calculates equipment and computer storage medium at device |
| CN109165200B (en) * | 2018-08-10 | 2022-04-01 | 北京奇虎科技有限公司 | Data synchronization method and device, computing equipment and computer storage medium |
| CN109284334A (en) * | 2018-09-05 | 2019-01-29 | 拉扎斯网络科技（上海）有限公司 | Real-time data base synchronous method, device, electronic equipment and storage medium |
| CN109284334B (en) * | 2018-09-05 | 2022-01-28 | 拉扎斯网络科技（上海）有限公司 | Real-time database synchronization method and device, electronic equipment and storage medium |
| CN109885614A (en) * | 2019-01-04 | 2019-06-14 | 平安科技（深圳）有限公司 | A kind of method and apparatus that data are synchronous |
| CN110334141A (en) * | 2019-05-30 | 2019-10-15 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium of data conversion |
| CN110334141B (en) * | 2019-05-30 | 2023-11-21 | 平安科技（深圳）有限公司 | Method, device, computer equipment and storage medium for data conversion |
| CN110765069A (en) * | 2019-09-29 | 2020-02-07 | 北京淇瑀信息科技有限公司 | Data migration method, device and system |
| CN110765069B (en) * | 2019-09-29 | 2024-05-17 | 北京淇瑀信息科技有限公司 | Data migration method, device and system |
| CN110851511A (en) * | 2019-10-09 | 2020-02-28 | 上海易点时空网络有限公司 | Data synchronization method and device |
| CN110909075A (en) * | 2019-10-30 | 2020-03-24 | 北京达佳互联信息技术有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN110909075B (en) * | 2019-10-30 | 2022-08-09 | 北京达佳互联信息技术有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN111090640A (en) * | 2019-11-13 | 2020-05-01 | 山东中磁视讯股份有限公司 | ETL data cleaning method and system |
| CN111259104A (en) * | 2020-01-22 | 2020-06-09 | 山东浪潮通软信息科技有限公司 | Report data synchronization method and device capable of defining rules |
| CN111459943A (en) * | 2020-04-03 | 2020-07-28 | 中国建设银行股份有限公司 | Data processing method, device, system, equipment and storage medium |
| CN112817990B (en) * | 2021-01-28 | 2024-03-08 | 北京百度网讯科技有限公司 | Data processing method, device, electronic equipment and readable storage medium |
| CN112817990A (en) * | 2021-01-28 | 2021-05-18 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and readable storage medium |
| CN113377871A (en) * | 2021-06-22 | 2021-09-10 | 特赞（上海）信息科技有限公司 | Data synchronization method, device and storage medium |
| CN113377871B (en) * | 2021-06-22 | 2024-03-22 | 特赞（上海）信息科技有限公司 | Data synchronization method, device and storage medium |
| CN116909688B (en) * | 2023-09-14 | 2024-01-26 | 中移(苏州)软件技术有限公司 | Database calling method and device, storage medium and electronic equipment |
| CN116909688A (en) * | 2023-09-14 | 2023-10-20 | 中移(苏州)软件技术有限公司 | Database calling method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108121757A (en) | A kind of method of data synchronization, device, system, computing device and storage medium | |
| CN105335412B (en) | For data conversion, the method and apparatus of Data Migration | |
| CN105446966B (en) | The method and apparatus that production Methods data are converted to the mapping ruler of RDF format data | |
| CN106055584B (en) | Manage data query | |
| CN107038262B (en) | A kind of Semantic Modeling Method based on data map, Information Atlas and knowledge mapping frame that association frequency calculates | |
| US7613734B2 (en) | Systems and methods for providing data sets using a store of albegraic relations | |
| CN106575246A (en) | Machine learning service | |
| CN106663038A (en) | Feature processing recipes for machine learning | |
| US7769754B2 (en) | Systems and methods for data storage and retrieval using algebraic optimization | |
| JP2017509971A (en) | Specify and apply logical validation rules to data | |
| US20070266000A1 (en) | Systems and Methods for Data Storage and Retrieval Using Virtual Data Sets | |
| CN106547766A (en) | A kind of data access method and device | |
| US20130091184A1 (en) | Correlating independent schema mappings | |
| Dragomir et al. | Model checking kernel P systems | |
| CN106445645B (en) | Method and apparatus for executing distributed computing task | |
| Alhazov et al. | Space complexity equivalence of P systems with active membranes and Turing machines | |
| CN106933859A (en) | The moving method and device of a kind of medical data | |
| CN107016115A (en) | Data export method, device, computer-readable recording medium and electronic equipment | |
| Leporati et al. | A survey on space complexity of P systems with active membranes | |
| CN107688609A (en) | A kind of position label recommendation method and computing device | |
| US20190303478A1 (en) | Path query evaluation in graph databases | |
| CN109446268A (en) | A kind of method of data synchronization and device | |
| EP3306540A1 (en) | System and method for content affinity analytics | |
| CN108701153A (en) | Use the inquiry response of the mapping to parametrization report | |
| Deharbe et al. | The omniscient garbage collector: A resource analysis framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200902 Address after: 310052 room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: 510627 Guangdong city of Guangzhou province Whampoa Tianhe District Road No. 163 Xiping Yun Lu Yun Ping square B radio tower 15 layer self unit 02 Applicant before: GUANGZHOU UC NETWORK TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180605 |
|
| RJ01 | Rejection of invention patent application after publication |