CN110347693A - Method of data synchronization, device and storage medium - Google Patents
Method of data synchronization, device and storage medium Download PDFInfo
- Publication number
- CN110347693A CN110347693A CN201910627681.7A CN201910627681A CN110347693A CN 110347693 A CN110347693 A CN 110347693A CN 201910627681 A CN201910627681 A CN 201910627681A CN 110347693 A CN110347693 A CN 110347693A
- Authority
- CN
- China
- Prior art keywords
- data
- tables
- database
- synchronization
- synchronous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The present invention relates to data processing technique, a kind of method of data synchronization, electronic device and storage medium are proposed.This method comprises: the synchronization request of the first tables of data to source database of response user's triggering, obtains the synchronous path and database information of first tables of data stored in the configuration file of the source database；Library information logs in source database and target database respectively based on the data, and it is whether consistent with the table structure of the second tables of data of the target database to compare first tables of data；If inconsistent, according to the second data table name of default naming rule modification target database, in the newly-built third tables of data identical as first data list structure and identical with the table name of second tables of data of the target database；The data read are synchronized to the third tables of data according to the synchronous path by the data for reading the first tables of data storage.Using the present invention, the efficiency of data deposit database is improved.
Description
Technical field
The present invention relates to technical field of data processing more particularly to a kind of method of data synchronization, device and storage medium.
Background technique
The fast development of internet so that the data of application are presented the trend that is skyrocketed through, enterprise need by storage this
These data are synchronized in big data platform by a little data by different types of service database, more convenient after being conducive to
Ground uses these data.In data synchronization process, it is frequently encountered the changed feelings of table structure for the database being synchronized
Condition, this causes difficulty to data synchronous working.
At present in the industry in data synchronization process, when encountering the table structure of distinct type data-base and changing, often need
The artificial tool carried using database is reminded, data are exported into file, then file is handled and is imported into again separately
One database.This method needs continuous manual intervention, not only low efficiency, but also is easy to appear mistake.
Summary of the invention
The present invention provides a kind of method of data synchronization, device and storage medium, and main purpose is to promote data deposit number
According to the efficiency in library.
To achieve the above object, the present invention provides a kind of method of data synchronization, method includes the following steps:
Response of step: the synchronization request of the first tables of data to source database of response user's triggering obtains the source number
According to the synchronous path and database information of first tables of data stored in the configuration file in library, wherein the database letter
Breath includes the account of source database and the account and password of password and target database；
Comparison step: library information logs in source database and target database respectively based on the data, obtains described first
The table structure of second tables of data of tables of data and the target database compares the table of first tables of data, the second tables of data
Whether structure is consistent, and the table structure of the tables of data includes the field name data type corresponding with field name of tables of data；
Amendment step: if first tables of data, the table structure of the second tables of data are inconsistent, according to default naming rule
The table name for modifying second tables of data, it is newly-built identical as first data list structure in the target database and with it is described
The identical third tables of data of the table name of second tables of data；And
Synchronizing step: within a preset time, reading the data of first tables of data storage, according to the synchronous path,
The data read are synchronized to the third tables of data.
It preferably, is increment synchronization by the method for synchronization that the data of first tables of data are synchronized to the third tables of data
Or full dose is synchronous.
Preferably, synchronous for full dose when the data of first tables of data to be synchronized to the mode of the third tables of data
When, the synchronizing step includes:
The data volume for inquiring first tables of data, according to query result according to preset batch rule by first data
Data in table are in turn divided into multiple batches, and save obtained data batch number is divided into default log sheet；
Data successively are read from first tables of data according to the data batch number in the log sheet.
Preferably, the table structure variation of the tables of data includes: newly to add, delete column, the one or more column of any adjustment
The combination of position.
Preferably, when data synchronously complete, first tables of data, the data volume of third tables of data are counted respectively, if
The data volume of first tables of data is equal with the data volume of the third tables of data, then shows that data synchronize successfully prompt letter
Breath feeds back to the user.
In addition, to achieve the above object, the present invention also provides a kind of electronic device, which includes memory and place
Device is managed, is stored with the data synchronization program that can be run on the processor, the data synchronization program quilt in the memory
The processor realizes following steps when executing:
Response of step: the synchronization request of the first tables of data to source database of response user's triggering obtains the source number
According to the synchronous path and database information of first tables of data stored in the configuration file in library, wherein the database letter
Breath includes the account of source database and the account and password of password and target database；
Comparison step: library information logs in source database and target database respectively based on the data, obtains described first
The table structure of second tables of data of tables of data and the target database compares the table of first tables of data, the second tables of data
Whether structure is consistent, and the table structure of the tables of data includes the field name data type corresponding with field name of tables of data；
Amendment step: if first tables of data, the table structure of the second tables of data are inconsistent, according to default naming rule
The table name for modifying second tables of data, it is newly-built identical as first data list structure in the target database and with it is described
The identical third tables of data of the table name of second tables of data；And
Synchronizing step: within a preset time, reading the data of first tables of data storage, according to the synchronous path,
The data read are synchronized to the third tables of data.
It preferably, is increment synchronization by the method for synchronization that the data of first tables of data are synchronized to the third tables of data
Or full dose is synchronous.
Preferably, synchronous for full dose when the data of first tables of data to be synchronized to the mode of the third tables of data
When, the synchronizing step includes:
The data volume for inquiring first tables of data, according to query result according to preset batch rule by first data
Data in table are in turn divided into multiple batches, and save obtained data batch number is divided into default log sheet；
Data successively are read from first tables of data according to the data batch number in the log sheet.
Preferably, when data synchronously complete, the data volume and the third data of first tables of data are counted respectively
The data volume of table shows that data synchronize successfully prompt letter if first tables of data, the data volume of third tables of data are equal
Breath feeds back to the user.
Method of data synchronization, server and computer readable storage medium proposed by the present invention, pair of response user's triggering
The synchronization request of first tables of data of source database obtains first data stored in the configuration file of the source database
The synchronous path and database information of table；Library information logs in source database and target database respectively based on the data, obtains
The table structure of second tables of data of first tables of data and the target database compares first tables of data, the second number
It is whether consistent according to the table structure of table；If inconsistent, the table name of second tables of data is modified according to default naming rule, in institute
State the newly-built third number identical as first data list structure and identical with the table name of second tables of data of target database
According to table；The data read are synchronized to described the according to the synchronous path by the data for reading first tables of data storage
Three tables of data effectively improve the efficiency of data deposit database.
Detailed description of the invention
Fig. 1 is the flow chart of one embodiment of method of data synchronization of the present invention；
Fig. 2 is the schematic diagram of internal structure of one embodiment of electronic device of the present invention；
Fig. 3 is the Program modual graph of one embodiment of data synchronization program in Fig. 2.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
Shown in referring to Fig.1, for the flow diagram for the method for data synchronization that one embodiment of the invention provides.This method can be with
It is executed by an electronic device, which can be by software and or hardware realization.
In the present embodiment, which includes:
Step S10, the synchronization request of the first tables of data to source database of response user's triggering, obtains the source data
The synchronous path and database information of first tables of data stored in the configuration file in library, wherein the database information
The account and password of account and password and target database including source database.
In one embodiment of the invention, if receiving the synchronization request of client initiation, the user need to be verified whether
It has permission and implements data simultaneously operating, if user does not implement the permission of data simultaneously operating, user is prompted to have no right operation
Information.In the present embodiment, configuration file is previously stored with the synchronous path and database information of the first tables of data.
Step S20, library information logs in source database and target database respectively based on the data, obtains first number
According to table and the table structure of the second tables of data of the target database, the table knot of first tables of data, the second tables of data is compared
Whether structure is consistent, and the table structure of the tables of data includes the field name data type corresponding with field name of tables of data.
Step S30, if first tables of data, the table structure of the second tables of data are inconsistent, according to default naming rule
The table name for modifying second tables of data, it is newly-built identical as first data list structure in the target database and with it is described
The identical third tables of data of the table name of second tables of data.
In one embodiment of the invention, naming rule is preset are as follows: modify the second data table name of target database
For former table name+time (year-month-day)+syn.
For example, the first tables of data is " user clicks statistics " tables of data, being somebody's turn to do " user clicks statistics " data sheet field includes:
Serial number, clicks time, click location at user alias.It is desirable to which " user institute is added in " user clicks statistics " tables of data
In region " this field, since user region is added " field leads to " user click statistics " tables of data of source database
Table structure changes, and leads to not complete data simultaneously operating.When the table structure of comparison the first tables of data and the second tables of data
When changing, the table name of " user clicks statistics " table of target database is changed to " user's click according to default naming rule
+ xx years xx months xx day+syn of statistics ".
Step S40 reads the data of the first tables of data storage within a preset time, will according to the synchronous path
The data read are synchronized to the third tables of data.
In one embodiment of the invention, the data of first tables of data are synchronized to the same of the third tables of data
Step mode is that increment synchronization or full dose are synchronous.Full dose synchronizes to be to whole system include system file and application data carry out complete
Full backup.The synchronous advantage of full dose is short the time required to data are restored.The disadvantage is that have in synchrodata a large amount of contents be it is duplicate,
These duplicate data occupy a large amount of memory space, virtually increase the synchronous cost of data；Furthermore it is same due to needing
The data volume of step is very big, therefore synchronous required time is relatively long.
Increment synchronization refers to that the last time is only comparable to per hyposynchronous data synchronizes data that are rear increased and modifying.This
The advantages of kind method of synchronization is no duplicate synchrodata, saves memory space, and shorten synchronization time.But the disadvantage is that extensive
Complex data is more troublesome.
Few for data volume, change measures big data source and uses full dose backed up in synchronization, big to data volume, changes small data
Increment synchronization is taken to back up in source.
Such as:
A1. the people of Sunday to Monday access website is fewer, backs up Sunday in such a way that full dose is synchronous to Monday therebetween
The data for changing or increasing newly.
A2. the people of Monday to Tuesday access website is relatively more, and backed up Monday by the way of increment synchronization is become between Tuesday
The data more crossed or increased newly.
In the same embodiment of the invention, when data synchronously complete, the data of first tables of data are counted respectively
The data volume of amount and second tables of data, if the data volume phase of the data volume of first tables of data and second tables of data
Deng then showing that data synchronize successful prompt information and feed back to the user.
The present invention also proposes a kind of electronic device.It is the electronic device that one embodiment of the invention provides referring to shown in Fig. 2
Schematic diagram of internal structure.
In the present embodiment, the electronic device 1 includes at least memory 11, processor 12, network interface 13, Yi Jitong
Believe bus.
Wherein, the memory 11 includes at least a type of readable storage medium storing program for executing, and the readable storage medium storing program for executing includes
Flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory etc.), magnetic storage, disk, CD etc..It deposits
Reservoir 11 can be the internal storage unit of electronic device 1, such as the hard disk of the electronic device 1 in some embodiments.Storage
Device 11 is also possible to the grafting being equipped on the External memory equipment of electronic device 1, such as electronic device 1 in further embodiments
Formula hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card
(Flash Card) etc..Further, memory 11 can also both including electronic device 1 internal storage unit and also including outside
Store equipment.Memory 11 can be not only used for the application software and Various types of data that storage is installed on electronic device 1, such as data
The code etc. of synchronization program 10 can be also used for temporarily storing the data that has exported or will export.
Processor 12 can be in some embodiments a central processing unit (Central Processing Unit,
CPU), controller, microcontroller, microprocessor or other data processing chips, the program for being stored in run memory 11
Code or processing data, such as execute data synchronization program 10 etc..
Network interface 13 optionally may include standard wireline interface and wireless interface (such as WI-FI interface), be commonly used in
Communication connection is established between the electronic device 1 and other electronic equipments.
Communication bus is for realizing the connection communication between these components.
Optionally, the electronic device 1 can also include user interface, user interface may include display (Display),
Input unit such as keyboard (Keyboard), optional user interface can also include standard wireline interface and wireless interface.It can
Selection of land, in some embodiments, display can be light-emitting diode display, liquid crystal display, touch-control liquid crystal display and OLED
(Organic Light-Emitting Diode, Organic Light Emitting Diode) touches device etc..Wherein, display can also be appropriate
Referred to as display screen or display unit, for showing the information handled in the electronic apparatus 1 and for showing visual user
Interface.
Fig. 2 illustrates only the electronic device 1 with component 11-13 and data synchronization program 10, those skilled in the art
It is understood that Fig. 2 shows structure do not constitute the restriction to electronic device 1, may include less or more than illustrating
More components perhaps combines certain components or different component layouts.
In 1 embodiment of electronic device shown in Fig. 2, data synchronization program 10, processor 12 are stored in memory 11
Following steps are realized when executing the data synchronization program 10 stored in memory 11:
Response of step: the synchronization request of the first tables of data to source database of response user's triggering obtains the source number
According to the synchronous path and database information of first tables of data stored in the configuration file in library, wherein the database letter
Breath includes the account of source database and the account and password of password and target database；
Comparison step: library information logs in source database and target database respectively based on the data, obtains described first
The table structure of second tables of data of tables of data and the target database compares the table of first tables of data, the second tables of data
Whether structure is consistent, and the table structure of the tables of data includes the field name data type corresponding with field name of tables of data；
Amendment step: if first tables of data, the table structure of the second tables of data are inconsistent, according to default naming rule
The table name for modifying second tables of data, it is newly-built identical as first data list structure in the target database and with it is described
The identical third tables of data of the table name of second tables of data；And
Synchronizing step: within a preset time, reading the data of first tables of data storage, according to the synchronous path,
The data read are synchronized to the third tables of data.
Concrete principle please refers to introduction of following Fig. 3 about the Program modual graph of data synchronization program 10, does not make to tire out herein
It states.
Optionally, in other examples, data synchronization program can also be divided into one or more module, and one
A or multiple modules are stored in memory 11, and are held by one or more processors (the present embodiment is by processor 12)
For row to complete the present invention, the so-called module of the present invention is the series of computation machine program instruction section for referring to complete specific function,
For describing the implementation procedure of data synchronization program in an electronic.
It is the journey of the data synchronization program in 1 one embodiment of electronic device provided by the invention for example, referring to shown in Fig. 3
Sequence module diagram, in the embodiment, data synchronization program 10 can be divided into respond module 110, contrast module 120, repair
Change module 130, synchronization module 140, illustratively:
Respond module 110: for responding the synchronization request of the first tables of data to source database of user's triggering, institute is obtained
State the synchronous path and database information of first tables of data stored in the configuration file of source database, wherein the number
It include the account of source database and the account and password of password and target database according to library information.
In one embodiment of the invention: source database can for oracle database (oracle database also known as
Oracle RDBMS or abbreviation Oracle is a relational database management system of Oracle), MySQL database
(the small-sized correlation data base management system that MySQL is an open source code), the target database can be hive data
Library (hive is a Tool for Data Warehouse based on Hadoop).
Target database can be Hive database, Hive database include: data warehouse be divided into ODS data storage layer,
DWD data detail layer, DWS data summarization layer, ADS data application layer；
ODS data storage layer is data buffer storage layer, for storing the initial data obtained, when retaining a regular length
Between, any processing is not done to data；
DWD data detail layer is used to clean the data of ODS data storage layer, transcoding, increment turn full dose, to table name
Word stores after carrying out unified standard with field name；
DWS data summarization layer is used for subject-oriented group organization data, according to requirements of service constructs multidimensional model data, carries out related
The fractionation of Data Integration, related service in subject area summarizes；
ADS data application layer is used to need to construct multidimensional model data according to service application, and the data obtained is directly used in point
Analysis shows, this layer also takes on the construction of thematic class data model.
Contrast module 120: source database and target database are logged in respectively for library information based on the data, obtains institute
The table structure of the second tables of data of the first tables of data and the target database is stated, first tables of data, the second data are compared
Whether the table structure of table is consistent, and the table structure of the tables of data includes the field name data corresponding with field name of tables of data
Type.
Modified module 130: if inconsistent for first tables of data, the table structure of the second tables of data, according to default
Naming rule modifies the table name of second tables of data, newly-built identical as first data list structure in the target database
And third tables of data identical with the table name of second tables of data.
Synchronization module 140: within a preset time, reading the data of the first tables of data storage, according to described same
The data read are synchronized to the third tables of data by step diameter.
In one embodiment of the invention, the data of first tables of data are synchronized to the same of the third tables of data
Step mode is that increment synchronization or full dose are synchronous.Full dose synchronizes to be to whole system include system file and application data carry out complete
Full backup.The synchronous advantage of full dose is short the time required to data are restored.The disadvantage is that have in synchrodata a large amount of contents be it is duplicate,
These duplicate data occupy a large amount of memory space, virtually increase the synchronous cost of data；Furthermore it is same due to needing
The data volume of step is very big, therefore synchronous required time is relatively long.
Increment synchronization refers to that the last time is only comparable to per hyposynchronous data synchronizes data that are rear increased and modifying.This
The advantages of kind method of synchronization is no duplicate synchrodata, saves memory space, and shorten synchronization time.But the disadvantage is that extensive
Complex data is more troublesome.
Few for data volume, change measures big data source and uses full dose backed up in synchronization, big to data volume, changes small data
Increment synchronization is taken to back up in source.
When the mode that the data of first tables of data are synchronized to the third tables of data is that full dose is synchronous, institute is inquired
The data volume for stating the first tables of data, according to query result according to preset batch rule by the data in first tables of data successively
Multiple batches are divided into, and are saved obtained data batch number is divided into default log sheet；
Data successively are read from first tables of data according to the data batch number in the log sheet.
Synchronous data are needed there are 30000 for example, inquiring, every time synchronous 10000 datas of most multipotency, preset batch
Rule can synchronize for synchronous data will be needed uniformly to be divided into 3 batches according to arrangement serial number, per subsynchronous 10000
Data, data can be synchronously completed by executing 3 subsynchronous operations.
In another embodiment of the invention, third tables of data store historical data is stored in the per hyposynchronous data
In the different subregions of three tables of data.In addition, creating the 4th tables of data in target database is used to store current newest synchronization
Data.After each third data table data synchronously completes, the hyposynchronous data of third tables of data sheet are synchronized to the 4th tables of data,
Cover the data of the 4th tables of data storage.
For example, third tables of data receives point 3 batches of hyposynchronous 30000 numbers after this executes data simultaneously operating
According to.This 30000 data is also backuped to the 4th tables of data by data synchronization program, and the 4th data only store current newest
Upper one synchronous data of 30000 datas, the storage of the 4th tables of data have been capped.
In addition, the embodiment of the present invention also proposes a kind of storage medium, data synchronization program is stored on the storage medium,
The data synchronization program can be executed by one or more processors, to realize following steps:
Response of step: the synchronization request of the first tables of data to source database of response user's triggering obtains the source number
According to the synchronous path and database information of first tables of data stored in the configuration file in library, wherein the database letter
Breath includes the account of source database and the account and password of password and target database；
Comparison step: library information logs in source database and target database respectively based on the data, obtains described first
The table structure of second tables of data of tables of data and the target database compares the table of first tables of data, the second tables of data
Whether structure is consistent, and the table structure of the tables of data includes the field name data type corresponding with field name of tables of data；
Amendment step: if first tables of data, the table structure of the second tables of data are inconsistent, according to default naming rule
The table name for modifying second tables of data, it is newly-built identical as first data list structure in the target database and with it is described
The identical third tables of data of the table name of second tables of data；And
Synchronizing step: within a preset time, reading the data of first tables of data storage, according to the synchronous path,
The data read are synchronized to the third tables of data.
Storage medium specific embodiment of the present invention and above-mentioned electronic device 1 and each embodiment of system are essentially identical, herein
Do not make tired state.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, device, article or the method that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, device, article or method institute it is intrinsic
Element.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone,
Computer, server or network equipment etc.) execute method described in each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of method of data synchronization is applied to electronic device, which is characterized in that the described method includes:
Response of step: the synchronization request of the first tables of data to source database of response user's triggering obtains the source database
Configuration file in the synchronous path and database information of first tables of data that store, wherein the database information packet
Include the account of source database and the account and password of password and target database；
Comparison step: library information logs in source database and target database respectively based on the data, obtains first data
The table structure of second tables of data of table and the target database compares first tables of data, the table structure of the second tables of data
Whether consistent, the table structure of the tables of data includes the field name data type corresponding with field name of tables of data；
Amendment step: it if first tables of data, the table structure of the second tables of data are inconsistent, is modified according to default naming rule
The table name of second tables of data is newly-built identical as first data list structure in the target database and with described second
The table name of tables of data just as third tables of data；And
Synchronizing step: within a preset time, the data for reading the first tables of data storage will be read according to the synchronous path
The data got are synchronized to the third tables of data.
2. method of data synchronization as described in claim 1, which is characterized in that the data of first tables of data are synchronized to institute
The method of synchronization for stating third tables of data is that increment synchronization or full dose are synchronous.
3. method of data synchronization as claimed in claim 2, which is characterized in that be synchronized to when by the data of first tables of data
When the mode of the third tables of data is that full dose is synchronous, the synchronizing step includes:
The data volume for inquiring first tables of data, will be in first tables of data according to preset batch rule according to query result
Data be in turn divided into multiple batches, and save obtained data batch number is divided into default log sheet；
Data successively are read from first tables of data according to the data batch number in the log sheet.
4. method of data synchronization as described in claim 1, which is characterized in that the table structure variation of the tables of data includes: new
It adds, delete column, any combination for adjusting one or more column positions.
5. the method for data synchronization as described in Claims 1-4 any one, which is characterized in that when data synchronously complete, point
First tables of data, the data volume of third tables of data are not counted, if the data volume of first tables of data and the third number
It is equal according to the data volume of table, then show that data synchronize successful prompt information and feed back to the user.
6. a kind of electronic device, including memory and processor, which is characterized in that include the synchronous journey of data in the memory
Sequence, the data synchronization program realize following steps when being executed by the processor:
Response of step: the synchronization request of the first tables of data to source database of response user's triggering obtains the source database
Configuration file in the synchronous path and database information of first tables of data that store, wherein the database information packet
Include the account of source database and the account and password of password and target database；
Comparison step: library information logs in source database and target database respectively based on the data, obtains first data
The table structure of second tables of data of table and the target database compares first tables of data, the table structure of the second tables of data
Whether consistent, the table structure of the tables of data includes the field name data type corresponding with field name of tables of data；
Amendment step: it if first tables of data, the table structure of the second tables of data are inconsistent, is modified according to default naming rule
The table name of second tables of data is newly-built identical as first data list structure in the target database and with described second
The table name of tables of data just as third tables of data；And
Synchronizing step: within a preset time, the data for reading the first tables of data storage will be read according to the synchronous path
The data got are synchronized to the third tables of data.
7. electronic device as claimed in claim 6, which is characterized in that the data of first tables of data are synchronized to described
The method of synchronization of three tables of data is that increment synchronization or full dose are synchronous.
8. electronic device as claimed in claim 7, which is characterized in that described when being synchronized to the data of first tables of data
When the mode of third tables of data is that full dose is synchronous, the synchronizing step includes:
The data volume for inquiring first tables of data, will be in first tables of data according to preset batch rule according to query result
Data be in turn divided into multiple batches, and save obtained data batch number is divided into default log sheet；
Data successively are read from first tables of data according to the data batch number in the log sheet.
9. the electronic device as described in claim 6 to 8 any one, which is characterized in that when data synchronously complete, unite respectively
The data volume of first tables of data, the third tables of data is counted, if the data volume of first tables of data and the third number
It is equal according to the data volume of table, then show that data synchronize successful prompt information and feed back to the user.
10. a kind of computer readable storage medium, which is characterized in that include that data are synchronous in the computer readable storage medium
Program when the data synchronization program is executed by processor, realizes that the data as described in any one of claims 1 to 5 are synchronous
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910627681.7A CN110347693A (en) | 2019-07-11 | 2019-07-11 | Method of data synchronization, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910627681.7A CN110347693A (en) | 2019-07-11 | 2019-07-11 | Method of data synchronization, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110347693A true CN110347693A (en) | 2019-10-18 |
Family
ID=68175877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910627681.7A Pending CN110347693A (en) | 2019-07-11 | 2019-07-11 | Method of data synchronization, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347693A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111324614A (en) * | 2020-03-18 | 2020-06-23 | 浩云科技股份有限公司 | Table configuration method and device, terminal equipment and storage medium |
| CN111367924A (en) * | 2020-02-26 | 2020-07-03 | 杭州数梦工场科技有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN112925835A (en) * | 2019-12-05 | 2021-06-08 | 北京金山云网络技术有限公司 | Data synchronization method and device and server |
| CN113672639B (en) * | 2021-08-27 | 2024-04-26 | 成都长城开发科技股份有限公司 | Multi-type database table structure comparison method, system, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060136443A1 (en) * | 2004-12-16 | 2006-06-22 | International Business Machines Corporation | Method and apparatus for initializing data propagation execution for large database replication |
| CN101419616A (en) * | 2008-12-10 | 2009-04-29 | 阿里巴巴集团控股有限公司 | Data synchronization method and apparatus |
| CN106570086A (en) * | 2016-10-19 | 2017-04-19 | 上海携程商务有限公司 | Data migration system and method |
| CN107515933A (en) * | 2017-08-29 | 2017-12-26 | 平安科技（深圳）有限公司 | Change method, server, system and the storage medium of source data database table structure |
| CN108446315A (en) * | 2018-02-07 | 2018-08-24 | 中国平安人寿保险股份有限公司 | Big data moving method, device, equipment and storage medium |
| CN109885581A (en) * | 2019-03-14 | 2019-06-14 | 苏州达家迎信息技术有限公司 | Synchronous method, device, equipment and the storage medium of database |
-
2019
- 2019-07-11 CN CN201910627681.7A patent/CN110347693A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060136443A1 (en) * | 2004-12-16 | 2006-06-22 | International Business Machines Corporation | Method and apparatus for initializing data propagation execution for large database replication |
| CN101419616A (en) * | 2008-12-10 | 2009-04-29 | 阿里巴巴集团控股有限公司 | Data synchronization method and apparatus |
| CN106570086A (en) * | 2016-10-19 | 2017-04-19 | 上海携程商务有限公司 | Data migration system and method |
| CN107515933A (en) * | 2017-08-29 | 2017-12-26 | 平安科技（深圳）有限公司 | Change method, server, system and the storage medium of source data database table structure |
| CN108446315A (en) * | 2018-02-07 | 2018-08-24 | 中国平安人寿保险股份有限公司 | Big data moving method, device, equipment and storage medium |
| CN109885581A (en) * | 2019-03-14 | 2019-06-14 | 苏州达家迎信息技术有限公司 | Synchronous method, device, equipment and the storage medium of database |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112925835A (en) * | 2019-12-05 | 2021-06-08 | 北京金山云网络技术有限公司 | Data synchronization method and device and server |
| CN111367924A (en) * | 2020-02-26 | 2020-07-03 | 杭州数梦工场科技有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN111367924B (en) * | 2020-02-26 | 2024-01-02 | 杭州数梦工场科技有限公司 | Data synchronization method, device, electronic equipment and storage medium |
| CN111324614A (en) * | 2020-03-18 | 2020-06-23 | 浩云科技股份有限公司 | Table configuration method and device, terminal equipment and storage medium |
| CN113672639B (en) * | 2021-08-27 | 2024-04-26 | 成都长城开发科技股份有限公司 | Multi-type database table structure comparison method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112534396B (en) | Diary watch in database system | |
| EP3563268B1 (en) | Scalable database system for querying time-series data | |
| CN109997126B (en) | Event driven extraction, transformation, and loading (ETL) processing | |
| US8312037B1 (en) | Dynamic tree determination for data processing | |
| US10762075B2 (en) | Database interface agent for a tenant-based upgrade system | |
| Gupta et al. | Mesa: Geo-replicated, near real-time, scalable data warehousing | |
| US10176235B2 (en) | Techniques for data retention | |
| US8150889B1 (en) | Parallel processing framework | |
| CA2858680C (en) | Systems and methods for improving database performance | |
| US7512639B2 (en) | Management of time-variant data schemas in data warehouses | |
| US9600269B1 (en) | Deployment of database objects | |
| US10114626B2 (en) | Database level containers | |
| CN110347693A (en) | Method of data synchronization, device and storage medium | |
| US20070156849A1 (en) | Systems and methods for delivering software upgrades in a provider-tenant environment | |
| US20080162622A1 (en) | Systems and methods to implement extensibility of tenant content in a provider-tenant environment | |
| US11354284B2 (en) | System and method for migration of a legacy datastore | |
| WO2009017534A1 (en) | Persistent query system for automatic on-demand data subscriptions from mobile devices | |
| US7167974B2 (en) | Multiple saved kernel configurations | |
| US10289705B2 (en) | Zero-outage database reorganization | |
| CN107515933A (en) | Change method, server, system and the storage medium of source data database table structure | |
| US20130297577A1 (en) | Database element versioning system and method | |
| CN103440245A (en) | Line and column hybrid storage method of database system | |
| CN108874924A (en) | Creation method, device and the computer readable storage medium of search service | |
| Gupta et al. | Mesa: A geo-replicated online data warehouse for Google's advertising system | |
| US8131884B1 (en) | Reusing system configuration information and metadata for related operations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191018 |