CN103455599A - Method, equipment and system for data synchronism - Google Patents
Method, equipment and system for data synchronism Download PDFInfo
- Publication number
- CN103455599A CN103455599A CN2013103948800A CN201310394880A CN103455599A CN 103455599 A CN103455599 A CN 103455599A CN 2013103948800 A CN2013103948800 A CN 2013103948800A CN 201310394880 A CN201310394880 A CN 201310394880A CN 103455599 A CN103455599 A CN 103455599A
- Authority
- CN
- China
- Prior art keywords
- local data
- file
- described local
- data base
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a method, equipment and a system for data synchronism among multiple databases. The multiple databases comprise a local database and at least one remote database. The method comprises the following steps of: (a) generating a local data text based on data in the local data base at a preset synchronism time; (b) combining the local data text and a file header generated based on the local data text, and forming a local data file; (c) uploading the local data file to at least one middle equipment, and enabling the at least one remote database to acquire the local data file from the at least one middle equipment at the other preset synchronism time.
Description
Technical field
The present invention relates to database field, relate more specifically to the method, apparatus and system for synchrodata between database.
Background technology
Along with the development of social informatization, people need to carry out interchange, transmission and the storage of various information, and the information on the equipment such as the computer that people use, mobile phone, PDA has certain independence.In order to facilitate us the data that these are very at random reasonably to manage, these information organically need to be united, in order to can effectively use after handling data.Therefore, just need database to carry out these data of maintenance and management.
Database be according to data structure organize, the software of store and management data.Data in database are normally organized with the model relevant to real world applications.Along with the scale of the data that need store and management is increasing and the restriction of the network bandwidth, distributed data base has become the main flow in database.In distributed data base, data are dispersed on different nodes with redundant fashion.For example, the data D of first of store and management database on first node
1, the second portion data D of store and management database on Section Point
2... the N partial data D of store and management database on N node
n, and D
1, D
2..., D
nunion be the data complete or collected works of this distributed data base, and D
1, D
2..., D
nin can have certain redundancy between any two, have a certain amount of identical data.As an extreme case, D
1, D
2..., D
ncan be identical, each in them has all been preserved all data contents of this database.Yet, in order to make the redundant data of each node in distributed data base be consistent, or the node (this node may not have the data division that the user wants access) that approaches the user in this distributed data base most in order to facilitate the user to access, therefore in distributed data base, the data that inevitably will relate between different nodes are synchronous.
In the prior art, disclose and a kind ofly by event, triggered the data of different nodes are carried out to synchronous scheme.This synchronization scenario is realized under the environment with variety classes database node, different pieces of information model.It need to realize one or more source databases (source node) are synchronized to one or more target database (destination node) in time, and the Fast synchronization of data in enormous quantities.The method comprises: during data change in source database, produce the synchronous altering event of request; When altering event being detected, just trigger data that the propelling data module will change specifically and form packet according to the data structure of the reception data module of destination node, and issue this reception data module; This receives data module after packet is resolved, and according to the content of change data, operates corresponding target database.
Yet the problem of this scheme is: (1) need to trigger by altering event that just to carry out data synchronous; (2) secondly,, if altering event is too much, data synchronously will be very frequent, greatly waste the System and Network resource; And (3) data security degree is not high.
Summary of the invention
In order to address the above problem, provide according to the method, apparatus and system synchronous for data of the present invention.
According to a first aspect of the invention, provide a kind of method for synchronous a plurality of databases.Described a plurality of database comprises local data base and at least one remote data base.Described method comprises: a) in predetermined lock in time, the data based in described local data base, generate the local data text; B) file header that merges described local data text and generate based on described local data text, to form local data file; And c) upload described local data file at least one intermediate equipment, for described at least one remote data base, in another predetermined lock in time, from described at least one intermediate equipment, obtain described local data file.
In certain embodiments, the data in described local data base comprise following at least one: the market price of number of policy, mobile device, prolong the duration of insurance limit, prolong the lattice of supporting value, sales date, address name, subscriber mailbox, subscriber directory number, user the country one belongs to, user are bought mobile device the country one belongs to, the user buys currency, mobile device model, sequence number and Payment Methods.
In certain embodiments, the data in described local data base are newly-increased during section, data of revising and/or deleting at the appointed time in described local data base.
In certain embodiments, step a) comprising: create local clear data file; And, in described local clear data file, with the data layout of predesignating, the data in described local data base are inserted in described local clear data file, to form described local data text.
In certain embodiments, step b) comprising: the digital digest that calculates described local data text; And, at least according to the size of expection filename and the described local data text of described digital digest, described local data file, generate the file header of described local data file.
In certain embodiments, described digital digest is the MD5 value.
In certain embodiments, at step b) and c) between, described method also comprises: described local data file is encrypted.
In certain embodiments, step c) comprising: use the encrypted transmission agreement to upload described local data file to described at least one intermediate equipment.
In certain embodiments, described encrypted transmission agreement is Secure File Transfer Protocol (SFTP).
In certain embodiments, at step c) in, if upload failure, carry out the re-transmission of pre-determined number.
In certain embodiments, at step c) in, if all failures of described re-transmission send corresponding alarm email to the related personnel.
In certain embodiments, at step c) afterwards, described method also comprises: the historical record of record and this sync correlation in described local data base.
In certain embodiments, described historical record comprises following at least one: the digital digest value of the filename of described local data file, the creation-time of described local data file, described local data text, the local storage directory of uploading whether success, uplink time, on described at least one intermediate equipment, depositing the storage directory of described local data file and deposit described local data file.
According to a second aspect of the invention, provide a kind of equipment for synchronous a plurality of databases, described a plurality of databases comprise at least one remote data base and the local data base carried by described equipment.Described equipment comprises: generation unit, and in predetermined lock in time, the data based in described local data base, generate the local data text; Merge cells, for the file header that merges described local data text and generate based on described local data text, to form local data file; And uploading unit, at least one intermediate equipment, uploading described local data file, for described at least one remote data base, in another predetermined lock in time, from described at least one intermediate equipment, obtain described local data file.
In certain embodiments, the data in described local data base comprise following at least one: the market price of number of policy, mobile device, prolong the duration of insurance limit, prolong the lattice of supporting value, sales date, address name, subscriber mailbox, subscriber directory number, user the country one belongs to, user are bought mobile device the country one belongs to, the user buys currency, mobile device model, sequence number and Payment Methods.
In certain embodiments, the data in described local data base are newly-increased during section, data of revising and/or deleting at the appointed time in described local data base.
In certain embodiments, described generation unit also for: create local clear data file; And, in described local clear data file, with the data layout of predesignating, the data in described local data base are inserted in described local clear data file, to form described local data text.
In certain embodiments, described merge cells is also for the digital digest that calculates described local data text; And, at least according to the size of expection filename and the described local data text of described digital digest, described local data file, generate the file header of described local data file.
In certain embodiments, described digital digest is the MD5 value.
In certain embodiments, described equipment also comprises: ciphering unit, and for described local data file is encrypted.
In certain embodiments, described uploading unit also for: use the encrypted transmission agreement to upload described local data file to described at least one intermediate equipment.
In certain embodiments, described encrypted transmission agreement is Secure File Transfer Protocol (SFTP).
In certain embodiments, described uploading unit also for: if upload failure, carry out the re-transmission of pre-determined number.
In certain embodiments, described uploading unit also for: if described re-transmission all the failure, to the related personnel, send corresponding alarm email.
In certain embodiments, described equipment also comprises: record cell, and for the historical record with this sync correlation at described local data base record.
In certain embodiments, described historical record comprises following at least one: the digital digest value of the filename of described local data file, the creation-time of described local data file, described local data text, the local storage directory of uploading whether success, uplink time, on described at least one intermediate equipment, depositing the storage directory of described local data file and deposit described local data file.
According to a third aspect of the invention we, provide a kind of system for synchronous a plurality of databases, described a plurality of databases comprise local data base and at least one remote data base.At least one remote data base node and at least one intermediate equipment that this system comprises the equipment according to above-mentioned second aspect that carries described local data base and carries described at least one remote data base.
According to a forth aspect of the invention, provide a kind of method for synchronous a plurality of databases, described a plurality of databases comprise local data base and at least one remote data base.Described method comprises: a) monitor at least one intermediate equipment whether have the teledata file for the treatment of that the synchronous one or more remote data bases in described at least one remote data base are uploaded; B) in the situation that find to exist and treat to obtain described teledata file from described at least one intermediate equipment by synchronous teledata file on described at least one intermediate equipment; C), according to the file header of described teledata file, obtain the teledata text of described teledata file; And d) process the corresponding data in described local data base according to described teledata text.
In certain embodiments, described teledata text comprises following at least one: the Claims Resolution status information, send work information, logistics information, number of policy, the newly device identification of mobile device and the device identification of old mobile device.
The type of the processing steps d in certain embodiments) comprises: newly-increased data, Update Table and deletion data.
In certain embodiments, step c) also comprise: the digital digest that calculates described teledata text; And the digital digest that comprises of the file header of the digital digest relatively calculated and described teledata file: if two digital digests match determine that described teledata text is not tampered or loses and continue subsequent step; Otherwise determine that described teledata text is tampered or loses, described method directly finishes.
In certain embodiments, described digital digest is the MD5 value.
In certain embodiments, at step b) and c) between, described method also comprises: to described teledata file decryption.
In certain embodiments, step b) comprising: use the encrypted transmission agreement to obtain described teledata file from described at least one intermediate equipment.
In certain embodiments, described encrypted transmission agreement is Secure File Transfer Protocol (SFTP).
In certain embodiments, at step b) in, if obtain described teledata file failure, carry out the re-transmission of pre-determined number.
In certain embodiments, at step b) in, if all failures of described re-transmission send corresponding alarm email to the related personnel.
In certain embodiments, at step b) afterwards, described method also comprises: the historical record of record and this sync correlation in described local data base.
According to a fifth aspect of the invention, provide a kind of equipment for synchronous a plurality of databases, described a plurality of databases comprise at least one remote data base and the local data base carried by described equipment.Described equipment comprises: monitor unit, for monitoring at least one intermediate equipment whether have the teledata file for the treatment of that the synchronous one or more remote data bases by described at least one remote data base are uploaded; The file acquisition unit, in the situation that find to exist and to treat to obtain described teledata file from described at least one intermediate equipment by synchronous teledata file on described at least one intermediate equipment; The text acquiring unit, for the file header according to described teledata file, obtain the teledata text of described teledata file; And processing unit, for process the corresponding data of described local data base according to described teledata text.
In certain embodiments, described teledata text comprises following at least one: the Claims Resolution status information, send work information, logistics information, number of policy, the newly device identification of mobile device and the device identification of old mobile device.
In certain embodiments, the type of the processing of described processing unit comprises: newly-increased data, Update Table and deletion data.
In certain embodiments, described text acquiring unit is also for the digital digest that calculates described teledata text; And the digital digest that comprises of the file header of the digital digest relatively calculated and described teledata file: if two digital digests match determine that described teledata text is not tampered or loses and continue subsequent step; Otherwise determine that described teledata text is tampered or loses, the processing of described equipment directly finishes.
In certain embodiments, described digital digest is the MD5 value.
In certain embodiments, described equipment also comprises: decryption unit, and for to described teledata file decryption.
In certain embodiments, described file acquisition unit also for: use the encrypted transmission agreement to obtain described teledata file from described at least one intermediate equipment.
In certain embodiments, described encrypted transmission agreement is Secure File Transfer Protocol (SFTP).
In certain embodiments, described file acquisition unit also for: if obtain the failure of described teledata file, carry out the re-transmission of pre-determined number.
In certain embodiments, described file acquisition unit also for: if described re-transmission all the failure, to the related personnel, send corresponding alarm email.
In certain embodiments, described equipment also comprises: record cell, and for the historical record with this sync correlation at described local data base record.
According to a sixth aspect of the invention, provide a kind of system for synchronous a plurality of databases, described a plurality of databases comprise local data base and at least one remote data base.At least one remote data base node and at least one intermediate equipment that this system comprises the equipment according to above-mentioned the 5th aspect that carries described local data base and carries described at least one remote data base.
Use method of the present invention, equipment or system, realized the Timing Synchronization of data, reduced the impact on the System and Network performance.In addition, logarithm reportedly is input into capable encryption, thereby completes in time data and guaranteed in synchronous the security of data.
The accompanying drawing explanation
By below in conjunction with accompanying drawing explanation the preferred embodiments of the present invention, will make of the present invention above-mentioned and other objects, features and advantages are clearer, wherein:
Fig. 1 shows the schematic diagram according to the application scenarios of data synchronous system of the present invention.
Fig. 2 shows the process flow diagram of synchronous method for data according to the embodiment of the present invention.
Fig. 3 shows the process flow diagram of synchronous other method for data according to the embodiment of the present invention.
Fig. 4 shows the block diagram of synchronous equipment for data according to the embodiment of the present invention.
Fig. 5 shows the block diagram of another synchronous equipment for data according to the embodiment of the present invention.
Embodiment
To a preferred embodiment of the present invention will be described in detail, in the description process, having omitted is unnecessary details and function for the present invention with reference to the accompanying drawings, to prevent that the understanding of the present invention from causing, obscures.Below, the scene that the present invention of take is applied to wired communication system is example, and the present invention be have been described in detail.But the present invention is not limited thereto, the present invention also can be applied to fixed communications, mobile communication system, or is applied to any mixed structure of mobile communication system, fixed communications, wired communication system etc.With regard to wired communication system, the present invention is not limited to the concrete communication mode of related each wire communication terminal (node), can include, but is not limited to: fixed telephone network, cable TV network, xDSL, Ethernet, optical-fiber network etc., different catv terminals can adopt identical communication protocol, also can adopt different communication protocol.The present invention is not limited to the specific operating system of catv terminal, can include, but is not limited to Windows, Unix, Linux, OS X etc., and different catv terminals can adopt identical operating system, also can adopt different operating system.In addition, in the situation that the present invention is applied to mobile communication system, the specific operating system of mobile terminal can include, but is not limited to iOS, Android (Android), Windows Phone, Symbian etc.
Fig. 1 shows the schematic diagram according to the application scenarios of data synchronous system 1000 of the present invention.As shown in Figure 1, data synchronous system 1000 can comprise: (this locality) database node (equipment) 100, (long-range) database node (equipment) 200 and intermediate equipment 300.Hereinafter, when not specifying, (this locality) database node 100 and (long-range) database node 200 are referred to as to database node.
For the sake of clarity, only show two database nodes 100 and 200 in figure, but the present invention is not limited thereto, can comprise database node of three or more numbers etc.Database node 100 and 200 can belong to the user or can be operated by the user.Similarly, for the sake of clarity, only show an intermediate equipment 300 in figure, but the present invention is not limited thereto, also can comprise the intermediate equipment 300 of two or more numbers or there is no intermediate equipment 300.In addition, in the present embodiment, the user selects database node 100 is directly accessed, and conducts interviews by the data in 100 pairs of whole distributed data bases of database node.Now, for this user, database node 100 is user's local data base node, and database node 200 is owing to being that the user visits by database node 100, so database node 200 is regarded as the remote data base node.Therefore on the meaning of actual physics distance and virtual network distance, local data base node 100 all not necessarily more approaches the user than remote data base node 200, but in the present embodiment, local data base node 100 is depicted as than remote data base node 200 and more approaches the user.But the invention is not restricted to this, in other embodiments, the user can select database node 200 as its local data base node, and correspondingly database node 100 is considered as to the remote data base node, and carries out equally follow-up various operation.
Hereinafter, will various embodiments of the present invention be described respectively mainly for local data base node 100 and remote data base node 200.Carry respectively local data base and remote data base on local data base node 100 and remote data base node 200.These two databases are the part of whole distributed data base, or as mentioned above, the content of these two databases should be consistent in theory.Therefore need between these two databases, carry out data synchronous.
In the embodiment shown in fig. 1, for the data to local data base node 100 and remote data base node 200 are carried out synchronously, data sync client end 150 and 250 (being designated hereinafter simply as client 150 and 250) is arranged on respectively on local data base node 100 and remote data base node 200 according to an embodiment of the invention.Client 150 and 250 can be arranged in database node with the white row of the form of software by the user, or can with the form of hardware or firmware, be arranged on respectively in database node 100 and 200 by server production firm.In certain embodiments, client 150 and 250 can be the application software of the present invention that is specifically designed to of for example having downloaded from network after the user has bought database node 100 and 200.In further embodiments, client 150 and 250 can be for example by production firm, with firmware or example, in hardware, to be arranged in advance the application program in database node.In other embodiment, client 150 and 250 can be hardware module or the database node itself of being produced by production firm.
Hereinafter, will take mobile phone (mobile device) insurance describes embodiments of the invention in detail as application scenarios.Yet should recognize: the invention is not restricted to this concrete application, in fact it can be applied to any needs and carries out the synchronous application scenarios of data.
In the present embodiment, local data base node 100 is and sells mobile phone to the user and provide the retailer of mobile phone insurance service/insurance provider to be associated.After working as a retailer and selling mobile phone to the user, itself and/or/insurance provider can provide the mobile phone insurance service to the user.That is, the user whether can select be that mobile phone is insured.After having bought the mobile phone insurance, if user's mobile phone is lost unintentionally, is damaged, retailer (or insurance provider) can provide for the cellphone subscriber is corresponding mobile phone insurance service (for example, gratuitously provide new cell-phone, data Resume service etc. on mobile phone is provided).Therefore, need to be in local data base node 100 the record information content relevant to mobile phone and insurance service thereof.
In certain embodiments, this information content can comprise following at least one: the market price of number of policy, mobile phone (mobile device), prolong the duration of insurance limit, prolong the lattice of supporting value, sales date, address name, subscriber mailbox, subscriber directory number, user the country one belongs to, user are bought mobile device the country one belongs to, the user buys model, sequence number and Payment Methods of currency, mobile phone (mobile device) etc.Certainly, the information content be not limited to above-mentioned these, can be also any other information relevant to user, mobile phone, insurance service etc.
In addition, for example, in order to facilitate the user (to carry out insurance business in strange land, lost mobile phone owing to going on business in other places, and the business of need to handling insurance in locality), therefore the user need to be synchronized to (for example, the remote data base node 200) in each other database nodes by these data messages in local data base node 100.Next, will the example flow of this synchronizing process be described.
1, local data base node side
At first, local data base node 100 is according to the setting of user or insurance provider, inquire for example, data in fixed time section (this time period can arrange, and, specifies all related datas that the time buying is yesterday) in the data of the local data base of local data base node.These data can comprise various data listed above.
Next, local data base node 100 is according to the data creation local data file inquired.Idiographic flow is as follows:
(1) at first, local data base node 100 creates the local data file of a blank.The filename of this local data file can form with particular prefix and date created.The form of this local data file can meet the form of local data base node 100 and remote data base node 200 agreements, and uses the separator of agreement to be cut apart each data item.
(2) next, 100 pairs of all data that inquire of local data base node are carried out batch processing, to form the local data text of this local data file.These data that inquire can be one or more complete data recording, also can be based on the delta file of historgraphic data recording.That is, for synchronous data, can be the full release of related data record, can be also the difference version (increment version) of data synchronous with respect to the last time.Usually when synchronous for the first time, full release that must usage data, so that do not have the full content that other database nodes of these data can obtain these data.And, when subsequent synchronisation, by using the increment version, reduce the data volume of required transmission.Certainly, the invention is not restricted to this, in some embodiments of the invention, in fact can want synchronous any data by transmitting locally database node 100.
In addition, by local data base node 100 or by user/service provider etc., provide the filename of expection or the filename automatically generated according to specified rule, and calculate the digital digest (for example, MD5 value) of whole local data text.Then, by that obtain and size digital digest value, expection filename and local data text etc., each relevant information writes the file header of this local data file, and the formation local data file.In one embodiment, can, by merging local data text and the above-mentioned file header generated based on this local data text, form local data file.
(3), after calculating the digital digest value, can be alternatively all the elements of local data file be encrypted the final local data file of encrypting that forms.
After having created local data file, local data base node 100 can pass through encrypted transmission agreement (for example, Secure File Transfer Protocol (SFTP)) it is uploaded to the assigned catalogue of intermediate equipment 300 (for example, SFTP server).If upload failure, (for example, 3 times) of carrying out certain number of times retransmit.If these retransmit all failures, send out alarm email and notify the related maintenance personnel.
In addition, can upload relevant historical record to this at local data base node 100 and/or other specified location record.This is uploaded historical record and can comprise: this file name of uploading, its creation-time, digital digest (MD5) value, this uploads whether success, uplink time, this remote directory of uploading (destination directory) and local directory (source directory) etc.This historical record can be for maintainer's inquiry, to guarantee 100 normal operations of local data base node.
In the data upload by local data base node 100 after intermediate equipment 300, remote data base node 200 can be at the appointed time (for example, every day fixed point or according to predetermined time interval obtain) obtain this local data file from the assigned catalogue of middle equipment 300, and complete data synchronous (in lower joint, will describe this flow process in detail).In one embodiment, remote data base node 200 can obtain local data file in the following manner: periodically detect in the assigned catalogue of intermediate equipment 300 new data file whether occurs, and in the situation that this data file is downloaded in the appearance of new data file.
Next, specific descriptions remote data base node 200 is obtained to the flow process of the data synchronous by local data base node 100.
2, remote data base node side
Note that for the user of remote data base node 200, this database node 200 is actually its user's local data base node.But to understand in order helping reader, the treatment scheme at database node 200 places still to be described with the embodiment shown in Fig. 1 herein.Therefore, in this flow process, in fact local data base node 100 and remote data base node 200 can be exchanged, and do not affected the practical function of this flow process.
In one embodiment, by local data base node 100 as described above by its data synchronous (uploading) for example, under the assigned catalogue of intermediate equipment 300 (, SFTP server).In this case, whether remote data base node 200 need to by new data file occurring in the assigned catalogue on this intermediate equipment 300 of sequential monitoring.In one embodiment, this supervision can be periodically.In another embodiment, this supervision can occur in by the random time of user's appointment etc., the invention is not restricted to this.When detecting while new data file occurring, continue follow-up flow process.Otherwise, continue to monitor.
Note that in the process of local data base node 100 upload files, the suffix of the filename of this document can for example, end up with character string (, " tmp ").This means that this file is synchronous, and for example only has after uploading fully ability, by its RNTO standard name (, removing the tmp suffix), whether to distinguish file in synchronous regime.Therefore, remote data base node 200, before the data file with standard name being detected, can't be thought and have new data file to occur.Certainly, in other embodiments of the invention, also can avoid remote data base node 200 just to download uncompleted partial document data before local data base node 100 has been uploaded by other modes.For example, in one embodiment, remote data base node 200 can judge whether to have uploaded with in the SFTP server, to transfer files, whether completing relevant sign.In another embodiment, remote data base node 200 can judge to upload whether complete by the text length in the file header of upload file.In yet another embodiment, the length that remote data base node 200 can be by repeatedly obtaining this document also compares to judge whether just in upload procedure.In a word, in an embodiment of the present invention, remote data base node 200 can judge whether local data base node 100 uploading datas complete by variety of way, to avoid obtaining in advance uncompleted uploading data.
After the supervisory programme of remote data base node 200 is found new data file, it will attempt this data file is downloaded to local assigned catalogue automatically.For example, if failed download, can carry out the retry of certain number of times (this number of times can be arranged by user or service provider, 3 times).If all failures of retry, send out alarm email and notify the related maintenance personnel.
In addition, can be recorded in remote data base node 200 to this, downloading relevant historical information.With upload historical information similarly, this download historical record comprises: the file name of this download, its creation-time, digital digest (MD5) value, this downloads the remote directory (source directory) of whether success, download time, this download and local directory (destination directory) etc.This historical record can be for maintainer's inquiry, to guarantee 200 normal operations of remote data base node.
In one embodiment, after data file is downloaded successfully, if this data file is encrypted data file, remote data base node 200 carries out corresponding deciphering to it.After deciphering, remote data base node 200 need to be resolved the content of data file.Content after parsing comprises file header and the local data body part of above-mentioned local data file.Digital digest (MD5 value), filename and the document size information that in this document head, can contain as mentioned above, the local data text.
The local data text that 200 pairs of remote data base nodes parse carries out digital digest (MD5) to be processed, and obtains its digital digest value, and the digital digest value contained in this digital digest value and file header information is mated.If consistent, prove that the content of this local data file is not tampered or loses.On the contrary, if inconsistent, prove that the content of this local data file is tampered or loses, its data are unreliable.In parsing or comparison process, if the data of noting abnormalities can be sent out the mail notification related maintenance personnel to carry out the corresponding operation of processing.
After definite digital digest value is consistent, according to the above-mentioned file layout of appointing, be remote data base one or more data item to understand on remote data base node 200 by the Data Analysis of the local data body part of local data file, and the corresponding data in remote data base node 200 is operated.Operation for example can comprise: newly-increased data, Update Table and deletion data.In addition, except the data content of describing in above, these data can also comprise following at least one: the Claims Resolution status information, send work information, logistics information, number of policy, the newly device identification of mobile device and the device identification of old mobile device.
In addition, after resolving and processing data, will for example, with analytically dependent information (, resolving whether success, parsing time etc.), also as the download historical information, be kept in remote data base node 200.So far, the data synchronization process between local data base node 100 and remote data base node 200 completes.
Use said method, realized the Timing Synchronization of the data between local data base node 100 and remote data base node 200, reduced the impact on the System and Network performance.In addition, by logarithm, reportedly be input into capable encryption (comprise file encryption and transmission encrypt), thereby complete in time data and guaranteed in synchronous the security of data.
In addition, although used in the above-described embodiments an intermediate equipment 300, but in other embodiments of the invention, can distinguish synchronous a part of data by a plurality of intermediate equipments 300, perhaps also can there do not is intermediate equipment 300, and directly upload the data to (one or more) remote data base node 200 by (one or more) local data base node 100, and without the active process (comprising the processing of above-mentioned remote data base node side) of remote data base node 200.In addition, also can obtain data file from (one or more) local data base node 100 by (one or more) remote data base node 200, and without the active process (comprising the processing of above-mentioned local data base node side) of local data base node 100.
Fig. 2 shows the process flow diagram according to the method for data synchronization 400 of the embodiment of the present invention.As shown in Figure 2, method of data synchronization 400 can comprise step S410, S420 and S430.According to the present invention, execution can be carried out separately or combine to some steps of method 400, and can executed in parallel or order carry out, be not limited to the concrete operations order shown in Fig. 2.In certain embodiments, the local data base node (data synchronizer) 100 that method 400 can be as shown in Figure 1 or the client 150 be arranged in local data base node 100 are carried out.
In addition, Fig. 3 shows the process flow diagram according to another method of data synchronization 450 of the embodiment of the present invention.As shown in Figure 3, method of data synchronization 450 can comprise step S460, S470, S480 and S490.According to the present invention, execution can be carried out separately or combine to some steps of method 450, and can executed in parallel or order carry out, be not limited to the concrete operations order shown in Fig. 3.In certain embodiments, the remote data base node (data synchronizer) 200 that method 450 can be as shown in Figure 1 or the client 250 be arranged in remote data base joint 200 are carried out.
Fig. 4 shows the block diagram of the equipment for synchronous a plurality of databases (local data base node) 100 according to the embodiment of the present invention, and the plurality of database can comprise at least one remote data base and the local data base on this equipment.As shown in Figure 4, equipment 100 can comprise: generation unit 110, merge cells 120 and uploading unit 130.
The file header of merge cells 120 for merging this local data text and generating based on this local data text, to form local data file.Merge cells 120 can be CPU (central processing unit) (CPU), digital signal processor (DSP), microprocessor, microcontroller of equipment 100 etc., it can merge local data text and the file header generated based on the local data text, and then forms local data file.
Uploading unit 130, at least one intermediate equipment, uploading this local data file, is obtained local data file in another predetermined lock in time from this at least one intermediate equipment at least one remote data base.Uploading unit 130 can be CPU (central processing unit) (CPU), digital signal processor (DSP), microprocessor, microcontroller of equipment 100 etc., its can with the communications portion of equipment 100 (for example, radio receiving-transmitting unit, Ethernet card, xDSL modulator-demodular unit etc.) match, upload local data file at least one intermediate equipment, at least one remote data base, in another predetermined lock in time, from this at least one intermediate equipment, obtain local data file.
In addition, equipment 100 can also comprise unshowned other unit, for example ciphering unit and record cell in Fig. 2.The local data file encryption that ciphering unit generates for being combined unit 120, with the security during the enhancing data transmission.Record cell, for the historical record with this sync correlation at the local data base record, is inquired about and determines whether equipment 100 works for the maintainer.
Fig. 5 shows the block diagram of another equipment for synchronous a plurality of databases (remote data base node) 200 according to the embodiment of the present invention, and the plurality of database comprises at least one remote data base and the local data base on this equipment.Please note: although equipment 200 is in the above-described embodiments corresponding to remote data base node 200 herein, but still the database carried on it is called to " this locality " database, to indicate this database, with respect to other databases, are the databases in equipment 200 this locality, and and then the use that conveniently the following describes.
As shown in Figure 5, equipment 200 can comprise: monitor unit 210, file acquisition unit 220, text acquiring unit 230 and processing unit 240.
In addition, equipment 200 can also comprise unshowned other unit, for example decryption unit and record cell in Fig. 3.The teledata file decryption of decryption unit for file acquisition unit 220 is got, with the security during the enhancing data transmission.Record cell, for the historical record with this sync correlation at the local data base record, is inquired about and determines whether equipment 200 works for the maintainer.
Below with reference to Fig. 2 and Fig. 4, the method for data synchronization 400 according to the embodiment of the present invention and data synchronizer (local data base node) 100 are described in detail.
In step S420, the file header that can be merged the local data text and be generated based on the local data text by the merge cells 120 of equipment 100, to form local data file.
In step S430, can upload local data file at least one intermediate equipment by the uploading unit 130 of equipment 100, obtain local data file in another predetermined lock in time from this at least one intermediate equipment at least one remote data base.
In certain embodiments, the data in local data base can comprise following at least one: the market price of number of policy, mobile device, prolong the duration of insurance limit, prolong the lattice of supporting value, sales date, address name, subscriber mailbox, subscriber directory number, user the country one belongs to, user are bought mobile device the country one belongs to, the user buys currency, mobile device model, sequence number and Payment Methods.
In certain embodiments, the data in local data base can be newly-increased during section, the data revising and/or delete at the appointed time in local data base.
In certain embodiments, step S410 can comprise: create local clear data file; And, in local clear data file, with the data layout of predesignating, the data in local data base are inserted in local clear data file, to form the local data text.
In certain embodiments, step b) can comprise: the digital digest that calculates the local data text; And, at least according to the size of expection filename and the local data text of this digital digest, local data file, generate the file header of local data file.
In certain embodiments, digital digest can be the MD5 value.
In certain embodiments, between step S420 and S430, the method can also comprise: local data file is encrypted.
In certain embodiments, step S430 can comprise: use the encrypted transmission agreement to upload local data file at least one intermediate equipment.
In certain embodiments, the encrypted transmission agreement can be Secure File Transfer Protocol (SFTP).
In certain embodiments, in step S430, if upload failure, can carry out the re-transmission of pre-determined number.
In certain embodiments, in step S430, if retransmit all failures, can send corresponding alarm email to the related personnel.
In certain embodiments, after step S430, the method can also comprise: the historical record of record and this sync correlation in local data base.
In certain embodiments, historical record can comprise following at least one: the digital digest value of the filename of described local data file, the creation-time of described local data file, described local data text, the local storage directory of uploading whether success, uplink time, on described at least one intermediate equipment, depositing the storage directory of described local data file and deposit described local data file.
Below with reference to Fig. 3 and Fig. 5, another method of data synchronization 450 according to the embodiment of the present invention and data synchronizer (remote data base node) 200 are described in detail.
In step S470, can be by the file acquisition unit 220 of equipment 200 in the situation that find to exist and treat to obtain the teledata file from this at least one intermediate equipment by synchronous teledata file at least one intermediate equipment.
In step S480, can be by the text acquiring unit 230 of equipment 200 file header according to the teledata file, obtain the teledata text of this teledata file.
In step S490, can process the corresponding data in local data base according to the teledata text by the processing unit 240 of equipment 200.
In certain embodiments, the teledata text can comprise following at least one: the Claims Resolution status information, send work information, logistics information, number of policy, the newly device identification of mobile device and the device identification of old mobile device.
In certain embodiments, the type of the processing in step S490 can comprise: newly-increased data, Update Table and deletion data.
In certain embodiments, step S490 can also comprise: the digital digest that calculates the teledata text; And the digital digest that comprises of the file header of the digital digest relatively calculated and teledata file: if two digital digests match determine that the teledata text is not tampered or loses and continue subsequent step; Otherwise determine that the teledata text is tampered or loses, the method directly finishes.
In certain embodiments, digital digest can be the MD5 value.
In certain embodiments, between step S470 and S480, the method can also comprise: to the teledata file decryption.
In certain embodiments, step S470 can comprise: use the encrypted transmission agreement to obtain the teledata file from least one intermediate equipment.
In certain embodiments, the encrypted transmission agreement can be Secure File Transfer Protocol (SFTP).
In certain embodiments, in step S470, if obtain the failure of teledata file, can carry out the re-transmission of pre-determined number.
In certain embodiments, in step S470, if retransmit all failures, can send corresponding alarm email to the related personnel.
In certain embodiments, after step S470, the method can also comprise: the historical record of record and this sync correlation in local data base.
So far invention has been described in conjunction with the preferred embodiments.Should be appreciated that, those skilled in the art without departing from the spirit and scope of the present invention, can carry out various other change, replacement and interpolations.Therefore, scope of the present invention is not limited to above-mentioned specific embodiment, and should be limited by claims.
Claims (27)
1. the method for synchronous a plurality of databases, described a plurality of databases comprise local data base and at least one remote data base, described method comprises:
A), in predetermined lock in time, the data based in described local data base, generate the local data text;
B) file header that merges described local data text and generate based on described local data text, to form local data file; And
C) upload described local data file at least one intermediate equipment, for described at least one remote data base, in another predetermined lock in time, from described at least one intermediate equipment, obtain described local data file.
2. method according to claim 1, wherein, the data in described local data base comprise following at least one: the market price of number of policy, mobile device, prolong the duration of insurance limit, prolong the lattice of supporting value, sales date, address name, subscriber mailbox, subscriber directory number, user the country one belongs to, user are bought mobile device the country one belongs to, the user buys currency, mobile device model, sequence number and Payment Methods.
3. method according to claim 1, wherein, the data in described local data base are newly-increased during section, data of revising and/or deleting at the appointed time in described local data base.
4. method according to claim 1, wherein, step a) comprising:
Create local clear data file; And
In described local clear data file, with the data layout of predesignating, the data in described local data base are inserted in described local clear data file, to form described local data text.
5. method according to claim 1, wherein, step b) comprising:
Calculate the digital digest of described local data text; And
At least, according to the size of expection filename and the described local data text of described digital digest, described local data file, generate the file header of described local data file.
6. method according to claim 5, wherein, described digital digest is the MD5 value.
7. method according to claim 1, wherein, at step b) and c) between, described method also comprises: described local data file is encrypted.
8. method according to claim 1, wherein, step c) comprising:
Use the encrypted transmission agreement to upload described local data file to described at least one intermediate equipment.
9. method according to claim 8, wherein, described encrypted transmission agreement is Secure File Transfer Protocol (SFTP).
10. method according to claim 1, wherein, at step c) in, if upload failure, carry out the re-transmission of pre-determined number.
11. method according to claim 10, wherein, at step c) in, if all failures of described re-transmission send corresponding alarm email to the related personnel.
12. method according to claim 1, wherein, at step c) afterwards, described method also comprises:
The historical record of record and this sync correlation in described local data base.
13. method according to claim 12, wherein, described historical record comprises following at least one: the digital digest value of the filename of described local data file, the creation-time of described local data file, described local data text, the local storage directory of uploading whether success, uplink time, on described at least one intermediate equipment, depositing the storage directory of described local data file and deposit described local data file.
14. the equipment for synchronous a plurality of databases, described a plurality of databases comprise at least one remote data base and the local data base carried by described equipment, and described equipment comprises:
Generation unit, in predetermined lock in time, the data based in described local data base, generate the local data text;
Merge cells, for the file header that merges described local data text and generate based on described local data text, to form local data file; And
Uploading unit, at least one intermediate equipment, uploading described local data file, obtain described local data file in another predetermined lock in time from described at least one intermediate equipment for described at least one remote data base.
15. equipment according to claim 14, wherein, the data in described local data base comprise following at least one: the market price of number of policy, mobile device, prolong the duration of insurance limit, prolong the lattice of supporting value, sales date, address name, subscriber mailbox, subscriber directory number, user the country one belongs to, user are bought mobile device the country one belongs to, the user buys currency, mobile device model, sequence number and Payment Methods.
16. equipment according to claim 14, wherein, the data in described local data base are newly-increased during section, data of revising and/or deleting at the appointed time in described local data base.
17. equipment according to claim 14, wherein, described generation unit also for:
Create local clear data file; And
In described local clear data file, with the data layout of predesignating, the data in described local data base are inserted in described local clear data file, to form described local data text.
18. equipment according to claim 14, wherein, described merge cells also for:
Calculate the digital digest of described local data text; And
At least, according to the size of expection filename and the described local data text of described digital digest, described local data file, generate the file header of described local data file.
19. equipment according to claim 18, wherein, described digital digest is the MD5 value.
20. equipment according to claim 14, wherein, described equipment also comprises:
Ciphering unit, for encrypting described local data file.
21. equipment according to claim 14, wherein, described uploading unit also for:
Use the encrypted transmission agreement to upload described local data file to described at least one intermediate equipment.
22. equipment according to claim 21, wherein, described encrypted transmission agreement is Secure File Transfer Protocol (SFTP).
23. equipment according to claim 14, wherein, described uploading unit also for: if upload failure, carry out the re-transmission of pre-determined number.
24. equipment according to claim 23, wherein, described uploading unit also for: if described re-transmission all the failure, to the related personnel, send corresponding alarm email.
25. equipment according to claim 14, wherein, described equipment also comprises:
Record cell, for the historical record with this sync correlation at described local data base record.
26. equipment according to claim 25, wherein, described historical record comprises following at least one: the digital digest value of the filename of described local data file, the creation-time of described local data file, described local data text, the local storage directory of uploading whether success, uplink time, on described at least one intermediate equipment, depositing the storage directory of described local data file and deposit described local data file.
27. the system for synchronous a plurality of databases, described a plurality of database comprise local data base and at least one remote data base, this system comprise the described local data base of carrying according to the described equipment of any one in claim 14～26 and at least one remote data base node and at least one intermediate equipment of carrying described at least one remote data base.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103948800A CN103455599A (en) | 2013-09-03 | 2013-09-03 | Method, equipment and system for data synchronism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013103948800A CN103455599A (en) | 2013-09-03 | 2013-09-03 | Method, equipment and system for data synchronism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103455599A true CN103455599A (en) | 2013-12-18 |
Family
ID=49737962
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2013103948800A Pending CN103455599A (en) | 2013-09-03 | 2013-09-03 | Method, equipment and system for data synchronism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103455599A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104978324A (en) * | 2014-04-03 | 2015-10-14 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN106227822A (en) * | 2016-07-22 | 2016-12-14 | 深圳市中润四方信息技术有限公司 | A kind of data distributing method and data synchronous system |
| CN107566119A (en) * | 2017-09-28 | 2018-01-09 | 中国—东盟信息港股份有限公司 | A kind of guard method of eSIM cards data safety and system |
| CN108038127A (en) * | 2017-11-08 | 2018-05-15 | 中国平安人寿保险股份有限公司 | A kind of method, apparatus, terminal device and the storage medium of data synchronization |
| CN110933025A (en) * | 2019-10-21 | 2020-03-27 | 武汉神库小匠科技有限公司 | Multi-source heterogeneous data cross-domain synchronous shared storage method, device, equipment and medium |
| CN110990481A (en) * | 2019-10-22 | 2020-04-10 | 珠海许继芝电网自动化有限公司 | Synchronization method and system for historical data of power distribution automation system |
| CN113656366A (en) * | 2021-08-03 | 2021-11-16 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
| CN116633951A (en) * | 2023-07-19 | 2023-08-22 | 中国电子科技集团公司第十五研究所 | Cross-network data synchronization method and device |
| CN113656366B (en) * | 2021-08-03 | 2024-05-03 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094026A (en) * | 2006-06-19 | 2007-12-26 | 上海全成通信技术有限公司 | Method for synchronizing, transmitting mass data, and inputting mass data to database |
| CN102082807A (en) * | 2009-12-01 | 2011-06-01 | 突触计算机系统（上海）有限公司 | File transmitting method and device based on multiple protocols |
| CN102355497A (en) * | 2011-09-28 | 2012-02-15 | 中国铁道科学研究院电子计算技术研究所 | Data synchronization method |
| CN102930225A (en) * | 2012-10-25 | 2013-02-13 | 中国航天科工集团第二研究院七〇六所 | Electronic document access control method based on confidential identifier |
-
2013
- 2013-09-03 CN CN2013103948800A patent/CN103455599A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101094026A (en) * | 2006-06-19 | 2007-12-26 | 上海全成通信技术有限公司 | Method for synchronizing, transmitting mass data, and inputting mass data to database |
| CN102082807A (en) * | 2009-12-01 | 2011-06-01 | 突触计算机系统（上海）有限公司 | File transmitting method and device based on multiple protocols |
| CN102355497A (en) * | 2011-09-28 | 2012-02-15 | 中国铁道科学研究院电子计算技术研究所 | Data synchronization method |
| CN102930225A (en) * | 2012-10-25 | 2013-02-13 | 中国航天科工集团第二研究院七〇六所 | Electronic document access control method based on confidential identifier |
Non-Patent Citations (1)
| Title |
|---|
| 宫涛: ""异构数据库准实时同步技术研究"", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104978324A (en) * | 2014-04-03 | 2015-10-14 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN106227822A (en) * | 2016-07-22 | 2016-12-14 | 深圳市中润四方信息技术有限公司 | A kind of data distributing method and data synchronous system |
| CN107566119A (en) * | 2017-09-28 | 2018-01-09 | 中国—东盟信息港股份有限公司 | A kind of guard method of eSIM cards data safety and system |
| CN108038127A (en) * | 2017-11-08 | 2018-05-15 | 中国平安人寿保险股份有限公司 | A kind of method, apparatus, terminal device and the storage medium of data synchronization |
| CN108038127B (en) * | 2017-11-08 | 2020-03-20 | 中国平安人寿保险股份有限公司 | Data synchronization method, device, terminal equipment and storage medium |
| CN110933025A (en) * | 2019-10-21 | 2020-03-27 | 武汉神库小匠科技有限公司 | Multi-source heterogeneous data cross-domain synchronous shared storage method, device, equipment and medium |
| CN110990481A (en) * | 2019-10-22 | 2020-04-10 | 珠海许继芝电网自动化有限公司 | Synchronization method and system for historical data of power distribution automation system |
| CN113656366A (en) * | 2021-08-03 | 2021-11-16 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
| CN113656366B (en) * | 2021-08-03 | 2024-05-03 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
| CN116633951A (en) * | 2023-07-19 | 2023-08-22 | 中国电子科技集团公司第十五研究所 | Cross-network data synchronization method and device |
| CN116633951B (en) * | 2023-07-19 | 2023-09-29 | 中国电子科技集团公司第十五研究所 | Cross-network data synchronization method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103455599A (en) | Method, equipment and system for data synchronism | |
| CN109522330B (en) | Cloud platform data processing method, device, equipment and medium based on block chain | |
| US11569982B2 (en) | Blockchain compression using summary and padding blocks | |
| CN102307210B (en) | Data downloading system and data management and downloading method thereof | |
| CN106612285B (en) | Distributed cloud data management method and system based on peer-to-peer network | |
| CN101854392B (en) | Personal data management method based on cloud computing environment | |
| CN104426975A (en) | Method, equipment and system for data synchronization | |
| CN109522363B (en) | Cloud platform synchronization method, system, equipment and storage medium based on block chain | |
| CN104798052B (en) | The matched method and apparatus of consumer | |
| CN109302495A (en) | A kind of date storage method and device | |
| CN101647006A (en) | Be used for method of data backup and system | |
| CN113157648A (en) | Block chain based distributed data storage method, device, node and system | |
| JP2019091477A (en) | Distributed data system with document management and access control | |
| JP2017201546A (en) | Predictive storage service | |
| CN111597567B (en) | Data processing method, data processing device, node equipment and storage medium | |
| CN109194651B (en) | Identity authentication method, device, equipment and storage medium | |
| CN103841180B (en) | A kind of network data synchronizing method, apparatus, terminal device and server based on operational order | |
| CN108769153B (en) | Data processing method and system for network application | |
| CN110061887B (en) | Block chain-based traffic statistical method, device and equipment | |
| CN108134696A (en) | A kind of micro services configuration update method and device | |
| JP2007299284A (en) | Log collection system, client device, and log collection agent device | |
| CN111488372A (en) | Data processing method, device and storage medium | |
| CN111740966A (en) | Data processing method based on block chain network and related equipment | |
| CN109491645A (en) | Point methods, device, computer equipment and storage medium are buried based on interface element | |
| CN104202387A (en) | Metadata recovery method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20131218 |