CN103067483A - Remote data increment synchronization method and device based on data package - Google Patents
Remote data increment synchronization method and device based on data package Download PDFInfo
- Publication number
- CN103067483A CN103067483A CN2012105688610A CN201210568861A CN103067483A CN 103067483 A CN103067483 A CN 103067483A CN 2012105688610 A CN2012105688610 A CN 2012105688610A CN 201210568861 A CN201210568861 A CN 201210568861A CN 103067483 A CN103067483 A CN 103067483A
- Authority
- CN
- China
- Prior art keywords
- packet
- data
- file
- database
- increment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a remote data increment synchronization method and a device based on a data package. The method comprises a first step of obtaining data manipulation languages (DML) of a database, a second step of leading out the DML into the data package, a third step of sending the data package to a target database in a remote synchronization mode, a fourth step of analyzing the data package into structural query languages (SQL) and changing data, and a fifth step of updating the SQL and the changing data to the target database. According to the remote data increment synchronization method and the device based on the data package, increment records, deleted records and changed records in the database can be extracted in fixed time to form the data package including the increment records. According to network situations and data synchronization demands, the data package is transmitted to a target database system in a java message service (JMS) or a file transfer protocol (FTP) manner by utilizing message-oriented middleware or an FTP client side/ server terminal product and being supported by a certain allocation management, and therefore consistency of data of a remote terminal and a near-terminal databases is kept.
Description
Technical field
The present invention relates to database technical field, particularly a kind of teledata increment synchronization method and device of based on data bag.
Background technology
Teledata refers to that synchronously two Database Systems of cross-network segment keep data's consistency.Origin system is generally the operation system database that is deployed in front end, and the purpose system is generally for OLAP Data Mart or the data warehouse of supporting Analysis of Policy Making.The front end service system database needs firewall-penetrating with being connected of Back end data center.Existing teledata synchro system has following problem:
1, remote data transmission means fixed single can't be tackled various complex situations.
Remote data transmission can adopt the variety of ways such as Web service, JMS message-oriented middleware and FTP.The mode of Web service can realize the real-time synchronization transmission of data, but can only transmit low volume data, is applicable to the fragmentary scene that changes of data, and can't guarantee the quality of data delivery; The JMS message-oriented middleware is general to be applicable to transmit low volume data, but can guarantee that data also can achieve the goal at abominable net environment; FTP can transmit fairly large data volume, and can support breakpoint transmission.These three kinds of modes respectively have quality, and existing remote data transmission method can only be taked wherein a kind of mode, can't tackle various complicated situations under different network environments and the data synchronisation requirement.
2, the incremental data collector of remote data transmission front end employing has certain limitation.
In database, often need to insert, delete initial data or the operation such as modification, the object of these operations is called as incremental data.In some cases, the incremental data of some original tables need to be synchronized in some other table, the data of these tables are consistent; Also have certain situation, need to understand in detail the variation of some original tables, in time the data with these variations are submitted in the data warehouse, are used for data analysis and excavation, in the above-mentioned situation, need to carry out the incremental data collection.
The method that existing incremental data collection realizes mainly contains trigger, timestamp, table extracts, entirely shows comparison, log sheet, flash back inquiry (flashback) etc. entirely.There is following shortcoming and defect in these mechanism:
The trigger mode needs origin system to invade, the performance of meeting major injury origin system, and also this intrusion is forbidden by the client sometimes.
The mode of timestamp requires all tables of origin system need to possess the timestamp field, and is larger to the intrusion of origin system if do not need extra the interpolation, even needs to revise the service code of origin system.In addition, this mode realizes complicated to the judgement of data deletion action with renewal.
Full table extracts and is actually the thinking of abandoning increment extraction, directly entirely shows to extract, and extraction efficiency is very low for large table.
The mode of full table comparison needs one by one the relatively record of original table and object table, and record newly-increased and that revise is read out, and efficient is minimum.
The way of log sheet is to create the business diary table in source database, when the business datum of specific needs monitoring changes, comes updating maintenance log sheet content by corresponding operation system program module.Which during increment extraction, determine to load data and how to load by reading the log sheet data.The maintenance of log sheet need to be finished with code by the operation system program.Origin system is had great invasion, and increment extraction efficient is high but realization is complicated.
Flash back inquiry (flashback): the Database Systems of the above version of Oracle 9i provide the flash back inquiry mechanism, and the permission user inquires about over database positioning sometime.Like this, the extraction process can compare the current state of source database and the state constantly that extracted last time, draws fast the situation of change of source table data record.But this mode needs specific DBMS(Database management system, data base management system) dependence, and need origin system support flash back inquiry, range of application is narrower.
Summary of the invention
Main purpose of the present invention is, for above-mentioned deficiency of the prior art, a kind of teledata increment synchronization method and device of based on data bag are provided, by analyzing the Operation Log of source database, regularly the record that increases in the database, deletes, changes is extracted and form the packet that comprises incremental record, then according to network condition and data synchronisation requirement, with the mode of packet with JMS or FTP, utilize message-oriented middleware or ftp client/server end product, and under certain configuration management is supported, with data packet transmission to the target database system.
For achieving the above object, the present invention by the following technical solutions.
The invention provides a kind of teledata increment synchronization method of based on data bag, may further comprise the steps:
Obtain the DML of source database;
Described DML is exported as packet;
With described packet remote synchronization to target database;
Described packet is resolved to SQL statement and change data;
With described SQL statement and the change Data Update to target database.
The present invention also provides a kind of teledata increment synchronizer of based on data bag, comprising:
Acquisition module is for the DML that obtains source database;
The packet generation module is used for described DML is exported as packet;
Synchronization module is used for described packet remote synchronization to target database;
Parsing module is used for described packet is resolved to SQL statement and change data;
Enter library module, be used for described SQL statement and change Data Update to target database.
Than above-mentioned prior art, the present invention has following beneficial effect.
The packet that front end extraction program of the present invention forms can be selected to send to destination in the mode of JMS or FTP.JMS can guarantee the long-range real-time delivery of fragmentary and low volume data based on message-oriented middleware; FTP supports breakpoint transmission, is fit to the remote transmission of large-scale data, and both complement one another, and have guaranteed the stability of remote synchronization process.And, packet is the interface point of leading portion extraction program and remote synchronization program, again as the backup of source end system incremental data, can get up by persistence, even in extremely abominable in network environment like this even the situation that access to netwoks is limited, the physical medium that also can take mode under the line will preserve packet is sent to destination, has the synchronous or not available function of transmitting device of traditional teledata.
Description of drawings
Fig. 1 is the flow chart of the teledata increment synchronization method of based on data bag in the embodiment of the invention.
Fig. 2 is the flow chart that obtains the DML of source database in embodiment illustrated in fig. 1.
Fig. 3 is the structure chart of the teledata increment synchronizer of based on data bag in the embodiment of the invention.
The realization of the object of the invention, functional characteristics and advantage are described further with reference to accompanying drawing in connection with embodiment.
Embodiment
Describe technical scheme of the present invention in detail below with reference to drawings and the specific embodiments, so as clearer, understand invention essence of the present invention intuitively.
Fig. 1 is the flow chart of the incremental data acquisition method that Log-based is caught in the embodiment of the invention; Fig. 2 is the flow chart that obtains the DML of source database in embodiment illustrated in fig. 1.
Shown in seeing figures.1.and.2, the embodiment of the invention at first provides a kind of teledata increment synchronization method of based on data bag, may further comprise the steps:
S1: the DML that obtains source database;
S2: DML is exported as packet;
S3: with described packet remote synchronization to target database;
S4: described packet is resolved to SQL statement and change data;
S5: with described SQL statement and the change Data Update to target database.
DML refers to Data Manipulation Language, i.e. DML, and it is one of classification of SQL, is used for the state variation of database of record.SQL statement refers to database statement, be used for record to the data of database increase, operation and the operands such as deletion, modification and inquiry.
In step S1, the step of obtaining the DML of source database specifically comprises following substep:
S11: obtain the table of required monitoring, and create list of modification;
S12: create log sheet, be used for record DML;
S13: the daily record of monitored data storehouse obtains the Data Update operation;
S14: the Data Update operation is reduced to complete SQL statement and change data;
S15: will change data and write list of modification, SQL statement is write log sheet.
In the above steps, the table of required monitoring is called object table.Object table can be specific some tables of data in the source database system, also can be a collection of tables of data of all specified type.List of modification is corresponding one by one with the object table that will monitor, and structure is identical, is used for the change data that interim storage object table produces in this snoop procedure.
In step S1, obtain the table of required monitoring, and the step that creates list of modification be by the reading database dictionary to obtain the structure of object table, then create the list of modification corresponding with object table.
Implement this step in the oracle database, need to carry out following statement:
select?table_name?from?dba_tables?where?owner=....
select?name,length,type#,charsetform?from?sys.col$?where?obj#=(select?obj#?...)?order?by?intcol#
If database is SQL server, need to carry out following statement:
select?name?from?sysobjects?where?xtype='U'?and?uid=(select?uid?from?sysusers?where?name=...)
select?name,length,xtype?from?syscolumns?where?id=(select?id?frmo?sysobjects?where?name=tableName?and?uid=(select?uid?from?sysusers?where?name=username))。
In fact, before carrying out step S1, the DBA also needs to carry out register, carries out identification, guarantees the fail safe of Database Systems.
In step S12, in Database Systems, create the log sheet of MY_CDC_LOG_TABLE by name or similar title, be used for blotter increment DML.
In step S13, the daily record concrete operations of monitored data storehouse are:
Listening period according to the user arranges starts watcher thread.For the ORACLE database, watcher thread uses LOGMNR bag correlation function, monitors the log recording of ORACLE, obtains data change operation (comprising insert, update and delete statement); For SQL SERVER, watcher thread uses the related data dictionary to monitor log recording, obtains the data change operation.
Read log information in the oracle database and need to carry out following statement:
select?name,COMPLETION_TIME?from?v$archived_log?where...
select?v$logfile.member?from?v$logfile,v$log?where?v$logfile.group#=v$log.group#?and?v$logfile.status='CURRENT'
Select seg_owner, table_name, sql_redo, timestamp, row_id, operation, data_obj# from v $ logmnr_contents where DATA_OBJ# in (table sequence number ..) and operation in (' INSERT', ' DELETE', ' UPDATE')
select?seg_owner,seg_name,sql_redo,timestamp,row_id,operation,?data_obj#?from...。
Read log information in the SQL Server database and need to carry out following statement:
select?allocunitname,operation,[Rowlog?Contents?0]?as?r0,[Rowlog?Contents?1]?as?r1,[Rowlog?Contents?2].....[R...4]?as?r4,[Current?LSN]?from?::fn_dblog(null,null)?where?allocunitname?like?userName.tableName?and?not?allocunitname?like?userName.tableName_cdc?and?operation?in?('Lop_Insert_Rows','Lop_Delete_Rows','Lop_Modify_Row','Lop_Modify_Columns')...
select?[object?Name],operation,[Current?LSN]?frmo?::fn_dblog(null,null)?where?[object?Name]?like?...。
In step S14, the step that the Data Update operation is reduced to complete SQL statement and change data is specially, and watcher thread is reduced to complete SQL statement and change data by analyzing daily record and related data dictionary with the data change operation.
Change data reduction SQL statement needs to carry out following statement in Oracle:
select?obj#?from?sys.obj$?where?name='table_name'?and?owner#=(select?user#?from?sys.user$?where?name='user_name')
select?c.column_name?from?dba_cons_columns?c,dba_constraints?t?where?c.constraint_name=t.constraint_name?and?t.constraint_type='P'?and?t.table_name='table_name'?and?c.owner='user_name'。
SQL Server then needs to carry out following statement:
select?column_name?from?information_schema.constraint_column_usage?where?constraint_name=(select?constraint_name?from?information_schema.table_constraints?where?constraint_type='PRIMARY?KEY'?and?table_schema=userName.tableName)。
In step S15, will change data and write list of modification, when SQL statement was write log sheet, watcher thread write list of modification according to the change data that some cycles will get access to, and SQL statement is write the MY_CDC_LOG_TABLE log sheet.
In step S2, export as packet, this packet comprises XML file and csv file, the SQL statement of all Data Update of XML file record operation wherein, CSV records the change data of all increases and modification.Packet can be used as the input of downstream application to finish follow-up transfer of data and to extract and process.
In step S3, the step of packet remote synchronization to target database specifically comprised:
According to network condition and synchronous urgency demand, select to adopt the mode of JMS or FTP to carry out remote synchronization.
Particularly, for the FTP mode, in target database, set up the packet catalogue, dispose ftp client program and ftp server end program, transmitting terminal is according to cycle regular hour, executing data scanning if find to have new packet file in the packet catalogue, is logged into fileinfo first in the local configuration database of transmitting terminal, then pass through File Transfer Protocol, access long-range ftp server end, with the packet File Upload, and wait for the receiving terminal return information.After receiving terminal is received described packet file, the packet file is preserved and described fileinfo is logged in the local configuration database of receiving terminal, then feed back receipt by the mode of FTP order to receiving terminal, namely send confirmation.
As follows based on the synchronous handling process of File Transfer Protocol remote data:
1, ftp client scan-data;
2, packet information to be sent is counted the repository of ftp client, be used for error correction or re-transmission;
3, connect ftp server;
4, uploading data bag, the concurrent order of losing one's life;
5, ftp server receives mentioned order, and the downloading data bag;
6, disconnecting FTP connects;
7, the packet information that receives is counted the repository of ftp server, be used for error correction or re-transmission;
8, save data bag.
For the JMS mode, front-end acquisition device is disposed receiver terminal, destination is disposed sender terminal, transmitting terminal is according to cycle regular hour, and executing data scanning is in case find to have new packet file in the packet catalogue, first fileinfo is logged in the local configuration database of transmitting terminal, then by the JMS agreement, utilize message-oriented middleware that the packet file is delivered as message attachment, and wait for the receiving terminal return information.After receiving terminal is received information, file is preserved and fileinfo is logged in the local configuration database, then to receiving terminal feedback receipt, namely send confirmation.
As follows based on JMS protocol data remote synchronization handling process:
1, JMS transmitting terminal scan-data;
2, packet information to be sent is counted the repository of JMS transmitting terminal, be used for error correction or re-transmission;
3, JMS message producer services;
4, the ActiveMQ of JMS transmitting terminal is with attachment version uploading data bag;
5, the ActiveMQ downloading data bag of JMS receiving terminal;
6, JMS information consumption person service;
7, the packet information that receives is counted the repository of JMS receiving terminal, be used for error correction or re-transmission;
8, save data bag.
In above-mentioned steps S4 and S5, packet is resolved to SQL statement is specially with the change data with SQL statement and the step of change data loading to target database:
After target database receives packet, for the XML form, resolve SQL statement, and put in storage be converted into self database format according to pre-defined rule after; For the CSV form, then carry out in-stockroom operation according to the corresponding data dictionary information structuring of target database in-stockroom operation statement.
By the above, the teledata increment synchronization method of present embodiment is by analyzing the Operation Log of source database, not only can regularly the record that increases in the database, deletes, changes be extracted and form the packet that comprises incremental record, and can be according to network condition and data synchronisation requirement, with the mode of packet with JMS or FTP, utilize message-oriented middleware or ftp client/server end product, and under certain configuration management is supported, with data packet transmission to the target database system.
Fig. 3 is the structure chart of the teledata increment synchronizer of based on data bag in the embodiment of the invention.
With reference to shown in Figure 3, present embodiment provides a kind of teledata increment synchronizer of based on data bag, and it mainly comprises:
Acquisition module 1 is for the DML that obtains source database;
Parsing module 4 is used for packet is resolved to SQL statement and change data;
Enter library module 5, be used for SQL statement and change data loading are arrived target database.
Particularly, above-mentioned packet generation module specifically is used for:
The derived data APMB package comprises XML file and csv file, the SQL statement among all DML of XML file record wherein, and CSV records the change data of all increases and modification.
Synchronization module specifically is used for: according to network condition and synchronous urgency demand, select to adopt the mode of JMS or FTP to carry out remote synchronization.
For the FTP mode, set up the packet catalogue, front-end acquisition device is disposed receiver terminal, and destination is disposed sender terminal, transmitting terminal is according to cycle regular hour, executing data scanning if find to have new packet file in the packet catalogue, is logged into fileinfo first in the local configuration database of transmitting terminal, then pass through File Transfer Protocol, access long-range ftp server end, with the packet File Upload, and wait for the receiving terminal return information.After receiving terminal is received described packet file, the packet file is preserved and described fileinfo is logged in the local configuration database of receiving terminal, then feed back receipt by the mode of FTP order to receiving terminal.
For the JMS mode, set up the packet catalogue, dispose receiver terminal and sender terminal, transmitting terminal is according to cycle regular hour, and there is new packet file if find in executing data scanning in the packet catalogue, first fileinfo is logged in the local configuration database of transmitting terminal, then by the JMS agreement, utilize message-oriented middleware that the packet file is delivered as message attachment, and wait for the receiving terminal return information.After receiving terminal is received information, file is preserved and fileinfo is logged in the local configuration database, then to receiving terminal feedback receipt, namely send confirmation.
The library module that enters of present embodiment specifically is used for:
After target database receives packet, for the XML form, resolve SQL statement, and put in storage be converted into self database format according to pre-defined rule after; For the CSV form, then carry out in-stockroom operation according to the corresponding data dictionary information structuring of target database in-stockroom operation statement.
In sum, the packet that front end extraction program of the present invention forms can be selected to send to destination in the mode of JMS or FTP.JMS can guarantee the long-range real-time delivery of fragmentary and low volume data based on message-oriented middleware; FTP supports breakpoint transmission, is fit to the remote transmission of large-scale data, and both complement one another, and have guaranteed the stability of remote synchronization process.And, packet is the interface point of leading portion extraction program and remote synchronization program, again as the backup of source end system incremental data, can get up by persistence, even in extremely abominable in network environment like this even the situation that access to netwoks is limited, the physical medium that also can take mode under the line will preserve packet is sent to destination, has the synchronous or not available function of transmitting device of traditional teledata.
The above only is the preferred embodiments of the present invention; be not so limit its claim; every equivalent structure or equivalent flow process conversion that utilizes specification of the present invention and accompanying drawing content to do; directly or indirectly be used in other relevant technical fields, all in like manner be included in the scope of patent protection of the present invention.
Claims (10)
1. the teledata increment synchronization method of a based on data bag is characterized in that, may further comprise the steps:
Obtain the DML of source database;
Described DML is exported as packet;
With described packet remote synchronization to target database;
Described packet is resolved to SQL statement and change data;
With described SQL statement and the change Data Update to target database.
2. teledata increment synchronization method as claimed in claim 1 is characterized in that, the described step that DML is exported as packet specifically comprises:
The derived data APMB package comprises XML file and csv file, the SQL statement among all DML of XML file record wherein, and CSV records the change data of all increases and modification.
3. teledata increment synchronization method as claimed in claim 1 is characterized in that, described the step of packet remote synchronization to target database is comprised:
According to network condition and synchronous urgency demand, select to adopt the mode of JMS or FTP to carry out remote synchronization.
4. teledata increment synchronization method as claimed in claim 3 is characterized in that, described the step of packet remote synchronization to target database is specifically comprised:
For the FTP mode, set up the packet catalogue, dispose ftp client program and ftp server end program, transmitting terminal is according to cycle regular hour, executing data scanning, there is new packet file if find in the packet catalogue, first fileinfo is logged in the local configuration database of transmitting terminal, then by File Transfer Protocol, access long-range ftp server end, with the packet File Upload, and wait receiving terminal return information, after receiving terminal is received described packet file, the packet file is preserved and described fileinfo is logged in the local configuration database of receiving terminal, then feed back receipt by the mode of FTP order to receiving terminal;
For the JMS mode, set up the packet catalogue, dispose receiver terminal and sender terminal, transmitting terminal is according to cycle regular hour, executing data scanning, there is new packet file if find in the packet catalogue, first fileinfo is logged in the local configuration database of transmitting terminal, then by the JMS agreement, utilize message-oriented middleware that the packet file is delivered as message attachment, and wait for the receiving terminal return information, after receiving terminal is received information, file is preserved and fileinfo is logged in the local configuration database, then to receiving terminal feedback receipt.
5. teledata increment synchronization method as claimed in claim 1 is characterized in that, described SQL statement and change data loading is comprised to the step of target database:
After target database receives packet, for the XML form, resolve SQL statement, and put in storage be converted into self database format according to pre-defined rule after; For the CSV form, then carry out in-stockroom operation according to the corresponding data dictionary information structuring of target database in-stockroom operation statement.
6. the teledata increment synchronizer of a based on data bag is characterized in that, comprising:
Acquisition module is for the DML that obtains source database;
The packet generation module is used for described DML is exported as packet;
Synchronization module is used for described packet remote synchronization to target database;
Parsing module is used for described packet is resolved to SQL statement and change data;
Enter library module, be used for described SQL statement and change Data Update to target database.
7. teledata increment synchronizer as claimed in claim 6 is characterized in that, described packet generation module specifically is used for:
The derived data APMB package comprises XML file and csv file, the SQL statement among all DML of XML file record wherein, and CSV records the change data of all increases and modification.
8. teledata increment synchronizer as claimed in claim 6 is characterized in that described synchronization module specifically is used for:
According to network condition and synchronous urgency demand, select to adopt the mode of JMS or FTP to carry out remote synchronization.
9. teledata increment synchronizer as claimed in claim 8 is characterized in that described synchronization module also is used for:
For the FTP mode, set up the packet catalogue, dispose ftp client program and ftp server end program, transmitting terminal is according to cycle regular hour, executing data scanning, there is new packet file if find in the packet catalogue, first fileinfo is logged in the local configuration database of transmitting terminal, then by File Transfer Protocol, access long-range ftp server end, with the packet File Upload, and wait receiving terminal return information, after receiving terminal is received described packet file, the packet file is preserved and described fileinfo is logged in the local configuration database of receiving terminal, then feed back receipt by the mode of FTP order to receiving terminal;
For the JMS mode, set up the packet catalogue, dispose receiver terminal and sender terminal, transmitting terminal is according to cycle regular hour, executing data scanning, there is new packet file if find in the packet catalogue, first fileinfo is logged in the local configuration database of transmitting terminal, then by the JMS agreement, utilize message-oriented middleware that the packet file is delivered as message attachment, and wait for the receiving terminal return information, after receiving terminal is received information, file is preserved and fileinfo is logged in the local configuration database, then to receiving terminal feedback receipt.
10. teledata increment synchronizer as claimed in claim 6 is characterized in that, describedly enters library module and specifically is used for:
After target database receives packet, for the XML form, resolve SQL statement, and put in storage be converted into self database format according to pre-defined rule after; For the CSV form, then carry out in-stockroom operation according to the corresponding data dictionary information structuring of target database in-stockroom operation statement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210568861.0A CN103067483B (en) | 2012-12-25 | 2012-12-25 | Teledata increment synchronization method based on packet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210568861.0A CN103067483B (en) | 2012-12-25 | 2012-12-25 | Teledata increment synchronization method based on packet |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103067483A true CN103067483A (en) | 2013-04-24 |
| CN103067483B CN103067483B (en) | 2017-04-05 |
Family
ID=48109937
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210568861.0A Active CN103067483B (en) | 2012-12-25 | 2012-12-25 | Teledata increment synchronization method based on packet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103067483B (en) |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103607470A (en) * | 2013-11-29 | 2014-02-26 | 北京中电普华信息技术有限公司 | File transfer system and method |
| CN104063301A (en) * | 2014-07-02 | 2014-09-24 | 网神信息技术（北京）股份有限公司 | Monitoring method and device |
| CN104317921A (en) * | 2014-10-29 | 2015-01-28 | 北京思特奇信息技术股份有限公司 | Cross-database data synchronizing system and method |
| CN104468274A (en) * | 2014-12-16 | 2015-03-25 | 深圳大学 | Cluster monitor and management method and system |
| CN104618434A (en) * | 2014-12-23 | 2015-05-13 | 上海斐讯数据通信技术有限公司 | Synchronization method and system for mPortal system data |
| WO2015188488A1 (en) * | 2014-06-12 | 2015-12-17 | 中兴通讯股份有限公司 | Method and apparatus for generating database dictionary of word documents |
| CN105183860A (en) * | 2015-09-10 | 2015-12-23 | 北京京东尚科信息技术有限公司 | Data synchronization method and system |
| CN105335170A (en) * | 2014-06-05 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Distributed system and incremental data updating method |
| CN105373621A (en) * | 2015-12-07 | 2016-03-02 | 高新兴科技集团股份有限公司 | Rapid database-system-across data increment migration method |
| CN105391791A (en) * | 2015-11-26 | 2016-03-09 | 努比亚技术有限公司 | Upgrade system and upgrade method thereof |
| CN106682098A (en) * | 2016-12-01 | 2017-05-17 | 无线生活（杭州）信息科技有限公司 | DML processing method and device |
| CN107357848A (en) * | 2017-06-27 | 2017-11-17 | 中国电子科技集团公司第二十八研究所 | Database synchronization method based on driving encapsulation |
| CN107491558A (en) * | 2017-09-08 | 2017-12-19 | 北京奇艺世纪科技有限公司 | Metadata updates method and device |
| CN107562883A (en) * | 2017-09-04 | 2018-01-09 | 马上消费金融股份有限公司 | Data synchronization method and system |
| CN108052681A (en) * | 2018-01-12 | 2018-05-18 | 毛彬 | The synchronous method and system of structural data between a kind of relevant database |
| CN108200220A (en) * | 2018-04-08 | 2018-06-22 | 武汉斗鱼网络科技有限公司 | A kind of method of data synchronization, server and storage medium |
| CN108334621A (en) * | 2018-02-08 | 2018-07-27 | 平安科技（深圳）有限公司 | Database operation method, device, equipment and computer readable storage medium |
| CN108733823A (en) * | 2018-05-22 | 2018-11-02 | 浪潮软件股份有限公司 | A kind of system and method that ORACLE database intranet and extranet data double-ways are synchronized |
| CN110084335A (en) * | 2019-03-26 | 2019-08-02 | 天津大学 | A kind of both-end commodity information management system based on bar code scanning |
| CN110134851A (en) * | 2019-05-05 | 2019-08-16 | 北京科技大学 | A kind of search engine system and construction method based on field Intranet |
| CN110232097A (en) * | 2019-06-21 | 2019-09-13 | 北京奇艺世纪科技有限公司 | A kind of method of data synchronization and device |
| CN110502584A (en) * | 2019-08-28 | 2019-11-26 | 北京三快在线科技有限公司 | The synchronous method and apparatus of data |
| CN110543513A (en) * | 2018-05-29 | 2019-12-06 | 中兴通讯股份有限公司 | incremental data synchronization method, equipment and storage medium for distributed database |
| CN110569142A (en) * | 2019-08-29 | 2019-12-13 | 天津大学 | ORACLE data increment synchronization system and method |
| CN112163037A (en) * | 2020-09-17 | 2021-01-01 | 彩讯科技股份有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN113656366A (en) * | 2021-08-03 | 2021-11-16 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
| CN113986914A (en) * | 2021-09-30 | 2022-01-28 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Tree type cascade database synchronization system and method based on data increment |
| CN115391457A (en) * | 2022-08-23 | 2022-11-25 | 昆仑数智科技有限责任公司 | Cross-database data synchronization method and device and storage medium |
| CN115442361A (en) * | 2022-08-30 | 2022-12-06 | 重庆长安汽车股份有限公司 | Synchronous transmission method, system, equipment and medium for large-batch files |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289469A (en) * | 2011-07-26 | 2011-12-21 | 国电南瑞科技股份有限公司 | Method for supporting universal database to synchronize data based on physical isolating equipment |
| CN102752372A (en) * | 2012-06-18 | 2012-10-24 | 天津神舟通用数据技术有限公司 | File based database synchronization method |
-
2012
- 2012-12-25 CN CN201210568861.0A patent/CN103067483B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289469A (en) * | 2011-07-26 | 2011-12-21 | 国电南瑞科技股份有限公司 | Method for supporting universal database to synchronize data based on physical isolating equipment |
| CN102752372A (en) * | 2012-06-18 | 2012-10-24 | 天津神舟通用数据技术有限公司 | File based database synchronization method |
Cited By (41)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103607470A (en) * | 2013-11-29 | 2014-02-26 | 北京中电普华信息技术有限公司 | File transfer system and method |
| CN105335170A (en) * | 2014-06-05 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Distributed system and incremental data updating method |
| WO2015188488A1 (en) * | 2014-06-12 | 2015-12-17 | 中兴通讯股份有限公司 | Method and apparatus for generating database dictionary of word documents |
| CN104063301A (en) * | 2014-07-02 | 2014-09-24 | 网神信息技术（北京）股份有限公司 | Monitoring method and device |
| CN104317921A (en) * | 2014-10-29 | 2015-01-28 | 北京思特奇信息技术股份有限公司 | Cross-database data synchronizing system and method |
| CN104317921B (en) * | 2014-10-29 | 2017-10-17 | 北京思特奇信息技术股份有限公司 | A kind of integration across database data synchronous system and method |
| CN104468274A (en) * | 2014-12-16 | 2015-03-25 | 深圳大学 | Cluster monitor and management method and system |
| CN104618434A (en) * | 2014-12-23 | 2015-05-13 | 上海斐讯数据通信技术有限公司 | Synchronization method and system for mPortal system data |
| CN105183860B (en) * | 2015-09-10 | 2018-10-19 | 北京京东尚科信息技术有限公司 | Method of data synchronization and system |
| CN105183860A (en) * | 2015-09-10 | 2015-12-23 | 北京京东尚科信息技术有限公司 | Data synchronization method and system |
| CN105391791A (en) * | 2015-11-26 | 2016-03-09 | 努比亚技术有限公司 | Upgrade system and upgrade method thereof |
| CN105373621A (en) * | 2015-12-07 | 2016-03-02 | 高新兴科技集团股份有限公司 | Rapid database-system-across data increment migration method |
| CN106682098A (en) * | 2016-12-01 | 2017-05-17 | 无线生活（杭州）信息科技有限公司 | DML processing method and device |
| CN107357848A (en) * | 2017-06-27 | 2017-11-17 | 中国电子科技集团公司第二十八研究所 | Database synchronization method based on driving encapsulation |
| CN107357848B (en) * | 2017-06-27 | 2019-11-26 | 中国电子科技集团公司第二十八研究所 | Database synchronization method based on driving encapsulation |
| CN107562883A (en) * | 2017-09-04 | 2018-01-09 | 马上消费金融股份有限公司 | Data synchronization method and system |
| CN107562883B (en) * | 2017-09-04 | 2018-10-26 | 马上消费金融股份有限公司 | Data synchronization method and system |
| CN107491558A (en) * | 2017-09-08 | 2017-12-19 | 北京奇艺世纪科技有限公司 | Metadata updates method and device |
| CN107491558B (en) * | 2017-09-08 | 2020-09-15 | 北京奇艺世纪科技有限公司 | Metadata updating method and device |
| CN108052681A (en) * | 2018-01-12 | 2018-05-18 | 毛彬 | The synchronous method and system of structural data between a kind of relevant database |
| CN108334621A (en) * | 2018-02-08 | 2018-07-27 | 平安科技（深圳）有限公司 | Database operation method, device, equipment and computer readable storage medium |
| CN108200220A (en) * | 2018-04-08 | 2018-06-22 | 武汉斗鱼网络科技有限公司 | A kind of method of data synchronization, server and storage medium |
| CN108733823A (en) * | 2018-05-22 | 2018-11-02 | 浪潮软件股份有限公司 | A kind of system and method that ORACLE database intranet and extranet data double-ways are synchronized |
| CN110543513A (en) * | 2018-05-29 | 2019-12-06 | 中兴通讯股份有限公司 | incremental data synchronization method, equipment and storage medium for distributed database |
| CN110543513B (en) * | 2018-05-29 | 2024-03-15 | 金篆信科有限责任公司 | Incremental data synchronization method, equipment and storage medium for distributed database |
| CN110084335A (en) * | 2019-03-26 | 2019-08-02 | 天津大学 | A kind of both-end commodity information management system based on bar code scanning |
| CN110134851B (en) * | 2019-05-05 | 2021-10-15 | 北京科技大学 | Search engine system based on domain intranet and construction method |
| CN110134851A (en) * | 2019-05-05 | 2019-08-16 | 北京科技大学 | A kind of search engine system and construction method based on field Intranet |
| CN110232097A (en) * | 2019-06-21 | 2019-09-13 | 北京奇艺世纪科技有限公司 | A kind of method of data synchronization and device |
| CN110502584A (en) * | 2019-08-28 | 2019-11-26 | 北京三快在线科技有限公司 | The synchronous method and apparatus of data |
| CN110502584B (en) * | 2019-08-28 | 2021-09-28 | 北京三快在线科技有限公司 | Data synchronization method and device |
| CN110569142A (en) * | 2019-08-29 | 2019-12-13 | 天津大学 | ORACLE data increment synchronization system and method |
| CN112163037A (en) * | 2020-09-17 | 2021-01-01 | 彩讯科技股份有限公司 | Data synchronization method and device, computer equipment and storage medium |
| CN113656366A (en) * | 2021-08-03 | 2021-11-16 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
| CN113656366B (en) * | 2021-08-03 | 2024-05-03 | 北京数码大方科技股份有限公司 | Data processing method and device for electronic warehouse |
| CN113986914A (en) * | 2021-09-30 | 2022-01-28 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Tree type cascade database synchronization system and method based on data increment |
| CN113986914B (en) * | 2021-09-30 | 2023-01-24 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Tree type cascade database synchronization system and method based on data increment |
| CN115391457A (en) * | 2022-08-23 | 2022-11-25 | 昆仑数智科技有限责任公司 | Cross-database data synchronization method and device and storage medium |
| CN115391457B (en) * | 2022-08-23 | 2023-09-12 | 昆仑数智科技有限责任公司 | Cross-database data synchronization method, device and storage medium |
| CN115442361A (en) * | 2022-08-30 | 2022-12-06 | 重庆长安汽车股份有限公司 | Synchronous transmission method, system, equipment and medium for large-batch files |
| CN115442361B (en) * | 2022-08-30 | 2024-06-04 | 重庆长安汽车股份有限公司 | Synchronous transmission method, system, equipment and medium for large-batch files |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103067483B (en) | 2017-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103067483A (en) | Remote data increment synchronization method and device based on data package | |
| US10956362B1 (en) | Searching archived data | |
| EP3602341B1 (en) | Data replication system | |
| US11520772B2 (en) | Tracking change data in a database | |
| US10891297B2 (en) | Method and system for implementing collection-wise processing in a log analytics system | |
| CA2665951C (en) | Asynchronous information sharing system | |
| US7565381B2 (en) | Smart synchronization using created manifest | |
| US8374966B1 (en) | In memory streaming with disk backup and recovery of messages captured from a database redo stream | |
| US7031974B1 (en) | Replicating DDL changes using streams | |
| US20140172886A1 (en) | System and method for conversion of jms message data into database transactions for application to multiple heterogeneous databases | |
| US20170139956A1 (en) | Dynamic data-ingestion pipeline | |
| US20070255763A1 (en) | Database replication method and system | |
| US9131010B2 (en) | Delay-tolerant and loss-tolerant data transfer for mobile applications | |
| CN111125260A (en) | Data synchronization method and system based on SQL Server | |
| CN109284312A (en) | A kind of heterogeneous database change real-time informing method | |
| CN101222449A (en) | Publish/subscribe system | |
| CN104765840A (en) | Big data distributed storage method and device | |
| CN103488793A (en) | User behavior monitoring method based on information retrieval | |
| CN107291938B (en) | Order inquiry system and method | |
| CN116980475B (en) | Data pushing system based on binlog and double annular buffer areas | |
| CN112053150A (en) | Data processing method, device and storage medium | |
| Donselaar | Low latency asynchronous database synchronization and data transformation using the replication log. | |
| CN117609315B (en) | Data processing method, device, equipment and readable storage medium | |
| CN118277118A (en) | Data acquisition method, data synchronization method and system of server cluster | |
| CN116578616A (en) | Data management method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |