CN106485465A - A kind of configurable timing data synchronous method - Google Patents
A kind of configurable timing data synchronous method Download PDFInfo
- Publication number
- CN106485465A CN106485465A CN201610842182.6A CN201610842182A CN106485465A CN 106485465 A CN106485465 A CN 106485465A CN 201610842182 A CN201610842182 A CN 201610842182A CN 106485465 A CN106485465 A CN 106485465A
- Authority
- CN
- China
- Prior art keywords
- data
- configuration
- information
- node
- synchronous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/109—Time management, e.g. calendars, reminders, meetings or time accounting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Operations Research (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Marketing (AREA)
- Computing Systems (AREA)
- Economics (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of configurable timing data synchronous method, belongs to Data Interchange Technology field, the present invention comprises the steps：1), synchrodata information configuration DbCrmJob.config；2) data syn-chronization task timing configured file jobs.xml；3) daily record configuration file JobScheduler.exe.config；4), SQL Server generates field configuration SQL.Solve the data synchronization problems between enterprises heterogeneous system.It is achieved in that the data exchange realizing based on data base's aspect between system, existing business system is no affected.
Description
Technical field
The present invention relates to Data Interchange Technology, more particularly, to a kind of configurable timing data synchronous method.Solve enterprise
Data synchronization problems between internal heterogeneous system.Mainly utilize computer multiple thread technology, XML technology, database technology real
Existing.There is easy configuration, rapid deployment, flexible expansion, and for particular system (such as CRM and ERP system it
Between), after once configuring, reusable, be conducive to the accumulation of configuration template.
Background technology
In enterprise, due to the difference of development time or developing department, often have multiple isomeries, operate in different soft
Information system on hardware platform is run simultaneously, and the data source of these systems is independent of one another, mutually closing is so that data is difficult to
Exchange, shared and fusion between system, thus define " information island ".With deepening continuously of information-based application, in enterprise
The demand that portion, enterprise are interacted with external information is increasingly strong, needs in a hurry existing information is integrated, and " information is lonely for UNICOM
Island ", shared information.
Data integration：Data integration pass through application between data exchange thus reaching integrated, mainly solve data distribution
Property and isomerism problem, its premise be integrated applying must public data structure, must disclose table structure, close between table
System, implication of coding etc..
The common solution for data set, one is the side directly accessing the data base to method, system in our systems
Formula.Such mode real-time property is good, does not postpone.Shortcoming is that also ratio is larger for poor, the interactive risk of safety,
Method, system may be worried to upset data, affect their system, whether our systems also may be used because of other side's system database
With and impacted.Second is taking back to the data of method, system, be put into the lane database of our system.
Content of the invention
For above-mentioned second situation, the present invention proposes a kind of configurable timing data synchronous method, and it can be by
Data timing from certain system is synchronized in goal systems.Data syn-chronization it should be understood that 5 aspect information, the source of data,
Data in source, target, should be put into target where also have synchronizing frequency.
1st, synchrodata information configuration DbCrmJob.config.
Need the information configuring to include the source of data, purpose in this file, need synchronous data, data syn-chronization
Direction and step.Main node is as follows：
Database information configuration node, configuration data carry out source and destination database linkage information, according to actual items
Environment modification configuration.
WebService information configuration node, configuration data come source and destination be UpdateCrm Interface company
Connect information, according to the environment modification configuration of actual items, typically only need to change the IP address of interface server.
Synchrodata configuration node, configuration needs synchronous data.Each InfObject is equivalent to one of data base
Table, Field is equivalent to the field in table.If the table structure of synchronous both sides is identical, only one can be configured, if the table of both sides
Structure difference is required for configuring.
Data syn-chronization task configuration node, the synchronous direction of configuration data, the step of synchronization, certain data will be derived from
What data syn-chronization in storehouse or UpdateCRM is to purpose data base or UpdateCRM.Each Task represents that a synchronization is appointed
Business, can comprise multiple synchronizing step Item, and Item is the specific data needing synchronization.
2nd, data syn-chronization task timing configured file jobs.xml.
The timing execution information of the main data syn-chronization task to DbCrmJobCfg.xml configuration configures.Configuration section
Point is as follows：
Main configuration jobs node Job and Trigger of time node trigger.Task pair in Job and DbCrmJobCfg
Should, the name of Job needs, other reference explanatory notes identical with the taskName of Task in DbCrmJob.Config.
Trigger configures the execution time of operation, and job-name therein needs the name of job node identical.Trigger
Timing be divided into two kinds of Simple and cron, different demands can be met.
Allow task only sometime executing once if necessary, or, sometime starting, then according to certain when
Between be spaced and repeat, briefly, if you wish to allowing trigger in August in 2007 morning 11 on the 20th：23：Execution in 54 seconds, so
Each repeated once every 10 seconds afterwards, and was so repeated 5 times.So Simple just can meet your requirement.
If you need to trigger task by schedule as calendar, rather than as Simpler between specific
Every Time Triggered, cron mode is usually used and configures timing.Using cron, you can specify such as " in each Friday
Noon ", or " the 9 of every workday:30 " or " from each Monday, Wednesday, Friday the morning 9：00 to the morning 10：Between 00
Every five minutes " so schedule to trigger.Even, as Simple, Cron also has a StartTime to specify day
When, from the beginning of journey is from, (optional) EndTime is also had not continue to specify when schedule.
Cron expression formula is illustrated
The simple expression formula triggering once every 5 minutes of example 1 one
"0 0/5***？"
Example 2 expression formula (such as .10 once every triggering in 5 minutes behind per minute 10 seconds:00:10am,10:05:
10 etc.).
"10 0/5***？"
Example 3 is the 10 of each Wednesday and Friday：30,11：30,12：The expression formula of 30 triggerings.
"0 30 10-13？*WED,FRI"
Example 4 in every month No. 5, between 8 points of No. 20 and 10 points, every half an hour, triggering once and not includes 10
Point, simply 8：30,9：00 and 9：30 expression formula.
"0 0/30 8-9 5,20*？" note, for independent trigger, some schedule demands may excessively complicated and
Can not be stated with expression formula, for example：9：00 to 10：Between 00 every triggering in 5 minutes once, afternoon 1：00 to 10 o'clock every 20 minutes
Clock triggers once.This solution is exactly to create two triggers, and two triggers all run identical task.
3rd, daily record configuration file JobScheduler.exe.config.
Daily record output configuration to the operation information of program, tasks carrying information, error message.Program default configuration is mistake
False information export file (run catalogue under log catalogue) and by error log send mail to specify maintenance people
Member.Running log for program can run logView.bat and checks real-time logs.
Mainly it is amended as follows information during deployment：
<Subject value=" JobScheduler Log [test event] "/>
<To value=" * * *@inspur.com "/>
[test event] is revised as the project name of reality
***@inspur.comThe addresses of items of mail of modification attendant
4th, SQL Server generates field configuration SQL
The invention has the beneficial effects as follows
Based on XML configuration, flexible configuration, use are simple；
Support that full dose updates and incremental update；
Two-way Time data syn-chronization between support system；
Flexible timing strategies, repeat and particular point in time execution including the fixing frequency, support cron expression formula；
Prefabricated crm system and ERP system data processing template, facilitate the rapid deployment between particular system, improve efficiency；
Using service arrangement mode, stable, automatically restart after os starting, need not artificially participate in；
, mistake and automatically sends mail to attendant in complete feedback mechanism, in time solution, Debugging message and operation
Information can decide whether to export by configuring.
Brief description
Fig. 1 is DbCrmJob.xml and Jobs.xml each configuration node relation schematic diagram.
Specific embodiment
Below more detailed elaboration is carried out to present disclosure：
A kind of configurable timing data synchronous method and device that the present invention provides, are achieved in that based on database layer
Face, to realize the data exchange between system, no affects on existing business system.Support SqlServer/Oracle with
Two-way Time data syn-chronization between SqlServer/Oracle, between SqlServer/Oracle and UpdateCRM system.
Mainly realize code as follows：
Feature of the present invention
1st, complete data syn-chronization in data base's aspect, support two kinds of data bases of Oracle, SqlServer, prefab data storehouse
Access layer interface, other data bases can be supported by extension；
2nd, dispose a set of program, by configuring, the data exchange between any number of systems can be supported；
3rd, can be configured by xml, Sql and three kinds of modes of storing process obtain data；
4th, can according to the time format of the requirement configuration field of different system, such as 20160819,2016-08-19,2016/
08/09；
5th, a lot number can be generated for the batch of data of each data syn-chronization, with the process after being easily introduced into；
6th, LSN can be generated for every batch data；
7th, for master-salve table, master meter data can be entered with guide, then import from table data, after then being done by this batch number
Continuous process；
8th, for target data record, can be with configuration data lock in time information (when synchronous data)；
9th, can the every data of automatic decision be to update or add；
10th, support that full dose updates and incremental update；
11st, deposit and miss the journey flexible temporal expression of support；
12nd, support to call the storing process of source database before synchronization, to do pretreatment to this hyposynchronous data；
13rd, after synchronization can with the storing process of invocation target data base, so that post processing is done to this hyposynchronous data,
Such as data conversion is done to this lot data；
14th, data syn-chronization timing, supports Cron expression formula, can such as not have execution in 5 seconds by the fixing frequency, and specific
Time dot cycle executes, such as expression formula " 00 23**？" represent 11 points of execution every night.
15th, can be with the output of configuration log, such as debugging log, running log, error log etc.
16th, after mistake occurring, error log is sent mail to safeguard people, to solve a problem promptly.
Claims (5)
1. a kind of configurable timing data synchronous method is it is characterised in that comprise the steps：
1), synchrodata information configuration DbCrmJob.config；
2) data syn-chronization task timing configured file jobs.xml；
3) daily record configuration file JobScheduler.exe.config；
4), SQL Server generates field configuration SQL.
2. method according to claim 1 it is characterised in that
Wherein step 1), need the information configuring to include the source of data, purpose in this file, need synchronous data,
The direction of data syn-chronization and step；Main node is as follows：
Database information configuration node, configuration data carry out source and destination database linkage information, according to the environment of actual items
Modification configuration；
WebService information configuration node, the connection that source and destination is UpdateCrm Interface that comes of configuration data is believed
Breath, according to the environment modification configuration of actual items；
Synchrodata configuration node, configuration needs synchronous data；Each InfObject is equivalent to one of data base table,
Field is equivalent to the field in table；If the table structure of synchronous both sides is identical, only can configure one, if the table knot of both sides
Structure difference is required for configuring；
Data syn-chronization task configuration node, the synchronous direction of configuration data, synchronous step, will from a data base or
Data syn-chronization in UpdateCRM is to purpose data base or UpdateCRM；Each Task represents a synchronous task, comprises
Multiple synchronizing step Item, Item are the specific data needing synchronization；
3. method according to claim 1 it is characterised in that
Wherein step 2), mainly the timing execution information of the data syn-chronization task of DbCrmJobCfg.xml configuration is configured；
Configuration node is as follows：
Main configuration jobs node Job and Trigger of time node trigger；Task in Job with DbCrmJobCfg is corresponding,
The name of Job needs identical with the taskName of Task in DbCrmJob.Config；
Trigger configures the execution time of operation, and job-name therein needs the name of job node identical；Trigger determines
When be divided into two kinds of Simple and cron.
4. method according to claim 1 it is characterised in that
Wherein step 3), the daily record output configuration to the operation information of program, tasks carrying information, error message；Program is given tacit consent to
Be configured to error message export file and by error log send mail to specify attendant；Fortune for program
Row daily record can run logView.bat and check real-time logs；
Mainly it is amended as follows information during deployment：
<Subject value=" JobScheduler Log [test event] "/>
<To value=" * * *@inspur.com "/>
[test event] is revised as the project name of reality
***@inspur.comThe addresses of items of mail of modification attendant.
5. method according to claim 1 it is characterised in that
Wherein step 4, code is as follows
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610842182.6A CN106485465A (en) | 2016-09-22 | 2016-09-22 | A kind of configurable timing data synchronous method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610842182.6A CN106485465A (en) | 2016-09-22 | 2016-09-22 | A kind of configurable timing data synchronous method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106485465A true CN106485465A (en) | 2017-03-08 |
Family
ID=58268798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610842182.6A Pending CN106485465A (en) | 2016-09-22 | 2016-09-22 | A kind of configurable timing data synchronous method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106485465A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107193579A (en) * | 2017-03-29 | 2017-09-22 | 武汉斗鱼网络科技有限公司 | The implementation method and device of plan target |
| CN107704607A (en) * | 2017-10-17 | 2018-02-16 | 武汉楚鼎信息技术有限公司 | A kind of method of database in phase |
| CN108154427A (en) * | 2017-12-01 | 2018-06-12 | 上海富利通信息系统有限公司 | A kind of data detection method, device and electronic equipment |
| CN108170576A (en) * | 2017-12-26 | 2018-06-15 | 广东欧珀移动通信有限公司 | log processing method, device, terminal device and storage medium |
| CN108170585A (en) * | 2017-12-26 | 2018-06-15 | 广东欧珀移动通信有限公司 | log processing method, device, terminal device and storage medium |
| CN112148854A (en) * | 2020-09-16 | 2020-12-29 | 出门问问(苏州)信息科技有限公司 | Conversation management method and device |
| CN112307114A (en) * | 2019-07-31 | 2021-02-02 | 北京中关村科金技术有限公司 | Data exchange method, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617176A (en) * | 2013-11-04 | 2014-03-05 | 广东电子工业研究院有限公司 | Method for achieving automatic synchronization of multi-source heterogeneous data resources |
| CN105760174A (en) * | 2016-03-09 | 2016-07-13 | 浪潮软件集团有限公司 | Suggested data extraction method based on Windows timed task |
| CN105956207A (en) * | 2016-07-01 | 2016-09-21 | 杭州帕拉迪网络科技有限公司 | Binlog-based configurable mysql database real-time synchronization method |
-
2016
- 2016-09-22 CN CN201610842182.6A patent/CN106485465A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617176A (en) * | 2013-11-04 | 2014-03-05 | 广东电子工业研究院有限公司 | Method for achieving automatic synchronization of multi-source heterogeneous data resources |
| CN105760174A (en) * | 2016-03-09 | 2016-07-13 | 浪潮软件集团有限公司 | Suggested data extraction method based on Windows timed task |
| CN105956207A (en) * | 2016-07-01 | 2016-09-21 | 杭州帕拉迪网络科技有限公司 | Binlog-based configurable mysql database real-time synchronization method |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107193579A (en) * | 2017-03-29 | 2017-09-22 | 武汉斗鱼网络科技有限公司 | The implementation method and device of plan target |
| CN107193579B (en) * | 2017-03-29 | 2021-04-27 | 武汉斗鱼网络科技有限公司 | Method and device for realizing planned tasks |
| CN107704607A (en) * | 2017-10-17 | 2018-02-16 | 武汉楚鼎信息技术有限公司 | A kind of method of database in phase |
| CN108154427A (en) * | 2017-12-01 | 2018-06-12 | 上海富利通信息系统有限公司 | A kind of data detection method, device and electronic equipment |
| CN108170576A (en) * | 2017-12-26 | 2018-06-15 | 广东欧珀移动通信有限公司 | log processing method, device, terminal device and storage medium |
| CN108170585A (en) * | 2017-12-26 | 2018-06-15 | 广东欧珀移动通信有限公司 | log processing method, device, terminal device and storage medium |
| CN108170576B (en) * | 2017-12-26 | 2021-12-07 | Oppo广东移动通信有限公司 | Log processing method and device, terminal equipment and storage medium |
| CN112307114A (en) * | 2019-07-31 | 2021-02-02 | 北京中关村科金技术有限公司 | Data exchange method, device and storage medium |
| CN112148854A (en) * | 2020-09-16 | 2020-12-29 | 出门问问(苏州)信息科技有限公司 | Conversation management method and device |
| CN112148854B (en) * | 2020-09-16 | 2024-01-26 | 出门问问创新科技有限公司 | Dialogue management method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106485465A (en) | A kind of configurable timing data synchronous method | |
| Wohed et al. | Pattern based analysis of BPEL4WS | |
| US6898791B1 (en) | Infospheres distributed object system | |
| US20180225194A1 (en) | Method and system for implementing an automation software testing and packaging framework with entitlements | |
| CN104423953B (en) | A kind of SCADA timings data processing script execution system and method | |
| US20110022190A1 (en) | System for scheduling using an external calendaring service | |
| CN102291464A (en) | System and method for dynamically generating Web Service by business flow in BPM (Business Process Management) | |
| CN102867219A (en) | System and method for automatically scheduling business | |
| CN109992571A (en) | A kind of intelligent substation configuration file line closed loop check system and method for calibration | |
| CN102724304A (en) | Information warehouse federation in subscription/release system and data synchronization method | |
| CN113258669B (en) | Networked intelligent scheduling method and terminal based on distribution network maintenance | |
| CN109743399B (en) | Internal and external network data transmission method and system for multi-task scheduling of physical examination center | |
| CN108108460A (en) | A kind of standardized work flows intelligent management system framework and system | |
| Taweel et al. | Communication, knowledge and co-ordination management in globally distributed software development: Informed by a scientific software engineering case study | |
| CN110287006A (en) | A kind of data dispatching method and system | |
| Ranno et al. | 25 A system for specifying and coordinating the execution of reliable distributed applications | |
| WO2024032298A1 (en) | Hosting system for scenarios and configurations in power system, and hosting method | |
| CN100547967C (en) | Mesh operation supervisor and grid job management method | |
| CN105227648A (en) | A kind of subway comprehensive monitoring system time sequence triggering method based on time bucket | |
| Beder et al. | On applying coordinated atomic actions and dependable software architectures for developing complex systems | |
| Wang et al. | A state synchronization mechanism for orchestrated processes | |
| CN113592453A (en) | Information system operation compliance examination method and system based on block chain | |
| Donini et al. | Testing complex safety-critical systems in SOA context | |
| Gritskevich et al. | Research of the mechanisms for coordinating the work of personnel when implementing 3D technologies | |
| Shi et al. | Research and Implementation of Nuclear Power Overhaul Schedule Automatic Control Platform Based on WebService |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170308 |