CN105117441A - Data work order processing method and system - Google Patents
Data work order processing method and system Download PDFInfo
- Publication number
- CN105117441A CN105117441A CN201510490867.4A CN201510490867A CN105117441A CN 105117441 A CN105117441 A CN 105117441A CN 201510490867 A CN201510490867 A CN 201510490867A CN 105117441 A CN105117441 A CN 105117441A
- Authority
- CN
- China
- Prior art keywords
- data
- work order
- data work
- sql statement
- interface table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Abstract
The present invention discloses a data work order processing method and system. The method comprises: searching for a corresponding data work order interface table according to an identifier of a current target database; scanning and reading a data work order in the data work order interface table; analyzing a data packet in the data work order, and according to an analysis result, constructing an executable SQL statement by using the data packet; and executing the SQL statement in the current target database, and synchronizing the data work order to the current target database. According to the technical scheme, a method for moving data work orders to several target databases in a unified mode is provided. Compared with the prior art, wherein moving a data work order into each single target database requires separate processing, the data work order processing method provided by the present invention reduces development and operation costs; provided that a newly-added target database and a corresponding data work order interface table are configured, separate development for the newly-added target database is no longer needed, and only one set of programs needs to be maintained, thereby moving a data work order into a database more conveniently and quickly.
Description
Technical field
The present invention relates to technical field of data processing, be specifically related to a kind of data worksheet method and system.
Background technology
Each telecommunication system achieves the decoupling zero of CRM and form, resource, sales service and data, therefore after CRM accepts business, need the data increment of correspondence to be synchronized to object library system, in object library system, need resolution data work order, data work order is risen and lands to object library.
A business is accepted in CRM business side, such as to open an account participation marketing activity, need customer data feeding base library of opening an account, form storehouse, such as, marketing data is sent to marketing storehouse, resource data is sent to resources bank, be process separately each object library at present, this needs, for corresponding each object library develops a set of program, to increase cost of development and O&M cost.
Summary of the invention
Technical matters to be solved by this invention is to provide a kind of data worksheet method and system, decreases exploitation and the operation cost of data processing.
The technical scheme that the present invention solves the problems of the technologies described above is as follows:
On the one hand, the invention provides a kind of data worksheet method, described method comprises:
S1, the data work order interface table corresponding according to current target data storehouse identified query, described target database mark identifies one_to_one corresponding with data work order interface table;
Data work order in S2, the described data work order interface table of scanning reading;
S3, the data message of resolving in described data work order, and according to analysis result, described data message is configured to executable SQL statement;
S4, perform described SQL statement in described current target data storehouse, described data work order is synchronized to described current target data storehouse.
On the other hand, the invention provides a kind of data worksheet system, described system comprises background server and at least one target database;
Described background server comprises:
Interface table enquiry module, for the data work order interface table corresponding according to current target data storehouse identified query, described target database mark identifies one_to_one corresponding with data work order interface table;
Data read module, for scanning the data work order read in described data work order interface table;
Data message parsing module, for resolving the data message in described data work order;
SQL statement constructing module, for according to analysis result, is configured to executable SQL statement by described data message;
Described target database comprises:
Execution module, for performing described SQL statement, moves into described current target data storehouse by described data work order.
A kind of data worksheet method and system provided by the invention, when needing the data work order in source database to be synchronized to several corresponding target databases, data work order interface table that can be corresponding according to current target data storehouse identifier lookup, and data work order is read in scanning from the data work order interface table of correspondence, data message in resolution data work order, according to analysis result, data message is configured to executable SQL statement, then on current target data storehouse, this SQL statement is performed, directly data work order is synchronized in current target database after executing SQL statement.By method provided by the invention, when data work order is synchronized to several target databases by needs, only need the newly-increased target database of configuration and corresponding data work order interface table, do not need to develop separately for each target database again, decrease exploitation and operation cost.
Accompanying drawing explanation
Fig. 1 is a kind of data worksheet method flow diagram of the embodiment of the present invention one;
Fig. 2 is the whole data handling procedure process flow diagram of the embodiment of the present invention one;
Fig. 3 is a kind of data worksheet system schematic of the embodiment of the present invention two.
Embodiment
Be described principle of the present invention and feature below in conjunction with accompanying drawing, example, only for explaining the present invention, is not intended to limit scope of the present invention.
Embodiment one, a kind of data worksheet method.Below in conjunction with Fig. 1-Fig. 2, the method that the present embodiment provides is described in detail.
See Fig. 1, S1, the data work order interface table corresponding according to current target data storehouse identified query, wherein, target database mark identifies one_to_one corresponding with data work order interface table.
Concrete, when source database exist more new data work order (comprise newly-increased data work order, delete data work order and Update Table work order etc.) time, the data work order of renewal can be synchronized to different target database by source database respectively, such as, there is the data work order of renewal in source database A, now need these data work orders upgraded to be synchronized in target database B, target database C and target database D.In the present embodiment, background server gives each target database correspondence configuration a data work order interface table, and wherein, data work order interface table mark identifies with target database exists relation one to one.The data work order of renewal can back up in data work order interface table corresponding to target database by source database.Therefore, see Fig. 2, when source database needs when more new data work order is synchronized to multiple target database, first can identify according to current target data storehouse and inquire about the data work order interface table corresponding with it.
Data work order in S2, the described data work order interface table of scanning reading.
Concrete, after above-mentioned background server is according to target database identified query to corresponding data work order interface table, the data work order in this data work order interface table is read in scanning.The mode that the present embodiment can adopt multi-process to walk abreast to read to read the data work order in data-interface table, can provide like this read data work orders speed and efficiency.Each process is responsible for some the data work orders reading data-interface table, and the present embodiment is also provided with the data-quantity threshold that each process reads data work order at every turn, and data congestion when reading data work order so that impact can be avoided like this to read the efficiency of data.In addition, the data work order in data-interface table generally carries timestamp, and each process, when scanning reading data work order, can be carried out order according to the timestamp of data work order and read data work order, in order to avoid read the confusion of data, accomplish without any confusion.
Data message in S3, resolution data work order, is configured to executable SQL statement according to analysis result by described data message.
Concrete, get the data work order in corresponding data work order interface table when scanning after, the data message in resolution data work order, the data message in the present embodiment is XML (ExtensibleMarkupLanguage, extend markup language) message.Because the data work order read from data work order interface table may for the data work order inserted, also may for the data work order of deleting, also may be the data work order of amendment, therefore, in the process of the data message of resolution data work order, the back end type of resolution data message, the back end type of data message can be judged according to back end mark, such as, when the back end of data message is designated " I ", then represent that this data work order is for inserting type; When the back end of data message is designated " D ", then represent that this data work order is for deleting type; When the back end of data message is designated " X ", then represent that this data work order is for amendment type.
Subsequently, according to the back end type of resolving the data message obtained, call the SQL assembly of corresponding types, data message is configured to the SQL statement of respective type, such as, for the data work order inserting type, call insertsql assembly and the data message in data work order is assembled into insert statement; For the data work order of deleting type, call deletesql assembly and the data message of data work order is assembled into delete statement; For the data work order of amendment type, call updatesql assembly and the data message of data work order is assembled into update statement.
S4, perform described SQL statement in described current target data storehouse, described data work order is synchronized to described current target data storehouse.
Concrete, when the data message in data work order is resolved, and according to after in data work order, the back end type structure of data message becomes the executable SQL statement of respective type, this SQL statement is performed in current target database, make the data work order in data work order interface table be synchronized in current target data storehouse, be also synchronized in corresponding target database by the data work order upgraded in source database.
Embodiment two, a kind of data worksheet system.Below in conjunction with Fig. 3, the system that the present embodiment provides is described in detail.
See Fig. 3, the system that the present embodiment provides comprises background server 10 and target database 20, wherein, background server 10 comprises interface table enquiry module 101, threshold setting module 102, data read module 103, data message parsing module 104 and SQL statement constructing module 105, and target database 20 comprises execution module 201.
Interface table enquiry module 101 in background server 10 is mainly used in the data work order interface table corresponding according to current target data storehouse identified query, and described target database mark identifies one_to_one corresponding with data work order interface table.
Concrete, in the present embodiment, background server 10 gives each target database 20 correspondence configuration a data work order interface table, and wherein, data work order interface table mark identifies with target database 20 exists relation one to one.The data work order of renewal can back up in the data work order interface table of target database by source database.Therefore, when source database needs when more new data work order is synchronized to multiple target database 20, the interface table enquiry module 101 in background server 10 inquires about the data work order interface table corresponding with it according to target database 20 mark.
Data read module 103 is mainly used in scanning the data work order read in data work order interface table.
Concrete, after the interface table enquiry module 101 in above-mentioned background server 10 is according to target database 20 identified query to corresponding data work order interface table, data read module 103 scans the data work order read in this data work order interface table.The mode that the present embodiment can adopt multi-process to walk abreast to read to read the data work order in data-interface table, can improve like this read data work orders speed and efficiency.Each process is responsible for some the data work orders reading data-interface table, threshold setting module 102 in background server 10 is also provided with the data-quantity threshold that each process reads data work order at every turn, and the efficiency of data is read in data congestion when data read module 103 can be avoided like this to read data work order so that impact.In addition, the data work order in data-interface table generally carries timestamp, and each process, when scanning reading data work order, can be carried out order according to the timestamp of data work order and read data work order, in order to avoid read the confusion of data, accomplish without any confusion.
Data message parsing module 104 is mainly used in the data message in resolution data work order.
SQL statement constructing module 105 is mainly used according to analysis result, and this data message is configured to executable SQL statement.
Concrete, after data read module 103 scanning reads the data work order in corresponding data work order interface table, the data message in data message parsing module 104 resolution data work order.Because the data work order read from data work order interface table may for inserting the data work order of type, also may for deleting the data work order of type, also may be the data work order of amendment type, therefore, data message parsing module 104 is in the process of the data message of resolution data work order, the back end type of resolution data message, can judge the back end type of data message according to back end mark.Subsequently, SQL statement constructing module 105, according to the back end type of resolving the data message obtained, calls the SQL assembly of corresponding types, data message is configured to the SQL statement of respective type.
Execution module 201 in target database 20 is mainly used in performing described SQL statement, described data work order is synchronized in current target data storehouse 20.
Concrete, after the data message in data work order to be configured to the executable SQL statement of respective type by SQL statement constructing module 105 according to the node type of data work order, execution module 201 in target database 20 performs this SQL statement, data work order in data work order interface table is synchronized in corresponding target database, is also synchronized in corresponding target database by the data work order upgraded in source database.
A kind of data worksheet method and system provided by the invention, when needing the data work order in source database to be synchronized to several corresponding target databases, data work order interface table that can be corresponding according to current target data storehouse identifier lookup, and data work order is read in scanning from the data work order interface table of correspondence, data message in resolution data work order, according to analysis result, data message is configured to executable SQL statement, then on current target data storehouse, this SQL statement is performed, directly data work order is synchronized in current target database after executing SQL statement.By method provided by the invention, when data work order is synchronized to several target databases by needs, only need the newly-increased target database of configuration and corresponding data work order interface table, do not need to develop separately for each target database again, decrease exploitation and operation cost; The data message of the data work order needing warehouse-in is configured to executable SQL statement, and directly in target database, perform SQL statement, just can be synchronized in target database by the data work order of correspondence, simple and convenient, the warehouse-in efficiency of data work order is high.
Claims (10)
1. a data worksheet method, is characterized in that, described method comprises:
S1, the data work order interface table corresponding according to current target data storehouse identified query, described target database mark identifies one_to_one corresponding with data work order interface table;
Data work order in S2, the described data work order interface table of scanning reading;
S3, the data message of resolving in described data work order, and according to analysis result, described data message is configured to executable SQL statement;
S4, perform described SQL statement in described current target data storehouse, described data work order is synchronized to described current target data storehouse.
2. a kind of data worksheet method as claimed in claim 1, it is characterized in that, described step S2 specifically comprises:
Employing multi-process parallel scan reads the data work order in described data work order interface table, and arranges the data-quantity threshold that each process reads data work order at every turn.
3. a kind of data worksheet method as claimed in claim 2, is characterized in that, described process reads the data work order in described data work order interface table according to the scanning of data work order timestamps ordering.
4. a kind of data worksheet method as claimed in claim 1, it is characterized in that, described step S3 specifically comprises:
Resolve the data message in described data work order, according to the back end type of described data message, described data message is configured to the performed SQL statement of respective type, wherein, described back end type at least comprises inserts type, deletion type and amendment type.
5. a kind of data worksheet method as claimed in claim 4, is characterized in that, the performed SQL statement that described data message is configured to respective type specifically comprised:
According to the back end type of described data message, this data message is assembled into the performed SQL statement of corresponding types by the SQL assembly calling corresponding types.
6. a kind of data worksheet method as claimed in claim 1, it is characterized in that, described step S4 also comprises:
When performing the failure of described SQL statement in described current target data storehouse, prompting error message, and the data work order of correspondence is stored in the failure record table in described current target data storehouse.
7. a data worksheet system, is characterized in that, described system comprises background server and at least one target database;
Described background server comprises:
Interface table enquiry module, for the data work order interface table corresponding according to current target data storehouse identified query, described target database mark identifies one_to_one corresponding with data work order interface table;
Data read module, for scanning the data work order read in described data work order interface table;
Data message parsing module, for resolving the data message in described data work order;
SQL statement constructing module, for according to analysis result, is configured to executable SQL statement by described data message;
Described target database comprises:
Execution module, for performing described SQL statement, is synchronized to described current target data storehouse by described data work order.
8. a kind of data worksheet system as claimed in claim 7, is characterized in that, described data read module, specifically comprises for scanning the data work order read in described data work order interface table:
Employing multi-process parallel scan reads the data work order in described data work order interface table;
Described background server also comprises:
Threshold setting module, reads the data-quantity threshold of data work order at every turn for arranging each process.
9. a kind of data worksheet system as claimed in claim 7, is characterized in that, described SQL statement constructing module, for according to analysis result, described data message is configured to executable SQL statement and specifically comprises:
According to the back end type of resolving the data message obtained, described data message is configured to the performed SQL statement of corresponding types, wherein, described back end type at least comprises inserts type, deletion type and amendment type.
10. a kind of data worksheet system as claimed in claim 9, is characterized in that, the performed SQL statement that described data message is configured to corresponding types specifically comprised:
According to the back end type of described data message, this data message is assembled into the performed SQL statement of corresponding types by the SQL assembly calling corresponding types.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510490867.4A CN105117441A (en) | 2015-08-11 | 2015-08-11 | Data work order processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510490867.4A CN105117441A (en) | 2015-08-11 | 2015-08-11 | Data work order processing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105117441A true CN105117441A (en) | 2015-12-02 |
Family
ID=54665431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510490867.4A Pending CN105117441A (en) | 2015-08-11 | 2015-08-11 | Data work order processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105117441A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598721A (en) * | 2016-12-27 | 2017-04-26 | Ut斯达康(深圳)技术有限公司 | Media asset data circulating method and device |
| CN107276790A (en) * | 2017-05-27 | 2017-10-20 | 北京思特奇信息技术股份有限公司 | A kind of processing method and system of business work order |
| CN108829876A (en) * | 2018-06-26 | 2018-11-16 | 山东汇贸电子口岸有限公司 | The solution and device that the pl/sql client request time-out of Oracle does not respond |
| CN109905475A (en) * | 2019-02-28 | 2019-06-18 | 上海浪潮云计算服务有限公司 | A method of cloud computing monitoring data is exported with prescribed form based on SQL |
| CN110334156A (en) * | 2019-07-10 | 2019-10-15 | 北京启迪区块链科技发展有限公司 | A kind of database synchronization method, device, server and storage medium |
| CN112131846A (en) * | 2020-09-07 | 2020-12-25 | 北京思特奇信息技术股份有限公司 | Method and system for parameter transmission based on mapping relation, electronic device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101000619A (en) * | 2006-12-25 | 2007-07-18 | 华为技术有限公司 | Data synchronous method and device based on SQL |
| CN101038591A (en) * | 2007-04-11 | 2007-09-19 | 华为技术有限公司 | Method and system for synchronizing data base |
| CN104301421A (en) * | 2014-10-24 | 2015-01-21 | 浪潮集团有限公司 | Cloud calculation real-time data synchronization method |
| CN104317921A (en) * | 2014-10-29 | 2015-01-28 | 北京思特奇信息技术股份有限公司 | Cross-database data synchronizing system and method |
| CN104572672A (en) * | 2013-10-15 | 2015-04-29 | 北大方正集团有限公司 | Heterogeneous database synchronization method and application system |
-
2015
- 2015-08-11 CN CN201510490867.4A patent/CN105117441A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101000619A (en) * | 2006-12-25 | 2007-07-18 | 华为技术有限公司 | Data synchronous method and device based on SQL |
| CN101038591A (en) * | 2007-04-11 | 2007-09-19 | 华为技术有限公司 | Method and system for synchronizing data base |
| CN104572672A (en) * | 2013-10-15 | 2015-04-29 | 北大方正集团有限公司 | Heterogeneous database synchronization method and application system |
| CN104301421A (en) * | 2014-10-24 | 2015-01-21 | 浪潮集团有限公司 | Cloud calculation real-time data synchronization method |
| CN104317921A (en) * | 2014-10-29 | 2015-01-28 | 北京思特奇信息技术股份有限公司 | Cross-database data synchronizing system and method |
Non-Patent Citations (1)
| Title |
|---|
| 鄂旭等编著: "《软件项目开发与管理》", 31 May 2013, 清华大学出版社 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106598721A (en) * | 2016-12-27 | 2017-04-26 | Ut斯达康(深圳)技术有限公司 | Media asset data circulating method and device |
| CN106598721B (en) * | 2016-12-27 | 2020-01-03 | 优地网络有限公司 | Media asset data streaming method and device |
| CN107276790A (en) * | 2017-05-27 | 2017-10-20 | 北京思特奇信息技术股份有限公司 | A kind of processing method and system of business work order |
| CN107276790B (en) * | 2017-05-27 | 2020-05-19 | 北京思特奇信息技术股份有限公司 | Method and system for processing service work order |
| CN108829876A (en) * | 2018-06-26 | 2018-11-16 | 山东汇贸电子口岸有限公司 | The solution and device that the pl/sql client request time-out of Oracle does not respond |
| CN109905475A (en) * | 2019-02-28 | 2019-06-18 | 上海浪潮云计算服务有限公司 | A method of cloud computing monitoring data is exported with prescribed form based on SQL |
| CN109905475B (en) * | 2019-02-28 | 2022-03-15 | 上海浪潮云计算服务有限公司 | Method for outputting cloud computing monitoring data in specified format based on SQL |
| CN110334156A (en) * | 2019-07-10 | 2019-10-15 | 北京启迪区块链科技发展有限公司 | A kind of database synchronization method, device, server and storage medium |
| CN112131846A (en) * | 2020-09-07 | 2020-12-25 | 北京思特奇信息技术股份有限公司 | Method and system for parameter transmission based on mapping relation, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105117441A (en) | Data work order processing method and system | |
| CN110908997B (en) | Data blood relationship construction method and device, server and readable storage medium | |
| CN108427684B (en) | Data query method and device and computing equipment | |
| CN108228814B (en) | Data synchronization method and device | |
| CN102497454B (en) | Method for realizing gray publication of application in application service platform system | |
| CN104899295B (en) | A kind of heterogeneous data source data relation analysis method | |
| CN111666326B (en) | ETL scheduling method and device | |
| US9311175B2 (en) | Method and system for processing log information | |
| US20160171505A1 (en) | Extract, transform, and load (etl) processing | |
| US20200356624A1 (en) | Collaborative document editing system and method | |
| CN109241184B (en) | Data synchronization method, device, computer equipment and storage medium | |
| CN102402559A (en) | Database upgrade script generating method and device | |
| CN108121827B (en) | Full data synchronization method and device | |
| CN105468720A (en) | Method for integrating distributed data processing systems, corresponding systems and data processing method | |
| CN104765840A (en) | Big data distributed storage method and device | |
| CN109086382B (en) | Data synchronization method, device, equipment and storage medium | |
| CN111984264B (en) | Static library generation method and device | |
| CN104899134A (en) | Automatic domain name registration server testing system and method | |
| CN114661823A (en) | Data synchronization method and device, electronic equipment and readable storage medium | |
| CN110941629A (en) | Metadata processing method, device, equipment and computer readable storage medium | |
| CN112395307A (en) | Statement execution method, statement execution device, server and storage medium | |
| CN113407511A (en) | Log aggregation method, log aggregation equipment and computer program product | |
| CN113377789A (en) | Processing method and device for database change data, computer equipment and medium | |
| CN101206649B (en) | Multitask analysis system and method | |
| CN112699183A (en) | Data processing method, system, readable storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151202 |